In this paper, we studied the effect of microscopic surface roughness on heat transfer between aluminum and water by molecular dynamic (MD) simulations and macroscopic surface roughness on heat transfer between aluminum and water by finite element (FE) method. It was observed that as the microscopic scale surface roughness increases, the thermal boundary conductance increases. At the macroscopic scale, different degrees of surface roughness were studied by finite element method. The heat transfer was observed to enhance as the surface roughness increases. Based on the studies of thermal boundary conductance as a function of system size at the molecular level, a procedure was proposed to obtain the thermal boundary conductance at the mesoscopic scale. The thermal boundary resistance at the microscopic scale obtained by MD simulations and the thermal boundary resistance at the mesoscopic scale obtained by the extrapolation procedure can be included and implemented at the interfacial elements in the finite element method at the macroscopic scale. This provides us a useful model, in which different scales of surface roughness can be included, for heat transfer analysis.