If you would like to receive your free copy of ElectronicsCooling click here to subscribe.

Download the December 2011 issue here.

As I contemplate what I want to write, it is with the realization that when you read these words, we will have come to the end of another year. Even more significantly for me, it is also with the realization that this will be my last opportunity to communicate with you, the readers, from this forum. This is because after much deliberation, I have decided to leave ElectronicsCooling.

Naturally, as I progressed through my decision process, my thoughts turned back to the many years that I have been associated with the magazine. I first joined the editorial staff in December 2000 at the invitation of the then editor-in-chief, Kaveh Azar. Although I have served 11 years as an Associate Technical Editor, in a sense my association goes back as much as 15 years, when I contributed my first article to the magazine for the May 1996 issue. I still remember the article very well, its title was “Direct Liquid Immersion Cooling for High Density Micro-Electronics” and it drew upon my many years of experience in this area at IBM. For those who may be interested, the article is still available on the ElectronicsCooling website by searching “direct liquid immersion simons.” So, I owe a debt of gratitude to my longtime friend Kaveh, for his invitation to submit that first article (as well as some subsequent articles) and especially for inviting me to join the editorial staff.

Many of you may be aware from my Calculation Corner article “A Useful Catalog of Calculation Corner Articles” in the September 2011 issue, that I have shared the writing of this column with my friend and colleague Bruce Guenin, almost since when I first joined the editorial staff. I have found it both challenging and interesting to come up with suitable analysis topics of interest to readers and amenable to explanation in two to three pages. I hope that you have found them as interesting to read as it has been for me to prepare and write. I have always derived a great sense of satisfaction when at a conference such as SEMI-THERM, readers would introduce themselves to me and tell me how much they liked and how useful they found the Calculation Corner articles. I want to sincerely thank those readers for having done so.

As you may have gathered from my words so far, I have truly enjoyed my years of working on the magazine and I have derived a true sense of accomplishment from the fruits of our (i.e. my co-editors and myself) labor. I particularly enjoyed my close working relationship with my fellow editors (Clemens Lasance and Bruce Guenin who I have already mentioned and Jim Wilson) over all these years. I have known them not only through our association as co-editors, but even before that through the many SEMI-THERM Symposia in which we were involved. I regard each of them not only as first-class thermal professionals, but equally importantly, as friends.

So, you may be wondering why I have chosen to leave at this time. As has been said by others and in particular Kaveh Azar when he gave up his position as editor-in-chief, “Every good thing must come to an end.” My decision is one that has been brewing for some time now. I find that between my engineering work, as well as other activities I would like to pursue during my leisure time, I simply no longer have the amount of time to devote to the magazine as the position of associate technical editor warrants.

In case you are wondering who will take over my position on the editorial staff, it has not yet been decided. In any event, I am confident that whoever is chosen will do a first-class job in contributing to the fine reputation ElectronicsCooling magazine has earned over the years.

In closing, I want to extend my personal thanks to you, our readers. It is your interest and support that has helped ElectronicsCooling to become the premier publication of the electronics cooling community it is today. In particular, I want to single out for recognition and thanks, the many readers over the years who have contributed articles for publication in the magazine. It is these articles that make up the content and substance of the magazine.

So, thank you all!

Figure 1. Simple heat sink.

The use of heat exchanger theory provides a good example where there is a possible benefit from thinking about the problems we are solving from more than one viewpoint. Consider the simple illustration of a heat sink shown in Figure 1. Heat sink suppliers and designers, especially in air cooled electronics, like to use a thermal resistance type description such as

R_th = 1/hA = (T_surf -T_cool-in)/q

Since the resistance can vary with the coolant velocity, information about how R_th varies with velocity may be provided. The coolant temperature of reference is the inlet temperature. While this approach is convenient, there isn’t much need to think about using a minimum amount of coolant and other constraints such as noise and prime power to move the coolant may dictate the flow rate. Engineers that come from an avionics background typically consider that the coolant will change temperature as the waste heat is added. Often, the coolant flow rate is specified in terms of flow rate per KW of heat such that the temperature rise of the coolant from inlet to exit is constant for different electronic assemblies. The mindset is to use the coolant as efficiently as possible because additional coolant is either not possible or very expensive. One of the potential benefits of applying heat exchanger theory to electronics cooling is that it can provide one way of looking at efficiency.

Figure 2. Plot of equation 4.

Moffat [2] provides a good discussion on heat exchanger theory applied to air cooled heat sinks and general heat exchanger theory can be found in most heat transfer textbooks [3]. While the theory was mostly developed to deal with two fluids, a bounding case where one of the fluid temperatures did not change (such as with condensation or evaporation) simplifies the equations and can be representative of heat sinks used to cool electronics. A uniform temperature for T_surf is only an approximation but useful for illustration. Heat exchanger analysis frequently uses what is known as the effectiveness-NTU method where the effectiveness represents the actual heat transfer divided by the maximum possible heat transfer. The NTU, or Number of Transfer Units, is a dimensionless parameter that relates the heat transfer convective resistances to the coolant flow heat capacity. While the details are beyond the scope of this short column, a typical heat sink (or cold plate) can be described with the following equations (assuming that the simplifying assumption of one surface temperature is reasonable).

1  Actual heat transfer = C_cool*(T_cool-out – T_cool-in);

        (C_cool is the mass flow times the heat capacity for the coolant)

2  Maximum possible heat transfer = C_cool*(T_surf – T_cool-in)

3  Effectiveness = E = (T_cool-out – T_cool-in)/ (T_surf – T_cool-in)

4  Effectiveness = 1 – exp(-NTU), where NTU = hA/C_cool

At this point, someone used to using resistance based concepts for heat sinks might ask why go to all this trouble. The reason for thinking about the problem in these terms is the form of equation 4 which is shown graphically in Figure 2. Note that if we want to increase the effectiveness of our heat sink we need to increase NTU. One way to increase the NTU term is to decrease the coolant heat capacity but while our effectiveness increased, the resulting temperature for the surface may not be acceptable. The other way is to increase the hA term which means either larger area or a higher effective heat transfer coefficient. The engineering challenge is to minimize the decrease in effectiveness as coolant flow rates increase. Note that the limit is when the surface temperature and the coolant exit are at the same temperature, or the system has an effectiveness of 1. This limit is formally only for a constant temperature boundary condition (as opposed to uniform flux or mixed boundary condition) and is only a theoretical maximum, but it does provide a basis for comparison. More complex systems such as cold plates with multiple heat sources or significant coolant temperature variation may require more detail to assess but the general trends are similar. Historical electrical analogy treatment of cold plates (typical in avionics) sub-divides the problem into zones and accounts for the coolant rise but make the assumption that a heat transfer coefficient relative to a local coolant temperature is known. Some heat sink resistance calculations add in a “fluid resistance” but the reader is cautioned that this terminology can be problematic [2].

We are often asked to help design more efficient systems but sometimes the definition of efficient is vague. From the perspective of coolant in and out temperatures, the heat transfer process is 100% efficient and the heat ended up in the coolant. However, the real question may be, can the cooling job be accomplished with a lower flow rate and still have acceptable temperatures? Thermal engineers can participate and even lead discussions on developing even more efficient systems but we benefit when we look at problems from multiple viewpoints and even venture out into the research from related fields.

References

Bureau of Ships NavShips 900,190, Guide Manual of Cooling Methods for Electronic Equipment, 1 November, 1956.

Moffat, R, “Modelling Air-Cooled Heat Sinks at Heat Exchangers,” ElectronicsCooling, February 2008.

F. P. Incropera & D. P. DeWitt 1990 Fundamentals of Heat and Mass Transfer, 3rd edition, Wiley, New York.  l

Liquid cooling of electronics is not a new technology. The need to further increase packaging density and reduce signal delay between communicating circuits led to the development of multi-chip modules beginning in the late 1970s. The heat flux associated with bipolar circuit technologies steadily increased from the very beginning and really took off in the 1980s [1]. IBM had determined that the most effective way to manage chip temperatures in these systems was through the use of indirect water-cooling [2]. Several other mainframe manufacturers also came to the same conclusion [3-7]. The decision to switch from bipolar to Complementary Metal Oxide Semiconductor (CMOS) based circuit technology in the early 1990s led to a significant reduction in power dissipation and a return to totally air-cooled machines. However, this was but a brief respite as power and packaging density rapidly increased, matching then exceeding the performance of the earlier bipolar machines. These increased packaging densities and power levels have resulted in unprecedented cooling demands at the package, system and data center levels necessitating a return to water-cooling [1, 8-9].

Table 1. Cold plate description summary. * Dh = 4A/P, where A is the cross-sectional area of the channel and P is the wetted perimeter.

This article focuses on some of the design trade-offs associated with the processor cold plate. Two general types of cold plates are considered: a relatively high performance, high cost machined copper cold plate and a relatively low performance, low cost copper tube embedded in aluminum cold plate. Thermal performance in the context of the product application is also addressed.

Cold Plate Descriptions

A summary of the cold plates under consideration in this exercise can be found in Table 1.  Cold plates (1), (2), and (3) are copper blocks with rectangular channels for the water to flow through. Cold plates (1) and (2) have 0.5 mm wide channels while cold plate (3) has 0.25 mm wide channels. Cold plate (1) is a 3-pass flow design in that flow passes sequentially through 3 groupings of channels (Figure 1); each with a third of the total number of channels. By contrast, cold plates (2) and (3) are 1-pass flow designs; water flows through all the channels in parallel. Cold plates (2) and (3) are shown in plan view (transparent to show channels and fins) in Figure 1.  Also depicted in Figure 1 is cold plate (7) with pin fin geometry.

Figure 1. Plan (transparent) view; machined copper cold plates (1), (2), and (3) along with molded copper pin fin cold plate (7).

Cold plates (4), (5), and (6) are comprised of 6.35mm (¼”) outside diameter (OD) copper tubes embedded in an aluminum or copper plate. Cold plate (4) has its tubes flattened slightly to provide favorable thermal contact with the module lid. The formed tube cross section can be seen in Figure 2. The tubes in cold plates (5) and (6) are flattened to a greater degree for increased thermal performance: a greater fraction of the module lid area is in contact with the copper tubes along with a reduced hydraulic diameter for increased convective heat transfer. The price of this enhancement is increased pressure drop. Note, however, how inexpensive the copper tube cold plates (4), (5), and (6) are in comparison to the machined copper cold plates (1), (2), (3), and (7). The 0.25 mm channel cold plate (3) is by far the most expensive as it required wire electrical discharge machining (EDM) to form the channels.

Analysis Results and Discussion

Figure 2. Cold plates with ¼” OD copper tubes embedded in aluminium or copper plates.

Conjugate thermal analyses were performed on the cold plates described in the previous section using a commercially available computational fluid dynamics code [11]. In order to objectively compare cold plate thermal performance, a uniform heat flux boundary condition was applied to each cold plate’s active cooling area, which is defined for the purposes of this comparison as the cold plate area which is conduction coupled to the module lid by a thermal interface material (TIM). The cold plate thermal performance is defined by a total unit thermal resistance (mm² C/W),

                           (1)

where _base is the average temperature of the cold plate base active cooling area (^oC), T_in is the water inlet temperature (^oC), and q” is the heat flux applied to the cold plate base active cooling area. R”_total, which captures both convective and advective heat transfer behavior, does not completely capture the true nature of the cold plate’s thermal performance. The copper tube cold plates, having larger characteristic dimensions, rely on transitional or turbulent flows to achieve heat transfer performance while the machined cold plates with smaller characteristic dimensions rely on large wetted areas to achieve performance. Since the two types of cold plates compared in this study have very different pressure drop characteristics, implementations of the two types would necessarily require disparate flow rates (high flow rates for the formed tubes, and much lower flow rates for the small machined channels). The thermal resistance calculated in equation 1 includes the temperature rise of the fluid within the cold plate implicitly. Consider a micro-scale channeled cold plate, with extremely enhanced area but a high pressure drop and a pressed tube cold plate, both having equivalent thermal performance at the module level. For this to be true, the microchannel cold plate would have to be at a low flow rate, and the pressed tube at a relatively high flow rate; clearly most of the thermal resistance of the microchannel in this case is temperature rise of the fluid, whereas in the pressed tube the resistance is dominated by the heat transfer coefficient on the tube walls and the available area. To better observe this differing behavior, we also compare the cold plates by their convective performance only, and so define a convective unit thermal resistance (mm² C/W),

(2)

where  is the water mass flow rate (kg/s) and C_p is the water specific heat (J/kgK).

Figure 3. Cold plate convective unit thermal resistance (R”conv) as a function of volumetric flow rate.

Figure 4. Cold plate pressure drop as a function of volumetric flow rate.

The convective performance of the cold plates is depicted in the graph in Figure 3. The machined (plus pin fin) cold plates achieve a high level of performance with relatively little flow.  The slightly formed copper tube cold plate (4) performed the worst with comparable behavior to the machined cold plates only at much higher flows. The crushed tube (5) and (6) fa

ired much better although the resulting pressure drop, seen in Figure 4, is an order of magnitude higher than that associated with the 0.5 mm channel cold plates (1) and (2) or the slightly formed copper tube cold plate (4).

It is also useful to compare the cold plates using the total resistance as a function of flow power (Figure 5) to capture the convective and advective resistance as well as the pumping power cost associated with the varying flow rates and pressure drops inherent to these different geometries. Flow power (W) is defined as,

(3)

where DP is pressure drop (Pa) and is volumetric flow rate (m³/s). Clearly, the higher pressure drop, higher flow cold plates required much higher flow power (2 orders of magnitude in some cases).

Finally, consideration must also be made to how the cold plate performs in the actual product application. Several of the cold plates in this study were incorporated into two different product module applications (depicted in Figure 6). The cold plate thermal resistance in the context of the module application is determined by

(4)

where T_pr is a point reference temperature on the cold plate base corresponding to the x-y center of the processor and q is the total processor heat load. This thermal resistance is compared to the thermal resistance resulting from the uniform heat flux boundary condition previously discussed and is defined as

(5)

where A_mod is the module lid (active) area. By graphing in Figure 6 the ratio of these resistances, (R_UHF / R_prod), it can be seen that the actual performance can be considerably less (by a factor of 2) than the idealized uniform heat flux case; the R_UHF / R_prodmetric as shown represents an efficiency, where values less than 100% indicate to what extent the final design under-performs the idealized uniform heat flux case. The magnitude and variation with flow will vary with cold plate type and module application. One interesting feature of these packages is that as the overall thermal resistance of the package decreases, less spreading is taking place

Figure 5. Cold plate total unit thermal resistance (R”total) as a function of flow power.

in the lid and cold plate base; conversely, as the cold plate resistance increases, the heat flux from the lid becomes more uniform. In the case of a laminar flow cold plate, such as (1) and (7), the heat transfer coefficient between the channel walls and the coolant is not a function of flow rate; only the advective resistance is affected by increasing flow, resulting in a characteristic R_UHF/R_prod that decreases as flow increases. Alternatively, a cold plate with a larger hydraulic diameter and higher flow rates can be in transitional or turbulent flow, such as cold plates (4), (5), and (6); with transitional and turbulent flows the heat transfer coefficient between the tube walls and the coolant increases with increased flow rate, lowering the resistance of the path from the lid through the interface to the cold plate base and into the side walls of the tube.

Conclusions

Figure 6. Comparison of the uniform heat flux cold plate thermal resistance to its thermal resistance in a product application.

Two differing cold plate technologies for cooling processor modules have been compared. The following observations and conclusions can be drawn from the analysis results presented herein:

1. Cold plate thermal resistance performance, whether inclusive or exclusive of advective resistance, does not translate to a better design (i.e. one that most closely satisfies all of the design constraints of pressure drop, cost, and thermal performance). A more costly cold plate does not guarantee a better design.

2. The entire system design must be taken into account when designing / specifying a cold plate. Choices at the system level can elevate design requirements on the cold plate (i.e. allow for a lower performance / lower cost design). Specifically, given a thermal resistance target that can be met with a low flow, expensive, high pressure drop cold plate or a high flow, inexpensive, low pressure drop alternative, the coolant routing within the system (parallel versus serial paths, for example) and the available pump performance are key to the cold plate selection.

3. Cold plate thermal resistance based on a uniform heat flux (UHF) boundary condition can not be applied directly to a product application specification. Performance in the product application can be very different from what the UHF resistance would suggest.

References

[1]      Ellsworth, Jr., M.J., Campbell, L.A., Simons, R.E., Iyengar, M.K., Schmidt, R.R., Chu, R.C., “The Evolution of Water Cooling for IBM Large Server Systems: Back to the Future,” Proceedings of the 2008 ITherm Conference, Orlando, FL, USA, May 28-31.

[2]      Simons. R.E., ”The Evolution of IBM High Performance Cooling Technology,” Proceedings of the Eleventh Annual IEEE Semiconductor Thermal Measurement and Management Symposium, 1995, pp. 102-112.

[3]      Kaneko, K., Seyama, K., and Suzuki, M. “LSI Packaging and Cooling Technologies for Fujitsu VP2000 Series,” Fujitsu Scientific & Technology Journal, v. 41, no. 1, 1990, pp. 12-19.

[4]      Kaneko, K., Kuwabara, K., Kikuchi, S. and Kano, T. “Hardware Technology for Fujitsu VP2000 Series,” Fujitsu Scientific & Technology Journal, v. 37, no. 2, 1991, pp. 158-168.

[5.]      Kobayashi, F., Watanabe, Y., Yamamoto, M., Anzai, A., Takahashi, A.,Daikoku, T., Fujita, T., “Hardware Technology for Hitachi M-880 Processor Group,” Proceedings of the 41st Electronic Components and Technology Conference, 1991, pp. 693-703.

[6.]     Watari, T., Murano, H.,” Packaging Technology for the NEC SX Supercomputer,” IEEE Transactions on Components, Hybrids, and Manufacturing Technology, Volume 8, Issue 4, 1985, pp.462 – 467

[7.]     Murano, H., Watari, T., “Packaging technology for the NEC SX-3 Supercomputers,” IEEE Transactions on Components, Hybrids, and Manufacturing Technology, Volume 15, Issue 4, 1992, pp. 411 – 417.

[8.]     Ellsworth, Jr., M.J., Goth, G.F., Zoodsma, R.J., Arvelo, A., Campbell, L.A., Anderl, W.J., “An Overview of the IBM Power 775 Supercomputer Water Cooling System,” Proceedings of the ASME 2011 Pacific Rim Technical Conference & Exposition on Packaging and Integration of Electronic and Photonic Systems (InterPACK 2011), Portland, Oregon, USA, July 6-8.

[9.]     Wei, J., “Hybrid Cooling Technology for Large-Scale Computing Systems – From Back to the Future,” Proceedings of InterPACK 2011, Portland, Oregon, USA, July 6-8.

[10.]   Sahan, R.A., Rahima, M.K., Xia, A., and Pang, YF, “Advanced Liquid Cooling Technology Evaluation for High Powered CPUs and GPUs,” Proceedings of InterPACK 2011, Portland, Oregon, USA, July 6-8.

[11.]     Fluent, Distributed by ANSYS, Inc.; Southpointe, 275 Technology Drive, Canonsburg, PA, 15317, www.ANSYS.com l

This article seeks to provide greater insight into transient thermal phenomena in high-power IC packages and examines a number of approaches for predicting transient thermal behavior.

OVERVIEW

This work extends the effort described in two recent installments of this column devoted to the steady-state thermal analysis of high-power IC packages attached to heat sinks [2, 3].

Figure 1. Diagram of high-power package attached to a heatsink. Components in bold color are explicitly represented in the model. Those in a faint color are part of the physical assembly, but are not represented in the model.

The present effort is also divided into two parts. Part 1 focuses on nuances of transient heat transfer and explores three different methodologies using a simplified model of the package and heat sink.

Part 2 will extend this analysis to more practical examples such as those treated in the recent columns as well as the JEDEC junction-to-case thermal resistance test. It will be published in the Spring 2012 issue.

This article explores three analysis methods: 1) finite element analysis (FEA), 2) an analytical, multi-stage RC model, and 3) a numerical, multi-stage RC model (RC = Resistor Capacitor). A commercial code was used to implement the FEA work [4]. Methods 2 and 3 were implemented in spreadsheets.

The current analysis assumes the same package construction as in the recent columns, namely a high-power IC package attached to a heat sink, as depicted in Figure 1. In this model, two simplifying assumptions were made: 1) heat flow to the package substrate and to the PCB is neglected and 2) the cooling effect of heat sink fins is represented by the application of a suitable heat transfer coefficient directly to the heat sink base. Hence, the only components explicitly represented in the model are the die, TIM1, lid, TIM2, and the heat sink base (where TIM = Thermal Interface Material).

A further simplifying assumption for Part 1 only of this article is that the width of all components is equal to that of the die, making the heat flow one-dimensional. This will simplify the task of evaluating the relative accuracy of the analytical and numerical multi-stage RC models.

Finite Element Analysis Solution

Assumptions

Figure 2a depicts the solid model representing the five aforementioned components. It is a one-quarter model, because of the symmetry in the component geometry, heat load, and boundary condition. In the subsequent text, comments regarding up and down directions (or top and bottom) in the component stack-up are made with reference to this figure.

The quantitative assumptions of the model are listed in Tables 1, 2, and 3. Note that the stated values of die width, thickness of the all components,  material composition, and thermal resistance are consistent with those in the recent articles [2, 3]. Differences involve the width of the lid, TIM2, and heat sink base, and value of heat transfer coefficient, h. The current value of h is much larger than the ones assumed in the cited articles. It was chosen to produce a value of heat sink-to-air thermal resistance comparable to those in the earlier work despite having a much smaller surface area for heat removal. All material properties are assumed to be independent of temperature. Note that, for convenience, the ambient temperature was assumed to equal zero. Hence, the reported component temperatures correspond to the temperature rise with respect to the ambient. The power is assumed to increase in a stepwise fashion from 0W to 1W at time 0.

Results

The results of the FEA solution are shown in Figures 2 and 3. Figure 2a shows the thermal contour maps for elapsed times in the range 2E-7 to 100 seconds. As expected, the heat flow is one-dimensional.  Interpretation of the contour plots is assisted by Figure 2b, which attempts to capture the spatial and temporal variations of temperature in the model. It plots the temperature for each node at a given z-axis location for each FEA solution illustrated in Figure 2a. (The z-axis is assumed to have an “up” orientation).

Clearly, the temperature rise at all levels in the model proceeds most rapidly in the early portions of the time interval explored. The steady state is nearly achieved after only 8.5 seconds. There is only a slight further increase in temperature after 100 seconds have elapsed.

Most of the temperature rise occurs within the TIM layers and at the heat sink-to-air interface. In contrast, the temperature variation within the die, lid, and heat sink base is relatively small. Hence the thermal behavior of the package/heat sink stack can be efficiently captured by tracking the temperature of nodes at the top of the die, lid, and heat sink base.

Figure 3a is a linear plot of the temperatures at these three locations versus time. It displays the behavior normally expected in a transient thermal situation, with the rapid initial temperature rise as discussed in the context of Figure 2b. Unfortunately, since most of the graph represents time intervals during which the system is near or at the steady state, it is not the best vehicle for documenting the nuances of a transient thermal model.

In contrast, the log-log plot in Figure 3b expands the scale considerably in the very short time scales, down to 1E-7. It clearly shows that, up to approximately 5E-4 seconds, the only heat flow occurs within the die. Heat flow doesn’t reach the heat sink base until about 2E-3 seconds. Because of their advantages, log-log plots are used in the subsequent analyses.

Lumped-Parameter Thermal Circuit Models

A further examination of Figure 2a indicates that there are alternating regions of different transient behavior: 1) small thickness and large temperature rise (TIMs and heat-sink-to- air thermal resistance) and 2) larger thickness and small temperature rise across them (die, lid, and heat sink base). The first class of regions has a dominant resistive behavior while the second exhibits capacitive behavior.

The one-dimensional heat flow situation allows us to easily calculate the thermal resistance, R, and heat capacity, C, of each layer using the following formulas:

(1)

(2)

C = Specific Heat * Density * Volume (3)

Figure 2a and 2b. Results of FEA transient thermal calculations: (top) a) solid model and overlaid temperature contours calculated at elapsed times of 2E-7 sec. to 100 sec. (bottom) b) Plot of nodal temperature vs z coordinate at different values of elapsed time.

Table 4 lists the calculated values of R_CONDUCTIVE and C for each component and for the heat sink base-to-air thermal resistance, R_CONVECTIVE, are listed in Table 4. The stated values confirm the dominant behavior of each region as described above. This suggests the possibility that a multi-stage RC circuit model could be used to represent the thermal behavior of the package/heat sink stack. Figure 4 represents a template for an RC ladder circuit that could be scaled to an arbitrary number of RC stages.

As indicated in the lower part of Table 4, it is possible to lump the various layers in the package stack-up into RC ladder models consisting of various numbers of stages. The thermal resistance and heat capacity of each stage would each

equal the sum of these parameters for each of the components lumped into that stage. The thermal time constant for each stage equals the product of its total R and C values. It should be noted that up to 3 stages, the lumped elements are delineated by component interfaces. To create 4 stages, the die is divided into upper and lower parts. In the subsequent sections, these lumped parameter values are used for both analytical and numerical transient calculations. It should be noted that, as the number of stages is increased, the time constant for the first stage (containing the die) gets smaller and smaller and, hence, should enable the model to better track the short-time-scale transient behavior of the die.

Analytical Multi-Stage RC Model

Equation 4 provides a convenient method of calculating the transient behavior of the junction temperature, TJ, in response to stepwise increase in power from zero to a constant value, beginning at time = 0.
(4)

where Θ_JA, the junction-to-air thermal resistance, is equal to the sum of all the R_i values and τ_i is the thermal time constant of the ith stage, equal to R_i*C_i.

Note that this equation is not an exact solution for this type of problem. However, it often provides adequate accuracy in situations such as the present one involving a multi-layer structure, with each layer having a significantly different time constant from the others [5]. The reader is directed to Ref. [6] for a more rigorous derivation of multi-stage thermal networks.

Figure 5 compares predictions of the analytical model with those of the FEA model (and the numerical model, to be discussed below). Obviously, increasing number of stages in the analytical model improves accuracy. The calculation using the 4-stage analytical model is in good agreement with the FEA prediction. The maximum discrepancy between them is 0.2˚C, occurring at time = 0.1 second.

Figure 3a and 3b. FEA transient analysis, temperature at 3 different locations versus time. a) linear plot. b) log-log plot.

Numerical Multi-Stage RC Model

The preceding analysis has demonstrated acceptable accuracy using the analytical model. It has the virtue of being very easy to implement for a simple power on/power off situation. The main problem, however, is that it becomes much more complicated to use it to calculate the effect of an arbitrary change in power on the junction temperature. The normal approach to dealing with this situation is to perform a Laplace transform transfer function analysis, which can be quite challenging [6].

Numerical methods, such as the one described below, are more difficult to implement initially. However, they can readily be adapted to account for the effect of an arbitrary power waveform on the junction temperature.

Figure 4. Diagram of multi-stage RC thermal network.

Methodology

Assume a series RC circuit containing the desired number of stages. The heat flow from the junction to the ambient at a given value of instantaneous power can be calculated by discretizing time into sufficiently small steps, and applying the following procedure [7].

Figure 5. Transient thermal results from 1-, 2-, 3-, and 4-stage analytical, FEA, and 1- and 4-stage numerical models. Log-log plot of junction temperature vs time.

•     At time < 0, zero power is applied to the die. The temperature of all nodes is equal to that of the ambient.

•     For time ≥ 0, in a given time step, δt, a quantity of thermal energy is injected into the left side of the ladder network by the die, as indicated in Figure 4.

•     This quantity of energy, Q, is equal to δt*P, where P is the instantaneous dissipated power.

•     The heat then propagates toward the right end of the network by flowing through each stage in the network ladder.

•     The quantity of heat flowing from nodes iand i+1 during time step j is calculated with reference toFigure 6 using the following expression. It is directly proportional to the difference in the nodal temperatures and

Figure 6. Diagram indicating heat flow into and out of a single RC stage, for nodes i and i + 1 during time step j.

inversely proportional to the resistance, R_i.

(5)

•     The change in temperature at a given node i, during time step j, is calculated using the net heat remaining in the node and the heat capacity of that node according to the expression:

(6)

•     Finally, the temperature at node i can be calculated at the next time step, j + 1 from:

T_i,j+1 = T_i,j + ∆T_i,j                                                      (7)

Once the updated temperatures are calculated for all nodes, then the process repeats by sequentially applying equations 5, 6, and 7 to all nodes. It is repeated until the process steps though the time interval of interest.

Figure 7a and 7b. Results of FEA transient analysis compared with equivalent numerical RC model. Temperature at different locations versus time: a) 3-stage numerical model. b) 4-stage.

Spreadsheet Structure

The spreadsheet structure to perform this calculation as described above is shown in Table 5a. Note that the calculation requires a fixed time step to converge. As we have seen, a transient solution can span several decades of time. This spreadsheet uses a technique to deal with this efficiently. There are three different sheets implementing the calculation with different time steps: Sheet 1, 1E-6 sec.; Sheet 2, 1E-4 sec.; and Sheet 3, 1E-2 sec. The calculation on each sheet begins at 0 time and proceeds at different rates per the chosen time step. Hence, Sheet 1 provides a much finer time resolution, suitable for the early portion of the transient. Each successive sheet lends itself to calculating the temperatures with a larger time step and, therefore, covering a larger span of elapsed time. The calculated results from all the calculation sheets, at the desired time steps, can be readily aggregated on the “Summary” sheet (described below) for graphing and further analysis.

Figure 8. Results of 4-stage numerical RC model, with variable power: junction temperature and instantaneous power vs time.

Note that the power in a given time step is entered directly into the appropriate cell in Column C. As such, it can be varied at any desired time step(s).

A spreadsheet for RC circuits with more than one stage can be generated by the following recursive process:

•     To create a 2-stage model, copy the cell contents from the range: [Col E, row 2: Col G, last row] and pasting it into the range: [Col G, row 2: Col I, last row]. Re-label cells G2 and F2 as “R2” and “C2”, respectively. Input correct values of these parameters into cells G3 and F3.

•     To create a 3-stage model from the 2-stage model, copy the cell contents from the range: [Col G, row 2: Col I, last row] and paste it into the range: [Col I, row 2: Col K, last row], etc.

This spreadsheet has a fourth sheet described as the “Summary” sheet. Its structure is described in Table 5b. It uses the nested formulas “INDIRECT(ADDRESS(row,col,,,sheet))” to access an arbitrary solution temperature on any of the calculation sheets, per the row and sheet inputs to the ADDRESS function, in columns A and B. The present example only shows links to values on Sheet 1. However, as used by this author, these formulas are copied to many different rows to selectively access calculated values from all of the sheets.

The reader might be interested in reading a description of a much more sophisticated approach dealing with the issue of nested time scales (and spatial ones as well) with the objective of achieving computational efficiency in the spirit of the highly simplified example described here [8].

Results

Figure 5 compares the numerical results for the 1-stage and 4-stage models with both the FEA and analytical results.  The 1-stage results for the numerical and analytical methods are nearly identical. The difference between them depends on the value of the time step. It was largest for the largest time step, equal to  1E-2.  In this case it is 5E-4˚C. Since the 1-stage analytical result represents an exact solution, this precise correlation with the 1-stage numerical method is a confirmation of its accuracy.

Figures 7a and 7b compare the 3- and 4-stage numerical results with the FEA solution. Clearly, the 4-stage method provides a calculation of the junction temperature that is in better agreement with the FEA result than either the 3-stage numerical or the 4-stage analytical result. The lid temperature calculations for the numerical method are in good agreement with the FEA results. The calculated heat sink base temperatures are in slightly worse agreement. It is likely that the agreement could be improved by subdividing the heat sink base into an upper and lower half, as was done with the die.

Numerical 4-Stage RC Model with Variable Power

The 4-stage model, whose results with constant power are presented in Figure 7b, was modified simply by manually changing the power level at selected time steps in Col C of Sheet 3. In some cases a linear function relating power and time was input. The choices for instantaneous power were quite arbitrary and are intended mainly to exercise the numerical model.

The results, extracted by the Summary sheet from Sheet 3 only, are presented in Figure 8. As expected, the process for modifying the spreadsheet to accommodate the variable power input took only a few minutes.

Conclusions

With a sufficient number of RC stages, both a simple analytical model and a straightforward numerical give adequate accuracy when accounting for a simple power on/power off situation with a typical high-power IC package. However, if there is a need to account for the effect of a time-varying power level, the use of the numerical model provides the more convenient path to a solution.

In Part 2, the numerical methods applied here to one-dimensional heat flow situations will be extended to more typical situations involving diverging heat flows, using the methods recently developed in this column for steady-state thermal analyses of high-power packages.

References

1.  JEDEC Standard, JESD-14, “Transient Dual Interface Test Method for the Measurement of the Thermal Resistance Junction to Case of Semiconductor Devices with Heat Flow Through a Single Path.” Available for free download at www.jedec.com.

2.  B. Guenin, “Thermal interactions Between High-Power Packages and Heat Sinks, Part 1,” ElectronicsCooling, Vol. 16, No. 4, Winter, 2010.

3.  B. Guenin, “Thermal interactions Between High-Power Packages and Heat Sinks, Part 2,” ElectronicsCooling, Vol. 17, No. 1, Spring, 2011.

4.  ANSYS®, Version 13.0

5.  B. Guenin, “Simplified Transient Model for IC Packages,” ElectronicsCooling, Vol. 8., No. 3, August, 2002.

6.  Y-L Xu, R. Stout, and D. Billings, “Electronic Package Thermal Response Prediction to Power Surge,” Proceedings, I-Therm Conference, May, 2000, 366-371.

7.  B. Guenin, “Transient Thermal Model for the MQUAD Microelectronic Package,” Proceedings SEMI-THERM X Conference, February, 1994, pp. 86-95.

[8]  J. S. Wilson and P. E. Raad, “A Transient Self-Adaptive Technique for Modeling Thermal Problems with Large Variations in Physical Scales,”  International Journal of Heat and Mass Transfer, Vol. 47, pp. 3707-3720, 2004.

Electro-Thermal Simulation

Electro-thermal simulation at system level is a combined simulation of electrical and thermal aspects of the system as schematically shown in Figure 1.

Figure 1. Simple electro-thermal simulation with conservative coupling between thermal and electrical subsystems.

The circuit model calculates power dissipation, which is used by the thermal subsystem to evaluate the temperatures. Temperatures in return influence the circuit parameters, thus requiring a two-way coupling.

The thermal model in Figure 1 is a simple, lumped-element model. For general complex geometries, a more accurate, physical model in form of heat transfer partial differential equation is required:

(1)
where K(r) is the thermal conductivity in W/m/K at the position r, c_p (r) is the specific heat capacity in J/kg/K, ρ(r) is the mass density in kg/m³, T(r,t) is the temperature distribution and Q(r,t) is the heat generation rate per unit volume in W/m³. Spatial discretisation (via e. g. finite element method – FEM) of (1) results in a large-scale ordinary differential

equation (ODE) system with n equations of the form:

Figure 2. The idea of model order reduction: automatic way from a finite element thermal model to electro-thermal simulation at the system level

(2)

where E Є R^nxn is a heat capacity matrix, K Є R^nxn is a heat conductivity matrix, T Є Rⁿ is a vector of nodal temperatures in time and F Є Rⁿ is a heat source load vector. (1) can also be considered as an electrical network where the vector T will be equivalent to unknown voltages, the matrix E will be a capacity matrix and the matrix K the resistance matrix (see [1] for methods to synthesize electrical networks). In both cases, (2) is not compatible with system level simulation, as vector T usually contains several hundred thousand degrees of freedom. The remedy is to apply modern mathematical algorithms of model order reduction [2],[3], which enable a formal transformation of (2) to a low-dimensional ODE system of the same form but with much smaller dimensions, as described in the following section.

Model Order Reduction

Model order reduction is an area of mathematics that enables a formal approximation of the physical model and hence, the generation of a compact model suitable for system level simulation (see Figure 2).

Model order reduction starts with the ordinary differential equation system (2), which is slightly modified as follows

(3)

Figure 3. Model order reduction as a projection of the system onto the low-dimensional subspace, defined by the matrix V. Different MOR algorithms define V in different ways. The approximation is good if the error ε is small. For thermal models, E is the heat capacity matrix, K is the heat conductivity matrix, and F is a heat source load vector.  is a vector of nodal temperatures in time.

In (3) the load vector F Є Rⁿ from (2) was split into a product of a constant input matrix B Є R^nxp and a vector u(t) Є R^p of p input functions (heat sources) and a second, output equation has been added. The output equation in (3) defines certain linear combinations of the state vector T that are of interest for the particular application in system level simulation. This equation has to be defined by the user. Note, that in a case when the complete temperature field is required, the output matrix C Є R^mxn (where m is the number of outputs of interest) has to be defined as a unity matrix.

Model reduction is based on an assumption that the “movement in time” of a high dimensional state vector T can be well approximated by a lower dimensional subspace (ε in Figure 3 should be small). Such subspace is defined by the projection matrix V. Provided this subspace is known, the original system can be projected on it, as shown in Figure 3.

The definition of such low dimensional subspace, i. e. matrix V, is subject of a particular MOR algorithm. For thermal systems, we propose to use the Krylov-subspace based Arnoldi algorithm [7], which defines the projection matrix in such a way, that the transfer function of the original dynamic system is well approximated by the Taylor-series of the transfer function of the reduced system. Mathematically speaking this approach belongs to the Padé approximation and is also known under the name moment-matching method, where the Taylor-series coefficients are called moments. The detailed description of the algorithm and the theorems proving its moment matching properties can be found in [2][3] and [7].

Figure 4. Solid model of the power switch, SOIC package mounted on PCB, de-capsulated.

For performing model order reduction of finite element models, we employ the software tool MOR for ANSYS [4]. It reads the system matrices created by the finite element simulator ANSYS [5], runs the Arnoldi algorithm and writes the reduced system, i. e. its matrices, E_r, K_r, B_r and C_r (see Figure 3). The reduced matrices can be read directly in Simplorer [6], MATLAB/Simulink, and other system level simulation tools. It is also possible to write down the reduced model as a Spice model or a template for the use in VerilogA and VHDL-AMS.

Electro-thermal System Simulation of a power MOSFET

The self-heating of a power MOSFET can cause a significant temperature rise associated with temperature gradients and thermal stresses in the particular device. Thermal cross-talk between transistor channels as well as temperature dependent heat generation rates will accelerate the self heating phenomenon, which can occasionally even cause a thermal runaway instance. A good understanding of the electro-thermal behavior of the power switch, electronic control unit and the load is a key for the system design.

Figure 5. Schematics of a power switch driving four light bulbs, only the first light bulb is shown, the other three are identically connected to T2, … , T4.

The model generation path and electro-thermal system simulation are demonstrated for a novel High-Side Switch device [8] with four independent channels. The device is packaged in a Small Outline IC (SOIC) package, which itself is assembled onto a printed circuit board. Figure 4 shows a model of the de-capsulated package (plastic molding compound removed) with the wire bonds connecting each transistor channel with its associated external lead fingers.

The corresponding electro-thermal system model (Figure 5) contains two distinct thermal submodels for power switch and printed circuit board (PCB), and five electrical submodels, one for each transistor channel and one for the light bulb. Thermal submodels have been extracted from the 3D-finite element models by means of model order reduction, as described in section 2. Power switch and PCB share one common node at the system level, which represents the thermal port between SOIC exposed pad and the adjacent PCB land pad. The PCB model also contains the heat convection of the system into the ambient air.

The electrical transistor models are ready to consider dynamic self heating phenomena. In addition to the electrical IO/s, gate, drain and source, these models each contain one electro-thermal port T interconnecting electrical and thermal domains of the power switch.

Figure 6. Comparison of the transient thermal response of a transistor model: the reduced model order 30 vs. the original model order 300.000.

In Figure 6, the thermal response of a full-scale model and of a reduced order model of a single transistor is shown. The difference is less than one percent. Model reduction takes only 80 s, while transient simulation with 60 time steps requires about 2000 s. At the same time, system level simulation lasts less than a second.

In the system level model shown in Figure 7 a single transistor has been loaded sequentially with one, two or three lamps. The light-bulb model is represented using an existing Spice model which was imported directly. In Figure 8 the junction temperatures are shown as functions of time for a different number of lumps. One can see that with three lumps the temperature for short time of about 5 ms goes over 200 degrees Celsius. The thermal model considered in this study cannot tell us whether this is acceptable or not, but it demonstrates the need for transient electrothermal simulation. The stationary temperatures are acceptable for all three curves in Figure 8 and only transient simulation can reveal that the temperature goes over the critical temperature.

More information can be found in [9] and [10].

Figure 7. Simulation model with bulb lamps.

Conclusion

We have shown that model order reduction is an efficient tool to automatically generate an accurate compact thermal model for power electronic devices. The advantages of the method are as follows:
The method uses the system matrices from the accurate finite element model directly.
The method is automatic. The user should set the dimension of the reduced model and define the outputs of interest. According to our experience the accuracy of one percent is achieved with 10-15 degrees of freedom per input.
The model reduction process is fast. The time for model reduction is comparable with that of a static solution and usually, depending on the chosen integration time step, shorter than a single transient run with the original finite element model.

Figure 8. Junction temperature on the transistor vs. time as a function of number of lamps.

REFERENCES

[1] J. T. Hsu, L. Vu-Quoc, A rational formulation of thermal circuit models for electrothermal simulation. Part I: Finite element method. IEEE Trans. Circ. Syst. – I: Fund. Theor. Appl. v. 43, N 9, p. 721-32, 1996.

[2] T. Bechtold, E. B. Rudnyi, J. G. Korvink. Fast Simulation of Electro-Thermal MEMS: Efficient Dynamic Compact Models, Springer 2006, ISBN: 978-3-540-34612-8.

[3] A. C. Antoulas, Approximation of Large-Scale Dynamical Systems. Society for Industrial and Applied Mathematic, 2005, ISBN: 0898715296.

[4] E. B. Rudnyi and J. G. Korvink. Model Order Reduction for Large Scale Engineering Models Developed in ANSYS. Lecture Notes in Computer Science, v. 3732, pp. 349-356, 2006, http://ModelReduction.com.

[5] ANSYS® Mechanical, Release 13.0, http://www.ansys.com

[6] ANSYS Simplorer®, Release 9.0, http://www.ansys.com

[7] R. W. Freund, “Krylov-subspace methods for reduced order modeling in circuit simulation,” Journal of Computational and Applied Mathematics, vol. 123, pp. 395-421, 2000.

[8] MC15XS3400: Quad High Side Switch, http://www.freescale.com

[9] T. Hauck, W. Teulings, E. B. Rudnyi. Electro-Thermal Simulation of Multi-channel Power Devices on PCB with SPICE. THERMINIC 2009, 7-9 October, Leuven, Belgium, 6 p.

[10]
T. Hauck, I. Schmadlak, L. Voss, E. B. Rudnyi. Electro-Thermal Simulation of Multi-channel Power Devices: From Workbench to Simplorer by means of Model Reduction. NAFEMS, Seminar: Multi-Disciplinary Simulations – The Future of Virtual Product Development, 9-10 November 2009, Wiesbaden, Germany, paper #5, 10 p. l

This article will explain some of the challenges faced in the search for the perfect data center and why, given the variability of equipment design and the time varying nature of the data center load, CFD is not the only tool to be used in design and/or management of a data center, but is an essential tool to enable maximum performance.

Figure 1. Example showing poor airflow balancing with over-supply waste and under-supply and recirculation potentially causing overheating.

IT Equipment Airflow Is Important Design For Airflow As Well As kW

Traditional design simply ensured that the cooling system provides sufficient cooling in terms of kW per area or per cabinet. When power densities were low, this was sufficient since any re-circulation of air only resulted in moderately warmed air entering the equipment. Increasing heat densities in electronics often results in higher temperature recirculation putting equipment at risk.

The room cooling performance will be affected by the ratio of air supplied to the room compared with IT equipment airflow demand. Too much cooling air (Figure 1, left) results in wasted energy with cool air returning unused to the cooling system. Too little cooling air (right) will be compensated for by drawing potentially warm ‘used’ air in from the surrounding environment risking overheating. The designer must therefore consider airflow balance, as well as cooling power, and incorporate a strategy to address the changing needs likely during the life of the data center.

Figure 2. Example showing several different airflow regimes from equipment. Front to back, upward outflow and mixed side to side and front to back.

Impact Of Real Equipment On Cooling Performance

There are no generally accepted standards for equipment cooling. This provides a blank canvas for the equipment designer who can freely choose how to package the equipment and cool it. The equipment design will change the equipment’s demands on its environment and how best to configure the rack and the data center.

This variability in design affects the room cooling requirement by drawing air in from differing locations/faces of the equipment and similarly exhausting it from different locations and in different directions. Historically while servers have been generally designed with front to back airflow other equipment has often varied. Figure 2 shows some typical examples.

The choice of flow rate and temperature rise for the equipment also affects room airflow patterns and temperatures. A low flow (Figure 3, left) results in slow moving air that rises due to buoyancy. At higher flows (right) velocity dominates and the air shoots away without rising appreciably.

Flow rate will depend upon equipment design but also on operational details such as equipment utilization or environmental conditions. The designer and operator must recognize the need for the design to be flexible so it can accommodate the change over time. The choice of IT equipment and how it is utilized will affect the cooling performance of the data center, consequently airflow must be considered throughout data center life.

Figure 3. Example showing the impact of air volume and temperature rise on the flow from a server.

Configuring The Data Center Rack Configuration

The data center is an evolving entity in which deployments are easily made but often not easily reversed. In practice the consequences of these deployments are often not seen until later in the data center lifetime. Failure to plan, allowing for the consequences of deployments on airflow and cooling can, and commonly does, result in a gradual deterioration of cooling performance and hence capacity of the data center.  It is not unusual for a data center to experience cooling difficulties and hotspots at only two-thirds design capacity in terms of kW equipment deployed. This apparently unusable capacity is often called ‘stranded capacity.’ It is stranded because without further assessment and reconfiguration it may not be possible to use the full design capacity without creating hotspots.

This is not just a data center issue at room level alone but is affected by deployment decisions within the cabinet too. It is common to consider issues such as blanking when placing equipment in a cabinet but less common to consider equipment interaction. Figure 4 shows the air circulation and temperatures in a cabinet where only two IT deployments have been made. First a blade system has been deployed and then three 1U servers added in the slots immediately above. In this instance the hot air from the blade system is confined to the lower part of the cabinet and recirculates under the cabinet causing high inlet temperatures and reduced resilience.

Figure 4. Blade system deployed in cabinet below 1U servers.

If the servers are deployed in the opposite sequence (Figure 5), with the 1U servers in the lower 3 slots and the blade system above, then the recirculation is dramatically reduced resulting in lower inlet temperatures and greater resilience.

The configuration of equipment inside the cabinet can equally affect room conditions. Figure 6 shows that, with no modification to rack cooling strategy, the room conditions and equipment temperatures are dramatically changed when front to back ventilated servers are replaced by routers with mixed airflow of the same power.

Room Configuration

The primary challenges for the owner operator are:

a. How to deploy different types of equipment in the data center side by side that have different demands for airflow and cooling when the data center was intended to provide a relatively uniform cooling capability;

b. How to be prepared for and accommodate future generations of equipment with characteristics yet unknown.

It is unlikely that the cooling system will deliver airflow and cooling uniformly throughout the data center. This means that the owner/operator must be aware of the impact of location of deployment in the context of the actual performance of the data center. Consider a scenario where 2 new Sun M5000s are to be deployed. Figure 7 shows the impact is different depending on choice of installation location even though in principle all the chosen locations have space power and cooling.

Figure 5. Blade system deployed in cabinet above 1U servers.

The different results are not easily predicted using simple rules as they result from the combination of equipment heat load, equipment airflow but also the configuration, in particular the interaction between the equipment and room airflows.

The increase in power density and soaring energy costs, combined with the growing awareness of a need for environmental responsibility, has stimulated the fundamental design of data center cooling to be revisited. New approaches, such as aisle containment, are implemented to increase efficiency but the increased efficiency will only be achieved if equipment airflows are appropriately controlled/balanced. If this is not done potentially high temperature damaging recirculation can still occur, Figure 8.

Matters affecting the cooling performance are not limited to equipment deployment alone but also the infrastructure such as ACU & PDU deployment and location and size of cable routes.

So should CFD be used?

Figure 6. Example showing front to back servers cool while routers are hot in the same configuration.

Alternative approaches to design and analysis such as rules of thumb and hand calculations or more sophisticated models such as modeling using potential flow theory provide an insight into cooling performance. However, they all, to a greater or lesser degree, fail to account fully for the important issue of airflow and its momentum.

Computational Fluid Dynamics, or CFD for short, provides a unique tool capable of modeling the data center and equipment installation in conceptual design right through to detailed modeling for operation. Providing a 3-dimensional model of the facility can account for almost any feature and combination of features, in a manner very similar to its use in electronic equipment design. However, modeling a data center can be somewhat more challenging since the data center is a dynamic changing ‘electronics design.’ The above illustrations show the potential for CFD to predict the significant impact of these small variations.

So, what are the barriers that must be overcome for successful use of CFD? From an academic point of view people may point to the more theoretical aspects of CFD such as turbulence modeling, gridding, solution time and indeed including the full physics, but, in practice, these difficulties are normally far outweighed by the difficulty of capturing the true configuration (such as the features described above) in sufficient detail to predict the resulting environment.

Figure 7. Example showing 2 different spatial deployments of 2 items of equipment – only option 2 works.

CFD can relatively easily be used for concept design decisions but even here there is need for education about the significant risk of ignoring equipment type and resulting airflow, temperature affects and likely deployment locations. For real facilities it is even more challenging and it is essential that measurements are made alongside the modeling process in order to ascertain that the model reflects reality. Why? Because some details can never be represented precisely (e.g. unstructured cabling, damper settings …) and others may depend on operational factors such as equipment utilization. For the latter it is hard to gather airflow and heat dissipation data for equipment to be able to characterize them fully in deployment. Here the electronics industry can make an important contribution by publishing data more openly and indicating likely trends for planning purposes.

Figure 8. Recirculation in a poorly contained scenario.

Once this careful data gathering exercise has been achieved then, with the right tools, it is relatively straightforward to build a CFD model of the data center and check that it reflects reality. Once a ‘calibrated’ model is achieved it can be used with reasonable certainty to make deployment decisions and undertake other tests such as failure scenarios with confidence. In the view of the authors it is imperative, if the potential for stranded capacity is to be minimized, that simulations be undertaken frequently to understand the implications of deployment decisions and that monitoring and comparison with measured data is made regularly to ensure the model continues to reflect reality. With the increasing availability of (live) measured data, maturing CFD tools and increasing pressure on energy efficiency but still maintaining availability, using CFD with an appropriate methodology can be a critical tool for design and management.

Yet independent of increases in power density, advantages to improved cooling can be demonstrated. Leakage current has become an increasing fraction of processor power with each technology node. As leakage current is strongly temperature dependent, processor power dissipation can be reduced through improved cooling. Achievable processor frequency is strongly dependent on temperature and voltage. The voltage dependence is approximately proportional, while temperature dependence of performance has reduced with each technology node. In the near future, the temperature dependence of performance may almost disappear.

Figure 1. Package and heatsink.

Through improved cooling, temperature can be reduced while voltage and frequency are increased, resulting in higher system performance. For a given cooling configuration, a combination of voltage and temperature exists which maximizes system performance per watt. At higher powers, additional performance is achieved at the expense of energy. Such increases may be limited by electromigration and other failure mechanisms, which are functions of both temperature and voltage. The temperature dependence of the leakage current has been increasing through recent technology nodes. Significant decreases in energy consumption and/or increases in processor frequency can be achieved through improved cooling.

Figure 2. Thermal resistance components.

Leakage Current Effects

Leakage current is current which bypasses the transistor gate, which is always present regardless of whether the gate is active, and which increases with both voltage and temperature. It can be considered as wasted energy, unlike energy used to perform computation.

Leakage current effects have increased as semiconductor lithography has progressed well into the subcontinuum range [2]. In some ASICs, leakage current can be over half of the total power. As the industry moves from 45nm to 32nm to 22nm and beyond, designs are intended to limit leakage current to about one-third of total power dissipation. Some of this will be achieved by lower junction temperatures. The dependence of leakage current on junction temperature is also becoming stronger. At 65nm, leakage current would change by a factor of two over a temperature range of about 45^oC; at 32nm, the range may shrink to 22^oC [3]. So the energy savings due to a given temperature

Figure 3. Component temperatures at nominal frequency.

reduction will increase with technology improvements. From the viewpoint of effectiveness of improved cooling, a relevant metric is the percent reduction in total power versus temperature. Recent observations of processors on the market show this to range from about
0.5%/^oC up to almost 2%/^oC [4]

Improved Cooling

Ellsworth and Simons [5] discussed a variety of package cooling improvements which could be used to cool up to several hundred W/cm². These began with traditional air cooling of a lidded package as shown in Figure 1. As the thermal path through the package substrate offers a high resistance, nearly all the heat flows upwards and outwards while spreading through the package lid and heatsink base, then finally to the heatsink fins and into the air stream. The heat flow path is through the following components:

• Silicon die

• Thermal interface material 1 (TIM1)

• Package lid

• Thermal interface material 2 (TIM2)

• Heatsink base

• Heatsink fins

• Coolant

Figure 4. Processor frequency.

Nonuniformity of power dissipation within the die increases package thermal resistance above that of a package with a uniformly powered by a significant factor, typically two or more [6, 7]. For a given nonuniform power distribution, this factor will increase as the thermal path is improved, moving asymptotically toward the ratio of the highest local power density to the average power density.

Recent developments in packaging and cooling largely consist of replacing traditional materials with those having higher thermal conductivity. In the package, organic TIM1 with metallic or ceramic fillers can be replaced by a completely metallic material, typically indium or indium alloy. Copper package lids can be replaced by composite materials, such as aluminum-graphite, copper-graphite, aluminum-diamond or silicon carbide-diamond. External to the package, heatsinks feature embedded heatpipes or vapor chambers, and in some cases the vapor chamber shares a continuous vapor/liquid space with heatpipes extending into the fins. A cold plate can replace the heatsink or become an integral part of the package, acting as the lid. Ultimately, coolant can contact the silicon with no interfaces between. At the system level, liquid can be used to transfer heat to air through a liquid-to-air heat exchanger, to another liquid through an interface or liquid-to-liquid heat exchanger, or the liquid can flow across the system boundary to transfer heat to the data center and beyond. Such technologies are described in detail by Kang and Miller [8].

Figure 5. System performance.

Performance Modeling

The processor has a nominal power dissipation of 240W, operating at nominal conditions of 0.85V and 95^oC.

Thermal resistance components of various combinations of packaging and cooling are shown in Figure 2. Values shown are typical for a die approximately one square inch (645mm2) in modern flip-chip packaging. Design variations consist of the following:

Package thermal interface material (TIM1) – The TIM1 can be organic or metallic in a traditional lidded package.

Coolant on silicon removes the TIM1 and lid or cold plate, simplifying package construction.

Package types can be lidded, which requires another thermal interface material (TIM2) or cold plate lid, which eliminates TIM2.

The system cooling configuration can be one of three types:

Air, in which all heat is dissipated to the air stream without liquid as an intermediary. In this case the air stream is fixed at 35^oC, the allowable limit for a typical data center.

Internal liquid, in which liquid is used to transfer heat to the air stream.

External liquid, in which heat is transferred to liquid without air as an intermediary. In this case, the external liquid temperature is fixed at 25^oC, slightly above the allowable maximum dewpoint of a typical data center.

Electrical performance is based on a generic technology representative of recent and upcoming process nodes, and is modeled by a few simple assumptions:

In the region of interest on gate-dominated paths, achievable frequency is assumed to be proportional to voltage:

Freq αV                                                         [1]

Dynamic power to be proportional to frequency times voltage squared:

P_d αFreqV²                                                      [2]

So dynamic power becomes proportional to voltage cubed:

P_d αV³                                                                        [3]

Static power (leakage current) is also proportional to voltage cubed.

At nominal conditions, static power is fixed at 35% of dynamic power:

P_s / P_t = P_s / P_t = P_s / (P_s + P_d) = 0.35                           [4]

Dynamic power is assumed to change by a factor of two every 22^oC, expressed by:

P_d α exp[0.0315(T_j - 95)]                                            [5]

At nominal frequency (and voltage) the temperatures of various packaging components are shown in Figure 3. While most of the temperature reduction is due to improved cooling, some is due to the reduction in dynamic power. This positive feedback converges toward a lower operating temperature.

Figure 6. System efficiency.

Reliability Concerns

One practical limit to increasing power is reliability, which is typically dominated by gate oxide  failure. This is assumed to change by a factor of two every 10^oC (typical of modern technology nodes, and happens to be equal to an often misapplied rule of thumb which has been around for decades) and/or every 15mV in the present study, resulting in:

Fail α exp{[0.0693(T_j - 95)] +

[46.4(V - 0.85)]}                         [6]

As the cooling performance is increased and thermal resistance is reduced, the reliability limit is reached at decreasing temperatures, higher voltages and slightly higher power than nominal. Ultimately leakage current is reduced to only one-seventh of its nominal value.

Frequency Improvements

Permitting the processor to operate at the highest combination of temperature and voltage, while maintaining gate oxide reliability equal or better than that of the nominal case, results in frequency improvements as high as 19%, as shown in Figure 4. The slopes of the lines from the origin to the operating point are proportional to computational efficiency, defined as frequency divided by power. In this case, as voltage is lowered, efficiency improves monotonically, with packaging and cooling technology making very little difference at the lowest values.

System Performance

If the processor provided all or nearly all of the power dissipation in a system, very low voltage (and frequency) operation would be most efficient. But processors consume, in all but a few cases, less power than memory. Volume systems typically feature fewer dual inline memory modules (DIMMs) per socket than high-end systems. High-end systems have been described as large boxes of memory with other hardware to drive it, and this is a reasonable description in terms of power dissipation, volume occupied and component cost.

Adding a constant memory power load changes power per socket and system performance to that of Figure 5. In this case memory power is equal to twice that of nominal processor power, typical of a modern high-end system. System performance is assumed to scale with the square root of processor frequency. This exponent relating processor frequency and system performance is at the low end of the range seen, which approaches a value of one at the high end and is strongly dependent on the program being exercised.

Unlike the example of a processor only, an optimum operating point at which the slope of power versus performance is maximized falls close to the midpoint of the range evaluated. This is slightly different for each packaging and cooling configuration. Dividing performance by power provides computational efficiency, which is shown compared to the nominal configuration in Figure 6. Here the optimum operating condition shifts to lower power as cooling is improved.

Figure 7. System performance versus TCO.

Total Cost of Ownership

While maximization of computational efficiency in terms of energy has been demonstrated, the total cost of ownership should be considered. Until just a few years ago, capital expenditure on computing equipment dominated energy cost, and the two expenses were the concern of different organizations with little communication. Recently higher energy costs, much higher energy usage and environmental concerns have resulted in a more holistic view of data center cost and efficiency.

In the present study, computing equipment cost is fixed to be equal to that of energy cost. At 11.4 cents per kilowatt-hour, a continuous watt of electricity costs exactly one dollar per year. This is close to an average data center’s electricity cost, so a rough approximation can be made that lifetime energy cost of a server in dollars is equal to its average power consumption in watts times its operational lifetime in years. Equal capital energy costs could thus be a server with a five year lifetime and capital cost equal to five times power consumption, for example.

Figure 7 plots normalized total cost of ownership (TCO) versus economic computational efficiency, a term invented for this study and defined as system performance divided by TCO. All optimum values feature TCO below the nominal value, and improvement in economic computational efficiency can be as high as 7%. As this is a simple model, some factors are neglected, which will alter the results if included. The cost of floorspace may be an additional factor, tax incentives and deductions will alter TCO calculations. Additional equipment external to the computing must be added or removed.

Conclusions

A case study showing energy reduction and/or performance maximization through improved cooling of a typical modern high-end server has been performed. In both energy efficiency and economic computational efficiency or TCO minimization, an optimum frequency (and voltage) exists for each packaging and cooling configuration. As cooling is improved, the advantage becomes more favorable. Energy computational efficiency can be increased to 15% above nominal, and economic computational efficiency can be increased to 7% above nominal. More detailed models will shift the values but should follow the same general trend. Lower temperature operation of processors is an available design option with immediate rewards.

Acknowledgments

This article is based on the author’s keynote presentation of the same title at “The Heat Is On: Performance and Cost Improvements through Thermal Management Design,” 21st March, 2011 [9]. The author thanks his colleagues Vadim Gektin (now at Huawei), who provided Figure 1, and Bruce Guenin, who invited him to write this article.

References

[1] International Technology Roadmap for Semiconductors, 2010, http://www.itrs.net/Links/2010ITRS/Home2010.htm

[2] N. S. Kim, T. Austin, D. Blaauw, T. Mudge, K. Flautner, J. S. Hu, M. J. Irwin, M. Kandemir and V. Narayanan, 2003, “Leakage Current: Moore’s Law Meets Static Power,” IEEE Computer, Vol. 36, No. 12, pp. 68-75

[3] S. Mukhopadhyay, A. Raychowdhury and K. Roy,  2003, “Accurate Estimation of Total Leakage Current in Scaled CMOS Logic Circuits Based on Compact Current Modeling,” Proceedings of the Design Automation Conference, pp. 169-174

[4] J. Wei, 2007, “Challenges in Package Cooling of High Performance Servers,” Proceedings of the 2007 International Electronic Packaging Technical Conference and Exhibition, IPACK2007-33637, ASME

[5] M. J. Ellsworth and R. E. Simons, 2005, “High Powered Chip Cooling – Air and Beyond,” ElectronicsCooling, Vol. 11, No. 3, http://www.electronics-cooling.com/2005/08/high-powered-chip-cooling-air-and-beyond/

[6] J. Deeney, 2002, “Thermal Modeling and Measurement of Large High Power Silicon Devices with Asymmetric Power Distribution,” Proceedings of the 35th International Symposium on Microelectronics, IMAPS

[7] D. Copeland, 2005, “64-bit Server Cooling Requirements,” Proceedings of the IEEE Twenty-First Annual IEEE Semiconductor Thermal Measurement and Management Symposium, pp. 94-98

[8] S. Kang and D. Miller, 2011, “Thermal Management Architectures for Rack Based Electronics Systems,” Proceedings of The Heat Is On: Performance and Cost Improvements through Thermal Management Design, MicroElectronics Packaging and Test Engineering Council (MEPTEC)

[9] D. Copeland, 2011, “Energy Reduction and Performance Maximization through Improved Cooling,” Proceedings of The Heat Is On: Performance and Cost Improvements through Thermal Management Design, MicroElectronics Packaging and Test Engineering Council (MEPTEC) l

If you would like to receive your free copy of ElectronicsCooling click here to subscribe.

Download the September 2011 issue here.

So, what to do, I wondered? It is the privilege of an editor writing an editorial to discuss any subject that comes to mind, as long as it is (remotely) linked to the theme of the journal. My Thermal Facts and Fairy Tales column is, as already indicated above, devoted again to the danger of using correlations for thermal management design purposes. I keep wondering why so many people stick to their use. In my opinion, the main reason is that correlations on the one hand fulfill the need to reduce the significant complexity of the real thermal world to something that is manageable, while on the other hand correlations seem to be based on science because so many handbooks devote so many chapters to their derivation and use in practice (“… for engineering convenience …”).

Fortunately, my feelings on the abuse of correlations are shared by two famous Stanford scholars: Bill Kays and Bob Moffat, who more or less gave up on dimensionless correlations about 40 years ago. Let me quote Bob Moffat: “The habit of assembling “dimensionless groups” and trusting the Buckingham Pi theorem was very important in the early years of the development of physics and engineering, but it has become so ingrained in our teaching that the dangers of blind obedience (the religious aspect) has been forgotten.  When “irrelevant parameters” are included in the parameter list before the groups are formed, what comes out are correlations that have some elements that simply conceal the interactions among irrelevant parameters.  Sometimes simpler is better.”

Moffat’s using the phrase “religious aspect” reminded me of the following. I once coined the abundant use of correlations “correligion,” not without a reason, because it has some similarity to certain aspects that all religions share.  Holy books, priests and believers, preaching the pros but being blind to the cons. People who don’t share their beliefs are considered unreliable, at least. I realize that America is a very religious country, but I was really shocked by the following: the 2007 Gallup poll asked Americans whether they would vote for “a generally well-qualified” presidential candidate nominated by their party with each of the following characteristics: Jewish, Catholic, Mormon, an atheist, a woman, black, Hispanic, homosexual, 72 years of age, and someone married for the third time. The result? Atheists closed the row: 45%. A Mormon president would not raise objections by 72%.

What happened since the days of Jefferson and Adams, who wrote around 1800: “As the Government of the United States of America is not, by any sense, founded on the Christian religion; as it has in itself no character of enmity against the laws, religion, or tranquility, of Musselmen; and as the said States never have entered into any war or act of hostility against any Mehomitan nation, it is declared by the parties that no pretext arising from religious opinions shall ever produce an interruption of the harmony existing between the two countries.” This concerned a treaty with Tripoli, the capital of Libya! Indeed Mr. Dylan, times they are a-changin’.

What is the lesson to be learned, be it for the claims of any religion or the claims in heat transfer textbooks when we don’t talk fairy tales, but real life? As Timothy Leary once put it: “Think for yourself, Question authority”.

All of these Calculation Corner articles are still available on the ElectronicsCooling website (www.electronics-cooling.com) by entering the appropriate keyword(s). As an aid to the reader, following is a listing of all the Calculation Corner articles, organized chronologically in topic areas. These topic areas include:

1) Heat Transfer Fundamentals;

2) Thermal Spreading Formulas;

3) Heat Sink Analysis and Performance;

4) Package Component Analysis and Performance; and

5) System Cooling Analysis, Applications and Trade-Offs.

Each listing provides the article title, author and date of publication. Beneath each listing is the Internet address of the webpage on which the article may be found. As a further aid to the reader, in the electronic version of this article on the ElectronicsCooling website, the Internet address beneath each article is a link, which if clicked on, will take the reader directly to the article.

Heat Transfer Fundamentals

“One-Dimensional Heat Flow”, Bruce Guenin, September 1997.

www.electronics-cooling.com/1997/09/one-dimensional-heat-flow/

“Convection and Radiation”, Bruce Guenin, January 1998.

www.electronics-cooling.com/1998/01/convection-and-radiation/

“Convection and Radiation Loss From a Fin”, Bruce Guenin, January 1999.

www.electronics-cooling.com/1999/01/convection-and-radiation-loss-from-a-fin/

“Don’t Underestimate Radiation in Electronics Cooling”, Bruce Guenin, February 2001.

www.electronics-cooling.com/2001/02/dont-underestimate-radiation-in-electronic-cooling/

“Simplified Formula for Estimating Natural Convection Heat Transfer Coefficient on a Flat Plate”, Robert Simons, August 2001.

www.electronics-cooling.com/2001/08/simplified-formula-for-estimating-natural-convection-heat-transfer-coefficient-on-a-flat-plate/

“Estimating Natural Convection Heat Transfer for Arrays of Vertical Parallel Flat Plates”, Robert Simons, February 2002.

www.electronics-cooling.com/2002/02/estimating-natural-convection-heat-transfer-for-arrays-of-vertical-parallel-flat-plates/

“Calculations for Thermal Interface Materials”, Bruce Guenin, August 2003.

www.electronics-cooling.com/2003/08/calculations-for-thermal-interface-materials/

“The 45 Heat Spreading Angle — An Urban Legend?”, Bruce Guenin, November 2003.

www.electronics-cooling.com/2003/11/the-45-heat-spreading-angle-an-urban-legend/

“A Simple Thermal Resistance Model — Isoflux Versus Isothermal”, Robert Simons, February 2006.

www.electronics-cooling.com/2006/02/a-simple-thermal-resistance-model-isoflux-versus-isothermal/

“Comparing Heat Transfer Rates of Liquid Coolants Using the Mouromtseff Number”, Robert Simons, May 2006.

www.electronics-cooling.com/2006/05/comparing-heat-transfer-rates-of-liquid-coolants-using-the-mouromtseff-number/

Thermal Spreading Formulas and Calculations

“The 45 Heat Spreading Angle — An Urban Legend?”, Bruce Guenin, November 2003.

www.electronics-cooling.com/2003/11/the-45-heat-spreading-angle-an-urban-legend/

“Simple Formulas for Estimating Thermal Spreading Resistance”, Robert Simons, May 2004.

www.electronics-cooling.com/2004/05/simple-formulas-for-estimating-thermal-spreading-resistance/

“Heat Spreading Calculations Using Thermal Circuit Elements”, Bruce Guenin, August 2008.

www.electronics-cooling.com/2008/08/heat-spreading-calculations-using-thermal-circuit-elements/

Heat Sink Analysis and Performance

“Estimating Parallel Plate-Fin Heat Sink Thermal Resistance”, Robert Simons, February 2003.

www.electronics-cooling.com/2003/02/estimating-parallel-plate-fin-heat-sink-thermal-resistance/

“Estimating Parallel Plate-Fin Heat Sink Pressure Drop”, Robert Simons, May 2003.

www.electronics-cooling.com/2003/05/estimating-parallel-plate-fin-heat-sink-pressure-drop/

“Estimating the Effect of Flow Bypass on Parallel Plate-Fin Heat Sink Performance”, Robert Simons, February 2004.

www.electronics-cooling.com/2004/02/estimating-the-effect-of-flow-bypass-on-parallel-plate-fin-heat-sink-performance/

“Using an Equivalent Heat Transfer Coefficient to Model Fins on a Fin”, Robert Simons, May 2005.

www.electronics-cooling.com/2005/05/using-an-equivalent-heat-transfer-coefficient-to-model-fins-on-a-fin/

“Estimating Thermal Resistance for Fin-to-Fin Thermal Couplers”, Robert Simons, February 2008.

www.electronics-cooling.com/2008/02/estimating-thermal-resistance-for-fin-to-fin-thermal-couplers/

“A Simple Method to Estimate Boiling Heat Sink Performance”, Robert Simons, February 2009.

www.electronics-cooling.com/2009/02/a-simple-method-to-estimate-boiling-heat-sink-performance/

“Thermal Interactions Between High-Power Packages and Heat Sinks, Part 1”, Bruce Guenin, December 2010.

www.electronics-cooling.com/2010/12/calculation-corner-thermal-interactions-between-high-power-packages-and-heat-sinks-part-1/

“Thermal Interactions Between High-Power Packages and Heat Sinks, Part 2”, Bruce Guenin, March 2011.

www.electronics-cooling.com/2011/03/calculation-corner-thermal-interactions-between-high-power-packages-and-heat-sinks-part-2/

Package and Component Analysis and
Performance

“Conduction Heat Transfer in a Printed Circuit Board”, Bruce Guenin, May 1998.

www.electronics-cooling.com/1998/05/conduction-heat-transfer-in-a-printed-circuit-board/

“Convection and Radiation Heat Loss From a Printed Circuit Board”, Bruce Guenin, September 1998.

www.electronics-cooling.com/1998/09/convection-and-radiation-heat-loss-from-a-printed-circuit-board/

“Determining the Junction Temperature in a Plastic Semiconductor Package, Part I”, Bruce Guenin, May 1999.

www.electronics-cooling.com/1999/05/determining-the-junction-temperature-in-a-plastic-semiconductor-package-part-1/

“Determining the Junction Temperature in a Plastic Semiconductor Package, Part II”, Bruce Guenin, September 1999.

www.electronics-cooling.com/1999/09/determining-the-junction-temperature-in-a-plastic-semiconductor-package-part-ii/

“Determining the Junction Temperature in a Plastic Semiconductor Package, Part III: The Use of the Junction-to-Board Thermal Characterization Parameter”, Bruce Guenin, May 2000.

www.electronics-cooling.com/2000/05/determining-the-junction-temperature-in-a-semiconductor-package-part-iii-the-use-of-the-junction-to-board-thermal-characterization-parameter/

“Determining the Junction Temperature in a Plastic Semiconductor Package, Part IV: Localized Heat Generation on the Die”, Bruce Guenin, September 2000.

www.electronics-cooling.com/2000/09/determining-the-junction-temperature-in-a-semiconductor-package-part-iv-localized-heat-generation-on-the-die/

“Characterizing a Package on a Populated Printed Circuit Board”, Bruce Guenin, May 2001.

www.electronics-cooling.com/?s=characterizing+a+package+on+a+populated+printed+circuit+board&x=36&y=12

“Simplified Transient Model for IC Packages”, Bruce Guenin, August 2002.

www.electronics-cooling.com/2002/08/simplified-transient-model-for-ic-packages/

“Thermal Calculations for Multi-Chip Modules”, Bruce Guenin, November 2002.

www.electronics-cooling.com/2002/11/thermal-calculations-for-multi-chip-modules/

“Thermal Vias — A Packaging Engineer’s Best Friend”, Bruce Guenin, August 2004.

www.electronics-cooling.com/2004/08/thermal-vias-a-packaging-engineers-best-friend/

“Entrance Effects for Heat Flow Into a Multi-Layer Printed Circuit Board”, Bruce Guenin, November 2004.

www.electronics-cooling.com/2004/11/entrance-effects-for-heat-flow-into-a-multi-layer-printed-circuit-board/

“A Funny Thing Happened on the Way to the Heat Sink”, Bruce Guenin, August 2005.

www.electronics-cooling.com/2005/08/a-funny-thing-happened-on-the-way-to-the-heatsink/

“So Many Chips, So Little Time: Device Temperature Prediction in Multi-Chip Packages”, Bruce Guenin, August 2006.

www.electronics-cooling.com/2006/08/so-many-chips-so-little-time-device-temperature-prediction-in-multi-chip-packages/

“Toward a Thermal Figure of Merit for Multi-Chip Packages,” Bruce Guenin, November 2006.

www.electronics-cooling.com/2006/11/toward-a-thermal-figure-of-merit-for-multi-chip-packages/

“Thermal Strain in Semiconductor Packages, Part I”, Bruce Guenin, August 2007.

www.electronics-cooling.com/2007/08/thermal-strain-in-semiconductor-packages-part-i/

“Thermal Strain in Semiconductor Packages, Part II”, Bruce Guenin, November 2007.

www.electronics-cooling.com/2007/11/thermal-strain-in-semiconductor-packages-part-ii/

“Power Map Calculations Using Image Sources and Superposition”, Bruce Guenin, November 2008.

www.electronics-cooling.com/2008/11/power-map-calculations-using-image-sources-and-superposition/

“Using a Matrix Inverse Method to Solve a Thermal Resistance Network”, Robert Simons, May 2009.

www.electronics-cooling.com/2009/05/using-a-matrix-inverse-method-to-solve-a-thermal-resistance-network/

“A Spreadsheet Based Matrix Solution for a Thermal Resistance Network: Part 1”, Ross Wilcoxon, September 2010.

www.electronics-cooling.com/2010/09/calculation-corner-a-spreadsheet-based-matrix-solution-for-a-thermal-resistance-network-part-1/

“Using Vendor Data to Estimate Thermoelectric Module Cooling Performance in an Application Environment”, Robert Simons, July 2010.

www.electronics-cooling.com/2010/07/using-vendor-data-to-estimate-thermoelectric-module-cooling-performance-in-an-application-environment/

System Cooling Analysis, Applications and Trade-Offs

“Estimating Temperatures in a Water-to-Air Hybrid Cooling System”, Robert Simons, May 2002.

www.electronics-cooling.com/2002/05/estimating-temperatures-in-a-water-to-air-hybrid-cooling-system/

“Estimating Temperatures in an Air-Cooled Closed Box Electronics Enclosure”, Robert Simons, February 2005.

www.electronics-cooling.com/2005/02/estimating-temperatures-in-an-air-cooled-closed-box-electronics-enclosure/

“Using a Simple Air Recirculation Model to Explore Computer Rack Cooling”, Robert Simons, February 2007.

www.electronics-cooling.com/2007/02/using-a-simple-air-recirculation-model-to-explore-computer-rack-cooling/

“Estimating the Effect of Intercoolers for Computer Rack Cooling”, Robert Simons, May 2007.

www.electronics-cooling.com/2007/05/estimating-the-effect-of-intercoolers-for-computer-rack-cooling/

“Estimating Dew Point Temperature for Water Cooling Applications”, Robert Simons, May 2008.

www.electronics-cooling.com/2008/05/estimating-dew-point-temperature-for-water-cooling-applications/

“Measuring Heat Load to Water in a Rear Door Heat Exchanger Application,” Robert Simons, June 2011.

www.electronics-cooling.com/2011/06/calculation-corner-measuring-heat-load-to-water-in-a-rear-door-heat-exchanger-application/

It is hoped that you will find these listings helpful in finding earlier Calculation Corner articles which may be of interest to you. If you have any ideas for specific topics you would like to see covered in future articles, please send Bruce (bruce.guenin@oracle.com) or myself (resimons@att.net) a note.  Also, you may note that the September 2010 Calculation Corner article was written by a guest author, Ross Wilcoxon. If you, too, have an idea for an article you would like to submit as a guest author, for possible publication in the Calculation Corner, please contact us.

There are a number of methods of improving fan efficiency including speed control, optimized motor design, optimized electronic brushless motor drive design, and optimized system design. These are fairly well understood and have been deployed in modern forced convection cooling solutions. What is often not accounted for, even among sophisticated designs, is the optimization of the cooling fan for a specific flow pressure operating point. This Technical Brief will deal with the significance of fan selection and aerodynamic design optimization.

Figure 1. Air resistance curves for two applications and two possible fan designs.

There is a surprising amount of power savings available by optimizing the fan design around a particular flow pressure point. Many of the fan manufacturers’ data sheets provide a flow pressure air performance curve but only provide power data during free delivery air flow conditions (zero back pressure). This is an unrealistic operating point within electronic equipment since there is always some restriction to air flow and resulting back pressure. Every cooling application produces a characteristic air resistance curve which intersects the fan air performance curve as shown on Figure 1. There is a trend in the electronics industry for applications to have greater air resistance as density of electronics within the enclosure increases. This is represented by the air resistance curve in Figure 1 with higher pressure for any given air flow value. When an electronic cooling fan application has a relatively high air resistance, choosing a fan that has been optimized for free delivery can result in unnecessary wasted power. Every fan has a peak aerodynamic efficiency point somewhere on its flow-pressure curve and there can be significant energy savings by ensuring the actual operating point of the application matches the peak efficiency point of the fan. In Figure 1 the flow pressure curve developed by fan “A” is one such design optimized for high flow but the fan performance of the flow pressure curve for fan “B” is optimized for the actual high resistance operating point. Many of today’s standard, commercially available fans produced in high volume are not easily modified without significant investment. These fans are often selected by engineers since they produce a flow-pressure curve that passes through the desired fan performance operating point — but often not at the peak efficiency possible.

A typical example of an actual fan application is shown below where a customized design significantly reduced power consumption when compared to the standard production catalog fan. Both fan designs delivered the same air performance with the same space claim, but fan power consumption was significantly different, in fact, cut in half!

Figure 2. CFD results showing stall at high pressure and redesign.

Example 

One of the most commonly used cooling fans in the industry is the 120mm tube axial fan. This fan has generally been optimized for low pressure applications with very little air resistance.  An optimized version was developed for an airborne vapor cycle liquid chiller application where the density of heat exchanger fins requires the fan to operate in the high pressure portion of its flow-pressure curve. The flow and pressure performance point is:  38.6 mm H₂O 23.6 l/s (50 CFM, 1.45 in H₂O). The optimized fan required a new propeller design with more blades, lower pitch blades and a higher operating speed. Table 1 provides of comparison of these features along with test results showing input power cut in half when compared to the standard production fan operating at the same high pressure point. The basic motor design, electronic drive and venturi remained the same. The Computational Fluid Dynamics (CFD) simulation output showing velocity vectors for the two designs operating at the relatively high pressure, low flow point is provided in Figure 2. Note the airflow stall region in the existing production design just under the blade. Overall sound power was reduced by 2 dB even though the new design is operating with a significantly higher speed. In addition, an annoying pure tone was eliminated.

Table 1. Measured performance comparison (two designs delivering the same pressure and air flow).

Conclusion

We have provided a convincing example of opportunities to save power by optimizing the fan aerodynamics for specific application operating points. For the application shown, the original catalog production fan selected by the customer achieved its desired flow pressure operating point. The problem was that the fan was not performing very efficiently. The technology to develop more efficient fans is in place. The precision of the latest CFD software, when used by a skilled engineer, has been shown to replicate actual test data within a few percentage points, greatly reducing engineering time and cost for custom designs. The barrier to more widespread fan design optimization is primarily an economic one due to the increased cost of a non-standard customized design produced in lower volume. However, the greater initial purchase price may be offset by the lifetime energy savings of the product.

Looking to the future, the increased life cycle cost of energy and social awareness of energy production by-products is surely going to move the threshold of justification in favor of high efficiency cooling fans [3].

References

[1] Hardy, Q., “Switchcraft,” Forbes, pp. 69 -73, September 29, 2008.

[2] “The Green Grid Opportunity- Decreasing Datacenter and other IT Energy Usage Patterns”, WP#2” February 16, 2007.

[3] Smith, N., “High Efficiency Electronic Cooling Fans”, in Proceedings of  25th Semi-Therm Symposium, 2009.

A sort of mathematicians’ paradise

Where everything happens

As it does in textbooks

— Bertrand Russell

This is the third column devoted to the sense and nonsense of correlations. My earlier comments [1] regarding Russell’s quote are maybe worth repeating: “Generations of mechanical engineers have been educated to use the correlations that fill the textbooks Russell is mentioning because: ‘For convenience of engineering applications, correlation equations are developed over the entire range of dimensions and for any Pr-number.’ Unfortunately, for a thermal designer, real-life is more like hell than paradise, and hence it is high time to address the question, ‘How convenient are these correlations in real-life?’ When it comes to thermal management of electronic systems, the answer is that correlations are not only useless but also dangerous because their use by non-experts gives a false sense of safety.” Maybe “useless” is a bit exaggerated. As one reviewer put it: “Correlations can be used to provide a basic sense of how a system might react under some simple geometries and equivalent inputs and can potentially yield important information of general behavior.” The problem is that in practice simple geometries are not the rule, but the exception.

The paper goes on: “Suppose some alien wants to study the heat transfer behaviour of a flat heat sink cooled by air. He, she or it never had the opportunity to read standard textbooks on heat transfer. The alien varies the parameters of interest, for example the power q and the length L, measures the temperature rise, and comes up with the following equation:

providing exactly the information needed: the dependency of the temperature rise on the power and the heat sink dimensions.” The alien would be very surprised if someone told him to present the results in the following way, because it should be “more convenient for engineering applications”:

The author fully agrees with the comments made by Botterill [2] some 20 years ago: “Give preference for the most simple form of representation instead of the much more ambitious generalisations.” The same way of working Bill Kays and Bob Moffat at Stanford proposed already 40 years ago (personal communication): “Simpler is better, hence don’t plot Nu vs. Re but h vs. v.”

I would like to mention two other flaws of publishing correlations: loss of original data and danger of imposing a slope. The first flaw is easily understood. Read any article in ASME journals and try to get hold of the original test data. You can’t.  Additionally, because all data points are first made dimensionless and then put on double-log paper, details are lost. Following Bob Moffat: “Years of poorly controlled and inadequately described experiments have filled the literature with data that appear to be ‘comparable’ but are not, with the consequence that the average designer seems to accept the fact that there will be always ± 20-50% scatter in heat transfer data.” In general, you can be confident that the original data did not show this large scatter.

The second flaw is less obvious. In mathematics, everyone strives towards separation of variables because doing so simplifies the solution. Not so in heat transfer. This becomes immediately obvious when looking at the following general Nu-Ra equation describing general heat transfer for natural convection, Nu = C.Ra^N:

where ΔT, L and k are common to both sides of the equation. This confounding of parameters has led to a curious observation, originally formulated by Rowe [3] and discussed by Wilkie in two papers [4,5]. The results presented by Rowe are very illustrative. He used data points from random number tables for the parameters that are common to both sides of a set of dimensionless equations, and plotted the ‘results’ on log-log paper. After ignoring a few rogue points, combinations such as Nu-Re, Nu-Gr and several others clearly showed a correlation where obviously there can be none. Wilkie reinterpreted Rowe’s data as follows. If Equation (3) is plotted on log-log paper, a straight line is obtained. He showed that by varying L a slope of 1/3 is imposed. It is interesting to note that this slope is close to the often-quoted values in the literature for turbulent heat transfer. In his second paper, Wilkie extended the analysis to the situation where the three common variables L, ΔT and k are varied simultaneously, which could be the case in natural convection studies, and presented some convincing arguments that some often-used correlations (especially for turbulent natural convection heat transfer) should be regarded with caution. If the observed slope from an experiment is not significantly different from the imposed slope (as determined by proper statistical means), no conclusion can be drawn about the correlation between the variables that are common to both sides. As a consequence, randomly chosen values of the heat transfer coefficient could produce an apparent correlation with an index given by the imposed slope, or, alternatively, a slope of magnitude different from the imposed slope could remain undetected. Especially when a large data set is being used, for example when one of the common variables has been varied over a wide range, the danger of an imposed slope becomes relevant. This is clearly the case for the famous correlation for natural convection from horizontal cylinders spanning more than 13 decades in Gr presented by McAdams [6] almost 60 years ago. Note that the upper-half slope is 1/3, exactly the exponent for turbulent heat transfer, but also of the imposed slope.

Figure 1. Nu-Ra correlation for horizontal cylinders (McAdams).

Figure 2. Nu-Re for random h.

One of Wilkie’s recommendations is to vary only those parameters that are not common to both sides. One of my recommendations is to refrain from common-variable formulae altogether.

The following simple example illustrates the problem raised by Rowe and Wilkie in a convincing way. Suppose we want to study turbulent pipe flow in a strange world where all physical properties happen to be 1. We set the dissipation to 1 W, vary the diameter over quite a big range, the velocity at three levels and measure the temperature rise. The ratio q/ΔT provides us with the heat transfer coefficient h. For reasons yet unknown in this strange world to be explored, whatever we choose for D and v, h varies between 1 and 10 in a random way. Let us now plot Nu=h·D/k against Re=v·D/v, with k and v set to 1.

I hope the reader understands that the essential point here is not the value for Nu and Re (can be scaled of course to suit any realistic world) but the fact that D varies over a large range (but not as large as in the McAdams example cited above). From the graph a clear correlation between Nu and Re becomes apparent, namely Nu=C.Re^N, with C=1.67 and n=0.97. In other words, the graph suggests that h is about proportional to v, which is clearly not the case. Figure 3 shows a ‘traditional’ plot of h against D with v as parameter from which it is immediately obvious that there is no correlation at all.

Conclusions

It should be noted that correlations have proven their value over the last century in a wide range of applications, some of which are not only of academic interest. However, when focusing on electronics cooling, we face a different situation. There is one major reason why correlations are not recommended: the inherently complex geometries. Correlations make sense only when three conditions are fulfilled [1]: similarity, congruency and the boundary layer approximation. In most practical cases these conditions are not met, and if so, the number of required dimensionless groups becomes intractable. The bottom line is: geometrically and physically complex phenomena cannot be described by simple equations.

This column focuses on a peculiarity of correlations: a non-physical slope can be imposed when attempting to develop relations using common-variable formulae leading to wrong conclusions. A simple example

Figure 3. h as a function of D, with v as parameter.

convincingly demonstrates this phenomenon.

References

[1] Lasance C., “Sense and Nonsense of Heat Transfer Correlations Applied to Electronics Cooling,” Proc. 5th Eurosime, Berlin, 2005.

[2] Botterill J., “No Substitute for Experimentation – Despite the Fallibility of Experimenters,” Exp. and Thermal Fluid Science, Vol. 3, pp. 463-466, 1990.

[3] Rowe P., “The Correlation of Engineering Data,” The Chem. Eng., No. 166, March, pp. CE 69-CE76, 1963.

[4] Wilkie D., “The Correlation of Engineering Data Reconsidered,” Int. J. Heat & Fluid Flow, Vol. 6, pp. 99-103, 1985.

[5] Wilkie D., “Some Doubtful Natural Convection Correlations,” Proc. 9th Int. Heat Transfer Conf., Jerusalem, pp. 555-560, 1990.

[6] McAdams W., Heat Transmission, 3d Edition, McGraw-Hill, p. 176, 1954. l

Recently, alternatives to full CFD have been discussed in the technical literature [1-8], some of which even provide CFD-like 3D velocity, pressure, and temperature predictions. Such tools can be used directly or incorporated within a more comprehensive data center design and management software package [9].

Figure 1. Racks color-coded by capture index.

In the initial planning stages for a new facility or modification, what is most needed is a dynamic go/no-go indication of cooling performance as the design takes shape. For example, racks and perforated tiles can be re-colored instantly based on cooling performance and airflow rate, respectively, as you move objects and change settings. Although the attractive colored planes and airflow vectors familiar from CFD tools provide real engineering value, simple empirical predictions can be used to provide a quick estimate of rack cooling performance. For example, estimating a single average rack inlet or cooler return temperature for all such objects provides greater value-per-effort than predicting the temperature at all points in the data center. In Figure 1, racks are colored based on a metric termed “capture index” (see sidebar, “Capture Index,” on Page 16), which is a measure of the quality with which air is delivered to rack inlets and captured from rack exhausts. The rack-level-empirical model approach works particularly well when equipment is arranged in geometrically regular groupings surrounding a hot or cold aisle, extreme examples of which are cold and hot aisle containment systems.

Several empirical rack-level cooling prediction techniques have been developed, for example [1-3], and the simplest techniques fit algebraic models to the airflow physics as determined by simple CFD analyses. With this approach, thousands of CFD simulations of simpler building-block units of data centers are analyzed covering the range of input parameters in the tool, then the models are “tuned” to provide the best fit to the CFD data. In other words, the accuracy of a CFD solution is still leveraged but the time and complexity associated with running models is moved “off line”. The resulting models are robust and simple enough to run every time the model is changed. Using the rack-level algorithms discussed above, one can predict capture index and, by extension, average rack inlet temperatures, average cooler return temperatures, and cooler loads.

While rack-level empirical models can form the key “cooling calculation engine” of data center design and management tools, many users would also like to “see” airflow vectors, temperatures, and pressures – just like CFD. For this purpose, and also for the very practical purpose of predicting perforated tile airflow rates, a Potential Flow Model (PFM) can be employed. The term “potential flow” applies to an irrotational velocity field.  When the flow can also be assumed to be incompressible, velocities can be determined from Poisson’s Equation. For general room-airflow predictions (e.g., above the raised floor in the data center white space), pressures are never explicitly required; however, they are needed when modeling pressure-dependent flow boundary conditions such as perforated floor tiles. Finally, once airflow patterns are known, temperatures may be determined from the energy equation.

Figure 2. PFM-CFD capture index prediction comparisons for eight data center layouts.

Compared to the formidable set of Navier-Stokes equations solved by traditional CFD, the potential flow model is, indeed, straightforward. Solutions are well behaved and convergence is achieved quickly and predictably. By using an efficient unstructured grid and optimized solution-control parameters, even full 3D room-level solutions can be obtained in several seconds on common computing hardware.

What do we have to give up for this tremendous improvement in solution time and robustness? Since rotationality — neglected in PFM — is often present in highly-viscous regions, potential flow solutions lack the ability to capture strong jet-like flow features, turbulence, and recirculation zones. Further, since the flow field is determined independently from the momentum and energy equations, buoyancy is neglected. Is this a reasonable assumption? It would not be reasonable for predicting airflow in general buildings where buoyancy may be a primary flow driver, but, in data centers, airflow patterns are generally dominated by the large amounts of airflow driven by racks and coolers. A recent study [10] concluded that airflow is generally dominated by momentum when the typical temperature rise across a rack is on the order of 10°C but when that figure rises to 20°C or more buoyancy becomes important.

Accuracy requirements must be tied to overall objectives. For data center design and management purposes, it is often enough to generally predict the correct bulk movement of airflow around racks and coolers which will, at least, reveal which areas are sufficiently cooled, inadequately cooled, or marginal. A high level of airflow-prediction accuracy may be wasted in most practical scenarios where the input data is either poorly estimated or impractical to include in the model.

Figure 3. 2D plenum example.

Considering the issue of accuracy more quantitatively, several recent studies [4-8] have assessed the use of PFM for data center cooling prediction. A study of 8 data center layouts, based on actual modern facilities [7], produced the comparisons of the capture index of PFM to CFD shown above (Figure 2). For example, layout A represents a standard best-practices raised-floor facility utilizing alternating cold and hot aisles with the Computer Room Air Conditioners (CRACs) aligned with the hot aisles. In this case, over 90% of PFM predictions are better than 93% accurate and 10% of the predictions are better than 98% accurate. For every layout, predictions for over half the

racks are better than 80% accurate. Average inlet temperature predictions are of similar accuracy to capture index, however, as might be expected, maximum rack inlet temperatures, which are local in nature, are not predicted very accurately by PFM.

Of course, if server and other input data can be accurately captured, traditional CFD will, indeed, yield better predictions. Further, commercial CFD packages typically provide a rich set of tools for modeling such things as equipment with pressure or flow-dependent control, unique architectural features of the building, and transient processes; all of which may be beyond the scope of design and management tools which utilize PFM. Additionally, CFD provides particularly better accuracy in cases where there is strong coupling between the room and under-floor plenum, such as when large-open-area perforated tiles are used.

Potential Flow Model Example

First consider a plenum airflow application where the primary goal is to estimate perforated tile airflow rates. In this case, a depth-averaged 2D model is sufficient and calculations run instantaneously on any modern computer [6]. In other words, drag in a new cooling unit or move a perforated tile and the flow vectors and perforated tile flow rates (Figure 3) are updated instantly as new objects are added or settings are modified. This is a much different user experience than that of CFD which typically involves a lot of time waiting for results and subsequently comparing two or more static solutions.

While 2D plenum calculations run faster than full 3D room calculations, the plenum application actually involves extra steps. Since perforated tile airflows are a primary output of the analysis, they are unknown at the beginning of the analysis and must be related back to pressure differences across the tiles based on tile performance data. Consequently, plenum pressures need to be computed and coupled to the airflow solution necessitating additional iteration. 

Figure 4. 3D room example.

Once the perforated tile airflow rates are determined, they can be used as boundary conditions for the 3D PFM solution for the above-the-raised-floor portion of the facility. This calculation is not quite real time but runs in just a few seconds on common computers, even for very large rooms. Velocity vector and temperature results (Figure 4) look like those from traditional CFD and provide the same ability to quickly locate problem spots and understand the underlying causes.

Summary

Data center design and management software utilizing the methods discussed here can be used to quickly optimize a data center design with much less time, cost, and specialized training than required by CFD. Further, at least one such package [9] provides the ability to export models to an XML neutral file format or directly to the native file format of a data center and buildings modeling CFD package [12]. This allows the user to leverage the advantages of both PFM and CFD tools.

Returning to our initial question: do you really need the accuracy of a full CFD package? Or are simple empirical models and PFM “good enough”? Given the poor quality of and limited time available for collecting the required input data and the huge improvements in speed and usability as compared to CFD, these new tools represent a very attractive alternative for many data center design and management applications.

References

[1] VanGilder, J., and Shrivastava, S.K., 2006, “Real-Time Prediction of Rack-Cooling Performance”, ASHRAE Transactions, Vol. 112, Part 2, pp. 151-162.

[2] VanGilder J., Zhang, X., and Shrivastava, S.K., 2007, “Partially Decoupled Aisle Method for Estimating Rack-Cooling Performance in Near-Real Time”, Proceedings of InterPACK’07, International Electronic Packaging Technical Conference and Exhibition, July, Vancouver, Canada.

[3] Shrivastava, S.K., VanGilder, J. and Sammakia, B.G., 2007, “Use of Artificial Neural Network in Data Center Cooling Prediction”, Proceedings of InterPACK’07, International Electronic Packaging Technical Conference and Exhibition, July, Vancouver, Canada.

[4] Lopez, V. and Hamann, H., 2010, “Measurement-Based Modeling for Data Centers”, Proceedings of ITHERM, June 2-5, Las Vegas, NV.

[5] Hamann, H., Lopez, V., and Stepanchuk, A., 2010, “Thermal Zones for More Efficient Data Center Energy Management”, Proceedings of ITHERM, June 2-5, Las Vegas, NV.

[6] VanGilder, J., Sheffer, Z., Zhang, X., and Healey, C., 2011, “Potential Flow Model for Predicting Perforated Tile Airflow in Data Centers”, ASHRAE Transactions, Vol. 117, Part 2.

[7] Healey, C., VanGilder, J., Sheffer, Z., and Zhang, X., 2011, “Potential-Flow Modeling for Data Center Applications”, Proceedings of InterPACK, July 6-8, Portland, OR.

[8] Toulouse, M., Doljac, G., Carey, V., and Bash, C., 2009, “Exploration of A Potential-Flow-Based Compact Model of Air-Flow Transport in Data Centers”, Proceedings of IMECE, November 13-19, Lake Buena Vista, FL.

[9] StruXureWare for Data Centers.  http://www.apc.com/site/software.

[10] Demetriou, D.W. and Khalifa, H.E., 2011, “Evaluation of a Data Center Recirculation Non-Uniformity Metric Using Computational Fluid Dynamics”, Proceedings of InterPACK, July 6-8, Portland, OR.

[11] VanGilder, J. and Shrivastava, S.K., 2007, “Capture Index: An Airflow-Based Rack Cooling Performance Metric,”ASHRAE Transactions, Vol. 113, Part 1, pp. 126-136.

[12]    FLOVENT by Mentor Graphics, http://www.mentor.com/products/mechanical/products/flovent.

Classification of Problem

The best place to begin deciding what type (or multiple types) of solution method may be preferred is to classify the problem to be solved. The two classification methods used for this article will be examining the geometry definition of the problem and the goals of the thermal model.

This means understanding two things clearly: what type of information is available at the problem definition; and the end goals of the solution. Examining these two areas will tend to point one in a specific direction. They also will point out the possibility of other solution methods and the advantages they may contain over one’s typical method.
Geometry definition is a statement about how much detail about the geometry is known at the time the problem is to be solved. Anyone who has been involved in a variety and significant number of projects has seen a wide range in this category. Often thermal analysts have been brought in late to solve an issue after the entire project is nearly complete; in these cases the geometry is well defined and often thoroughly detailed in explicit CAD models with complete bills of materials. One could label these as “fully defined” geometry definitions.

.”]

Figure 1. Sample fundamental equation network [5

Solution Methods for Problem Classes

With these classifications and goals in mind, one can then examine the solution methods available and see that there are some reasonable fits between the problem and type of solution method. This is an important step. One mistake people tend to make is to use and re-use methods they are most familiar with rather than what may be most suitable. This leads to forcing a method or tool to solve the problem. While this will still lead to solutions, it is not necessarily the best manner to go about this process. From this viewpoint, the authors propose the following matches of methods to the problem classes previously described:
Numerical analysis with systems of fundamental equations for nebulous geometry or multiple scenarios
When geometry is largely unknown, or multiple scenarios must be evaluated, it is most efficient to keep factors set to a variable or numerical value to allow for faster changes. For example, the convective surface area of an enclosure could be represented by number (say .25 m²), or it could be explicitly modeled in 3D CAD. If one wished to change this area to 0.3 m², this is an easy change if it is just a variable in an equation; the change becomes much harder if the 3D model must be changed (and this becomes cumbersome if several variables have a number of values to be evaluated).
To use this method, systems of fundamental, simultaneous equations are written and solved. In the authors’ experience this is tractable if the number of equations and unknowns is under 25 or so; beyond that may be difficult to set up and solve effectively, depending on the solution algorithm. The equations are forms of the three basic heat transfer equations for conduction, convection and radiation familiar to the reader:

kA

      l(T₁ - T₂)

Q=hA(T₁ - T₂)

Q=σAfε(T₁⁴ - T₂⁴)

A key point here is that the heat flow path must be visualized enough to write the equations correctly. The visualization exercise is an extremely powerful thought and discussion tool. The solved equations serve to quantify the relative heat flow paths, identifying trouble spots and opportunities for improvement. Of course, the accuracy of the results depends heavily on the accuracy of the thermal network. Sometimes several versions of the network are needed to arrive at a suitable representation — just as several versions of experimental or computational models are needed to achieve confidence in the model. Also, sometimes the accuracy of the solution is less important than the ability to quantify relative effects in order to make a good design decision. For the design decision, the ability of the network to capture the effect of a design variation is key.
To solve the system of equations, software that is essentially high level programming (e.g., commercial codes such as MATLAB or Mathcad, or network solvers such as the SPICE codes for solutions) is well suited to the analytical method. Figure 1 shows a sample problem set up using Mathcad. Author Petroski has used this method for small electronic devices dissipating a few watts to much larger electronic cabinets of 1kW dissipation with good correlation to product tests.
The network equations can also be implemented in spreadsheet form if needed, see [4]. There are many powerful programming tools available in spreadsheets far beyond simply coding formulas using cell references with row and column indicators. The biggest advantage of spreadsheet analysis is that the software is available on nearly every computer as part of an office software suite, requiring no additional purchase or installation. This feature facilitates sharing and discussion with team members who are not thermal specialists. A disadvantage of solving the network equations in a spreadsheet is that the solution equations must be coded afresh if there are changes in number of nodes, or in the way they are linked together. Also, since everything is done manually, debugging and assessing the suitability of the network are entirely up to the user. Another disadvantage to spreadsheets is that non-linearity in the equations (for example, properties that depend on temperature) can be tricky to handle, although there are ways to include these effects that are beyond the scope of this article.

Numerical analysis with discretization for nebulous geometry or multiple scenarios

Solving the same type of problem as the previous case is possible with a resistance network that would discretize the model into more regions. It also may allow for easier solutions with other programming methods; some non-linear areas of a problem may be translated into linear forms without much loss of fidelity. For example, a flat plate with a small heat source may contain spreading resistances; such a problem may be handled by discretizing the region into smaller regions with the appropriate conduction equations (see [3] for such equations). Such a problem could allow for multiple spreading scenarios to be examined without using a single equation for spreading and possibly violating its assumptions.  Viewed another way, perhaps a design may or may not incorporate spreading as a significant contribution to the heat transfer. A discretized region that can incorporate spreading, or not, by using the conduction equations may simplify solving the model for a variety of materials and geometries.

.”]

Figure 2. Sample hybrid network in a finite difference modeler [6

Hybrid solutions for partially known geometry and some trade-off studies
When some of a problem’s geometry is well-defined, a combination of a fully discretized solution and some ability to handle unknown geometry is useful. For example, one may have a PCB or electronics module whose design is complete, but whose surrounding chassis or enclosure design is not defined. The two previous methods work well for the unknown area of the design where single or small numbers of nodes represent major sections of it. A well-defined numerical model can be used for the known module.
These two types of models then need to be combined for a solution. With the proper heat transfer equations one can connect these two parts. Perhaps the simplest manner to model this is to use the finite difference modeler. A discretized model is feasible for the well-defined portion of the model, and simple thermal resistance elements make up the connections to the undefined areas and the undefined geometry as well. As an example, see Figure 2. A portion of the model is well discretized and defined, while the rest of the model is composed of few elements. Model creation, with the solutions for different scenarios, is a straightforward process. The flexibility this approach provides for the unknown geometry is useful to find a final design that should meet the system temperature requirements, while providing good fidelity for the known geometry thermal profile.

Automated numerical solutions for fully defined geometry and fully specified goals
The final case is where the geometry of the problem is fully defined. This occurs near the end of the development when full CAD-based geometry is complete. At this point, a fully automated numerical solution is feasible with a complete discretized grid or mesh and appropriate boundary conditions, material properties, thermal loads, etc., applied. Any of the commercial or academic codes from the finite element method, computational fluid dynamics or the finite difference method can be used for this solution type. For this solution, the goal is often a final or near-final analysis of the problem and iterations for geometry changes or different scenarios are usually few or none. The greatest detailed solution is found with this method, but at the cost of knowing the final design and often is the longest solution time.

Conclusions

There are several methods to solve any thermal analysis, but given the different levels of geometry definition one can face, and the different types of goals for the end analysis, it is best to choose a solution appropriate for the class of the problem. Different solution methods have different advantages, and one should choose a method best compatible with the end goal(s). Ideally one should choose a solution method that provides the best efficiency for the type of problem at hand. This will assist in avoiding approaches that resemble the proverb, “If you only own a hammer, everything looks like a nail.”  CFD is a fine problem solver, but if one is looking to evaluate multiple scenarios and many geometric conditions, a situation where dozens of analyses may result in weeks passing before everything is evaluated where another modeling approach would complete the task in hours or a few days.
Another drawback of using CFD directly is that fundamental limitations to the problem aren’t flagged, whereas the thought process required by model construction forces identification of the limiting factors. Thus, evaluating the basics first is also important — one could do a lot of modeling, only to find that the constraints are infeasible. This leaves the engineer doing a great deal of work with no result or poor results, and may be likened to using a hammer to pound in a screw, to alter the proverb. Simpler methods often identify important features and point to the solutions without resorting to extensive modeling.

References

[1] Luiten, G.A., “Cooling of a Flat TV Monitor”, ElectronicsCooling, Vol. 9 No. 2, May 2003.

[2] Luiten, G.A., “The Better Box Model”, ElectronicsCooling, Vol. 15 No. 3, August 2009.

[3] Lasance, C., “How to Estimate Heat Spreading Effects in Practice”, Journal of Electronics Packaging, 031004, Vol. 132, September 2010.

[4] Wilcoxon, R., “Calculation Corner: A Spreadsheet Based Matrix Solution for a Thermal Resistance Network, Part 1”, ElectronicsCooling, Vol. 16 No. 3, September 2010.

[5] Equation system developed and solved in Mathcad software by PTC.

[6] Model produced in Sauna MS software by Thermal Solutions.

[7] Belady, C., and Minichiello, A., “Effective Thermal Design for Electronic Systems”, ElectronicsCooling, Vol. 9 No. 2, May 2003.