Figure 1: Closed box electronics enclosure with thermoelectrically enhanced heat rejection.

For those that are unfamiliar with TE modules, a thermoelectric cooler, sometimes called a Peltier cooler, is a solid-state heat pump that transfers heat from one side of the device to the other side depending on the direction of the applied electric current [2,3]. For example in Figure 1, electric current, I, may be applied to the thermoelectric modules such that the side in contact with the base of the heat sink within the box becomes cooler, thereby augmenting the transfer of heat from the air circulating within the box to the internal heat sink. The opposite side of the thermoelectric module becomes hotter and both the heat pumped by the thermoelectric cooler (qp), and the heat dissipated (qte) by the thermoelectric cooler in performing its heat pumping function, is transferred to outside cooling air via the external heat sink.

This article is intended to present to the reader, by example, a methodology to estimate the cooling air temperatures that can be achieved by thermoelectric augmentation. However, to illustrate the value of thermoelectric augmentation, we will first consider some equations to calculate the air temperature (Ti1) entering the portion of the box housing the electronic components without thermoelectric augmentation. Working from the air temperature (T01) outside the box to the temperature of the wall (Tw) of the box (or a base temperature common to the external and internal heat sinks) gives,

where q is the heat dissipation of the electronic components and R2 is the total thermal resistance of the external heat sink, neglecting the contribution of the rest of the wall. It should be noted that throughout this analysis and in the subsequent calculation examples, heat sink resistances, R1 and R2, are assumed to include both the convective part and any associated thermal interface resistance between the heat sinks and the wall or later in the article, the TE modules. The air temperature entering the internal heat sink passages is given by,

which, substituting in eq. (1) for Tw, gives

The temperature drop of the air passing through the internal heat sink is given by,

where C1 is the air heat capacity rate inside the box, given by the product of the mass flow rate, , and specific heat, cp, of the inside air cooling stream. So, the temperature, Ti1, of the cooling air entering the electronics compartment (without thermoelectric augmentation) becomes,

Now, we will proceed to determine the temperature of the air entering the electronics compartment when thermoelectric cooling augmentation is incorporated. To do this we must include equations to account for thermoeletric heat pumping and the heat dissipation of thermoelectric modules. In an earlier article, Luo [4] presented the following equation for heat pumping with a thermoelectric cooling module

and another equation for the corresponding TE heat dissipation

in terms of the TE parameters at the module as defined in the nomenclature and discussed by Luo [4]. It should be emphasized here that the temperatures, Th and Tc, in equations (6) and (7), as well as in equations (6a) and (7a) to follow, must be expressed in Kelvin temperature units (i.e. Kelvin temperature = Centigrade temperature + 273.16).

In addition to the above two thermoelectric equations we have the following heat transfer equations,

It should be noted here that under steady-state conditions heat load, q, in equations (8-10) is equal to the heat pumped, qp, by the thermoelectric cooler. Considering equations (8–10) together with thermo-electric equations (6) and (7) we have five linear algebraic equations. Given that we know or can assume values for q, R1, R2 , C1, Sm, Km, Rm and I, we are left with five unknown variables. The unknown variables are qte, Th, Tc, Ti2 and Ti1 (which is the variable we are really after). It is possible to solve these equations via algebraic substitution, as the author has done with the aid of software [5] capable of performing symbolic algebraic manipulation. However, the author found that the algebraic solution obtained for Ti1 by this method was so long (about two or three screen-widths) and so complex as to be of practically no value other than demonstrating that a solution could be obtained.

A method of practical utility is to solve these equations for numerical values using a matrix method as employed in an earlier article addressing the solution of a thermal resistance network [6]. To do this we first rearrange the equations (6-10) so that the unknown variables are on the left-hand side of the equation and the constant terms are on the right-hand side,

In matrix notation, these same equations can be compactly represented as

The coefficients of the unknown variables and the corresponding constant terms for each equation may be grouped in a tabular form as shown in Table 2. So doing, the top row represents the column vector of unknowns, the columns beneath each unknown variable make up the coefficient matrix and the rightmost column the constant vector.

The coefficient matrix below comprises the known coefficients of the unknown variables of the equations, as shown in Table 2,

the column vector of unknown variables which we seek to solve is,

and the column vector comprising the known constants on the right hand side of each equation is,

A matrix equation such as (11) may be solved for the unknown variables by multiplying the constant vector by the inverse of the coefficient matrix,

Many computational aids such as MathCad [5], Matlab [7], and EXCEL [8] can be used to obtain the matrix inverse of the coefficient matrix and multiply by the constant vector to obtain the desired solution vector.

To demonstrate the potential enhancement with thermoelectrically augmented cooling, we now turn our attention to some numerical results obtained using the method described above. For purposes of illustration, a small enclosure with an allowable opening in one side of 100 mm x 100 mm is assumed. It is further assumed that the base dimensions of the internal and external air-cooling heat sinks are 100 mm x 100 mm. Before proceeding further it is necessary to assign values to the thermoelectric module cooling parameters, Sm, Km,and Rm. The same 40 mm x 40 mm TE module considered by the author in an earlier article [9] will be considered here. In the earlier article, the author illustrated the calculation of single module TE parameters from vendor data, using the method discussed by Luo [4]. The values for the example TE module were found to be:

Sm = 0.068 V/K
Km = 0.712 W/K
Rm = 2.307 Ω

However, to increase the overall heat pumping capacity, four of these TE modules can be sandwiched in a 2 x 2 array between the bases of the internal an external heat sink. Consequently, the parameters for the array of TE modules will simply be four times the value for a single TE module or,

Sm = 0.272 V/K
Km = 2.848 W/K
Rm = 9.228 Ω

Also, if we assume that the TE modules are wired in series, the current, I, through each module will be the same and equal to the total current.

Calculations were performed for a range of heat sink resistances to show the effects of this parameter, coupled with the thermoelectric cooling effects. The heat sink thermal resistances used in the calculations were 0.075, 0.125 and 0.175 oC/W and were considered to have the same value for the heat sink within and outside the box (i.e. R1 = R2). The internal air flow rate within the box was assumed to be 20 CFM (0.00944 m3/s).

As noted earlier, equations (6) and (7) are in Kelvin temperature units. Therefore, the temperature of the outside cooling air, T01, used in the calculations, must also be in degrees Kelvin and the temperatures obtained from equation (12) will be in degrees Kelvin. However, for ease of understanding, the temperatures reported in the following figures have been converted to degrees Centigrade.

Figure 2 illustrates both the effect of increasing electric current through the TE modules and heat sink thermal resistance, with a heat load of 100 watts from the electronics and an outside air temperature of 35 oC. The solid lines represent the cooling air temperature, Ti1, within the box with the TE modules sandwiched between the heat sinks and the dashed lines represent the temperatures obtained without the TE modules. As can be seen, at low values of electric current, the presence of the TE modules result in higher cooling air temperatures within the box. This is because at low electric current, the Peltier heat pumping effect is offset by the thermal resistance across the TE modules. As current is increased the Peltier heat pumping effect becomes more significant and the inside cooling air temperature can be decreased significantly. However, as electric current continues to be increased, the Joule heating (i.e. heat dissipation) within the TE modules becomes increasingly significant causing the inside cooling air temperature to bottom out and then begin to rise if current is increased still further.

Figure 2: Effect of increasing electric current to TE modules on internal cooling air temperature.

Figure 3. Effect of electronic heat load on internal cooling air temperature with (solid lines) and without (dashed lines) thermoelectric augmentation.

Figure 3 illustrates the effect of the electronics heat dissipation on cooling air temperature within the box, with a constant current of 3.5 amperes through the array of TE modules and an outdoor temperature of 35 oC. The internal and external air flow rate was held constant at 0.0094 cms (20 CFM). As can be seen, for the heat load range considered for this analysis, the cooling air temperatures obtained in all the cases are lower than could be achieved without employing TE enhancement. It should also be noted that, depending on heat load and heat sink thermal resistance, in many cases the cooling air temperature is even lower than could be obtained using outside air for cooling.

The results in Figure 4, show the effect on cooling air temperature within the box, of supplying a fixed current of 3.5 amperes to the TE modules and for a fixed electronics heat load of 100 watts. It can be seen that in this case, the resulting cooling air temperatures are always below the cooling air temperatures that could be achieved without thermoelectric enhancement.

Figure 4: Effect of outside air temperature on internal cooling air temperature, Ti1, with (solid lines) and without (dashed lines) thermoelectric augmentation.

The equations and solution methodology presented in this article can provide a useful tool with which to obtain a preliminary estimate of the effectiveness of thermoelectric augmentation in providing cooling to electronic components in a closed box.

Of course, when considering the use of thermoelectrics for cooling applications, the designer should be cognizant of the electrical power required to drive the thermoelectric elements, which is given by equation (7). The cooling efficiency of thermoelectric devices relative to their power consumption, as with other types of heat pumping devices, may be characterized in terms of Coefficient of Performance (COP). COP, is defined as the ratio of cooling (q) provided over the electrical energy consumed (qte) to produce the cooling effect. For example, for the results shown in Figure 3, the COPs ranged from 0.3 to 0.4 at heat loads of 50 or 60 W to around 1 at a heat load of 150 W. The range of heat sink thermal resistance values had only a little effect on the COP realized. Similarly, considering the results shown in Figure 4, COPs ranged from 0.6 to 0.7 with little effect due to heat sink thermal resistance or outside air temperature. As one might expect from equation (7) the principal effect on COP is caused by the electrical current required to drive the thermoelectrics. This is amply demonstrated considering the results presented in Figure 2. At electric currents around 1.25 A, the COPs calculated ranged from 6 to 6.4 depending on the value of heat sink thermal resistance. Although these COPs may seem good, in this case the cooling air temperature within the box is no lower than could be achieved without thermoelectrics. As current is increased in Figure 2, the COP realized decreases rapidly to about 0.65 at a current of 3.5 A. However, even though the COP has dropped, this coincides with the maximum reduction of air temperature within the box by as much as 20 to 30 Centigrade degrees below those achieved without thermoelectric augmentation. For comparison, the COP values realized by conventional vapor compression refrigeration systems may typically vary from 1 to 4.

Finally, it should be noted that a number of vendors provide thermoelectric heat exchangers for cooling air within in a closed box. The interested reader may find a number of vendor products and examples by conducting an internet web search using the terms “thermoelectric air cooler” or “thermoelectric air conditioner.

References

1. Simons, R.E., “Estimating Temperatures in an Air-Cooled Closed Box Electronics Enclosure,” ElectronicsCooling, February 2005.
2. Godfrey, S., “An Introduction to Thermoelectric Coolers” ElectronicsCooling, September 1996.
3. Simons, R.E., “Application of Thermoelectric Coolers for Module Cooling Enhancement,” ElectronicsCooling, May 2000.
4. Luo, Z., “A Simple Method to Estimate the Physical Characteristics of a Thermoelectric Cooler from Vendor Datasheets,” Electronics Cooling, August 2008.
5. http://en.wikipedia.org/wiki/Mathcad .
6. Simons, R.E., “Using a Matrix Inverse Method to Solve a Thermal Resistance Network,” Electronics Cooling, May 2009.
7. http://www.mathworks.com/products/matlab/ .
8. http://www.ehow.com/how_5898417solve-equations-matrix-method-excel.html .
9. Simons, R.E., “Using Vendor Data to Estimate Thermoelectric Module Cooling Performance in an Application Environment,” ElectronicsCooling, July 2010.

“Builders of quiet workstations and servers still have trouble finding quality cooling solutions for Intel’s LGA2011 socket with Narrow ILM mounting,” Mag. Roland Mossig, CEO of Noctua, said, “so we’ve decided to update our DX line of coolers to work with both Square ILM and Narrow ILM-based LGA2011 platforms.”

The NH-U12DX i4’s slim 45mm fin depth ensures easy access to RAM slots, while “the 125mm height of the NH-U9DX i4 makes it fully compatible with standard 4U server cases.”

PWM support has been added to both the NF-F12 120mm fan of the NH-U12DX i4 series and the two NF-B9 92mm fans of the NH-U9DX i4 series for “fully automatic speed control.” The fans come “fitted with the professional SecuFirm2 mounting system and bundled with Noctua’s industrial-grade NT-H1 thermal compound.”

(Image: Oven Industries)

Oven Industries, a supplier of custom temperature controllers and sensors for a variety of industries, has released the new 5R7-001 PID bi-directional temperature controller for independent thermoelectric modules or for use in conjunction with auxiliary or supplemental resistive heaters for heating and cooling applications.

Featuring a “full H-bridge and 25 Amps of thermoelectric module current,” the new temperature controller is PC-programmable via an RS232 communication port and can be equipped with an optional display for “temperature set and response.”

“All parameter settings are retained in non-volatile memory,” the company said. “Once the desired set parameters are established, the PC may be disconnected and Model 5R7-388 becomes a unique, stand alone controller.”

(Image: JMC Products)

JMC Products, a manufacturer of DC cooling fans and thermal solutions, has released three fans for low-noise, low-airflow applications.

Featuring a compact design for limited-space installations, “the 30×10, 35×10, and 40×10 PWM excel in low-profile set top boxes, stand alone storage units, spot coolers” and feature “sealed sleeve bearings (SSB), which aid in preventing leaks and dust entry for low-profile applications.” According to the company, the new fans are available with IP rating up to 56.

All three fans are UL- and TUV-certified and RoHS-compliant.

(Image: EIC Solutions)

EIC Solutions Inc., a manufacturer of thermoelectric air conditioners, electronic enclosures and transit cases, has released a digital temperature control (DTC) for fine-tuned temperature control inside air-conditioned electronic enclosures.

“[While] our standard bimetallic thermostat meets the vast majority of customer needs, some installations require tighter control over the internal enclosure temperature,” Dave Bates, operations manager at EIC, said.

According to the company, “EIC’s digital temperature controls operate with a tolerance of +/- 1 degree Fahrenheit through the use of a proportional-integral-derivative (PID) control scheme,” which “increases accuracy in temperature regulation when compared to traditional thermostats.”

The DTCs are fully programmable, providing users with the ability to “control internal enclosure environments over very specific temperature ranges and time periods.” EIC’s DTCs are designed to be mounted on the front panel of air-conditioned electronic enclosures, but can also be mounted in a variety of other configurations as needed.

According to the company, the newly patented Vortex A/C enclosure coolers are large enclosure coolers with a cooling capacity of 5000 BTU/hour and a “unique” two-stage cooling system designed to conserve compressed air.

“During regular usage, the coolers utilize only the first cooling stage. However, when there is increased demand for cooling, whether due to high heat load in the enclosure, high ambient manufacturing facility temperatures or hot weather conditions, the second cooling stage is activated,” ITW Vortec said.

“I cannot be more pleased that the years of research and innovation by the Vortec team have been rewarded with this patent. The Patent Office has recognized the novel way that Vortec’s air technology enhances working conditions and safety in manufacturing environments where cooling is of the utmost concern,” Barbara Stefl, general manager of ITW Air Management, a division of Illinois Tool Works and Vortec parent company, said.

“The acquisition of the Thermal Technology business adds a number of innovative and important products and technologies to our rapidly diversifying portfolio that will, we believe, allow us to accelerate our entrance into new markets,” Tom Gutierrez, president and CEO of GT Advanced Technology, said.

“This is an exciting moment for the company, our employees and our customers,” Matt Mede, president and CEO of Thermal Technology, said.  “The acquisition will open new opportunities for growth for our products and technology as we integrate them into GT’s business operations.”

The purpose of the present article is to suggest that accurate solution of thermal spreading problems for multilayer, edge-truncated geometry is easily accomplished using free, open-source finite element software. This should be especially attractive to designers and analysts who are full-time independent consultants, temporary contractors, or regularly employed engineers desiring to free themselves from the problems of justifying the cost of infrequently used software. This writer has studied some of the available open-source software and believes that many readers will profit by the information in the following paragraphs.

Figure 1: Spreader, submount, and source geometry. Planar source is embedded at top surface of submount. One quadrant of square device analyzed.

Background

Thermal spreading resistance is usually defined as the temperature difference per unit heat transfer, e.g. K/W, between a source and a defined isothermal plane, point, or ambient temperature. Math models for steady-state thermal spreading resistance have been under development for several decades. Early models considered conduction to an isothermal reference plane and were therefore of limited use for analyzing electronic packages [1]. Lee described a transistor-on-heat sink application with a convecting and/or radiating surface [2,3]. In this case, a uniform total heat transfer coefficient h was assumed for an ambient temperature T_A. The simplest of these models presumes a circular source centered on the surface of a circular substrate, a model that is accurate for many applications. Ellison published design curves and formulae for the maximum resistance (source center to ambient) for rectangular sources centered on rectangular substrates, either or both with non-unity aspect ratios [4]. Design curves were later added for source-averaged thermal resistances [5]. Time-dependence was added to the rectangular device problem by Rhee and Bhatt [6] who implemented a three-dimensional Green’s function solution based on a catalog of solutions by Cole et al. [7].

Both the steady-state and time-dependent models for three-dimensional conduction problems result in infinite series solutions: a single series for the Lee and Rhee solutions and a double series for the Ellison solutions. While the series formulae are not particularly difficult to implement in a math-scratchpad computer program, there is always the desire to use approximate closed-form formulae, i.e. non-infinite series based. Lee was successful in creating these approximations for his circular-shaped devices [2]. Continuing in the effort to obtain useful modeling results with approximate formulae, Lasance has proposed a method for calculating the spreading resistance for two-layer problems where the layers have unequal dimensions except for the thicknesses [8,9,10]. In these situations where the planar dimensions of the layers are not equal, there are no established closed-form analytical methods. Consequently, the engineer must often rely on more advanced numerical modeling using finite difference (FDM) or finite element (FEM) methods. Yovanovich and co-workers have also published numerous studies on spreading and closely related topics [11].

Sample Spreading Problem Geometry

Lasance considered the test case of a two-layer spreading problem with a truncated-silicon top layer submount (planar dimensions less than bottom layer), on a copper-tungsten heat spreader as shown in Figure 1 [9]. The source plane is at the top of the submount and all surfaces are adiabatic except the source region and the Z=0 plane, the latter location modeled by a uniform heat transfer coefficient h. Lasance refers to this h as effective as it may include the effect of an area-enlarging factor due to a heat sink, a common practice in modeling electronic equipment [5]. In the present problem, h = 250 W/(m²·K), the same value Lasance used for some of his work. An ambient T_A = 0 and a one watt total source dissipation mean that the maximum computed temperature is numerically identical to the thermal resistance. The square source is centered on a square, two-layer substrate, but symmetry allows the analysis of only one quadrant, thus indicating a corner source. Further advantage of symmetry could be used with the FEA model by halving this quadrant with a diagonal between the source and opposite corners. Table 1 lists the variable parameters used. A source of 1.0 W is uniformly distributed over the entire source area (8.0×10^-4 m)² as a heat flux of 1.5625×10⁶ W/m².

Case 1 is a benchmark comparison of analytical and FEA solutions and is used to justify the FEA model mesh for Cases 2-5. The top layer is not truncated for Case 1 and has the same conductivity as the bottom layer, i.e. this is a single layer material. Cases 2-5 are those in which we are primarily interested.

Simple Spreading Calculation for Case 1

The benchmark calculation for Case 1 uses rectangular device spreading theory from Ellison to obtain the dimensionless source center and source-averaged spreading resistances ψ_Sp = 0.59 and  ψ_Ave-Sp = 0.50, respectively [5]. Theory shows that the maximum total thermal resistance from the source center to ambient is the sum of terms for one-dimensional conduction, uniform h convection, and spreading [5]:

Similarly, using ψ_Ave-Sp, the total source-averaged resistance is

 R_Ave = 7.58 K/W

The source center resistance, R, will be compared with an FEA calculation in the Results section.

Software Used and the Spreading Model Described

Two free of cost, open-source programs were used: (1) Salome [12] and (2) Elmer [13]. In the Windows version, Salome is used primarily for model building and meshing [12]. The Salome software has a rather complete set of plate and solid model features, but is not a complete CAD system. Some users might find themselves frustrated by the geometry module, but it is quite adequate for many applications. Perhaps the most inconvenient aspect is that once added, an object’s dimensions cannot be changed. Thus if you have an incorrectly sized object, you must delete it and try again. This is not much of a problem and if you prefer, an open-source CAD program is also available [14].

Figure 2: Case 3 mesh for source, truncated submount, and a portion of the spreader. Source plane shown in red.

In the current spreading problem, Salome was used to build, mesh, and export the models. A look at Figure 2 for Case 3 shows that each model is constructed from three-dimensional blocks: a submount block, a spreader block, and a source block, the latter superimposed at a corner. The submount planar dimensions all differ for Cases 2-5. However, Cases 3-5 have truncated submount dimensions, i.e. the submount does not fully extend out to the spreader edges – a problem that is not managed by any of the analytical solutions of Ellison and Lee.

The source block has a total thickness that is the sum of the submount and spreader thicknesses. The top surface of the source block is the region to which the heat flux is applied (colored red in Figure 2). If the source, submount, and spreader blocks were left as-is in Salome, the geometry would consist of a single material. The “Geometry-Partition” operation must be used to separate the system into the three distinct objects. A later operation, “Geometry-Group” or “Mesh-Group,” is used to combine the lower portion of the source block with the spreader, and also to combine the upper portion of the source block with the submount, resulting in two distinct material groups. Two extruded circles are used to permit the creation of submesh regions with finer detail than that for the overall system. One of these cylinders is visible in Figure 3. Note that the mesh in Figure 2 shows the finest mesh in the source region, a not quite so fine mesh throughout the submount dimensions, and an even coarser mesh for the remainder of the spreader.

Figure 3: Case 3 FEA results.

An “Explode” option is used to identify the various solid blocks and faces that can be grouped into the two different materials as well as the source and convection faces. The geometry and meshing efforts are completed by exporting the mesh to some suitable file structure, a good choice being the universal UNV type. The next step is to convert this file to a mesh file that is recognizable by the solving program, Elmer [13]. The file converter is a command-type program known as “Elmergrid.” This program not only creates the correct input file for Elmer, but can also be used to clean up the UNV file.

The mesh file is loaded into Elmer, which has a very complete set of options for input, in this case heat conductivities, heat flux, and the heat transfer coefficient. The project is saved to the hard drive and a simple click of a solver icon begins the solution process. Once a successful solution is obtained, one can use either of two different Elmer options to display results. For example, the “run-vtk” post-processor was used to produce Figure 3.

Spreading Results

The theoretical and FEA results are listed in Table 2. The Case 1 benchmark, Theoretical and FEA results, have a discrepancy of 0.5%. The Case 1 theoretical and FEA discrepancy for the third term, R_Sp = ψ_Sp/kW_S, of the total resistance is 1.2%. These discrepancies are sufficiently small to justify the mesh. Note that for Cases 2-5 the resistance increases only slightly as the heat spreader planar dimensions decrease. A portion of the thermal surface contours in the vicinity of the source is shown in Figure 3, for which the legend is left with non-integral values so that the maximum temperature, i.e. the resistance, is displayed. In this case the submount is about six times the size of the source dimension and it is very clear that the size of the submount can be substantially decreased without significantly increasing the thermal resistance. The smallest submount configuration evaluated has dimensions of only twice those of the source and yet the thermal resistance is hardly different than that for Case 2, which has a non-truncated submount. These observations are entirely consistent with the surface temperature colors in Figure 3. The analysis also demonstrates the advantage that graphical results have over the single-value “theoretical” calculation used for the benchmark.

With regard to studies by Lasance in an evaluation of the application of single layer spreading formulae to multilayer problems, we can see in Figure 3 that use of the submount dimensions considered herein as a source for the second layer would be a poor modeling choice because such source dimensions would be too large.

No attempt has been made to address model approximations, but one of the omissions in this study is the absence of convection cooling from the submount and spreader planes on the source side of the device. This feature would be easy to add to the FEA model.

More Complex Conduction Problem – Single Chip Package on PCB Coupon

The thermal spreading problem considered in the preceding paragraphs has very simple geometry. As an example of a more complex problem, a single chip package attached to a circuit board coupon is shown with results in Figure 4. The geometry was constructed using the “Free-cad” open-source software [14]. While the author of this software stipulates that the program is under continuous development, it was more than sufficient for this problem and makes up for the limited model construction capabilities of Salome. In particular, it is helpful that Free-cad object dimensions are easily modified. The Free-cad STEP file export feature was used to create a file for import into Salome. From that point on, geometry meshing and model solving is similar to procedures used in the spreading example. Note that the substrate geometry was submeshed to obtain a finer mesh. The heat dissipating chip is not visible because it is mounted on the underside of the substrate.

Figure 4: Temperature rise above ambient for a single chip package on PCB. IC chip hidden at base of inverted substrate. Circuit board dimensions: 50.08 mm x 50.08 mm x 1.52 mm with orthotropic k. Ceramic substrate dimensions: 25.4 mm x 25.4 mm x 1.27 mm. Chip dimensions: 5.08 mm x 5.08 mm x 0.51 mm. All surfaces use h = 15.5 W/m2 ·K. Chip dissipation is 3.0 W or 116,250 W/m2.

The legend for the IC package results in Figure 4 is left with non-integral values so that the maximum temperature (at chip source center) is displayed. This same problem was constructed and analyzed with commercial CAD and FEA software, for which the model and results are shown in Chapter 1 of [5]. The two different FEA programs were used to obtain a maximum temperature rise above ambient with about one percent discrepancy, a value that is probably due to mesh differences.

Summary and Comments

The FEA method permits the analysis of geometry that is too complex for analytical spreading formulae, most of which are infinite series solutions that cannot accurately account for multilayer structures with one or more edge-truncated layers. The chip package on PCB in Figure 4 is a moderately complex example solvable using most FEA codes. The reader inexperienced in FEA is cautioned to check for grid independence of the numerical solution, particularly when there are large scale differences in the geometry.

Details of using the open-source software have been largely omitted in this article because of space limitations. However, it is hoped that the reader will be pleased to learn of the programs introduced. The web can be searched for tutorials, though some of these may not be in the field of interest. Nevertheless, there may be sections of any tutorial that are of general use for modeling and meshing. In particular, there are many Elmer tutorials provided as part of the download package. The number of Salome tutorials is more limited. Finally, don’t let the name “Free-cad” lead you to think that it is not a serious program. Though it may not yet meet the needs of someone requiring a commercial grade tool, it is still very useful. The Appendices will be of interest if you choose to evaluate the software for yourself.

Appendix i: Salome Options and Suggestions

1. In the Salome menu bar, you will need to select Geometry or Mesh from the drop down Salome list, depending on what you want to do.

2. When you create your first geometric entity or load a CAD file, you need to right click on Geometry and select “Show” in the Object Browser to visualize the problem. You will also have to click on a “magnifying glass” icon to fill the display window. A similar procedure applies to viewing a mesh.

3. Geometry – Operations – Partition: you need this operation to separate the various geometric blocks into distinct parts, which otherwise will be a single material.

4. Geometry – New Entity – Explode: use this to “explode” the various geometric entities into solid parts and faces, the latter being necessary to apply boundary conditions.

5. Mesh – Create Mesh – Algorithm (use Netgen 1D-2D-3D): in the Object Browser, select your partition (Partition_1 by default) on which to create the mesh.

6. Compute: in the Object Browser, right click Mesh_1 (the first default name) and select “Compute.”

7. While in the Mesh option, create mesh groups to isolate individual materials and boundary faces.

8. Note that in Figure 4 the substrate has a different mesh density than the PCB. This was constructed by first creating the overall mesh, selecting Mesh_1, then selecting Mesh-Create Submesh option (with a greater mesh density), and right clicking Mesh_1 and selecting Compute. The submesh parameters can be edited and the problem remeshed without beginning again.

9. Export to UNV file: do this by right clicking Mesh_1 in the Object Browser and making the selection.

Appendix ii: Elmer Options and Suggestions

1. After you install Elmer, you may need to drag an ElmerGUI.exe to create a desktop shortcut icon.

2. Add a path to the Elmer executable, Elmergrid: Using Windows Explorer, select Computer, Properties, Advanced System Settings, Advanced, Environment Variables, Path, Edit, and then add the path to the Elmer installation directory. This will let you run “elmergrid” from any Command Prompt in any directory.

3. Create a desktop Command Prompt for convenience.

4. Use the Command Prompt to change to your model file drive and directory, then run Elmergrid without any arguments to get a list of options. A suggestion is to use “elmergrid 8 2 mesh_1.unv -autoclean,” where mesh_1.unv is the model mesh file.

5. In your first use of a mesh file, use Elmer – File – Load Mesh: select the folder “mesh_1″ that was automatically created when you used Elmergrid. Don’t be confused by looking for a file. Elmer requires that you select a folder, not a file.

6. When you input model details for a heat problem, don’t forget to select Model – Equation – Add Heat Equation, and check the Active option.

7. After you have successfully solved a problem, if you use the Elmer “Post” icon, you may need to edit the background color to see all of the legend numbers.

Appendix iii: Other Suggestions

1. Free-cad, Salome, and Elmer are available as Windows binaries. Issues with regard to Windows 8 are presently unknown to the author.

2. The Ubuntu Linux distribution is easily downloaded as an ISO file, which in turn can be used to create a Linux boot-able system on a USB flash drive or DVD [15]. This distribution contains a large number of applications in addition to Free-cad, Salome, and Elmer. There is a Salome-Meca program that includes heat and elasticity modules, but the lack of complete manuals and the use of the French language in some parts of the software present obstacles for non-French readers.

3. The Linux CAE distribution is an alternate to MS Windows.

References^*

[1] D.P. Kennedy, “Spreading resistance in cylindrical semiconductor devices,” J. Applied Physics, vol. 31, pp. 1490-1497.

[2] S. Lee, “Optimum design and selection of heat sinks,” in Proc. 11^th Semiconductor Thermal Management and Measurement Symposium, San Jose, CA, pp. 48-54.

[3] S. Lee, “Calculating spreading resistance in heat sinks,” ElectronicsCooling, vol. 4, no. 1, Jan. 1998, pp. 30-33.

[4] G. Ellison, “Maximum thermal spreading resistance for rectangular sources and plates with non-unity aspect ratios,” IEEE Trans. Comp. and Pkg.Tech., vol. 26, June 2003, pp. 439-454.

[5] Gordon N. Ellison, Thermal Computations for Electronics, CRC Press, Nov. 2011.

[6] J. Rhee and A.D. Bhatt, “Spatial and temporal resolution of conjugate conduction-convection thermal resistance,” IEEE Trans. Comp. and Pkg. Tech., vol. 30, Dec. 2007, pp. 673-682.

[7] K. Cole, J. Beck, A. Haji-Sheikh, and B. Litkouhi, Heat Conduction Using Green’s Functions, Taylor & Francis, 2^nd Edition, July 2010.

[8] C. Lasance, “Heat spreading: not a trivial problem,” ElectroncsCooling, vol. 14, no. 2, May 2008, pp. 24-30.

[9] C. Lasance, “How to estimate heat spreading effects in practice,” ASME Journal of Electronic Packaging, vol. 132, Sept. 2010.

[10] C. Lasance, “Two-layer heat spreading approximations revisited,” in Proc. 28^th Semiconductor Thermal Management and Measurement Symposium, San Jose, CA, pp. 269-274.

[11] www.mhtl.uwaterloo.ca/MHTLarchive.html.

[12] www.salome-platform.org.

[13] www.csc.fi/english/pages/elmer.

[14] www.sourceforge.net/apps/mediawiki/free-cad.

[15] www.caelinux.com.

^*This list of references is not all-inclusive as there are other publications to be found on the subject.

Bibliography

Gordon Ellison is a retired Tektronix, Inc. Fellow. Following his retirement from Tektronix, he has taught Electronics Cooling at Portland State University and consulted with local companies. His publications include the books Thermal Computations for Electronic Equipment, Van Nostrand Reinhold Publishing Co., 1984, reprint edition by Krieger Publishing Co., 1989, and more recently, Thermal Computations for Electronics, CRC Press, 2011. He is a past winner of the IEEE Semi-Therm Significant Contributor Award. He received a B.A. degree in physics from the University of California at Los Angeles, and an M.A. in physics from the University of Southern California.

A recent solution to this “energy crisis” adopted by thermal designers of data centers is the confinement of the air cooled servers inside of racks with air-to-water cooling coils for heat removal, as an attempt to maximize the cooling performance and to reduce the overall thermal resistance between the chip and the external environment. Another solution is relying on the use of outside cold air and/or water for cooling (i.e., free cooling [2]), which is highly dependent of external environment conditions, and requires additional components, such as filters, ducts and fans, dampers, etc. This solution requires high levels of specialized controllers, continuous maintenance and is susceptible to errors [3].

A long-term solution is to upgrade to on-chip two-phase cooling [4], which besides providing very high cooling performance at the chip level without requiring a heat spreader with a large footprint, also eliminates the poorly performing air as a coolant all together [5, 6] and adds the capability to reuse the waste heat in a convenient manner, since higher evaporating and condensing temperatures of the two-phase cooling system (evaporating its dielectric refrigerant at the chip at temperatures up to 60°C whilst still maintaining the chip comfortably below 85°C) are possible with such a new green cooling technology.

Single-phase (water) on-chip cooling technologies have been implemented in new supercomputers, showing reductions in power consumption up to 45% when compared with air cooling technologies [7]. On-chip cooling has also yielded a significant increase in computing performance in terms of computing throughput (lower chip temperature, lower gate current leakage, lower voltage and higher frequency) and computing throughput per electrical energy use. Thus, the appeal here is to improve even more the computing performance using two-phase on-chip cooling, which due to the latent heat of the coolant, removes much higher heat fluxes while requiring smaller coolant flow rates than in the single-phase cooling [8]. Better temperature uniformity across the chips is also achievable.

In the present work (condensed version of the paper presented in [4]), two such two-phase cooling cycles using micro-evaporation technology were experimentally evaluated with specific attention being paid to energy consumption, overall energetic efficiency and more specifically controllability. The cooling cycles were comprised of a tube-in-tube counter flow condenser (heat rejection), two parallel micro-evaporator (ME)/pseudo chip packages (mimicking the cooling of the chips of blade servers) and a stepper motor valve (SMV) at the inlet of the MEs for flow control reasons. The two alternative drivers tested were a mini-vapor compressor (VC cycle) and a gear pump (LP cycle). Additionally, two internal heat exchangers were considered in the VC cycle to guarantee subcooling at the inlet of the MEs and superheating at the inlet of the minicompressor.

Figure 1: Hybrid cooling system test bench.

Figure 1 shows the multi-purpose test bench constructed to experimentally evaluate the performance of these cooling systems under various typical blade server operating conditions of transient, steady-state, balanced and unbalanced heat loads on the system’s two pseudo CPU’s. More details about the different cooling loops can be found in [9]. Since limited experience is available on two-phase cooling flow control for servers, this was the major objective to demonstrate here, implementing simple controllers.

FLOW CONTROL
The operational goal here is to maintain the chip temperature below a pre-established level by controlling the inlet conditions of the micro-evaporator cold plate (pressure, inlet subcooling and mass flow rate). Futhermore, it is imperative to keep the ME’s outlet vapor quality below that of the critical vapor quality, which is associated with the critical heat flux (premature dry out in the channels). Notably, the coolant flow rate is modulated to control the exit vapor quality to a target value and thus match the heat load fluctuation during the chips’ operation. Hence, greatly reduced energy consumption of the driver during normal operation and providing low energy consumption during standby operation and also off capability when the server is not in operation for even greater energy savings.

The condensing pressure must also be controlled since it sets the saturation temperature of the coolant. If the aim is to recover the energy dissipated by the coolant in the condenser to heat buildings, residences, district heating, pre-heat boiler feedwater, etc. (here represented by a thermal bus), this can be achieved using either a compressor (VC cycle) to reject the waste heat at a higher temperature (at the cost of higher energy consumption) or using a pump (LP cycle) to reject the heat at the ME’s exit saturation temperature, both without requiring any refrigeration chiller. Otherwise, using a pump as the driver, the ME’s exit saturation temperature can be modulated to follow the outside air temperature for heat dissipation into the ambient air via a compact air-cooled heat exchanger (viz. Figure 2).

Figure 2: Condensing temperature control considering (or not) waste heat recovery.

For the experimental evaluation, specific controllers were first designed and tested [9]. The variables controlled here were the ME’s outlet vapor quality, the condensing pressure (LP cycle) and the approach temperature in the condenser (VC cycle). The actuators used were a variable stroke length oil-free vapor compressor, a variable speed condenser water pump and an electronically controlled stepper motor valve (over-dimensioned to modulate the refrigerant mass flow with a negligible pressure drop).

Two ME’s in parallel (typical for blade server boards) assembled on two pseudo chips to emulate actual ones, each composed of 35 heaters and temperature sensors (2.5 mm by 2.5 mm in size made from a Delphi thermal test die), were used. The ME’s copper microchannel geometry consisted of 53 parallel channels having a height of 1.7 mm and a width of 0.17 mm, with the fins between channels being 0.17 mm thick. The effective “footprint” area of the ME’s is 12 mm length from inlet to outlet and 18 mm width. In the present work only uniform heat fluxes were considered and HFC134a (a common refrigerant that is a dielectric fluid) was tested as the working fluid.

Finally, for the present experimental campaign, only one SMV was considered for modulating the flow to both MEs. The outlet vapor quality used for control was that at the exit after both flows from the ME’s are mixed. The condenser used water as the secondary fluid, where the driver was a controllable speed gear pump.

EXPERIMENTAL RESULTS
Experiments for set point tracking (for each controller developed), disturbance rejection and non-uniform heat load (last two considering the developed controllers integrated / dual SISO, SISO and SIMO strategies) were developed and a short description is presented below. More details regarding the development of the controllers can be found in [9]. The results presented are for the LP cycle, but the authors highlighted that similar results were obtained with the VC cycle.

Figure 3: Different heat loads on the MEs.

Figure 4: Average temperatures on the pseudo chips.

A. Flow distribution for unbalanced heat loads
The experimental results showed that for different heat loads applied on the parallel ME’s an unbalanced flow exists, which generated a higher temperature on the pseudo chip with higher heat load. Temperatures of 75 °C against 60 °C were obtained when the difference in heat load was 60 W (90 W on ME1 against 30 W on ME2, respectively, emulating the maximum and idle clock speeds of real microprocessors). Despite this, it is important to mention that the temperatures obtained were lower than the typical CPU operating limit of 85 °C and that the difference of temperatures was reduced when the set point of the outlet vapor quality was reduced from 22% to 15% (viz. Figures 3 and 4). As can be seen, a total of eight different combinations of heat loads and three outlet vapor qualities were evaluated.

Regarding the controllability, the cooling systems were found to be fast and effective, controlling the condensing pressure or the secondary fluid temperature (more details in [9]) and the outlet vapor quality at the defined set points under steady state and transient conditions of heat load, and hence indirectly the chip temperature.

Figure 5: Heat load disturbance and pseudo chip temperatures.

Figure 6: Outlet vapor quality and SMV controller.

B. Heat load disturbance rejection tests
The heat loads on ME1 and ME2 were varied between 90 W and 75 W and 75 W and 60 W, respectively, with a periodic disturbance time of 1.4 s (emulating a fast and periodic change in the pseudo-microprocessors’ clock speed). Figure 5 shows the input power disturbance on the pseudo chips and the effect on the average temperature of each chip. The maximum temperature variation is only 1.5 °C, which is acceptable when compared to the temperature gradient along the chip for on-chip single-phase cooling using water (about 2-3 K for a uniform heat flux and without heat load disturbance [10]).

Figure 6 shows the controller’s reaction under the situation of a disturbance. It can be seen that the SMV controller was able to maintain the exit vapor quality to within ±5% of the set point. What is important to observe is that the controller was effective, i.e. it showed fast response for the induced disturbance and no instability was observed.
Finally, it can be highlighted that the control strategies adopted (SISO, dual SISO and SIMO) were simple but still effective for controlling the specific variables while maintaining the pseudo chips within a safe operating range. In fact, this is done without a temperature signal from the chip, which is very convenient because of the limited bandwidth available on actual CPU’s.

C. Energy comparison
To compare the performance of the liquid pumping and vapor compression cooling systems, which were experimentally evaluated and analyzed beforehand, a steady state condition was selected from the flow distribution tests.
Table 1 shows the results for the power consumption of the drivers, the two systems’ input and output energies associated with components and piping, and the thermodynamic conditions in the condenser for the main and secondary working fluids. The experimental condition selected for the comparison was that the input powers on pseudo chips 1 and 2 were 90 W (41.7 Wcm-2) and 75 W (34.7 Wcm-2), respectively.

Table 1: Energetic analysis for the VC and LP cooling systems and thermodynamic conditions in the condenser.

The results show a higher driver input power for the VC system, about 6 times, which naturally is associated with the energy expended to lift the pressure from the ME’s to the condenser. If one compares the results with a hypothetical air cooling system, considering a COP of 1.22 (45% of the total energy consumption for air cooling system [11, 12] ), the energy consumption would be 134.8 W versus 17.4 W when compared with the LP cycle and 237.8 W versus 102.1 W when compared with the VC cycle. This represents a reduction of 87% and 57% in energy consumption, respectively. The differences in air cooling system energy consumption are due to the input power on the post heater (which emulates the heat load of auxiliary electronics of servers, i.e. memories, DC/DC converters, etc.), which was only considered for the VC cycle (viz. Table 1). A pump or compressor optimized for this application would consume much less than 17.4 W and 102.1 W, probably less than one-half.
It can also be seen that 50.6% and 62.5% of the energy out of the VC and LP systems, respectively, are associated with heat losses. It shows that improvements can be done to improve the overall performance of the system, which would mainly be associated with the reduction of the driver and piping losses and, consequently, to increase the energy recovered in the condenser. The test bench here is a “plug-and-play” unit designed for versatile testing of components and flow control, not an optimized compact system.
The results showed a much higher temperature for the secondary fluid at the outlet of the condenser when using the VC system, which is related to the higher condensing temperature. This implies that a higher economic value is obtained for the waste heat available in the condenser. In Europe in particular, many cities have district heat lines (even the small city of Lausanne) and they are potential consumers for the waste heat.

CONCLUSIONS
The present study has demonstrated that simple control schemes are sufficient for management of two-phase on-chip cooling systems for servers, that the cooling is very effective and rapidly responds to step changes in heat dissipation rates, and that this technology provides a low energy consumption relative to air-cooling.

BIBLIOGRAPHY

Dr. Jackson Braz Marcinichen is a research post doc at the
Laboratory of Heat and Mass Transfer at the EPFL (Lausanne-Switzerland) and
has more than 20 years experience in HVAC & R systems. He received his BE in Mechanical Engineering from the Federal University of Santa Catarina, Brazil in 1996, and his Ph.D. in mechanical engineering from the same university in 2006. He has authored more than 30 scientific and technical papers in indexed journals and international peer-reviewed conferences, book chapters and US patents. He has designed and evaluated several experimental facilities characterizing the thermo-hydrodynamic and control of cooling systems (calorimeters, wind tunnel, hybrid systems etc). Today he is engaged in the development of new novel hybrid cooling systems (passive and active) to cool high heat flux electronics components using on-chip cooling.

Prof. John Richard Thome has been working at the Swiss Federal Institute of Technology Lausanne (EPFL) since 1998, where he is a director of the Laboratory of Heat and Mass Transfer (LTCM) and the director of Doctoral Program in Energy (EDEY). He received his Ph.D. in mechanical engineering at Oxford University in 1978 and worked as an assistant/associate professor in the US for five years at Michigan State University. He worked full-time as a consulting engineer for 15 years from 1984 through 1998 with his own firm. He has more than 170 journal papers and four books since joining the EPFL. His current main areas of research are two-phase flow and heat transfer in microchannels, two-phase flow control for electronics cooling using new hybrid cooling cycles; using either speed control of oil-free pumps and compressors or passive systems such as thermosyphons, and energy recovery systems.

ACKNOWLEDGEMENTS
Wolverine Tube Inc. (Huntsville, AL) provided MicroCool cold plates to our specification while Embraco (Joinville, Brazil) provided the linear oil-free mini-compressor.

REFERENCES
1. Marcinichen, J.B., Olivier, J.A., Lamaison, N., and Thome, J.R., Advances in Electronics Cooling. International Journal of Heat Transfer Engineering, 2013. Vol. 34(5-6): pp. 434-446.
2. Pawlish, M. and Varde, A.S. Free Cooling: A Paradigm Shift in Data Centers. in 5th International Conference on Information and Automation for Sustainability (ICIAFs). 2010.
3. Mulay, V., Humidity Excursions in Facebook Prineville Data Center, in Electronics Cooling. December 2012.
4. Marcinichen, J.B. and Thome, J.R. Two-Phase Flow Control of On-Chip Two-Phase Cooling Systems of Servers. in The 29th Annual Thermal Measurement, Modeling & Management Symposium SEMI-THERM 29. 2012. San Jose, CA, USA.
5. Samadiani, E., Joshi, S., and Mistree, F., The Thermal Design of a Next Generation Data Center: A Conceptual Exposition. Journal of Electronic Packing, 2008. Vol. 130: pp. 041104-1 – 041104-8.
6. Patel, C.D. A Vision of Energy Aware Computing from Chips to Data Centers. in The International Symposium on Micro-Mechanical Engineering – ISMME2003-K15. December 1-3, 2003. Tsuchiura and Tsukuba, Japan.
7. Campbell, L. and Ellsworth Jr, M.J., Back to the Future with a Liquid Cooled Supercomputer, in Electronics Cooling. August 2009.
8. Marcinichen, J.B., Olivier, J.A., and Thome, J.R., Reasons to Use Two-phase Refrigerant Cooling, in Electronics Cooling. March 2011. p. 22-27.
9. Marcinichen, J.B., Olivier, J.A., Oliveira, V., and Thome, J.R., A Review of On-Chip Micro-Evaporation: Experimental Evaluation of Liquid Pumping and Vapor Compression Driven Cooling Systems and Control. International Journal of Applied Energy, 2011. Vol. 92: pp. 147-161.
10. Brunschwiler, T., Meijer, G.I., Paredes, S., Escher, W., and Michel, B. Direct Wast Heat Utilization from Liquid-Cooled Supercomputers. in 14th Int. Heat Transfer Conference. 2010. Aug. 8-13, Washington, DC, USA.
11. Joshi, Y. and Kumar, P., eds. Energy Efficient Thermal Management of Data Centers. 1st ed. 2012, Springer New York Dordrecht Heidelberg London.
12. Rasmussen, N., Electrical Efficiency Measurement for Data Centers – WP 154 revision 2. 2010, American Power Conversion by Scheider Electric. p. 1-19.

The need to understand the system impact became evident to me when I was trying to use some thin thermoelectric coolers in a design to reduce local temperature gradients on some relatively low powered devices. While the addition of a thermoelectric cooler showed a thermal improvement compared to the original design, we had to allocate volume and interconnects for supplying a separate voltage for the coolers. A more fair comparison was to trade the thermal performance that would exist if this volume was used for thermal management with other approaches such as a high conductivity heat spreader. When examined at this level, the benefits of adding the thermoelectric cooler were minimal.

Figure 1: Microchannel cooling for an IC [1].

Figure 2: Chip level spray cooling [2].

Figure 3: Some other components needed for spray cooling.

A second example is shown in Figure 2 which illustrates a representative spray cooling concept from an overview article on chip cooling techniques beyond air. The system implications of implementing two-phase cooling can be significant. Figure 3 is a simplified diagram of a closed loop cooling system where the cooled electronics are represented by the box on the right side. While some of the other items are obvious, a complete implementation trade must consider filtration, maintenance, the size of the condenser, potential need to restrict orientation, etc. Specifically for two-phase with multiple heat sources, the system trade must also include the complexity of manifolding and controlling the flow to the different heat sources. The answer may still be that the more advanced cooling technique, such as spray cooling, is the right answer, but it is important that the full trade space be examined.

Sometimes the focus on just the chip cooling technology is driven by academic interests that fulfill the need to fully understand the process. Other times this focus can be marketing driven where the benefits of the advanced cooling technique are highlighted but potential complexities of implementing the technologies are minimized because they detract from the intended message. My reminder to the readers is to consider the complete electronics cooling system and the necessary other components that go along with a particular cooling technique.

REFERENCES

1. Colgan,E., et al, “A Practical Implementation of Silicon Microchannel Coolers”, Electronics Cooling, Nov 2007

2. Simons and Ellsworth, “High Powered Chip Cooling — Air and Beyond”, Electronics Cooling, Aug 2005

The computer industry is exploring various options for reducing the cost and energy consumption associated with cooling data centers. The use of liquid cooling in the data center has long been exploited as a more energy efficient means of heat removal than forced air cooling.  However,  more recently, there has been considerable interest in cooling data centers by means of circulating outside air through the data center to cool the computer hardware directly and then exhausting it to the outdoors.  This achieves energy efficiency by eliminating active refrigeration, for at least part of the year.  This cooling method is commonly called “free air cooling.”

Recent articles in this publication reflect the increased attention being directed at free air cooling.  In the December, 2012, issue, an article described the design, energy efficiency, and bring-up challenges of a predominately free-air-cooled data center in the Pacific Northwest region of the US [1].

In that same issue, a Technical Brief article provided an update of the activities of the ASHRAE Technical Committee 9.9, devoted to datacenter cooling technologies and the development of best practices in that regard [2].  The article described the evolution of the maximum allowable data center temperature from 25˚C in 2004 to the creation of additional environmental classes in 2011 that allow temperatures up to of 40˚C and 45˚C, at maximum values of relative humidity (RH) of 40% and 32%, respectively.  However, early experience in the deployment of free air cooled data centers indicates that, under more extreme weather conditions, the RH can reach levels exceeding the dew point [1, 3].  One expects that, over time, the technology will mature to the point that these episodes will occur less frequently.  However, one would anticipate that it will always be more difficult to control RH using free air cooling than with the more traditional, and energy intensive, vapor-compression cooled air conditioners.  Nonetheless, the expectation of this ASHRAE committee is that, over time, data centers will be migrating to higher temperature and relative humidity conditions than is now customary.

For reasons of economy and ease in manufacture, many of the components currently used in electronics systems employ organic materials.   As is well known, organic materials are, in general, permeable to moisture and will, over time, absorb moisture until a concentration level is reached that is in equilibrium with the that of the ambient air.  The precise value of the moisture concentration will depend on the ambient air temperature and RH and the temperature of the component in question and its material composition.

It has long been known that excessive levels of moisture in organic materials used in electronics can lead to reliability problems.  This phenomenon has been studied in particular for materials that are suddenly heated to a high temperature such as during the solder reflow process.   So-called “popcorn” cracking is a dramatic and typical failure mode in this situation.  More subtle moisture-induced failures can occur once an electronic system is in the field.  Examples are those caused by stress resulting from the swelling of polymers due to moisture intrusion or to electrochemical migration in the presence of electrical bias and moisture.  These effects have also been widely studied.  However, there has been much less study on the effect of moisture on the performance of passive components and active subsystems.

Case Study Analysis

Moisture Concentration in the Package Laminate Material — BT

A recent Calculation Corner column was devoted to the calculation methodology for modeling moisture diffusion [4]. It dealt with BT (bismaleimide triazine), a fiber-reinforced polymer commonly used in BGA (Ball Grid Array) packages.  The article demonstrated a method for calculating the equilibrium concentration of moisture at values of ambient temperature and RH within and slightly beyond the current ASHRAE limits.   Table 1 provides values of saturated moisture concentration in a BT substrate, assumed to be in an operating system, such that its temperature is 60˚C.  [Note that, as shown in the article, the elevated temperature of the BT leads to a lower moisture concentration than had it been at ambient temperature.]  All of the assumed values of ambient temperature and RH are within the current allowable ASHRAE range except for the 40˚C/60%RH value.   Using the concentration at 20˚C/40%RH, namely 0.2 mg/cm³, as a baseline value, one sees that, at the top end of the allowable range, the moisture concentration is 4 times that, at 0.8 mg/cm³.  We will return to these values in the discussion in the following section.

Another thing to note in the referenced analysis is that the time for the moisture to reach equilibrium in BT under these ambient conditions is on the order of only a few weeks.

Effect of Moisture Concentration on High-Speed Signal Propagation

A recent study measured the high-speed signal propagation along a copper trace, 21 µm wide and 15 µm thick and 50 mm long in a stripline configuration, typical of what would be used in a package substrate [5].  The trace is sandwiched between two layers of a low-loss, FR-4 type dielectric, each 130 µm thick, 1.4 mm wide, and 50 mm long.  Each dielectric layer has a copper plane bonded to its outer surface.  The signal propagation is measured using a vector network analyzer at frequencies between 2 and 16 GHz at two different moisture concentration levels in the dielectric.  Tests were performed at each of the specified frequencies over a range of temperature values from 20 to 80˚C.

The first moisture level was achieved by a “soak” exposure at 30˚C/60%RH for one week.  The calculated moisture concentration at the stripline location (along the centerline of the dielectric) is 1.8 mg/cm³.  This value is comparable to that calculated for the BT at the upper end of the range.

After the completion of the first set of tests, the sample was “baked out” and then retested over the same frequency and temperature ranges.  The bake out condition was 125˚C for 1 week.  The calculated moisture concentration at the stripline location following the bakeout is 0.3 mg/cm³.  This is comparable to the baseline moisture concentration in the preceding case study.

Figure 1: Graph of signal loss for a stripline structure versus temperature, frequency, and moisture content resulting from 1) soak process (1.76 mg/cm3) and 2) bake process (0.30 mg/cm3).

The results are plotted in Figure 1.  The graph compares the signal loss (attenuation) at a given temperature minus the loss measured at 20˚C.  There are two families of curves plotted, representing the sample in the “after soak” and the “after bake” conditions.  We see that the soaked sample shows more signal loss than the baked one.  The effect is more pronounced with increasing temperature and frequency.  In the worst case reported here, the loss at 16 GHz and 80˚C is 36% greater for the soaked sample, compared to the baked one.   In general, at these very high frequencies, the noise margins are tighter.  The increased attenuation measured here could potentially lead to increased bit error rates unless it was anticipated in the design phase and effectively accounted for.

Effect of Moisture Exposure on a High-speed Network Switch

Another recent study measured the data throughput of two different populations of 3 identical network switches over an 8 week period [6].  One was maintained under environmental conditions representative of a benign air conditioned data center environment: 20˚C/50%RH.  [Note that these conditions are close to the baseline values in the first case study.]

The second environment was chosen to be representative of conditions experienced in a free-air-cooled environment.   It was conducted in an environmental chamber in which the temperature/RH setting was varied between 10˚C/85%RH and 50˚C/15%RH.  A complete cycle was completed in 16 hours.  In that time period there was a 4 hour hold at the lower temperature, followed by 4 hour ramp to the higher temperature, holding there for 4 hours, and then followed by a 4 hour ramp to return to the lower temperature.

A baseline throughput rate was established for each population by averaging the rate over an first 10,000 data packets sent.  The duration of this part of the test was approximately 1 day.   The baseline was 93.7 Mbps for the air conditioned environment and slightly less, at 92.4 Mbps, for the temperature cycled environment.

For each population, the authors parsed the data acquired over the remainder of the 8 week period into 3 groups representing a 1%, 2%, 5%, 10%, and 20% throughput dropoff compared to the baseline.

Figure 2: Number of data packets per week experiencing indicated levels of dropoff in data throughput (1%, 2%, and 5%).

Most of the results are plotted in Figure 2.  The number of packets at a specified level of dropoff was higher for the population with the free-air-cooled  condition.  The ratios of these values (averaged over the 8-week period) at the 1%, 2%, and 5% levels, respectively were 2.5:1, 7.3:1, and 14.3:1, respectively.  Furthermore, there was a significant increase in the number of packets demonstrating dropoffs of 1 and 2% in the final week of the tests for the temperature cycled population.

There were no packets in the air conditioned environment at the 10% and 20% dropoff levels.  However, for the harsher environment there were 105 and 55 packets, on the average, per week.

The authors concluded that the level of performance variation of the switch in the simulated free air cooled environment might well be unacceptable to many data center customers.

Conclusions

This article highlights a number of published studies that address performance problems related to moisture absorption in individual electronic components and in subsystems.   In the experience of this author, the majority of moisture-related studies have to do with reliability not performance.  Indications are that the relaxed temperature and relative humidity ranges approved by ASHRAE will some day become the norm in the datacenter.  There is a risk that electronics companies will not anticipate the effect this change may have in the high-speed performance of their products.  This situation could be exacerbated  by the fact that performance degradation might well occur as soon as the moisture concentration achieves a critical value without the need for a secondary process to be triggered by the moisture absorption, as is usually the case with failure mechanisms.

It is hoped that this article will help to increase awareness of these issues in our industry and promote early action to effectively manage the risks detailed here.

References

1. V. Mulay, “Humidity Excursions in Facebook Prineville Data Center,” ElectronicsCooling, Vol.  18, No. 4., December, 2012.
2. R. Schmidt, “A History of ASHRAE Technical Committee TC9.9 and its Impact on Data Center Design and Operation,” ElectronicsCooling, Vol.  18, No. 4, December, 2012.
3. D. Atwood  and J. G. Miner, “Reducing Data Center Cost with an Air Economizer”, IT@Intel Brief, Intel Information Technology, Computer Manufacturing, Energy Efficiency, August 2008.
4. B. Guenin, “Calculation Corner – Application of Transient Thermal Methods to Moisture Diffusion Calculations, Part 2,” Vol.  19, No. 1., March, 2013.
5. J. Miller, Y. Li,  K. Hinckley,  G. Blando,  B. Guenin, and I. Novak,” Temperature and Moisture Dependence of PCB and Package Traces and the Impact on Signal Performance,” Proceedings DesignCon Conference, Santa Clara, January 30 – February 2, 2012
6. J. Dai, D. Das, M. Pecht, and M. Ohadi, “A Case Study on the Impact of Free Air Cooling on Telecom Equipment Performance,” Proceedings, SEMI-THERM XVIII Semiconductor Thermal Measurement, Modeling And Management Symposium, San Jose, CA, March 18-22, 2012, pp. 82-86.

Figure Captions

Figure 1: Graph of signal loss for a stripline structure versus temperature, frequency, and moisture content resulting from 1) soak process (1.76 mg/cm³) and 2) bake process (0.30 mg/cm³).

Figure 2: Number of data packets per week experiencing indicated levels of dropoff in data throughput (1%, 2%, and 5%).

To learn more visit Master Bond

In Part 1 we saw that the BGA type package benefited from an 81% drop of junction temperature rise over ambient whilst the TO package only dropped by 27%. To understand why such a difference let’s consider the heat flow within the package prior to a heatsink being deployed.

The black heat flux vectors show how the heat leaves its source on the active layer on the die and spreads out through the package. To clarify things I’ve clipped out low heat flux vectors. (There’s no such thing as a perfect insulator and the heat actually spreads everywhere, to massively varying extents though).

Thermal design engineers often characterise heat flow in terms of heat flow budgets. That is a more macro description of the dominant heat flow paths, what % of the total heat flows where. Such budgeting is also sometimes used to summarise the proportion of heat that passes through a surface by convection, conduction and radiation. In this case, looking at the budget for the die, 77% of the heat flows directly up through the die to pass near the top surface of the package, due in no small part to the nice metal spreader and slug designed specifically for that purpose. This is before a heatsink is placed on top, mind.

For the TO package 95% of the heat prefers to pass directly to the PCB. Again not surprising considering that was the design intent. As a piece of heat (eh?), metal is as attractive to you as would be the illicit love child of Jessica Alba+Mena Suvari or George Clooney+Leonardo DiCaprio [select as is appropriate to your leaning, or gender, or both]. To a piece of heat, low thermal conductivity epoxy based encapsulant is more like being forced to .. (I’ll stop there).

So, what happens to the budget when a nice fat metal heatsink is plonked on top of both packages?:

The 77% of the heat that was going up before the heatsink was placed on top has now been upped to 90%. The heatsink provides a welcome helping hand to a whole bunch of heat that was already waiting on the pier for the boat (so to speak, sorry, getting rather carried away with my similies). Thus the substantial reduction in junction temperature rise. The more readily the heat can leave the lower the temperature rise that backs up behind it.

For the TO package there’s only a very minor 3% shift in the budget. So why the 27% reduction in junction temperature rise? The answer to that lies not in the conductive effects of the heatsink (there are little) but in the effect the heatsink has on the air flow disruption that it causes.

The heatsink acts to funnel more air over the surface of the PCB around the component. This can be seen in the above plot of air speed, just above the board surface, comparing the faster moving air (>1m/s) with and without heatsinks. This increased air speed washing over the board surface more effectively removes the heat that has already passed down through the TO package and spread into the board, thus resulting in the 27% dT decrease.

[For those thermal experts out there; this is a fixed flow type environment with limited allowance of by-pass so there's no coupling in this example between pressure loss, flow rate and thermal performance as you'd find in reality]

Any good thermal engineer wouldn’t be happy with only a 27% drop in dT after having considered spending so much on a heatsink. There are better ways to assure an even bigger dT drop for the TO type package. More on that in Part 3.

15th May 2013, Ross-on-Wye

Previously, only the NF-A15 round-frame model was available with PWM support for automatic speed control. The NF-A14 “complies with Noctua’s Advanced Acoustic Orientation standard” and has a top speed of 1500rpm with speed control options. The fan also features “modular cabling, a Low-Noise Adaptor and six years manufacturer’s warranty.”

To learn more visit Noctua

These new i4 models support LGA2011, LGA 1356 and LGA2011 and also come with “PWM fans for automatic speed control.” Noctua also says “both the NF-F12 120mm fan of the NH-U12DX i4 and the two NF-B9 92mm fans of the NH-U9DX i4 now support PWM for fully automatic speed control. In addition, the maximum fan speed can be reduced using the supplied Low-Noise Adaptors for even quieter operation.”

With only a 45 mm fin design, the NH-U12DX “ensures easy access to RAM slots” and will not overhang the memory on the LGA1356, LGA1366, and LGA2011 models. The height of the NH-U9DX is only 125 mm, making it fully compatible with standard 4U servers. These models will be available in the near future.

To learn more visit Noctua