# Simulation Study of Influencing Factors of Immersion Phase-Change Cooling Technology for Data Center Servers

^{*}

## Abstract

**:**

## 1. Introduction

^{2}·K), leads to low heat transfer efficiency. The power usage effectiveness (PUE) value of data centers using air cooling as the mainstream cooling method is around 1.9 [9]. Liquid cooling systems are currently a more common cooling method. However, this method requires the installation of longer cooling pipelines and additional accessories such as pumps, which increase the floor space of data centers. To solve the problem of low cooling system efficiency in data centers, many researchers have proposed different solutions to control the cold and hot air flow organization, such as the closed cold aisle containment system (CACS), the closed hot aisle containment system (HACS), and the vertical exhaust duct system (VEDS) [10]. At the same time, new cooling systems have been developed based on the original air-cooling system, such as heat pipe cooling [11], steam radiator cooling, single-phase immersion cooling [12,13,14], immersion phase-change cooling, jet impingement cooling [15], and spray phase-change cooling [16,17]. Qiu et al. [18] used a combination of experiments and simulations to investigate the heat dissipation capabilities of forced air cooling and immersion cooling. The research results show that the measured thermal resistance of the three-dimensional stacked mold structure from the joint to the environment is reduced from 26 °C/W under natural convection to 7.6 °C/W under forced air cooling and 0.6 °C/W under immersion cooling. Additionally, when 1 W of power is applied to each chip layer, the temperature distribution variation rates for forced air cooling and immersion cooling are 5.8% and 7.4%, respectively. Bao K et al. [19] summarized the heat flux density range applicable to traditional air-cooling, heat pipe cooling, and liquid cooling technologies, as shown in Figure 1. When the heat flux density is greater than 100 W/cm

^{2}, air cooling and heat pipe cooling are no longer applicable. Immersion cooling shows great potential for cooling high-performance stacked mold structures [20].

^{2}). However, if two such heat sources are thermally coupled, the power of the upper heat source must be less than 185 W. Based on this study, the vertical coupling of heat sources can be set to reduce the size of the entire cabinet and thus the footprint of the entire server room.

## 2. Establishment of a Simulation Model

#### 2.1. Selection of Cooling Liquid

^{3}; and ${\rho}_{\mathrm{l}}$ is the density of the liquid phase of the electronic fluorine liquid, kg/m

^{3}.

^{2}. By substituting the various parameters of the electron-fluoride liquid into Equations (1) and (2), the corresponding wall superheat and critical heat flux (CHF) at boiling are shown in Table 2.

#### 2.2. Physical Model

#### 2.3. Numerical Model

#### 2.3.1. VOF Model

_{f}is the latent heat energy of the liquid phase and E

_{g}is the latent heat energy of the gas phase.

#### 2.3.2. Turbulence Models

#### 2.3.3. Evaporation-Condensation Model

#### 2.4. Preprocessing and Solving the Algorithm of the Model

^{2}. The pressure uses the PRESTO! interpolation method, the volume fraction uses the geometric reconstruction interpolation method, and energy and momentum use the second-order upwind interpolation format to improve computational accuracy. The time step is set to 0.001. Monitoring parameters are set as chip surface temperature, chip body temperature, pressure change, and heat flux density.

#### 2.5. Grid Independence Verification

^{−3}, and the calculation process converges. After the calculation process converges, the temperature averages of the two chips on the No. 3 motherboard under different grid number models are compared.

#### 2.6. Model Validation

#### 2.7. Evaluation Index

- (1)
- Maximum Temperature

- (2)
- Temperature Difference

- (3)
- Temperature uniformity coefficient: In modification to the temperature uniformity coefficient formula proposed by Sun et al. [1], the $\alpha $ coefficient is used to evaluate the thermal stability of the entire system, with its calculation formula shown in Equation (12):

## 3. Results and Discussion

#### 3.1. The Influence of Different Inlet Water Temperatures on the Cooling Effect

#### 3.2. The Effect of Different Cooling Water Inlet Flow Rates on Cooling Performance

#### 3.3. The Influence of Different Arrangement Intervals on the Heat Dissipation Performance of the System

## 4. Conclusions

- (1)
- When the cooling water inlet velocity in the condensing module is constant, as the cooling water inlet temperature decreases, the difference between its temperature gradient and the chip surface temperature gradient will increase, thereby increasing the heat transfer rate, and the chip surface temperature will drop significantly;
- (2)
- When the inlet water temperature is constant, as the inlet water flow rate increases, the time required for the chip to reach stability decreases, and the difference in maximum temperature after reaching stability is almost negligible. The maximum temperature and temperature difference also decrease;
- (3)
- Simulations on physical models with server spacing from 5 mm to 25 mm showed that increasing the spacing to 15 mm reduced the maximum temperature by 5.06%, the temperature difference from 16.5 °C to 11 °C, and the temperature uniformity coefficient by 52.6%. However, increasing the spacing from 15 mm to 25 mm only decreased the maximum temperature by 1.07%, the temperature difference from 11 °C to 10.3 °C, and the temperature uniformity coefficient by 22%. Thus, a 15 mm spacing is optimal for heat dissipation, reducing system size, and accommodating more servers in the same area.

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Sun, X.; Han, Z.; Li, X. Simulation study on cooling effect of two-phase liquid-immersion cabinet in data center. Appl. Therm. Eng.
**2022**, 207, 118–142. [Google Scholar] [CrossRef] - Ali, A.F. Thermal performance and stress analysis of heat spreaders for immersion cooling applications. Appl. Therm. Eng.
**2020**, 181, 115984. [Google Scholar] [CrossRef] - Li, J.; Zhou, G.; Tian, T.; Li, X. A new cooling strategy for edge computing servers using compact looped heat pipe. Appl. Therm. Eng.
**2021**, 187, 116599. [Google Scholar] [CrossRef] - Patankar, S.V. Airflow and Cooling in a Data Center. J. Heat Transf.
**2010**, 132, 073001. [Google Scholar] [CrossRef] - Khalaj, A.H.; Halgamuge, S.K. A Review on efficient thermal management of air- and liquid-cooled data centers: From chip to the cooling system. Appl. Energy
**2017**, 205, 1165–1188. [Google Scholar] [CrossRef] - Andrae, A.S.G. Total Consumer Power Consumption Forecast. Total consumer power consumption forecas. Nord. Digit. Bus. Summit
**2017**, 10, 69. [Google Scholar] - Lu, T.; Lü, X.; Remes, M.; Viljanen, M. Investigation of air management and energy performance in a data center in Finland: Case study. Energy Build.
**2011**, 43, 3360–3372. [Google Scholar] [CrossRef] - Nadjahi, C.; Louahlia, H.; Lemasson, S. A review of thermal management and innovative cooling strategies for data center. Sustain. Comput. Inform. Syst.
**2018**, 19, 14–28. [Google Scholar] [CrossRef] - Li, X.; Lv, L.; Wang, X.; Li, J. Transient thermodynamic response and boiling heat transfer limit of dielectric liquids in a two-phase closed direct immersion cooling system. Therm. Sci. Eng. Prog.
**2021**, 25, 100986. [Google Scholar] [CrossRef] - Lu, H.; Zhang, Z.; Yang, L. A review on airflow distribution and management in data center. Energy Build.
**2018**, 179, 264–277. [Google Scholar] [CrossRef] - Siedel, B.; Sartre, V.; Lefèvre, F. Literature review: Steady-state modelling of loop heat pipes. Appl. Therm. Eng.
**2015**, 75, 709–723. [Google Scholar] [CrossRef] - Shrigondekar, H.; Lin, Y.-C.; Wang, C.-C. Investigations on performance of single-phase immersion cooling system. Int. J. Heat Mass Transf.
**2023**, 206, 123961. [Google Scholar] [CrossRef] - Hnayno, M.; Chehade, A.; Klaba, H.; Polidori, G.; Maalouf, C. Experimental investigation of a data-centre cooling system using a new single-phase immersion/liquid technique. Case Stud. Therm. Eng.
**2023**, 45, 102925. [Google Scholar] [CrossRef] - Huang, Y.; Ge, J.; Chen, Y.; Zhang, C. Natural and forced convection heat transfer characteristics of single-phase immersion cooling systems for data centers. Int. J. Heat Mass Transf.
**2023**, 207, 124023. [Google Scholar] [CrossRef] - Sarkar, S.; Gupta, R.; Roy, T.; Ganguly, R.; Megaridis, C.M. Review of jet impingement cooling of electronic devices: Emerging role of surface engineering. Int. J. Heat Mass Transf.
**2023**, 206, 123888. [Google Scholar] [CrossRef] - Liu, P.; Kandasamy, R.; Ho, J.Y.; Wong, T.N.; Toh, K.C. Dynamic performance analysis and thermal modelling of a novel two-phase spray cooled rack system for data center cooling. Energy
**2023**, 269, 126835. [Google Scholar] [CrossRef] - Kim, J. Spray cooling heat transfer: The state of the art. Int. J. Heat Fluid Flow
**2007**, 28, 753–767. [Google Scholar] [CrossRef] - Qiu, D.; Cao, L.; Wang, Q.; Hou, F.; Wang, X. Experimental and numerical study of 3D stacked dies under forced air cooling and water immersion cooling. Microelectron. Reliab.
**2017**, 74, 34–43. [Google Scholar] [CrossRef] - Bao, K.; Wang, X.; Fang, Y.; Ji, X.; Han, X.; Chen, G. Effects of the surfactant solution on the performance of the pulsating heat pipe. Appl. Therm. Eng.
**2020**, 178, 115678. [Google Scholar] [CrossRef] - Deng, Y.; Zhang, M.; Jiang, Y.; Liu, J. Two-stage multichannel liquid–metal cooling system for thermal management of high-heat-flux-density chip array. Energy Convers. Manag.
**2022**, 259, 115591. [Google Scholar] [CrossRef] - Capozzoli, A.; Primiceri, G. Cooling Systems in Data Centers: State of Art and Emerging Technologies. Energy Procedia
**2015**, 83, 484–493. [Google Scholar] [CrossRef] [Green Version] - Matsuoka, M.; Matsuda, K.; Kubo, H. Liquid immersion cooling technology with natural convection in data center. In Proceedings of the IEEE 6th International Conference on Cloud Networking, Prague, Czech Republic, 25–27 September 2017; pp. 1–7. [Google Scholar] [CrossRef]
- Cheng, C.-C.; Chang, P.-C.; Li, H.-C.; Hsu, F.-I. Design of a single-phase immersion cooling system through experimental and numerical analysis. Int. J. Heat Mass Transf.
**2020**, 160, 120203. [Google Scholar] [CrossRef] - Kanbur, B.B.; Wu, C.; Fan, S.; Duan, F. System-level experimental investigations of the direct immersion cooling data center units with thermodynamic and thermoeconomic assessments. Energy
**2020**, 217, 119373. [Google Scholar] [CrossRef] - Gess, J.L.; Bhavnani, S.H.; Johnson, R.W. Experimental Investigation of a Direct Liquid Immersion Cooled Prototype for High Performance Electronic Systems. IEEE Trans. Compon. Packag. Manuf. Technol.
**2015**, 5, 1451–1464. [Google Scholar] [CrossRef] - Kanbur, B.B.; Wu, C.; Fan, S.; Tong, W.; Duan, F. Two-phase liquid-immersion data center cooling system: Experimental performance and thermoeconomic analysis. Int. J. Refrig.
**2020**, 118, 290–301. [Google Scholar] [CrossRef] - Zhang, C.; Sun, X.; Han, Z.; Li, X.; Dong, J. Energy saving potential analysis of two-phase immersion cooling system with multi-mode condenser. Appl. Therm. Eng.
**2023**, 219, 119614. [Google Scholar] [CrossRef] - Li, Y.; Bai, M.; Zhou, Z.; Lv, J.; Hu, C.; Gao, L.; Peng, C.; Li, Y.; Li, Y.; Song, Y. Experimental study of liquid immersion cooling for different cylindrical lithium-ion batteries under rapid charging conditions. Therm. Sci. Eng. Prog.
**2023**, 37, 101569. [Google Scholar] [CrossRef] - Wang, Z.; Zhao, R.; Wang, S.; Huang, D. Heat transfer characteristics and influencing factors of immersion coupled direct cooling for battery thermal management. J. Energy Storage
**2023**, 62, 106821. [Google Scholar] [CrossRef] - An, X.; Arora, M.; Huang, W.; Brantley, W.C.; Greathouse, J.L. 3D numerical analysis of two-phase immersion cooling for electronic components. In Proceedings of the 2018 17th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), San Diego, CA, USA, 29 May–1 June 2018; pp. 609–614. [Google Scholar]
- Luo, Q.; Wang, C.; Wen, H.; Liu, L. Research and optimization of thermophysical properties of sic oil-based nanofluids for data center immersion cooling. Int. Commun. Heat Mass Transf.
**2022**, 131, 105863. [Google Scholar] [CrossRef] - Tuma, P.E. Fluoroketone C2F5C(O)CF(CF3)2 as a Heat Transfer Fluid for Passive and Pumped 2-Phase Applications. In Proceedings of the 2008 Twenty-fourth Annual IEEE Semiconductor Thermal Measurement and Management Symposium, San Jose, CA, USA, 16–20 March 2008; pp. 173–179. [Google Scholar] [CrossRef]
- M. 3M Fluorinert ™ FC-72 Electronic Liquid. Available online: http://www.3m-novec-fluorinert.com/3M-Fluorinert-Fluids-FC-72.html (accessed on 9 April 2023).
- M. 3M NovecTM 649 Engineered Fluid. Available online: http://mulyimedia.3a.com/mws/media/569865O/3M-novec (accessed on 28 May 2023).
- M. 3M Novec™ HFE 7100 engineered fluid. Available online: http://www.3m-novec-fluorinert.com/3M-Novec-Engineer-Fluids-HFE-7100.html (accessed on 24 April 2023).
- M. 3M Fluorinert™ FC-77 Electronic Liquid. Available online: http://www.3m-novec-fluorinert.com/3M-Fluorinert-Fluids-FC-77.html (accessed on 24 April 2023).
- Lienhard, J.H.; Dhir, V.K.; Riherd, D.M. Peak Pool Boiling Heat-Flux Measurements on Finite Horizontal Flat Plates. J. Heat Transf.
**1973**, 95, 477–482. [Google Scholar] [CrossRef] - Wu, X.L.; Liu, Y.; Ni, H.; Huang, J.L.; Guo, H.W.; Zhuang, Y.; Han, X.H. Effects of Different Electronic Fluoride Liquids on the Performance of Immersion Phase Change Cooling Systems. J. Refrig.
**2021**, 42, 74–82. [Google Scholar] - Yang, M.; Li, M.-T.; Hua, Y.-C.; Wang, W.; Cao, B.-Y. Experimental study on single-phase hybrid microchannel cooling using HFE-7100 for liquid-cooled chips. Int. J. Heat Mass Transf.
**2020**, 160, 120230. [Google Scholar] [CrossRef] - Mulbah, C.; Kang, C.; Mao, N.; Zhang, W.; Shaikh, A.R.; Teng, S. A review of VOF methods for simulating bubble dynamics. Prog. Nucl. Energy
**2022**, 154, 104478. [Google Scholar] [CrossRef] - Vaishnavi, G.S.; Ramarajan, J.; Jayavel, S. Numerical studies of bubble formation dynamics in gas-liquid interaction using Volume of Fluid (VOF) method. Therm. Sci. Eng. Prog.
**2023**, 39, 101718. [Google Scholar] [CrossRef] - Ding, B.; Zhang, Z.-H.; Gong, L.; Xu, M.-H.; Huang, Z.-Q. A novel thermal management scheme for 3D-IC chips with multi-cores and high power density. Appl. Therm. Eng.
**2019**, 168, 114832. [Google Scholar] [CrossRef]

**Figure 1.**Effective heat transfer coefficients and applicable heat flux ranges for different cooling techniques.

**Figure 25.**Heat transfer coefficient on the surface of the chip under different inlet flow velocities.

**Figure 28.**Maximum temperature difference between chips and within the system under different spacing arrangements.

Coolant. | Boiling Point (°C) | Density (kg/m^{3}) | Surface Tension (N/m) | Latent Heat of Evaporation (kJ/kg) | Dielectric Constant | Viscosity (mm^{2}/s) | Flash Point (°C) |
---|---|---|---|---|---|---|---|

FC-40 | 155 | 1.85 | — | 68 | 1.9 | 0.18 | — |

FC-72 | 56.0 | 1.68 | 0.010 | 88 | 1.75 | 0.38 | 149.9 |

HFE-7100 | 61.0 | 1.52 | 0.136 | 112 | — | — | — |

HFE-7000 | 34.0 | 1.4 | 0.124 | 142 | 7.4 | 0.32 | — |

FC-77 | 97.0 | 1.78 | 0.13 | 89 | 1.9 | 0.72 | — |

Novec 649 | 49.0 | 1.6 | 0.108 | 88 | 1.8 | 0.40 | — |

Coolant | FC-72 | HFE-7100 | Novec 649 |
---|---|---|---|

Wall surface overheating (K) | 6.0 | 6.9 | 5.8 |

CHF (W/m^{2}) | 155,351 | 185,470 | 156,859 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Zhao, T.; Sun, R.; Hou, X.; Huang, J.; Geng, W.; Jiang, J.
Simulation Study of Influencing Factors of Immersion Phase-Change Cooling Technology for Data Center Servers. *Energies* **2023**, *16*, 4640.
https://doi.org/10.3390/en16124640

**AMA Style**

Zhao T, Sun R, Hou X, Huang J, Geng W, Jiang J.
Simulation Study of Influencing Factors of Immersion Phase-Change Cooling Technology for Data Center Servers. *Energies*. 2023; 16(12):4640.
https://doi.org/10.3390/en16124640

**Chicago/Turabian Style**

Zhao, Tiantian, Rongfeng Sun, Xukai Hou, Jikai Huang, Wenguang Geng, and Jianguo Jiang.
2023. "Simulation Study of Influencing Factors of Immersion Phase-Change Cooling Technology for Data Center Servers" *Energies* 16, no. 12: 4640.
https://doi.org/10.3390/en16124640