Study on Effects of Operating Parameters on a Water-Cooled Loop Thermosyphon System under Partial Server Utilization

Abstract

During the operation of a data center, servers are gradually installed in racks, causing most racks to work under a low heating load for a long time and affecting the cooling efficiency of the loop thermosyphon system (LTS). Thus, the effects of operating parameters on the thermal performance should be investigated. In this study, a water-cooled LTS was experimentally investigated under different airflow rates and heating loads. The results show that the additional liquid refrigerant reduced the heat transfer performance and aggravated a drop in cooling capacity when the airflow rate and heating load were decreased. To further reveal the effects of the operating parameters on the thermal performance and cooling efficiency, a steady-state distributed-parameter model was developed and validated based on the experimental data. The results show that the excessive cooling capacity was reduced by decreasing the airflow rate according to the upper limit of the server exhaust air temperature under partial server utilization. The excessive cooling capacity was reduced by 14.5–52.1% under 5–56.5% server utilization. To further reduce the excessive cooling capacity while ensuring thermal security, the water side operating parameters (including the supply chilled water temperature and water flow rate) were adjusted according to the upper limit of the rack’s average outlet air temperature, which reduced the excessive cooling capacity by more than 23.8% under partial server utilization.

1. Introduction

With digital and information technologies, the demands for data storage, transmission, and processing have increased dramatically, resulting in a dramatic increase in the number and scale of data centers. However, the energy consumption of data centers is also growing rapidly. In 2018, the total energy consumption of data centers reached 160.8 billion kWh in China, which accounted for 2.35% of the total national energy consumption [1]. One of the main reasons is that the energy consumption of servers increases dramatically with the improvement of chip manufacturing processes. The other main reason is that cooling systems use huge energy volumes to ensure IT equipments’ thermal safety in data centers. Because the energy consumption of servers is inevitably generated during the operation of the data center, improving the energy efficiency of the cooling system is the key strategy for saving energy. In the past decade, most data centers have used the computer room air conditioning system (CRACS) to cool IT equipment due to its wide application scenarios. However, the CRACS is a direct expansion split air conditioning system that cannot use natural cold sources, so it needs to operate at a high load all year round. In general, the CRACS accounts for more than 30% of the total energy consumption in data centers [2,3,4].

In order to improve the energy efficiency of the CRACS, the cold source uses side energy-saving technologies, such as direct or indirect airside free cooling technologies [5,6]. Limited by the low specific heat capacity and heat conductivity of air, these free cooling technologies are more commonly used in telecommunication base stations and small data centers with low cooling requirements. In addition, the local climate and air cleanliness would also limit the use of airside free cooling technologies. Therefore, more and more data centers use the water-cooled computer room air handler system (CRAHS) to replace the CRACS for energy saving. The CRAHS uses a chilled water system (composed of cooling towers, water pumps, heat exchangers, and chillers) as the cold source, which can use natural water [7], ariside economizers [8], and water-side economizers [9] for free cooling. Compared to the CRACS, the CRAHS can provide a greater cooling capacity and use a wider variety of free cooling techniques, which is more suitable to cool nearly all scales of data centers. Limited by the low-efficiency single-phase heat transfer from the terminal equipment and inefficient airflow distribution, the CRAHS can only use relatively low-temperature natural cold sources, limiting its improvement in energy efficiency. In order to improve the energy efficiency of terminal equipment, energy-saving technologies, such as airflow distribution and management technologies, have been proposed [10,11,12]. However, due to the different air supply distances of racks, room-level cooling systems such as the CRACS and CRAHS that use these technologies still struggle to accurately and efficiently cool IT equipment. In order to solve the above problems, the loop thermosyphon system (LTS) is a kind of terminal equipment for the cooling system proposed for rack-level cooling. The schematic diagram of a loop thermosyphon system is shown in Figure 1. The hot air generated by servers is driven by extractor fans on the backplate, which transfers the heat to the evaporator installed on the backplate. The refrigerant in the evaporator evaporates and enters the condenser through the vapor pipe. The vapor refrigerant is cooled into the liquid phase with the chilled water inside the condenser and returned to the evaporator by gravity. Because of the rack-level cooling and two-phase heat transfer, the LTS can accurately and efficiently cool IT equipment. Thus, the LTS can increase the supply of chilled water temperatures to make better use of the natural cold source for energy saving [13,14,15].

In order to improve the thermal performance and reduce energy consumption, several experimental and numerical studies were carried out to investigate the influencing factors of the LTS. Ling et al. [16], Ding et al. [17], and Zhang et al. [18] analyzed the effect of the filling ratio on the cooling capacity. The results showed that the LTS can achieve the maximum cooling capacity at an optimal filling ratio. At the optimal filling ratio, the effects of the geometrical parameters [19] and refrigerant type [20] on the thermal performance of the LTS were also studied. The cooling capacity can be increased via parameter optimization and refrigerant substitution. In addition to the above factors, the operating parameters, such as the airflow rate and supply chilled water temperature, have a noticeable impact on the thermal performance of the LTS. Ling et al. experimentally investigated the effect of the airflow rate and supply chilled water temperature on the cooling capacity of the LTS. The results showed that the cooling capacity increased by 42.8% when the airflow rate increased from 1712 m³/h to 2980 m³/h [16]; it also increased by 11.1% when the supply chilled water temperature decreased from 16 °C to 12 °C [21].

The above results are helpful in optimizing the design and operating parameters of an LTS under rated server utilization, but the operating conditions of servers in actual data centers are more complex. Since the servers are gradually installed in racks according to the user demand, most racks are operated under a low heating load for a long time. The data centers’ average utilization rate is only 50.61% compared to 29.01% for large-scale data centers in China [22]. The server utilization is also related to the CPU usage rate on the workdays [23]. However, the set value of the supply chilled water temperature was often designed according to the demand of the LTS under full server utilization. When an LTS is operated under a relatively low heating power and supply chilled water temperature, the cooling capacity of the LTS would obviously exceed the heating load, resulting in energy wastage. In our previous study [24], controlling extractor fans to adjust the airflow rate could reduce the excessive cooling capacity and power of extractor fans. The average increase rate of the LTS’s energy efficiency ratio (EER) was more than 432% under 2 kW of heating power (the heating power was 6 kW under full server utilization). However, adjusting the airflow rate for energy saving will affect the server’s exhaust air temperature, which in turn affects the thermal security of the chips under partial server utilization. According to the lectures [25,26], the chip-to-air temperature differences need to be larger than 45 °C under maximum heating loads; thus, the inlet air temperature of servers should be lower than 40 °C to keep a chip safe, but few studies have paid attention to this problem. In addition, the supply chilled water temperature and the chilled water flow rate also should be adjusted for energy saving under partial server utilization on the premise of ensuring the thermal safety of servers. Therefore, it is necessary to investigate the effects of the operating parameters (including the airflow rate, supply chilled water temperature, and water flow rate) on the performance of an LTS to improve the cooling efficiency and thermal security under partial server utilization.

This paper aims to conduct an experimental and numerical investigation on a water-cooled LTS under partial server utilization. Firstly, experimental studies were carried out in an enthalpy difference laboratory. The effects of the airflow rate and heating load on the cooling capacity, refrigerant mass flow rate, and rack’s average outlet air temperature were analyzed. Then, a steady-state distributed-parameter model was developed and validated based on the experimental data. To reduce the excessive cooling capacity and ensure thermal security, the effects of the operating parameters (including the airflow rate, supply chilled water temperature, and water flow rate) on the thermal safety and cooling efficiency were analyzed and evaluated, and several suggestions for operating parameter optimization were put forward.

2. Experimental Method

2.1. Experimental Setup

The experiments were carried out in an enthalpy difference laboratory, as shown in Figure 2. In order to obtain stable and uniform test conditions, the enthalpy difference laboratory provides hot air at dry and wet bulb temperatures of 35 °C and 21 °C, respectively, to simulate hot exhaust airflow generated by servers. The hot air was driven by extractor fans to carry the servers’ heat to the evaporator of an LTS mounted on the rack backplate and was cooled by the endothermic evaporation of the refrigerant. The vaporized refrigerant was condensed with 12 °C chilled water. The heating loads under different partial server utilizations were simulated by changing the airflow rate of extractor fans, and the airflow rate was measured by enthalpy difference laboratory. The structure parameters of the LTS are shown in

Table 1. The structure parameters of LTS.

Structure Parameters	Value	Structure Parameters	Value
Liquid/vapor collector diameter (mm)	30	Louver length Lle (mm)	7
Liquid outlet diameter (mm)	16	Louver pitch Pl (mm)	1
Liquid pipe length Llp (mm)	4250	Number of flat tubes	30
Liquid pipe height difference ΔH (mm)	2200	Number of rows	1
Vapor outlet diameter (mm)	19	Number of micro channels in flat tube	19
Transverse flat tube pitch Pf (mm)	8	Plate length (mm)	466
Micro channel pitch Pm (mm)	0.4	Plate width (mm)	111
Flat tube height Ht (mm)	2	Area of the plate (m2)	0.079
Flat tube width Fw (mm)	25.4	Plate thickness (mm)	0.4
Flat Tube distance (mm)	1580	Angle of the corrugation, βc (°)	30
Fin pitch Fp (mm)	2	Corrugation amplitude, b (mm)	2.4
Fin thickness δf (mm)	0.105	Plate pitch, (mm)	2.8
Fin width Bf (mm)	25	Number of plates	20
Fin height Hf (mm)	8	Number of plates effective in heat transfer	18
Louver angle θ (°)	30

.

Refrigerant temperatures and pressures at the inlet and outlet of the evaporator and condenser ①–④ were measured with PT100 platinum resistors and AKS32 pressure transmitters. Chilled water temperatures at the inlet and outlet of condenser ⑤–⑥ were also measured with PT100 platinum resistors. The air temperatures and humidity at the evaporator inlet and outlet ⑦–⑧ and the air temperatures and humidity at rack inlet ⑨ were measured by enthalpy difference laboratory. The refrigerant mass flow rate was measured by a Coriolis type meter, and the water flow rate was measured by a magnetic flow meter of the enthalpy difference laboratory. The measured ranges and uncertainties for test equipment are shown in

Table 2. Ranges and uncertainties of test equipment.

Parameters	Test Equipment	Range	Uncertainty
Temperature (°C)	Enthalpy difference laboratory	−20~80	±0.1
	PT100 platinum resistor	−80~150	±0.1
Relative humidity	Enthalpy difference laboratory	0~95%	±1%
Pressure (kPa)	AKS32 pressure transmitter	−100~300	±7.5
Refrigerant mass flow rate (kg/h)	DMF-1-3B Coriolis mass flowmeter	0~200	±1
Airflow rate (m3/h)	Enthalpy difference laboratory	600~3100	±0.3
Water flow rate (m3/h)	Enthalpy difference laboratory	0~3.2	0.5%

. The uncertainties were calculated using Moffat method [27], and the maximum uncertainty for the cooling capacity was 6.8%.

2.2. Test Conditions and Results

Since the refrigerant filling ratio was a main factor that determines the thermal performance of LTS [16,17], the optimal refrigerant filling ratio was tested at first. During the filling charge test, the airflow rate was constant at the rate of 1800 m³/h; the evaporator inlet air temperature (T_a,in) was 35°C, and the filling charge increased from 0.6 kg to 1.8 kg with each increase of 0.2 kg, corresponding to the filling ratio from 27.9% to 83.7%. In this study, the refrigerant filling ratio was defined using the following equation:

$$\mathrm{F}\mathrm{i}\mathrm{l}\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}=\frac{{\mathrm{m}}_{\mathrm{r}}}{{\mathsf{\rho }}_{\mathrm{l}\mathrm{q}}{\mathrm{V}}_{\mathrm{e}}}\times 100\mathrm{\%}$$

(1)

where m_r is the refrigerant filling charge, kg; ρ_lq is the density of liquid refrigerant, kg/m³; and V_e is the interior volume of the evaporator, m³. At the optimal refrigerant filling ratio, the thermal performance was tested under different airflow rates and partial heating loads. The test conditions are shown in

Table 3. Test conditions.

	Range
Refrigerant filling charge (kg)	0.6, 0.8, 1.0, 1.2, 1.4, 1.6, 1.8
Filling ratio (%)	27.9, 37.2, 46.5, 55.8, 65.1, 74.4, 83.7
Indoor air dry/wet bulb temperature (°C)	35/21
Supply chilled water temperature Tcw (°C)	12
Chilled water flow rate Vw (m3/h)	1.71
Airflow rate Va (m3/h)	600, 800,1000, 1200, 1400, 1600, 1800
Working fluid	R22

.

Figure 3 shows the cooling capacity under different filling ratios. The results showed that the cooling capacity increased with increasing filling ratio and reached the maximum value when the filling ratio reached 65.1%. With the further increase in filling ratio, the cooling capacity began to decrease. This indicates that the optimal filling ratio was 65.1%.

The thermal performance of LTS was tested at the optimal refrigerant filling ratio. The cooling capacity can be calculated with water side parameters:

$${\mathrm{Q}}_{\mathrm{e}}={{\mathsf{\rho }}_{\mathrm{w}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{r}}\mathrm{V}}_{\mathrm{w}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{r}}{\mathrm{C}\mathrm{p}}_{\mathrm{w}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{r}}({\mathrm{T}}_{\mathrm{r}\mathrm{w}}-{\mathrm{T}}_{\mathrm{c}\mathrm{w}})$$

(2)

where Q_e is the cooling capacity of LTS, W; ρ_water is the density of water, kg/m³; V_water is the chilled water mass flow rate, m³/h; Cp_water is the specific heat capacity of water, kJ/(kg·K); T_cw is the supply chilled water temperature, °C; T_rw is the return water temperature, °C.

The thermal performance was tested under T_a,in = 35 °C, and airflow rate ranged from 600 m³/h to 1800 m³/h, as shown in Figure 4. Under 1800 m³/h airflow rate, the Q_e was 7155 W. Then, Q_e decreased rapidly due to reduced airflow rate and heating load. When the airflow rate was reduced from 1800 m³/h to 1200 m³/h, the Q_e decreased by 15.4%. However, the decrement of Q_e reached 32.5% when the airflow rate varied from 1200 m³/h to 600 m³/h, which means the decrements of Q_e caused by airflow rate and heating load variation were nonlinear.

To better illustrate the above phenomenon, the refrigerant mass flow rate (G_r) is also shown in Figure 4. The refrigerant mass flow of the LTS was composed of two parts. The main part was the vaporized refrigerant, which reflected the thermal performance of LTS. The secondary part was the additional liquid refrigerant entrained by the vaporized refrigerant; it occupies a part of the two-phase heat transfer region in condenser, reducing the heat transfer performance of the whole system [28]. The effective refrigerant mass flow rate G_r,eff was estimated with:

$${\mathrm{G}}_{\mathrm{r},\mathrm{e}\mathrm{f}\mathrm{f}}=\frac{{\mathrm{Q}}_{\mathrm{e}}}{{\mathrm{h}}_{\mathrm{l}\mathrm{g}}}$$

(3)

where h_lg is the latent heat of vaporization, kJ/kg.

Under 1800 m³/h airflow rate, almost all G_r was G_r,eff, and the maximum oscillation amplitude of G_r was only 4.8%. With the decrease in airflow rate and heating load, the G_r,eff decreased, while the oscillation amplitude of G_r increased. When airflow rate decreased to less than 1200 m³/h, the G_r,eff was significantly lower than G_r, which meant a part of condensation heat transfer region was occupied by additional liquid refrigerant. Due to the mixture flow of vapor and additional liquid refrigerant being unstable, the oscillation amplitudes of G_r were increased to more than 15%. This well-explained why the Q_e dropped dramatically under 600–1200 m³/h airflow rate. However, the rack outlet air temperature changed less when airflow rate and heating load decreased. This suggested that LTS can still ensure the thermal safety of the data center under low airflow rates and partial server heating loads.

3. Numerical Model

To further reveal the effects of operating parameters on thermal performance and cooling efficiency, a steady-state distributed-parameter model was built for the water-cooled LTS. This model comprises three modules: a micro channel evaporator, a water-cooled condenser, and connection tubes. The modules of the evaporator and condenser used the finite volume method and ε-NTU method [29] to predict the cooling capacity (Q_e), airside heat transfer coefficient (h_a,i), refrigerant mass flow rate (G_r), pressure drop, and heat transfer coefficient. Each flow channel inside the evaporator and condenser was divided into 100 segments. The refrigerant thermodynamic properties were acquired from the REFPROP [30]. The working fluids along the system cycle abide by momentum (pressure), energy (enthalpy), and mass conservation.

Figure 5 illustrates the flow chart of the LTS modeling. The refrigerant mass flow rate, enthalpy, and pressure are unknown at the beginning of the calculation, and three iterations are required throughout the simulation. The heat transfer coefficient and pressure drop of working fluids are calculated with correlations in

Table 4. Heat transfer and pressure drop correlations for simulation.

Working Fluid	Item		Correlation
Refrigerant	Evaporator single-phase	Heat transfer coefficient	Gnielinski [31]
	Condenser single-phase	Heat transfer coefficient	Muley and Manglik [32]
	Single phase	Pressure drop	Gnielinski [31]
	Evaporator two-phase	Heat transfer coefficient	Shah [33]
		Pressure drop	Friedel [34]
	Condensation two-phase	Heat transfer coefficient	Han et al. [35]
		Pressure drop	Shah [36]
	Connection tubes	Pressure drop	Coleman [37]
Airside		Heat transfer coefficient	Kim and Bullard [38]
		Pressure drop	Kim and Bullard [38]
Water side		Heat transfer coefficient	Muley and Manglik [32]

.

For the developed model, several assumptions were made as follows:

• The working fluids in the evaporator and condenser have a one-dimensional homogeneous flow;
• The airflow was evenly distributed among the micro channel exchanger;
• There was no air backflow or leakage in the rack;
• The refrigerant flow rate, temperature, and pressure were evenly distributed in every channel of the heat exchanger;
• The acceleration pressure drop of the working fluids was neglected;
• The axial heat transfer and heat dissipation of the working fluids were neglected.

3.1. Evaporator Module

In the evaporator module, the airside and refrigerant side heat transfer coefficients of each segment were calculated with the heat transfer correlations. These correlations would affect the related prediction accuracies of the evaporator module and other modules. For the tested LTS, the lowest airside Reynolds number was lower than 50 when the airflow rate was 600 m³/h. However, the existing airside heat transfer correlations [38,39,40,41] used to predict the louver fin micro channel heat exchanger are primarily suitable for the prediction of airside heat transfer coefficients in regions with Reynolds numbers exceeding 100.

According to the comparisons in our previous study [42], Kim and Bullard’s correlation [38] was finally selected from four widely used airside heat transfer correlations, which can be calculated with:

$$\mathrm{j}={\mathrm{R}\mathrm{e}}_{\mathrm{a}}^{-0.487}{\left(\frac{\mathsf{\theta }}{90}\right)}^{0.257}{\left(\frac{{\mathrm{F}}_{\mathrm{p}}}{{\mathrm{P}}_{\mathrm{l}}}\right)}^{-0.13}{\left(\frac{{\mathrm{H}}_{\mathrm{f}}}{{\mathrm{P}}_{\mathrm{l}}}\right)}^{-0.29}{\left(\frac{{\mathrm{B}}_{\mathrm{f}}}{{\mathrm{P}}_{\mathrm{l}}}\right)}^{-0.235}{\left(\frac{{\mathrm{L}}_{\mathrm{l}\mathrm{e}}}{{\mathrm{P}}_{\mathrm{l}}}\right)}^{0.68}{\left(\frac{{\mathrm{P}}_{\mathrm{f}}}{{\mathrm{P}}_{\mathrm{l}}}\right)}^{-0.279}{\left(\frac{{\mathsf{\delta }}_{\mathrm{f}}}{{\mathrm{P}}_{\mathrm{l}}}\right)}^{-0.05}$$

(4)

$${\mathrm{h}}_{\mathrm{a}}=\mathrm{j}{\mathrm{R}\mathrm{e}}_{\mathrm{a}}{\mathrm{P}\mathrm{r}}_{\mathrm{a}}^{1/3}\frac{{\mathrm{k}}_{\mathrm{a}}}{{\mathrm{P}}_{\mathrm{l}}}$$

(5)

$${\mathrm{R}\mathrm{e}}_{\mathrm{a}}=\frac{{\mathrm{V}}_{\mathrm{a}}{\mathrm{D}}_{\mathrm{h},\mathrm{a}}}{{\mathsf{\nu }}_{\mathrm{a}}}$$

(6)

where j is the Coburn factor; θ is the louver angle, °; F_p is the fin pitch, mm; P_l is the louver pitch, mm; H_f is the fin height, mm; B_f is the fin width, mm; L_le is the louver length, mm; P_f is the transverse flat tube pitch, mm; δ_f is the fin thickness, mm; h_a is the airside heat transfer coefficient, W/(m²·K); Pr_a is the Prandtl number of air, k_a is the air thermal conductivity, W/(m·K); v_a is the maximum air velocity, m/s; ν_a is the air viscosity, m²/s; D_h,a is the airside hydraulic diameter, mm; Re_a is the airside Reynolds number.

For the LTS, there are mainly three flow patterns [43]: nuclear boiling, bubbly flow, and slug flow. Because there is no driving force, the refrigerant mass flow rate of the LTS is low, especially under partial server utilization. Inside the evaporator, the flow patterns are primarily nuclear boiling and bubbly flow. Thus, the Shah correlation [33] was chosen from four widely used boiling heat transfer correlations [33,44,45,46], which had a higher prediction accuracy for these flow patterns.

3.2. Condenser Module

In the condenser (plate heat exchanger) module, the Muley and Manglik correlation [32] was used to calculate the refrigerant side single-phase heat transfer coefficient and the water side heat transfer coefficient. Han et al.’s correlation [35] was chosen to predict the refrigerant two-phase heat transfer coefficient:

$${\mathrm{h}}_{\mathrm{c}}={\mathrm{G}\mathrm{e}}_1\left(\frac{{\mathrm{k}}_{\mathrm{r},\mathrm{l}}}{{\mathrm{D}}_{\mathrm{h},\mathrm{c}}}\right){\mathrm{R}\mathrm{e}}_{\mathrm{e}\mathrm{q}}^{{\mathrm{G}}_{\mathrm{e}2}}{\mathrm{P}\mathrm{r}}^{0.4}{\mathrm{B}\mathrm{o}}^{0.3}$$

(7)

$${\mathrm{G}\mathrm{e}}_1=2.81{\left(\frac{\mathrm{b}}{{\mathrm{D}}_{\mathrm{h},\mathrm{c}}}\right)}^{-0.041}{\left(\frac{\mathsf{\pi }}{2}-{\mathsf{\beta }}_{\mathrm{c}}\right)}^{-2.83}$$

(8)

$${\mathrm{G}\mathrm{e}}_2=0.746{\left(\frac{\mathrm{b}}{{\mathrm{D}}_{\mathrm{h},\mathrm{c}}}\right)}^{-0.082}{\left(\frac{\mathsf{\pi }}{2}-{\mathsf{\beta }}_{\mathrm{c}}\right)}^{0.61}$$

(9)

where h_c is the refrigerant side heat transfer coefficient of the condenser W/(m²·K); b is the corrugation amplitude, mm; Ge₁ and Ge₂ are the correlation factors, k_r,l is the liquid refrigerant thermal conductivity, W/(m·K); D_h,c is the flow channel hydraulic diameter of the plate heat exchanger, mm; Re_eq is the equivalent Reynolds number, the boiling number Bo = q(G_r1h_lg)⁻¹.

3.3. Connection Tube Module

In the connection tube module, the vapor pipe, liquid pipe, and collecting pipe in the LTS were considered adiabatic, so the refrigerant flows inside them were taken to be isoenthalpy flow. The pressure drop was calculated with Equations (9)–(13). Coleman’s correlation [37] was used to calculate the compact pressure drop at the collecting pipe.

$$\mathsf{\Delta }{\mathrm{P}}_{\mathrm{f},\mathrm{r}}={(\mathsf{\lambda }}_{\mathrm{f}}{\mathrm{L}}_{\mathrm{l}\mathrm{p}}+\sum {\mathsf{\xi }}_{\mathrm{f}})\frac{{\mathsf{\rho }}_{\mathrm{r}}{\mathrm{u}}_{\mathrm{r}}^2}{2\mathrm{g}}$$

(10)

$$\mathsf{\Delta }{\mathrm{P}}_{\mathrm{g}}={\mathsf{\rho }}_{\mathrm{r}}\mathrm{g}\mathsf{\Delta }\mathrm{H}$$

(11)

$${\mathrm{C}}_{\mathrm{c}}=1-\frac{1-{\mathsf{\sigma }}_{\mathrm{c}}}{2.08\left(1-{\mathsf{\sigma }}_{\mathrm{c}}\right)+0.5371}$$

(12)

$$\mathsf{\Delta }{\mathrm{P}}_{\mathrm{c}}=\frac{{\left(2.26{\mathrm{G}}_{\mathrm{r}}\right)}^2}{2{\mathsf{\rho }}_{\mathrm{r}}}\left[\left(1-{\mathsf{\sigma }}_{\mathrm{c}}^2{\mathrm{C}}_{\mathrm{c}}^2\right)-2{\mathrm{C}}_{\mathrm{c}}(1-{\mathrm{C}}_{\mathrm{c}})\right]$$

(13)

$$\mathsf{\Delta }{\mathrm{P}}_{\mathrm{l}\mathrm{p}}=\mathsf{\Delta }{\mathrm{P}}_{\mathrm{f},\mathrm{r}}+\mathsf{\Delta }{\mathrm{P}}_{\mathrm{g}}+\mathsf{\Delta }{\mathrm{P}}_{\mathrm{c}}$$

(14)

where ΔP_lp is the liquid pipe pressure drop, Pa; ΔP_f,r is the frictional pressure drop, Pa; ΔP_g is the gravitational pressure drop, Pa; ΔP_c is the compact pressure drop, Pa; λ_f is the frictional resistance coefficient; ξ_f is the local resistance coefficient; u_r is the refrigerant flow velocity, m/s; ΔH is the liquid pipe height difference, mm; σ_c is the ratio of the micro channel cross-sectional area to the collector cross-sectional area; C_c is the compact coefficient.

3.4. Model Verification

In practical application, the cooling capacity is the main parameter used to evaluate LTS performance followed by the refrigerant mass flow rate and pressure. The comparison results show that the prediction values agreed well with the experimental data, including the cooling capacity, refrigerant mass flow rate, and pressure, as shown in Figure 6. The mean absolute deviations (MADs) between the experimental and prediction results were less than 5.2% according to Equation (15).

$$\mathrm{M}\mathrm{A}\mathrm{D}=\frac{1}{\mathrm{N}}\sum _{\mathrm{i}=1}^{\mathrm{N}}\frac{\left|{\mathrm{y}\left(\mathrm{i}\right)}_{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}-{\mathrm{y}\left(\mathrm{i}\right)}_{\mathrm{e}\mathrm{x}\mathrm{p}}\right|}{{\mathrm{y}\left(\mathrm{i}\right)}_{\mathrm{e}\mathrm{x}\mathrm{p}}}$$

(15)

where ${\mathrm{y}\left(\mathrm{i}\right)}_{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}$ is the prediction result of point i, ${\mathrm{y}\left(\mathrm{i}\right)}_{\mathrm{e}\mathrm{x}\mathrm{p}}$ is the experimental data of point i, and N is the number of total data points.

4. Analysis and Discussion

4.1. Simulation Conditions

In an actual data center, the airflow distribution is a major impact factor for thermal safety. As shown in Figure 1, servers suck in cold air from the inlet aisle and exhaust hot air into the evaporator of the LTS inside the rack. The space inside a rack can be regarded as a hot aisle, the space outside the racks can be regarded as a cold aisle, and the hot aisle is enclosed by the racks’ walls. Therefore, the average outlet air temperature of the evaporator was assumed equal to the rack’s average outlet air temperature. In order to regulate the airflow, the rack seals well in applications, so the heat loss of the rack can be neglected in simulations.

During the simulation process, the total servers’ heating load of a rack under different server utilization can be calculated with Equations (16)–(18) [23]. A schematic diagram of a simulated LTS integrated with servers in a rack is shown in Figure 7. The server exhaust air temperature T_s,out can be regarded as the evaporator inlet air temperature T_a,in, which was calculated with Equation (19). The rack’s average outlet air temperature was calculated with Equation (20).

$${\mathsf{\alpha }}_{\mathrm{i}}=0.7848+0.02722{\mathrm{T}}_{\mathrm{s},\mathrm{i}\mathrm{n}}-0.001353{\mathrm{T}}_{\mathrm{s},\mathrm{i}\mathrm{n}}^2+0.00002381{\mathrm{T}}_{\mathrm{s},\mathrm{i}\mathrm{n}}^3$$

(16)

$${\mathsf{\eta }}_{\mathrm{h}}=0.01{\mathsf{\alpha }}_{\mathrm{i}}{\mathrm{u}}_{\mathrm{s}}\left(27.39+0.9027{\mathrm{u}}_{\mathrm{s}}-0.0017462704{\mathrm{u}}_{\mathrm{s}}^2\right)$$

(17)

$${\mathrm{P}}_{\mathrm{s},\mathrm{i}}={\mathsf{\eta }}_{\mathrm{h}}{\mathrm{P}}_{\mathrm{s},\mathrm{f}\mathrm{u}\mathrm{l}\mathrm{l}}$$

(18)

$${\mathrm{T}}_{\mathrm{s},\mathrm{o}\mathrm{u}\mathrm{t}}={\mathrm{T}}_{\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{k},\mathrm{i}\mathrm{n}}+\frac{{\mathrm{P}}_{\mathrm{s}}}{{{\mathrm{V}}_{\mathrm{a}}\mathsf{\rho }}_{\mathrm{a}}{\mathrm{C}\mathrm{p}}_{\mathrm{a}}}$$

(19)

$${\mathrm{T}}_{\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{k},\mathrm{o}\mathrm{u}\mathrm{t}}={\mathrm{T}}_{\mathrm{s},\mathrm{o}\mathrm{u}\mathrm{t}}-\frac{{\mathrm{Q}}_{\mathrm{e}}}{{{\mathrm{V}}_{\mathrm{a}}\mathsf{\rho }}_{\mathrm{a}}{\mathrm{C}\mathrm{p}}_{\mathrm{a}}}$$

(20)

where u_s is the server utilization, %; α_i is the heat generation increase factor; P_s is the total server heating load of a rack, W; P_s,full is the total rated heating load of servers in a rack, W; η_h is the server heat generation rate; ρ_a is the density of air, kg/m³; Cp_a is the specific heat capacity of air; kJ/(kg·K). In this study, P_s,full was assumed as 7200 W; T_rack,in is the rack inlet air temperature, which can be regarded as the indoor air temperature, °C.

Since the exhaust air of each rack will be mixed in the exhaust aisle, the influence of a single rack on the indoor air temperature of the whole data center is ignored, and the indoor air temperature is regarded as constant. According to the Code for Design of Data Centers (GB50174-2017) [47], the reference ranges of indoor air for dry, relative humidity, and dew-point temperatures are 18–27 °C, <60%, and 5.5–15 °C, respectively. In order to improve the operating efficiency of the cooling system, the indoor air temperature is usually set close to the upper limit of the reference value. Thus, the indoor air dry and wet bulb temperature was set as 26 °C and 14 °C, respectively. The details of the simulation conditions are shown in

Table 5. Simulation conditions.

	Range
Indoor air dry/wet bulb temperature (°C)	26/14
Server utilization us (%)	3–100
Server-rated heating load Ps,full (W)	7200
Supply chilled water temperature Tcw (°C)	10, 12, 14, 16, 18
Chilled water flow rate Vw (m3/h)	0.5, 0.9, 1.3, 1.71, 2.1
Airflow rate Va (m3/h)	600, 900, 1200, 1500, 1800

.

4.2. Effect of Airside Operating Parameter on Thermal Performance

Figure 8 shows the thermal performance under different airflow rates and server utilizations. The system temperature difference ΔT_sys under 1800 m³/h and 5% server utilization was 8.6 °C lower than that under 600 m³/h and 6.5% server utilization, but the cooling capacity was more than 45.3% higher (Figure 8a,b). This is because the airside thermal resistance R_a was the dominant part of the total system thermal resistance R_sys, as shown in Figure 8c (the calculation equations of the thermal resistances are shown in Figure 7). When the airflow rate decreased from 1800 m³/h to 600 m³/h, the R_a increased by 157%, resulting in a 122% increase in the R_sys. Under 1800 m³/h, it was also found that the decrements of the Q_e caused by server utilization variation were nonlinear (Figure 8b).

Figure 9 shows the rack’s average outlet air temperature and the excessive cooling capacity under different airflow rates and server utilizations. The excessive cooling capacity Q_e,ex reflects the cooling capacity supplied by the LTS over the server cooling demand in the rack, which is calculated with:

$${\mathrm{Q}}_{\mathrm{e},\mathrm{e}\mathrm{x}}={\mathrm{Q}}_{\mathrm{e}}-{\mathrm{P}}_{\mathrm{s}}$$

(21)

Under 5–12.5% server utilization, the Q_e,ex reduced by about 52% with the airflow rate, which decreased from 1800 m³/h to 600 m³/h. As shown in Figure 9, the T_rack,out was below the upper limit reference temperature T_ref,up of GB50174-2017 [47], but the T_s,out (T_a,in) reached 40 °C when the server utilization was 12.5% under a 600 m³/h airflow rate (Figure 8a). According to the lectures [25,26], the chip-to-air temperature differences need to be larger than 45 °C under maximum heating loads. Thus, the server exhaust air temperature T_s,out should not exceed 40 °C to keep a chip safe. If the control system further reduces the airflow rate, it increases the risk of server overheating even at a low server utilization. Therefore, the control system should reduce the airflow rate according to the upper limit of the server exhaust air temperature to reduce the excessive cooling capacity under partial server utilization. Keeping the T_s,out lower than 40 °C, the Q_e,ex was decreased by 14.5–52.1% with the decrease in airflow rate under 5–56.5% server utilization. It can be found that the lower the server utilization, the greater the effects of decreasing the airflow rate on reducing the Q_e,ex (Figure 9).

In summary, the control system can adjust the airflow rate of the LTS according to the upper limit of the server exhaust air temperature to reduce the excessive cooling capacity and help with energy saving. However, it is still difficult to eliminate the cooling capacity oversupply phenomenon by only adjusting the airflow rate under partial sever utilization (Figure 9). It is necessary to further investigate the effects of the water side operating parameters on the thermal performance to reduce the excessive cooling capacity.

4.3. Effects of Water Side Operating Parameters on Thermal Performance

The energy consumption of chillers, which is the main component of the total energy consumption, is effectively reduced by increasing the temperature of the supply chilled water temperature T_cw. Therefore, the control system can reduce the supply of chilled water temperature to reduce the excessive cooling capacity and the energy consumption of the chillers.

Figure 10 shows the thermal performance under different supply chilled water temperatures and server utilizations. Due to the decrease in the ΔT_sys, the cooling capacity decreased when the T_cw increased. Under 5% server utilization, the Q_e was 6559 W under 10 °C T_cw, and it decreased by 40.4% when the T_cw increased to 18 °C (Figure 10a). It was also found that even the ΔT_sys was similar, and the higher the server utilization, the lower the decrement of the Q_e. When the T_cw increased from 10 °C to 18 °C under 100% server utilization, the decrement of Q_e was only 15.3%. The influence of the T_cw on the thermal resistance is shown in Figure 10b. The T_cw had less effect on the thermal resistance of the condenser R_c, but the R_sys decreased with the T_cw increasing. The working state of the LTS can be judged with the overheating temperature ΔT_e,r, that is, the refrigerant outlet temperature minus the refrigerant outlet saturation temperature in the evaporator. When the evaporator is working properly without overheating, the refrigerant inlet saturation temperature should be slightly higher than the outlet temperature due to the pressure drop, and the ΔT_e,r ≤ 0. According to the Heat Pipe Qualification Requirements, if the ΔT_e,r > 5 °C, the evaporator is considered dried out [48]. Dried out means the absence of a sufficient liquid refrigerant and only vapor-phase heat transfer at the overheating region. It has been pointed out that the optimal refrigerant filling ratio of the LTS should increase with the ΔT_sys increasing to prevent the evaporator from overheating when other operating parameters are constant [16]. In this study, the optimal filling ratio was obtained from the experiments carried out under a 1800 m³/h airflow rate, 35 °C T_a,in, 12 °C T_cw, and ΔT_sys of 21.2 °C. Since the indoor air temperature was set close to the upper limit of the reference value in the simulation cases, when the server utilization reached 56.5%, the T_a,in and ΔT_sys were about 35 °C and 21.2 °C, respectively (Figure 10b). Under 100% server utilization and 10°C T_cw, the ΔT_sys and ΔT_e,r were 26.4 °C and 10.1 °C, respectively, which showed that the LTS was not operating at the optimal refrigerant filling ratio, and the evaporator was seriously dried out. There was a large vapor-phase region at the top of the evaporator, resulting in an increase in the R_e. When the T_cw increased from 10 °C to 18 °C, the evaporation rate of the refrigerant decreased with the ΔT_sys and Q_e decreasing, and the LTS operated again at the optimal refrigerant filling ratio. More liquid refrigerants re-entered the overheating region, thus decreasing the ΔT_e,r and R_e. This explained why the decrements of the Q_e decreased with the server utilization increasing when the T_cw increased.

Figure 11 shows the rack’s average outlet air temperature and the excessive cooling capacity under different supply chilled water temperatures and server utilizations. When the T_cw increased from 10 °C to 18 °C, the Q_e,ex dropped by more than 62.1%, which indicates that the cooling efficiency improved. The lower the server utilization, the greater the effects of decreasing the supply chilled water temperature on reducing the excessive cooling capacity, and the T_rack,out was still lower than the upper limit reference temperature under 5–100% server utilization. It is worth noting that the Q_e,ex < 0 when the T_cw increased to 18 °C under 95–100% server utilization. If the supply chilled water temperature is adjusted too high, the cooling capacity is less than the rack’s heat load; thus, it will cause a thermal risk to the data center. Therefore, the water flow rate needed to be adjusted according to the upper limit of the T_rack,out to maintain the thermal security of the data center.

In addition, the ΔT_sys also can be reduced by decreasing the chilled water flow rates V_water, as shown in Figure 12a. Under 5–100% server utilization, when the V_water decreased from 2.1 m³/h to 0.5 m³/h, the ΔT_sys decreased by 14.8–15.5%, but the Q_e decreased by more than 19.2%. This is because the water side heat transfer coefficient decreased as the water flow rate decreased. Under 2.1 m³/h V_water, the R_c remained in the range of 0.00037–0.00041 °C/W, and they increased by more than 134.7% when V_water decreased by 1.6 m³/h (Figure 12b). In addition, the ΔT_e,r decreased when V_water decreased, which caused the increments of the R_sys to decrease with the increase in server utilization when the V_water decreased.

Figure 13 shows the rack’s average outlet air temperature under different water flow rates and server utilizations. When the V_water decreased from 2.1 m³/h to 0.9 m³/h, the Q_e,ex reduced by more than 23.8% under partial server utilization, while the T_rack,out was lower than the upper limit reference temperature. It is worth noting that the T_rack,out exceeded the T_ref,up under 0.5 m³/h V_water and 95–100% server utilization, which will affect the thermal safety of the data center. In order to ensure thermal security, the water flow rate should be adjusted according to the upper limit of the T_rack,out.

In conclusion, the control system can adjust the water side operating parameters according to the upper limit of the rack’s average outlet air temperature to reduce the excessive cooling capacity of the LTS, which is difficult to deal with by only adjusting the airflow rate.

5. Conclusions

The operating parameters of the water-cooled LTS were designed according to the cooling demands of the rack under a full heating load, but most racks are operated under a low heating load for a long time, thus resulting in energy waste. In order to improve the cooling efficiency under partial server utilization while ensuring thermal safety, a water-cooled LTS was experimentally investigated under different airflow rates and heating loads. Then, a related steady-state distributed-parameter model was developed; the mean absolute deviations between the experimental and prediction results were less than 5.2%. Based on the developed model, the effects of the operating parameters (including the airflow rate, supply chilled water temperature, and chilled water flow rate) on the thermal safety and cooling efficiency were numerically investigated. The main results are concluded as follows:

• When the airflow rate and heating load decreased, the effective refrigerant mass flow rate was significantly lower than the measured refrigerant mass flow rate. The additional liquid refrigerant reduced the heat transfer performance of the LTS and aggravated a drop in cooling capacity. However, the LTS can still ensure the thermal safety of the data center under low airflow rates and partial server heating loads.
• The airflow rate of the LTS could be adjusted according to the upper limit of the server exhaust air temperature to accurately reduce the excessive cooling capacity under partial server utilization. The excessive cooling capacity was reduced by 14.5–52.1% with a decrease in the airflow rate under 5–56.5% server utilizations.
• Under partial server utilization, the water side operating parameters could be adjusted according to the upper limit of the rack’s average outlet air temperature to reduce the excessive cooling capacity. Increasing the supply chilled water temperature from 10 °C to 18 °C or decreasing the water flow rate from 2.1 m³/h to 0.9 m³/h reduced more than 23.8% of the excessive cooling capacity.

Further Study

• In this study, several suggestions for operating parameters optimization were put forward to reduce the excessive cooling capacity of the LTS and ensure thermal security of the data center. However, the specific energy-saving effects of these suggestions have not been studied, and further research is required.
• To reduce the excessive cooling capacity and ensure thermal safety, multiple operating parameters need to be adjusted quickly and precisely by the control system, which involves a multi-objective multi-parameter optimization problem. Therefore, the use of artificial intelligence or machine learning methods to solve this problem quickly deserves further research.