The Numerical Simulation of the Airflow Distribution and Energy Efficiency in Data Centers with Three Types of Aisle Layout

Abstract

The energy consumption of fast-growing data centers is drawing attentions from not only energy organizations and institutions all over the world, but also charity groups, such as Greenpeace, and research shows that the power consumption of air conditioning makes up a large proportion of the electricity cost in data centers. Therefore, more detailed investigations of air conditioning power consumption are warranted. Three types of airflow distributions with different aisle layouts (the open aisle, the closed cold aisle, and the closed hot aisle) were investigated with Computational Fluid Dynamics (CFD) methods in a typical data center of four rows of racks in this study. To evaluate the results of thermal and bypass phenomenon, the temperature increase index (β) and the energy utilization index (η_r) were used. The simulations show that there is a better trend of the β index and η_r index both closed cold aisle and closed hot aisle compared with free open aisle. Especially with high air flow rate, the β index decreases and the η_r index increases considerably. Moreover, the results prove the closed aisles (both closed cold aisle and closed hot aisle) can not only significantly improve the airflow distribution, but also reduce the mixture of cold and heat flow, and therefore improve energy efficiency. In addition, it proves the design of the closed aisles can meet the increasing density of installations and our simulation method could evaluate the cooling capacity easily.

1. Introduction

A huge amount of energy is consumed by data centers every year to meet large-scale integrated circuit computing, data exchanges, information communication, and data-storage [1]. By the end of 2017, in China, the annual electricity consumption of data center is more than 2% of their total social electricity consumption [2]. The whole electricity consumption of China’s data center reached 120 billion kWh, accounting for 1.8% of the total electricity consumption of the whole society [3]. In addition, data centers in U.S. will reach roughly 140 billion kWh annually by 2020, which is the equivalent of 50 large (500 megawatt) power plants [4].

In recent years, the proportion of the energy consumed by data centers has increased rapidly among the total energy uses. Researchers are now paying more attentions on the improvement of energy saving through adjusting the air conditioning in computer rooms, and focusing on the improvement of the thermal environment and energy efficiency. Many methods were proposed, such as hot and cold closed aisle, efficient floor tiles, optimization of temperature set points, heat and humidification controls, airflow thermal performance, and airflow management [5,6,7,8]. Most of all, airflow distribution and management are important factors for the design of cooling systems, and the reasonable layout not only helps to reduce data center local hot spots, but also improves the overall thermal environment and the utilization rate of cooling capacity. In addition, the analysis and evaluation of air distribution of the data center under different forms can help to optimize the suitable form of airflow arrangement, which is a green and low-carbon development path for the current data center.

Several researchers have studied the thermal management of data centers [8,9,10,11,12]. It is critical to design the air distribution to maintain a high-quality thermal environment in data centers and the optimization of airflow arrangement.

In order to reduce the inhomogeneity of airflow and inlet temperature in data centers, the structure of data centers have been studied, such as cold-closed aisles, partially closed aisles, and hot-closed aisles. For experimental techniques, Sundaralingam et al. [13] evaluated the characteristics of three types of cold aisle framework including open aisle, partially opened aisle, and fully closed aisle using experimental methods. Wang et al. [14] studied the influence of cold aisle partition on air rate and velocity. The exhaust air velocity can meet the demand of national standard, and the air velocity across IT devices is larger in the upper zones to ensure enough cooling capacity under the conditions of cold aisle partition. To achieve the criterion of no hot spots, different designs were suggested with a low pressure drop across servers and small velocity of the air in the cold aisle in data centers [15]. The thermal effects of implementing a cold aisle containment system on an open hot aisle/cold aisle arrangement were quantified under three different cabinet thermal load conditions in [16]. A case study showed that the use of cold channel sealing system can save about 22% of the cooling energy compared with the standard hot channel/cold channel layout. Three types of airflow distribution were tested by Nada et al. [17,18], including free open, semi-enclosed (two sides), and fully-enclosed cold aisle. Enclosed cold aisles improve the cooling efficiency of data centers, especially at high electricity density. For a semi-enclosed cold aisle with a raised floor, the recirculation and bypass of airflow around racks were reduced, and the performance of the cooling of data centers for different configurations were improved with about 18%–20% reduction of RTI and SHI and about 8%–18% increase of RCIHI [19]. Yuan et al. [20] presented the airflow profiles within the enclosed cold aisle. In racks, the design of the enclosed cold aisle could decrease the exhaust air temperatures, and increase the heat transfer and distribute the exhaust air, and the cooling efficiency was improved. Wang et al. [21] presented the effects of various enclosed frameworks on the energy efficiency of a data center. Framework with enclosing cold aisle shows a better distribution of temperature and the best overall performance with 99% RCI.

In order to improve the airflow/temperature distribution into racks, Wang et al. [22] presented a drawer-type rack which can increase the effective hot aisle space and reduce the cold aisle space. For traditional data centers, it is proposed that both hot air and cold air containment can improve the predictability and efficiency of cooling system [23]. Compared to cold-aisle containment, the choice of hot aisle containment can save 43% of energy cost annually, and reduce 15% of PUE annually. Pervila and Kangasharju [24] described the experimental results of the cooling technique of cold aisle containment, and showed the separation of airflow influences the temperatures of servers and could reduce the cooling energy consumption with roughly a fifth.

Existing works [5,8,25,26,27,28,29] have validated that the simulation method is the most common effective method to study air-flow distribution and management of data centers. CFD is a commonly-used software for the simulation of fluid dynamics, analyzing and solving problems that involve fluid flows, such as airflow distribution, temperature distribution, and calculations of thermal energy for specific environments and structures. Abdelmaksoud et al. [30] proposed an improved momentum source model for these perforated surfaces, which can improve the CFD results significantly. Experiments taken out by Fulpagare et al. [31] showed that the transient effect of racks are significant and cannot be ignored using CFD model of three racks in a raised floor data center with cold and hot aisle containment. Ham and Jeong [32] compared the simulation results of the thermal performance for two kinds of uncontained and contained aisles using CFD, which showed better performance of thermal management in both frameworks with enclosed aisles, and a greater energy saving with the contained framework. Lv et al. [33] analyzed and evaluated temperature and velocity distribution in different aisle types through CFD simulation method. The environmental temperatures of fully-enclosed cold aisle are greater than that of fully-enclosed hot aisle. Fully-enclosed cold or hot aisle can decrease the short pass of cold air and improve the cooling efficiency. Schaadt et al. [34] studied the effect of full containment, half containment, and no containment layout for hot aisle and cold aisle on the optimal operating parameters in a data center. Nada et al. [35] simulated the cooling efficiency of an enclosed cold aisle. Zhang et al. [36] proposed and simulated three optimization methods of airflow framework using CFD software, and suggested the enclosed cold aisle not only reduces short pass airflow, but also improves the thermal efficiency. In addition, the heat of the air in both sides of the hot aisle can quickly return and improve greatly the cooling performance, resulting in the 0.5~5 °C decrease in temperature. Demetriou et al. [37] showed a bypass branch installed within the CRAC itself would be a preferred approach.

Although there are many experts who have focused on the influencing factors of energy consumption in data centers, research on the cold and hot aisle closed conditions needs to be further improved, especially taking a representative data center as a paradigm to evaluate the impact between uniformity of airflow and energy cost. Moreover, the effect of various parameters on the airflow model of data centers is comprehensive, and there is a lack of detailed discussion and comparison under different conditions. Therefore, it is necessary to fully and clearly interpret and evaluate the three cases of airflow using mathematical simulation method with a typical model.

Three cases of airflow distribution for a data center with four row racks are presented in this study. The airflow arrangement is tested and analyzed. The comparison of these three cases is discussed on performance indexes, including the β index and the η_r index. The distributions of the airflow framework in a typical data center are studied, and the temperature distribution of environment and racks is presented.

2. Methodology

In this paper, a typical data center with a cooling air supplying system is chosen to be characterized as the simulation model, which is used to evaluate the airflow distribution under different cases. The thermal distribution characteristics are provided, and the better operation conditions are given.

2.1. The Model Framework of Data Center

Figure 1 shows a 3D model framework of the data center with free open aisle, which is used to simulate the airflow and thermal distribution under various operating conditions using Fluent software. Three types of aisle including free open aisle, closed cold aisle, and closed hot aisle are illustrated in Figure 2.

In our research, the 3D model of the analyzed data center is built according to its actual size with physical dimensions 12. 0 m × 10.0 m × 3.2 m (See

Table 1. The model framework size and device parameters.

	Length/m	Width/m	Height/m
Simulation of data center	12.0	10.00	3.20
Chassis	0.683	0.448	0.0875
Air conditioner	0.765	0.560	1.940

for the dimension parameters). According to the specific size, a corresponding mesh density is established with the Gambit software, which has two rows of standard 2U racks arranged in both sides of the aisle. The cold air is supplied from raised floor plenum through vent tiles by two air conditioners, with the flow rate 14.4–21.6 m³/s and exhausted hot air passing through racks returns along the hot aisles to the air conditioners. The data center contains four rows of racks (No. A, B, C, and D) with Ten R520 2U racks (No. Rack 1, Rack 2, Rack 3, Rack 4, …, Rack 10) are arranged for each, and every rack is equipped with 5 chassis (e.g., marked from bottom to top as A1, A2, A3, A4, and A5 for A row). The servers are Lenovo R520 GT containing two processors each. The total power consumption of the whole data center is 48.0 kW at idle state and 72.0 kW at full utilization state. The data center also contains two unit of computer room air conditioning, and this computer room air conditioning chilled air to 15 °C, and then into the raised floor at a flow rate.

In the process of simulation, the model room is isolated, and the influences of heat transfer outside the room are ignored. Adiabatic wall conditions are chosen for the vertical walls, the ceiling, and the floor according to the real situations. The chassis of the heat source is considered as a black box model, which allows no airflow in the servers. However, the loading chassis generates the exhaust thermal amount for exchanging with the airflow through the heat phase interface. It is proved the heat model processing method is feasible and available from results in references [38,39,40].

2.2. Mathematical Model

For the investigation of the design factors in a data center of small scale, commercial Fluent 6.3 software was used as the simulation tool to perform the airflow distribution simulation [5].

Most researchers consider the flow regime in room-level data center belong to the turbulent mix. The detailed discussion can be found in literatures [25,39,41]. Compared to the k-ω, SST, RSM, and RNG k-ε models, the k-ε turbulent model is used to simulate the flow characteristic, which has better performance and results. The relevant equations have been listed as follows.

Some assumed conditions for simulating process are presented in [42,43,44]: the air is regarded as incompressible fluid with uniform heat transfer interface, ignoring radiation influence, air leakage etc.

The continuity equation is given as follows:

$$\frac{\partial \rho }{\partial t}+\nabla ·(\rho \overrightarrow{u})=0$$

(1)

where u is the velocity of phase; ρ is density, kg/m³.

The momentum equation is listed as following:

$$\frac{\partial \overrightarrow{u}}{\partial t}+(\overrightarrow{u}·\nabla )\overrightarrow{u}=-\frac{1}{\rho }\nabla P+v{\nabla }^2\overrightarrow{u}+\overrightarrow{g}-\frac{1}{\rho }\nabla {\tau }_t$$

(2)

where g stands for vector of gravitational acceleration, m/s² [38]; v is the kinematic viscosity, N·s/m²; P is the pressure, N/m²; τ_t is the xyz component of the viscous stress tensor.

The energy equation is given as follows:

$$\rho c_p\left[\frac{\partial T}{\partial t}+\left(\overrightarrow{u}·\nabla \right)T\right]=k{\nabla }^{2}T+S$$

(3)

where T is the temperature, K; c_p is the specific capacity, J/g.K; k is the thermal transfer coefficient, m²/s; S is the viscous dissipation rate.

In order to solve the Reynolds stress in the turbulent momentum equations, the standard k-ε model is chosen for simulating the liquid phase. Two of these variants are available in Fluent software. The detailed turbulence model can be found in Reference [5], such as the equations of turbulence models, model constants, boundary conditions, initial conditions, and iteration scheme.

2.3. Calculation Method

Most of thermal metrics were proposed to assess the airflow characteristics and thermal efficiency [42,43], but a few additional summary metrics can instantly indicate whether a facility is working well from a thermal/airflow point of view [10,45]. In this study, two kinds of assessment index were used, namely the β index and the energy utilization index η_r, which refer to the literature [5] for details.

The β Index is proposed to evaluate the temperature rising in local rack [46] and this index could be defined as:

$$\mathsf{\beta }=\frac{T_{in}-T_{ref}}{T_{out}-T_{in}}$$

(4)

where T_in is the mean temperature in the inlet, T_out is the mean temperature in the outlet, T_ref is the reference cooling temperature. The β index ranges are between 0 and 1. When the β index value is 0, it illustrates no recirculation of airflow. While the β index value is above 1, it means the phenomenon of self-heating.

In order to meet the utilization energy efficiency of airflow, energy utilization index is corrected and its expression is [47]:

$${\mathsf{\eta }}_{\mathrm{r}}=\frac{T_{out}-T_{ref}}{\frac{T_{out}+T_{in}}{2}-T_{ref}}$$

(5)

where T_out is the air temperature in the outlet, T_in is the air temperature in the inlet and T_ref is the air temperature of the cooling in the rack. In Equation (5), the low η_r shows a bad airflow organization, and has a hot air self-circulation between racks.

Therefore, the β index and η_r index are used to evaluate the airflow distribution and energy efficiency.

2.4. Grid Independent and Model Validation

The mesh generation of geometric models has a great influence on the numerical simulation results [41]. With the increase of the number of grids, not only the performance requirements of computer can be improved, but also the computation time also increase. Therefore, in order to improve the accuracy and efficiency of numerical simulation, finding a suitable mesh generation is the foundation of numerical simulation.

A hybrid mesh technique is used in this study. The grid independence was checked using unstructured grids of hexes and quads with 110,560, 218,144, 327,424, 430,941, and 539,425 computational nodes. The difference of β and η_r in the Rack 1 and Rack 2 with the different grid sizes has a better accuracy within 8% in

Table 2. The influence of grid size on performance indexes.

Mesh (Nodes)	Temperature of Wind °C	Rack 1		Rack 2
		β	ηr	β	ηr
110,560	17.83	0.17	1.75	0.20	1.71
218,144	17.99	0.24	1.60	0.27	1.65
327,424	18.01	0.30	1.61	0.30	1.64
430,941	17.95	0.31	1.62	0.31	1.64
539,435	17.95	0.32	1.60	0.29	1.63

. According to the simulation procession, a typical grid with 430,941 nodes including around 405,045 hexes and 86,776 quads, in which the minimum volume 4.252 × 10⁻⁷ m³ and the maximum volume 2.34 × 10⁻³ m³ was used in the present work for the data center.

In order to verify the accuracy of grid mesh and CFD model, the exit temperature of Rack1-Rack10 in channel A was measured in an open aisle. In addition, the average simulation temperature at the outlet of A1–A5 from Rack 1 to Rack 10 using on-line thermocouple method (the measuring point is 0.08 m away from the center of each server) was compared according to the simulation results with 430,941 meshing, as shown in Figure 3. From the results, there is a maximum deviation of 0.92 °C temperature difference between the simulation and the measurements, which proved that the model and simulation method are feasible.

3. Results and Discussion

3.1. The Characteristic of Flow and Temperature Distribution

The most important thing in a data center is to ensure the safe and stable operation of equipment [48]. With an open aisle, because airflow causes non-uniformity in air temperature distribution around racks, airflow from the back to the front through the rack is expected. Therefore, to understand the actual conditions of the airflow distribution, heat and temperature distribution of the cabinet under different working conditions, it is necessary to simulate the temperature cloud and the average temperature results of the inlet and outlet surface of the central equipment according to the results of the numerical simulation. The simulation temperature result of free open aisle at XZ plane and YZ plane is shown in Figure 4.

From Figure 4, at the low flow rate, the hot airflow is mainly concentrated in the hot aisle, and the back mixing of airflow is less. The inlet temperature of racks is basically cold air (the blue region around the server is larger). At the high flow rate, the heat flow temperature has a diffusion area, and the mixing temperature is more serious than that of the cold current, which results in the inlet temperature has a tendency to increase.

In order to diminish the heterogeneity of airflow and inlet temperature, some new techniques have been widely used in data centers [49]. Two solutions are used to solve the distribution of airflow and temperature, which are the closed cold aisle and closed hot aisle. The simulation temperature result of the closed cold aisle at XZ plane and YZ plane is shown in Figure 5. From Figure 5, it shows that either at the low flow rate or the high flow rate, the cold airflow mainly concentrated in the cold aisle, and the back mixing of airflow is almost none. The inlet temperature of racks is basically cold air (the blue region around the server), and the heat flow is completely separated from the rack, and the mixture of cold and heat flows hardly occurs.

The other methods to solve the problem is the enclosed air pass or the separated air pass on the data center. Figure 6 shows the temperature distribution and flow vector for the three type of aisles. According to the results of the calculated data, the outlet temperature of the servers is not more than 50 °C, and the temperature range is acceptable. Through the analysis of the simulated temperature results of each row, it can be found that the temperature of the lowest server at each rack is the highest, closed to 50 °C, other severs almost have a uniform temperature distribution. It also shows that the servers at the closed hot aisle have a high temperature than other type aisles.

For example, Figure 6B,C, the airflow line drawing from the numerical simulation results can clearly show that the airflow distribution is reasonable. The cold airflow from the floor grid comes into the cabinet to take away the heat from severs at rack, then back from the upper space of the room, and finally back to the air conditioning unit. The flow pattern of the airflow is uniform and the temperature distribution is reasonable. The streamline of the cold airflow is clear, and there is no mismatch between cold and hot air.

3.2. The Energy Efficiency

Non-uniform airflow in open aisles can be reflected in energy efficiency problems. Figure 7 shows the evaluation of energy efficiency, at the low flow rate, the β index is around 0.5, which indicates there is a little air mixture between hot flow and cold flow. Meanwhile, the η_r is between 1.4 and 1.6. Therefore, there is a minimum β Index and a maximum η_r index, namely there is a better and more uniform distribution for airflow and temperature. At the high flow rate, the β value is above 1, which means the circulation of self-heating; the η_r index is closed to 1.0, which shows the circulation of the exhaust air among racks and reflects the local hot spot at racks.

For the closed aisle, the mixing of hot and cold air tends to decrease. Figure 8 shows the estimation of airflow distribution and energy efficiency. Either at low flow rate or high flow rate, the β index and the η_r index have the same tendency. But, at the high flow rate, the two indexes of the middle rack are not very good, and the cold and hot airflow obviously has a mixed phenomenon.

According to the numerical simulation results, the maximum temperature of each severs in the closed cold aisle is given in Figure 9.

First of all, from the overall analysis: The air temperature of each rack was basically stable, the average temperature of the air exit surface of the rack was controlled in the range of 25.0–42 °C. According to the engineering experience, the temperature range was basically in line with the requirements of the temperature control in the industry and the limit requirement of the temperature of 45 °C of the server running equipment. From the analysis and comparison of the spatial dimension of data center, it is found that the average air outlet temperature on two side of rack and the low sever on the rack was relatively high. Especially under the condition of A1 and C1 at row 1 and A1 and C1 at row 10, the local temperature of the location environment was up to 42 °C indicating that this position was a local hot spot. However, compared with free open aisle, the β index and the η_r index of the closed cold aisle show a better trend as seen in Figure 10. Especially for high flow rate, the β index decrease and the η_r index increased significantly, which indicate the distribution of airflow and temperature improved significantly. At the meantime, this proves the closed cold aisle can significantly improve the airflow distribution, reduce the mixture of cold and heat flow, and improve energy efficiency.

In order to evaluate the airflow distribution in date centers, the rack selection at row 1 can fully reflect the flow chart of the machine room. The β index and the η_r index of the closed hot aisle show a better trend as shown in Figure 11. Especially for the high flow rate, the β index decreases and the η_r index increases significantly, which indicates the airflow distribution and temperature distribution improves significantly. In addition, this proves the closed cold aisle can significantly improve the airflow distribution, reduce the mixture of cold and heat flow, and improve energy efficiency. As a result, the design of the closed aisles can successfully support the higher density of servers with more economical application of cold energy.

4. Conclusions

It is particularly important to improve energy efficiency in data centers with the ever-growing volume of big data. Using CFD simulation software, simulation investigations of a confined data room with high density racks have been presented for two kinds of closed aisle and operating conditions in this paper. According to the thermal and bypass phenomena, two kinds of evaluation indexes were used: the β index and the η_r index.

For the free open aisle, the temperature distribution at the low flow rate is more than one at the high flow rate. This point can also be seen from the airflow distribution and efficiency analysis of energy. At the low flow rate, there is some mixture of hot and cold airflow with a low β index and a high η_r index, which shows a better airflow distribution and a more even air distribution. In addition, at the high flow rate, the β value is above 1, which indicates circulation of air and self-heating; the η_r index is close to 1.0, which shows the circulation of the exhaust air among racks and reflects the local hot spot at racks.

For the closed cold aisle, the heat flow is completely separated from the rack, and the mixture of cold and heat flows hardly occurs from temperature cloud, which presented a better airflow distribution and a more uniform air distribution than the free open aisle. The β index and the η_r index have the same tendency with a better value. Moreover, the β index and the η_r index of the closed cold aisle show a better trend, and especially for the high flow rate, the β index decreases and the energy utilization index increases significantly, which indicates the distribution of airflow and temperature improves greatly.

For the closed hot aisle, the outlet temperature of the servers is not more than 50 °C, and almost have a homogeneous temperature distribution. Under the high flow rate, the β index decreases and the η_r index increases meaningfully, resulting in the distribution of the airflow and temperature improves greatly. In addition, the simulation results show the closed hot aisle can considerably improve the airflow distribution, reduce the mixture of cold and heat flow, and improve energy efficiency.

As a result, the closed aisle (closed cold aisle and closed hot aisle) is a better style to improve the airflow distribution, reduce the mixture of cold and heat flow, and increase energy efficiency. In addition, it proves the design of the closed aisles can support the higher density of servers with more economical application of cold energy.