Simulation of Thermal Distribution and Airflow for Efficient Energy Consumption in a Small Data Centers

Data centers have become ubiquitous in the last few years in an attempt to keep pace with the processing and storage needs of the Internet and cloud computing. The steady growth in the heat densities of IT servers leads to a rise in the energy needed to cool them, and constitutes approximately 40% of the power consumed by data centers. However, many data centers feature redundant air conditioning systems that contribute to inefficient air distribution, which significantly increases energy consumption. This remains an insufficiently explored problem. In this paper, a typical, small data center with tiles for an air supply system with a raised floor is used. We use a fluent (Computational Fluid Dynamics, CFD) to simulate thermal distribution and airflow, and investigate the optimal conditions of air distribution to save energy. The effects of the airflow outlet angle along the tile, the cooling temperature and the rate of airflow on the beta index as well as the energy utilization index are discussed, and the optimal conditions are obtained. The reasonable airflow distribution achieved using 3D CFD calculations and the parameter settings provided in this paper can help reduce the energy consumption of data centers by improving the efficiency of the air conditioning.


Introduction
With the rapid development of the Internet, the mobile Internet, cloud computing and big data applications, the pace of construction of data centers has accelerated worldwide.At the same time, environmental and energy-related problems arising from the growth of data centers have drawn increasing attention.To date, the worldwide energy consumption of data centers accounts for approximately 1.3% of the world's total electricity usage [1].In 2013, U.S. data centers consumed an estimated 91 billion kWh of electricity.Data center electricity consumption is projected to increase to roughly 140 billion kWh annually by 2020 [2,3].
Moreover, the energy consumption of data centers is expected to double every five years, which will incur a massive cost to both business and the environment [4,5].Therefore, both researchers and data center owners have made efforts to improve the energy efficiency of data centers to minimize their environmental impact, lower the cost of energy and optimize the operation of these centers [6].
Statistics pertaining to the energy consumption of data centers from several surveys are summarized in Figure 1, mainly covering the cost of computational and physical resources [7][8][9].Server computation accounts for approximately 40% of total cost [10], the energy consumption of the communication equipment for 5%, storage devices consume 5%, and power supply systems and miscellaneous factors account for approximately 10%.Surveys have also shown that refrigeration systems, mainly air conditioning, account for 40% of total energy consumption at data centers, which is also a major energy consumption part.
Sustainability 2017, 9, 664 2 of 16 miscellaneous factors account for approximately 10%.Surveys have also shown that refrigeration systems, mainly air conditioning, account for 40% of total energy consumption at data centers, which is also a major energy consumption part.Research has revealed that a primary cause of the high energy consumption of air conditioning systems in data centers is inefficient airflow.Investment in air conditioning equipment rises due to inefficient air distribution, which leads to higher cost.Therefore, the optimization of air distribution in data centers is important for improving the efficiency of air conditioning systems and reducing emissions.
In the last decade, most researchers have used numerical methods to study indoor airflow and analyze its energy saving effect.Different modeling methods have been used to simulate air circulation in data centers [7], including reduced order models (ROM), flow network modeling (FNM), and CFD.Tang et al. [11] presented an abstract heat flow model that used temperature information from onboard and ambient sensors, characterized hot air recirculation based on this information and accelerated the thermal evaluation process for high performance data centers.Simulation results confirmed that heat recirculation could be characterized as cross-interference in their abstract heat flow model.Moreover, fast thermal evaluation based on cross-interference can be used in online thermal management to predict temperature distribution in real time.Moreover, Tang et al. [12] provided the most elaborate way to describe recirculation through a computational fluid dynamics (CFD) model.They also claimed that CFD models are not suitable solutions for online decisions on task placement.Cho et al. [13] studied the profiles of temperature and airflow in a data center a using a CFD simulation, and used four performance metrics to analyze air management systems.Wang et al. [14] established a unified model for pricing and energy cost, and obtained the optimal conditions for the operation of the entire data center based on dynamic pricing.Zhang et al. [15] showed that air distribution improved with the addition of an intelligent fan system to regions with excessive heat.Annual energy consumption in normal ventilation systems can be reduced by 38% by using numerical models.
Axley et al. [16] and Dols et al. [17] evaluated the influence of demand-controlled ventilation systems on energy and indoor air quality by building airflow and thermal load models; subsequently, Dols and Emmerich developed coupled thermal/airflow solutions to simulate and verify a three-zone combined heat transfer/airflow analytical BESTEST case [18].Alissa et al. [19] provided a validated dynamic airflow prediction model and a testing methodology for server storage based on the RPM of fans and simple, facility-level pressure measurements, which are beneficial to the platform and facility-level control to assess the cooling needs of each storage unit.Wang et al. [20] simulated the airflow configuration of a computer room and computed the data for the velocity fields, temperature fields, PMV and PPD using CFD technology with two air supply systems for a computer room.Lang et al. [21] analyzed the influence of different inlet wind directions on airflow characteristics for an underground air distribution system and presented airflow characteristics resulting from experimental and numerical simulations.These characteristics were similar when the wind incidence angles were 60 and 90, and the maximum velocities were recorded at the center of the room.Indoor air was more mixed when the wind incidence angle was 60 than when it was 90.Hasson [22]  Research has revealed that a primary cause of the high energy consumption of air conditioning systems in data centers is inefficient airflow.Investment in air conditioning equipment rises due to inefficient air distribution, which leads to higher cost.Therefore, the optimization of air distribution in data centers is important for improving the efficiency of air conditioning systems and reducing emissions.

40%
In the last decade, most researchers have used numerical methods to study indoor airflow and analyze its energy saving effect.Different modeling methods have been used to simulate air circulation in data centers [7], including reduced order models (ROM), flow network modeling (FNM), and CFD.Tang et al. [11] presented an abstract heat flow model that used temperature information from onboard and ambient sensors, characterized hot air recirculation based on this information and accelerated the thermal evaluation process for high performance data centers.Simulation results confirmed that heat recirculation could be characterized as cross-interference in their abstract heat flow model.Moreover, fast thermal evaluation based on cross-interference can be used in online thermal management to predict temperature distribution in real time.Moreover, Tang et al. [12] provided the most elaborate way to describe recirculation through a computational fluid dynamics (CFD) model.They also claimed that CFD models are not suitable solutions for online decisions on task placement.Cho et al. [13] studied the profiles of temperature and airflow in a data center a using a CFD simulation, and used four performance metrics to analyze air management systems.Wang et al. [14] established a unified model for pricing and energy cost, and obtained the optimal conditions for the operation of the entire data center based on dynamic pricing.Zhang et al. [15] showed that air distribution improved with the addition of an intelligent fan system to regions with excessive heat.Annual energy consumption in normal ventilation systems can be reduced by 38% by using numerical models.
Axley et al. [16] and Dols et al. [17] evaluated the influence of demand-controlled ventilation systems on energy and indoor air quality by building airflow and thermal load models; subsequently, Dols and Emmerich developed coupled thermal/airflow solutions to simulate and verify a three-zone combined heat transfer/airflow analytical BESTEST case [18].Alissa et al. [19] provided a validated dynamic airflow prediction model and a testing methodology for server storage based on the RPM of fans and simple, facility-level pressure measurements, which are beneficial to the platform and facility-level control to assess the cooling needs of each storage unit.Wang et al. [20] simulated the airflow configuration of a computer room and computed the data for the velocity fields, temperature fields, PMV and PPD using CFD technology with two air supply systems for a computer room.Lang et al. [21] analyzed the influence of different inlet wind directions on airflow characteristics for an underground air distribution system and presented airflow characteristics resulting from experimental and numerical simulations.These characteristics were similar when the wind incidence angles were 60 • and 90 • , and the maximum velocities were recorded at the center of the room.Indoor air was more mixed when the wind incidence angle was 60 • than when it was 90 • .Hasson [22] presented an analysis of the airflow, the temperature and the pressure distribution of data centers using the CFD code Fluent, and investigated the effect of various parameters on the temperature distribution and the flow field.The model can evaluate airflow rates and thermal loads to optimize an existing data center or design a new one.
The above-mentioned research indicates that, as a common tool, CFD is widely used to analyze and optimize the airflow and temperature distribution in communication rooms and data processing equipment, the equipment layout and the effects of different layouts and frames on the computer room.However, studies on indoor air distribution to data rooms have either focused on temperature and airflow at the scale of a room or on vectors and contours of velocity and temperature in the macro scale.Airflow in a hot aisle/cold aisle remains an insufficiently explored field.
Solutions to energy saving in data centers equipped with air conditioning machines fall into three main categories: (1) selecting air conditioning machines with a high energy efficiency ratio; (2) managing auxiliary facilities or the automatic control of air conditioning to reduce energy consumption; and (3) designing a layout of the indoor air conditioners to optimize airflow to the server rooms, which is the most important approach.A perfect air conditioning system with optimal airflow according to the requirements of the equipment is the main problem in the design of data centers.Appropriately designed airflow can help achieve the best cooling effect, improve efficiency and reduce energy consumption [23][24][25][26].
To this end, a typical, small data center with tiles for a raised floor in an air supply system is used as the target model in this study.CFD is used to simulate airflow and thermal distribution in order to investigate the influence of the operating conditions on airflow in the hot aisle/cold aisle and optimize it to save energy using Fluent 6.3.The effect of the airflow outlet angle at the tile, the cooling temperature and the airflow rate on the beta index and the energy utilization index are discussed and optimum conditions are obtained.Furthermore, the problem of high energy consumption due to poor air distribution in the small data center is analyzed to improve air conditioning efficiency and reduce energy consumption.

Methodology
In this paper, a typical data center with an air supply system with a raised floor was chosen as target model to provide boundary conditions to simulate the pattern of airflow and confirm the results.

Physical Model of Data Center
Fluent 6.3 was used to obtain the thermal distribution of various scheduling algorithms and the cooling efficiency of the small-scale data center in a confined space, as shown in Figure 2.This methodology has been validated in prior studies [20][21][22][23][24][25][26][27][28].
Sustainability 2017, 9, 664 3 of 16 presented an analysis of the airflow, the temperature and the pressure distribution of data centers using the CFD code Fluent, and investigated the effect of various parameters on the temperature distribution and the flow field.The model can evaluate airflow rates and thermal loads to optimize an existing data center or design a new one.The above-mentioned research indicates that, as a common tool, CFD is widely used to analyze and optimize the airflow and temperature distribution in communication rooms and data processing equipment, the equipment layout and the effects of different layouts and frames on the computer room.However, studies on indoor air distribution to data rooms have either focused on temperature and airflow at the scale of a room or on vectors and contours of velocity and temperature in the macro scale.Airflow in a hot aisle/cold aisle remains an insufficiently explored field.
Solutions to energy saving in data centers equipped with air conditioning machines fall into three main categories: (1) selecting air conditioning machines with a high energy efficiency ratio; (2) managing auxiliary facilities or the automatic control of air conditioning to reduce energy consumption; and (3) designing a layout of the indoor air conditioners to optimize airflow to the server rooms, which is the most important approach.A perfect air conditioning system with optimal airflow according to the requirements of the equipment is the main problem in the design of data centers.Appropriately designed airflow can help achieve the best cooling effect, improve efficiency and reduce energy consumption [23][24][25][26].
To this end, a typical, small data center with tiles for a raised floor in an air supply system is used as the target model in this study.CFD is used to simulate airflow and thermal distribution in order to investigate the influence of the operating conditions on airflow in the hot aisle/cold aisle and optimize it to save energy using Fluent 6.3.The effect of the airflow outlet angle at the tile, the cooling temperature and the airflow rate on the beta index and the energy utilization index are discussed and optimum conditions are obtained.Furthermore, the problem of high energy consumption due to poor air distribution in the small data center is analyzed to improve air conditioning efficiency and reduce energy consumption.

Methodology
In this paper, a typical data center with an air supply system with a raised floor was chosen as target model to provide boundary conditions to simulate the pattern of airflow and confirm the results.

Physical Model of Data Center
Fluent 6.3 was used to obtain the thermal distribution of various scheduling algorithms and the cooling efficiency of the small-scale data center in a confined space, as shown in Figure 2.This methodology has been validated in prior studies [20][21][22][23][24][25][26][27][28].A three-dimensional (3D) model of an analyzed data center was built with dimensions 10.0 m × 9.6 m × 3.2 m (see Table 1 for the parameters).A corresponding physical model was created using the modeling software Gambit, which contained two rows of industry-standard 2U racks arranged in a typical cold aisle and a hot aisle.An unstructured hexahedral mesher was used to create the mesh, and the grid was refined around the openings, servers and vents.A total of 161,142 cells, 503,912 faces and 173,475 nodes were generated.The quality of the mesh was satisfactory because the calculated face alignment was greater than 0.7, which meant that no element had been severely distorted.Cold air was supplied by an air conditioner at a rate of 3.6-10 m 3 /s.The cold air rose from the raised floor plenum through vent tiles and hot air returned to the air conditioner.The data center contained two rows of racks, and four R520 2U racks (Rack 1, Rack 2, Rack 3, Rack 4, . . ., Rack 8) were arranged in each row; each rack was equipped with five chassis (marked from bottom to top as A, B, C, D and E).The servers were Lenovo R520 GT containing two processors each.The total power consumption of the data center was 9.6 KW when idle and 13.6 KW at full utilization.The data center also contained one CRAC unit that pushed chilled air at a temperature between 13 • C and 21 • C into the raised floor at a certain rate.
During the simulation, the room was isolated and the effects of heat transfer outside it were neglected.Adiabatic walls were chosen for the room.For the chassis as heat source, a black box model was used that allowed no flow through the servers, and a chassis load was added to generate the profile of the thermal exhaust and heat exchange with the environment through a heat transfer area.The black box method has been shown to be feasible in past work [29][30][31].

Mathematical Model
The commercial CFD software package Fluent 6.3 was used as basic tool for the simulation [32].Since design factors can affect the efficiency of airflow in the data center, several physical conditions based on the standard modulus of design and architecture variables were investigated.
In room-level data center airflow modeling, the major flow regime is commonly taken to be turbulent mixed convection because of the scale of the room, flow conditions through the perforations and the server racks [33,34].
Past investigations have indicated that the k-ε turbulent model exhibits better performance and results, in good agreement with experimental data, than the k-ω, SST, the RSM and the RNG k-ε models [29].The governing equations of the Euler-Euler model are provided in Section 2.2.1.
Since the equation involves the flow and characteristics of typical dimensions and flow turbulence, the simulation process with the equation model was assumed to be as follows [35][36][37]: (1) Air could be regarded as an incompressible fluid and the flow as steady turbulent flow because indoor air velocity was lower than the velocity of sound.(2) Heat transfer on the surface of the external wall and the server was uniform, and the effects of radiation were ignored in the room according to the steady heat assumption.(3) We ignored the impact of such disturbance as air leakage or personnel movement.

Continuity and Momentum Equations
The motion of each phase was governed by the corresponding mass and momentum conservation equations.The continuity equation is: where u i is the velocity of the i-th phase.The subscript i represents the x-, yor z-axis.
The equation for the conservation of momentum is: where ρ is density in kg/m 3 ; f i is the interfacial force acting on phase i due to the presence of phase j, which includes drag force, interphase turbulent dispersion force, virtual mass and lift force [32,34]; µ is shear viscosity in N•s/m 2 ; and p is pressure in N/m 2 .The energy equation is as follows: where T is the fluid temperature in K, c p is the specific heat capacity of the fluid at constant pressure in J/gK, k is the fluid thermal transfer coefficient in m 2 /s and S T is the term for viscous dissipation.

Turbulence Model
In order to solve Reynolds' stress in the turbulence momentum equations, the standard k-ε model was used to simulate introduction of the liquid phase with the Boussinesq assumption, obtained as follows: where i represents the x-, yor z-axis, G k is the turbulent kinetic energy at the mean velocity gradient ; µ t is the turbulent viscosity, ε ; and C 1ε , C 2ε , C 3ε , σ k and σ ε are parameters in the standard k-ε model.The turbulent viscosity of the liquid phase was modeled using the conventional shear-induced turbulent viscosity, which was obtained from the standard k-ε model as: and Boussinesq assumption, Sustainability 2017, 9, 664 6 of 16

Boundary Conditions
For the calculations of the CFD, a finite-volume approximation of the Reynolds-averaged Navier-Stokes equations and energy equations with the standard k-ε turbulence model were solved.The effects of buoyancy were included, but constant properties for the air were assumed otherwise.Grid generation is very important for CFD calculations because it is directly related to its success or failure.The grid should meet the following criteria: the change in density in the grid domain should be based on the variables, and the change in elements in the entire solution domain should be smooth.According to the geometry of the data room, the minimum volume was 2.961 × 10 −7 m 3 and the maximum was 8.001 × 10 −3 m 3 .This grid level was used to obtain all CFD data for a numerical comparison with the zonal modeling.
The boundary conditions, the initial conditions and the iteration scheme were set for the numerical simulations in the Fluent 6.3 software package.The continuity equations were discretized using the QUICK scheme and the others through the second-order upwind differencing scheme.The relaxation factors are shown in Table 2.In all simulations, a time step of 0.001 s was used.The convergence criterion for flow was set at 1 × 10 −3 and that for energy and H 2 O specie was set at 1 × 10 −6 .The simulation was stopped when it was determined to have reached a steady state.For validation purposes, a CFD simulation was also carried out in this study, for which the standard k-ε turbulence model was used.The following turbulence model constants were employed:

Evaluation Method
A data center is usually designed as a raised floor with racks arranged in a hot/cold aisle layout.In computer room air conditioning (CRACs), hot exhaust air from the racks is cooled and supplied with chilled air through the floor plenum.In the hot/cold aisle layout, several researchers have raised airflow and temperature management problems, such as the hotspot phenomenon, bypass airflow and recirculation air.A number of thermal metrics have been proposed to evaluate airflow and thermal management.Sharma et al. [38] proposed the dimensionless parameters of supply heat index and return heat index, and Herrlin [39,40] proposed the rack cooling index.These indices can provide a macroscopic evaluation of the recirculation phenomena but fail to present the effect of bypass on recirculation and other information pertaining to local IT equipment.Other thermal metrics were proposed in [41].
In response to the thermal and bypass phenomena, we used two kinds of evaluation indices, the beta index and the energy utilization index.The β index was proposed to evaluate temperature increase in local racks [42], and can be defined as: where T in is the airflow inlet mean temperature for each rack that corresponds with T out , the airflow outlet mean temperature for each rack, and T ref is the airflow cooling mean temperature for the floor.The temperature symbols are shown in Figure 3. Schmidt tried to solve this problem by proposing the β Index [42].This index provides information about local temperatures in a rack, and is obtained from the ratio of the punctual temperature difference across a rack to the average difference across the racks.The common value of β ranges between 0 and 1.A β value of 0 means no air recirculation, whereas a β value above 1 indicates self-heating.The other index is the energy utilization index [43,44].This index is defined as: where Tp is the temperature in the outlet and Ts is that in the inlet.Tn is the temperature in the working space.It indicates the efficiency of airflow in the workspace used to cool the equipment; the higher the value of , the higher the air cooling rate.The value of  can be used to evaluate the utilization efficiency of air distribution with respect to the air cooling capacity as well as the energy saving obtained through the air distribution scheme.
In order to calculate the energy efficiency of airflow in the data center, the energy utilization index was modified as [45]: where Tout is the exhaust air temperature and Tin the inlet air temperature.Tref is the cooling air temperature in the rack area.In Equation ( 13), the difference between the average temperature of the inlet and exhaust air and the reference temperature are used as reference to measure the intensity of the mixing of hot and cold air.It can reflect a local hotspot due to equipment failure.Smaller values represent higher the comprehensive evaluation of airflow organization.It represents the exhaust air circulation of the IT rack.

Effect of Airflow Outlet Angle of Tiles on Air Distribution
Early simulations of data centers focused on modeling a raised floor and perforated tiles to accurately predict the boundary conditions [46][47][48][49][50][51][52].We believe that not only does heterogeneous airflow in the cold aisle (hot air recirculation and cold air bypass) depend on airflow distribution from the tiles, the airflow outlet angle also influences the distribution of cold air.
The airflow vectors of the CFD simulation at airflow angles of 30 and 45 are shown in Figure 4.In the cold aisle, the vectors of the cooling air were clearly observed near the surface of the tile, flowing toward the cold aisle space, following a small volume of re-circulated cold air in the lower parts of the room.A majority of the flow streamed toward the right end and moved into the server inlets.However, a small amount of airflow, especially in the top parts of the room, simply streamed past the open cold aisle and mixed with the hot air in hot aisle in Figure 4A.As the figure shows, a similar bypass cooling air phenomenon was observed in the same region of Figure 4C.
In the hot aisle, airflow from the outlets of servers led directly into the main flow in the hot aisle from bottom to the top, as shown Figure 4B,D.However, the hot airflow mixed with a small part of the cold flow.Some degree of heterogeneous flow at the top of the room was also observed.The other index is the energy utilization index [43,44].This index is defined as: where T p is the temperature in the outlet and T s is that in the inlet.T n is the temperature in the working space.It indicates the efficiency of airflow in the workspace used to cool the equipment; the higher the value of η, the higher the air cooling rate.The value of η can be used to evaluate the utilization efficiency of air distribution with respect to the air cooling capacity as well as the energy saving obtained through the air distribution scheme.
In order to calculate the energy efficiency of airflow in the data center, the energy utilization index was modified as [45]: where T out is the exhaust air temperature and T in the inlet air temperature.T ref is the cooling air temperature in the rack area.In Equation (13), the difference between the average temperature of the inlet and exhaust air and the reference temperature are used as reference to measure the intensity of the mixing of hot and cold air.It can reflect a local hotspot due to equipment failure.Smaller values represent higher the comprehensive evaluation of airflow organization.It represents the exhaust air circulation of the IT rack.

Effect of Airflow Outlet Angle of Tiles on Air Distribution
Early simulations of data centers focused on modeling a raised floor and perforated tiles to accurately predict the boundary conditions [46][47][48][49][50][51][52].We believe that not only does heterogeneous airflow in the cold aisle (hot air recirculation and cold air bypass) depend on airflow distribution from the tiles, the airflow outlet angle also influences the distribution of cold air.
The airflow vectors of the CFD simulation at airflow angles of 30 • and 45 • are shown in Figure 4.In the cold aisle, the vectors of the cooling air were clearly observed near the surface of the tile, flowing toward the cold aisle space, following a small volume of re-circulated cold air in the lower parts of the room.A majority of the flow streamed toward the right end and moved into the server inlets.However, a small amount of airflow, especially in the top parts of the room, simply streamed past the open cold aisle and mixed with the hot air in hot aisle in Figure 4A.As the figure shows, a similar bypass cooling air phenomenon was observed in the same region of Figure 4C.
In the hot aisle, airflow from the outlets of servers led directly into the main flow in the hot aisle from bottom to the top, as shown Figure 4B,D.However, the hot airflow mixed with a small part of the cold flow.Some degree of heterogeneous flow at the top of the room was also observed.Figure 5 shows the contours of local temperature in the selected cross-sections through the computational domain.These figures provide useful insights into the thermal tendencies of the domain corresponding to the observed airflow.The thermal distribution at an airflow outlet angle of 45 at the tiles was better than that of 30.Particularly at the ceiling, a mixing of cold and hot airflows and the airflow in the cold aisle generated greater circulation.Moreover, the temperature of the upper region of the plotted area appeared slightly higher than the surrounding area.In Figure 6, the velocity of the air at the axis gradually weakened from the exit along the direction of flow.Compared with Figure 6A,B, at airflow outlet angles of 60 and 90 with the tiles, the airflow of the cold aisle at 60 was evenly mixed, and the difference in the inlet of the rack could be reduced.Moreover, airflow in the hot aisle had a uniform distribution, and only at the top part of the room Figure 5 shows the contours of local temperature in the selected cross-sections through the computational domain.These figures provide useful insights into the thermal tendencies of the domain corresponding to the observed airflow.The thermal distribution at an airflow outlet angle of 45 • at the tiles was better than that of 30 • .Particularly at the ceiling, a mixing of cold and hot airflows and the airflow in the cold aisle generated greater circulation.Moreover, the temperature of the upper region of the plotted area appeared slightly higher than the surrounding area.Figure 5 shows the contours of local temperature in the selected cross-sections through the computational domain.These figures provide useful insights into the thermal tendencies of the domain corresponding to the observed airflow.The thermal distribution at an airflow outlet angle of 45 at the tiles was better than that of 30.Particularly at the ceiling, a mixing of cold and hot airflows and the airflow in the cold aisle generated greater circulation.Moreover, the temperature of the upper region of the plotted area appeared slightly higher than the surrounding area.In Figure 6, the velocity of the air at the axis gradually weakened from the exit along the direction of flow.Compared with Figure 6A,B, at airflow outlet angles of 60 and 90 with the tiles, the airflow of the cold aisle at 60 was evenly mixed, and the difference in the inlet of the rack could be reduced.Moreover, airflow in the hot aisle had a uniform distribution, and only at the top part of the room In Figure 6, the velocity of the air at the axis gradually weakened from the exit along the direction of flow.Compared with Figure 6A,B, at airflow outlet angles of 60 • and 90 • with the tiles, the airflow of the cold aisle at 60 • was evenly mixed, and the difference in the inlet of the rack could be reduced.Moreover, airflow in the hot aisle had a uniform distribution, and only at the top part of the room was there a mixture of hot and cold air.On the other hand, at a 90 • angle with the perforated tiles, airflow first reached the ceiling, and then moved to the center along the ceiling.Part of the airflow occurred through the outlet, and the other part reached the center after the downward movement.In the upper part of the room, the indoor air formed a vortex, appearing in the position of circular flow vortex several times, and then continued to flow downward.Large air circulation typically occurs in this condition, which is not conducive to heat exchange in the server.was there a mixture of hot and cold air.On the other hand, at a 90 angle with the perforated tiles, airflow first reached the ceiling, and then moved to the center along the ceiling.Part of the airflow occurred through the outlet, and the other part reached the center after the downward movement.In the upper part of the room, the indoor air formed a vortex, appearing in the position of circular flow vortex several times, and then continued to flow downward.Large air circulation typically occurs in this condition, which is not conducive to heat exchange the server.To further illustrate the influence of the airflow outlet angle (as shown in Figure 3), the simulation results of the CFD showed the effect this angle on the  index and the energy utilization index as shown in Figure 7.Because the simulation of the two row racks was symmetrically structured, this paper only discusses the impact of one of them.In Figure 7, we see that the airflow outlet angle influenced flow and thermal distribution.With an increase in the airflow outlet angle, the β index decreased and the energy utilization index increased, which represented a clear difference between the punctual temperature and the average temperature across the rack.In addition, at an angle of 60, the  index was minimum and the energy utilization index maximum: the airflow was more uniform.

The Temperature of the Cold Air on the Airflow Distribution
Due to the complex nature of airflow inside the data center, some of the hot exhaust from the outlets of the servers was re-circulated into the inlets of the other servers.A reason for the extra cost of cooling systems is the mixture of cold air with the re-circulated hot air in the inlet.To provide an acceptable inlet temperature for all servers, the supplied air temperature needed to be reduced, which often increases cost.Thus, understanding and reducing the recirculation of hot air should be explored To further illustrate the influence of the airflow outlet angle (as shown in Figure 3), the simulation results of the CFD showed the effect this angle on the β index and the energy utilization index as shown in Figure 7.Because the simulation of the two row racks was symmetrically structured, this paper only discusses the impact of one of them.In Figure 7, we see that the airflow outlet angle influenced flow and thermal distribution.With an increase in the airflow outlet angle, the β index decreased and the energy utilization index increased, which represented a clear difference between the punctual temperature and the average temperature across the rack.In addition, at an angle of 60 • , the β index was minimum and the energy utilization index maximum: the airflow was more uniform.was there a mixture of hot and cold air.On the other hand, at a 90 angle with the perforated tiles, airflow first reached the ceiling, and then moved to the center along the ceiling.Part of the airflow occurred through the outlet, and the other part reached the center after the downward movement.In the upper part of the room, the indoor air formed a vortex, appearing in the position of circular flow vortex several times, and then continued to flow downward.Large air circulation typically occurs in this condition, which is not conducive to heat exchange in the server.To further illustrate the influence of the airflow outlet angle (as shown in Figure 3), the simulation results of the CFD showed the effect this angle on the  index and the energy utilization index as shown in Figure 7.Because the simulation of the two row racks was symmetrically structured, this paper only discusses the impact of one of them.In Figure 7, we see that the airflow outlet angle influenced flow and thermal distribution.With an increase in the airflow outlet angle, the β index decreased and the energy utilization index increased, which represented a clear difference between the punctual temperature and the average temperature across the rack.In addition, at an angle of 60, the  index was minimum and the energy utilization index maximum: the airflow was more uniform.

The Temperature of the Cold Air on the Airflow Distribution
Due to the complex nature of airflow inside the data center, some of the hot exhaust from the outlets of the servers was re-circulated into the inlets of the other servers.A reason for the extra cost of cooling systems is the mixture of cold air with the re-circulated hot air in the inlet.To provide an acceptable inlet temperature for all servers, the supplied air temperature needed to be reduced, which often increases cost.Thus, understanding and reducing the recirculation of hot air should be explored to reduce the energy cost of data centers.

The Temperature of the Cold Air on the Airflow Distribution
Due to the complex nature of airflow inside the data center, some of the hot exhaust from the outlets of the servers was re-circulated into the inlets of the other servers.A reason for the extra cost of cooling systems is the mixture of cold air with the re-circulated hot air in the inlet.To provide an acceptable inlet temperature for all servers, the supplied air temperature needed to be reduced, which often increases cost.Thus, understanding and reducing the recirculation of hot air should be explored to reduce the energy cost of data centers.
The temperature of the cold air was a parameter that influenced airflow.The effects of the temperature of the cold air on the β index and the energy utilization index are shown in Figure 8.With the increase in the temperature of the cold air, the β index decreased and the energy utilization index increased.While the temperature was over 17 • C, the β index occupied a stable region.However, for Rack A4, which was far from the air conditioner, only cold air at a temperature of 17 • C has a low value, which meant little recirculation.Under other conditions, when the β value was above 1, indicating a mixing of cold and hot airflows, local hot point in racks may present.The temperature of the cold air was a parameter that influenced airflow.The effects of the temperature of the cold air on the  index and the energy utilization index are shown in Figure 8.
With the increase in the temperature of the cold air, the  index decreased and the energy utilization index increased.While the temperature was over 17 °C, the  index occupied a stable region.
However, for Rack A4, which was far from the air conditioner, only cold air at a temperature of 17 °C has a low value, which meant little recirculation.Under other conditions, when the  value was above 1, indicating a mixing of cold and hot airflows, local hot point in racks may present.Furthermore, the energy utilization index increased with increase in the temperature of the cooling airflow.For cold air at 17 °C, the energy utilization index recorded its maximum valueairflow was more uniform.The results show that air at 15 °C and 17 °C was more conducive for the full utilization of the cooling capacity than at other temperatures.This showed that the variation in air temperature has a certain impact on the thermal environment and air distribution in a confined room.The unit energy consumption also changed.Hence, we needed to find the optimal operating conditions through the thermal and airflow simulations, including suitable airflow temperature, and the quantitative relationship between room temperature and the air distribution evaluation indices.
Figure 9 shows the inlet and outlet temperatures of the chassis at the reference temperatures of 15 °C and 17 °C.Regardless of the inlet temperature, chassis E at each rack had the worst mixing of cold and hot airflows: the hot air backflow phenomenon occurred.However, the thermal environment of the other chassis profile was better.At the same time, the higher is the rate of utilization rate of airflow, the higher the thermal safety of the rack.In addition, for the outlet temperature, the results in Figure 9B are better than those in Figure 9D.Thus, an inlet temperature of 15 °C is recommended as the temperature of the cold air, since it ensures high efficiency and a safe working environment for the heat transfer in the racks and the arrangement of the equipment, according to the results of the index for each location.Furthermore, the energy utilization index increased with increase in the temperature of the cooling airflow.For cold air at 17 • C, the energy utilization index recorded its maximum value-airflow was more uniform.The results show that air at 15 • C and 17 • C was more conducive for the full utilization of the cooling capacity than at other temperatures.This showed that the variation in air temperature has a certain impact on the thermal environment and air distribution in a confined room.The unit energy consumption also changed.Hence, we needed to find the optimal operating conditions through the thermal and airflow simulations, including suitable airflow temperature, and the quantitative relationship between room temperature and the air distribution evaluation indices.
Figure 9 shows the inlet and outlet temperatures of the chassis at the reference temperatures of 15 • C and 17 • C. Regardless of the inlet temperature, chassis E at each rack had the worst mixing of cold and hot airflows: the hot air backflow phenomenon occurred.However, the thermal environment of the other chassis profile was better.At the same time, the higher is the rate of utilization rate of airflow, the higher the thermal safety of the rack.In addition, for the outlet temperature, the results in Figure 9B are better than those in Figure 9D.Thus, an inlet temperature of 15 • C is recommended as the temperature of the cold air, since it ensures high efficiency and a safe working environment for the heat transfer in the racks and the arrangement of the equipment, according to the results of the index for each location.

The Effect of Airflow Rate on the Air Distribution
Floor air supply is related to heat transfer in servers in the data center, but the rate of airflow can also affect the circulation in the data center.The contours of local temperature in a z-axis 2.48 m plane is shown in Figure 10, which is indicate the contours of outlet temperature in Rack 1, Rack 2, Rack 3 and Rack 4. From the CFD simulation, the hot aisle presents a uniform distribution of temperature at small airflow and the temperature of chassis increase from the bottom to top of room; there is a heterogeneous distribution of temperature at larger airflow rate, and the temperature of bottom chassis close to 15 °C and the temperature of the top chassis 25-27 °C.In order to evaluate the thermal and bypass phenomena, the beta index and the energy utilization index were used, as shown Figure 11.

The Effect of Airflow Rate on the Air Distribution
Floor air supply is related to heat transfer in servers in the data center, but the rate of airflow can also affect the circulation in the data center.The contours of local temperature in a z-axis 2.48 m plane is shown in Figure 10, which is indicate the contours of outlet temperature in Rack 1, Rack 2, Rack 3 and Rack 4. From the CFD simulation, the hot aisle presents a uniform distribution of temperature at small airflow rate, and the temperature of chassis increase from the bottom to top of room; there is a heterogeneous distribution of temperature at larger airflow rate, and the temperature of bottom chassis close to 15 • C and the temperature of the top chassis 25-27 • C. In order to evaluate the thermal and bypass phenomena, the beta index and the energy utilization index were used, as shown Figure 11.

The Effect of Airflow Rate on the Air Distribution
Floor air supply is related to heat transfer in servers in the data center, but the rate of airflow can also affect the circulation in the data center.The contours of local temperature in a z-axis 2.48 m plane is shown in Figure 10, which is indicate the contours of outlet temperature in Rack 1, Rack 2, Rack 3 and Rack 4. From the CFD simulation, the hot aisle presents a uniform distribution of temperature at small airflow rate, and the temperature of chassis increase from the bottom to top of room; there is a heterogeneous distribution of temperature at larger airflow rate, and the temperature of bottom chassis close to 15 °C and the temperature of the top chassis 25-27 °C.In order to evaluate the thermal and bypass phenomena, the beta index and the energy utilization index were used, as shown Figure 11.  Figure 11 shows the relationships between airflow and the  index and the energy utilization index.The  index at an airflow rate of 2.0 m 3 /s was better than those at other velocities, which is consistent with the phenomenon in Figure 10A.Moreover, as shown in Figure 10B, the energy utilization index at this rate was the best.In China, the national standard of the thermal environment of data centers is outlined in the Code for the Design of Electronic Information System Rooms [53], which stipulates the temperature for data centers at 22-24 °C, but does not specify the sample points of the test temperature in the data center.Therefore, based on the room temperature requirements of the data center, the airflow rate required for the data center can be calculated through the cooling load on it, the air thermodynamic constant and the inlet and outlet temperatures, as shown in Equation ( 14): where n is the number of servers in the data center, Cp represents the specific heat of air (a thermodynamic constant), mi is the mass of the flow of air through servers and is the inlet temperature of the servers.
Therefore, a volume of airflow should be considered according to Equation ( 14) and the results of the simulation.
Figure 12 shows the velocity vectors and contours of the local temperature in the selected crosssections under different airflow conditions.There were some local hot spots at the top of the room, for which we need to consider the design of airflow in the mixing of hot and cold flows to increase thermal efficiency.Figure 11 shows the relationships between airflow and the β index and the energy utilization index.The β index at an airflow rate of 2.0 m 3 /s was better than those at other velocities, which is consistent with the phenomenon in Figure 10A.Moreover, as shown in Figure 10B, the energy utilization index at this rate was the best.In China, the national standard of the thermal environment of data centers is outlined in the Code for the Design of Electronic Information System Rooms [53], which stipulates the temperature for data centers at 22-24 • C, but does not specify the sample points of the test temperature in the data center.Therefore, based on the room temperature requirements of the data center, the airflow rate required for the data center can be calculated through the cooling load on it, the air thermodynamic constant and the inlet and outlet temperatures, as shown in Equation ( 14): where n is the number of servers in the data center, C p represents the specific heat of air (a thermodynamic constant), m i is the mass of the flow of air through servers and T in i is the inlet temperature of the servers.
Therefore, a volume of airflow should be considered according to Equation ( 14) and the results of the simulation.
Figure 12 shows the velocity vectors and contours of the local temperature in the selected cross-sections under different airflow conditions.There were some local hot spots at the top of the room, for which we need to consider the design of airflow in the mixing of hot and cold flows to increase thermal efficiency.
In addition, the increase in airflow can increase the heat transfer rate and decrease the local temperature of the chassis.However, the chance of the local mixing of cold and hot airflows increases, especially where there is obvious recirculation air at high airflow, which can lead to wasted energy and can increase the load on the CRAC.
Therefore, better airflow distribution and a uniform local temperature are needed.Hence, the CFD simulation, the models and the parameter settings in our research can provide secure foundations for further study.In addition, the increase in airflow can increase the heat transfer rate and decrease the local temperature of the chassis.However, the chance of the local mixing of cold and hot airflows increases, especially where there is obvious recirculation air at high airflow, which can lead to wasted energy and can increase the load on the CRAC.
Therefore, better airflow distribution and a uniform local temperature are needed.Hence, the CFD simulation, the models and the parameter settings in our research can provide secure foundations for further study.

Conclusions
The consumption of energy in data centers has consistently increased due to the rapid development of big data.A very important issue in the design of data centers is the balancing of the airflow distribution through the tiles to fulfill the cooling requirements of the adjacent computer equipment.By using CFD simulation software, a series of influence parameters for a confined data room were simulated in this paper.According to the thermal and the bypass phenomena, we used two kinds of evaluation indices, the beta index and the energy utilization index.By analyzing the relation between the airflow outlet angle with the tiles and airflow distribution, we found a minimum  index and a maximum energy utilization index at an angle of 60, which represented better, more uniform air distribution.The variation in air temperature affects the thermal environment and air distribution in the confined room, and the unit energy consumption also changed.The simulation results showed that an air supply temperature of 15-17 °C is most suitable for the full utilization of the cooling capacity.Moreover, floor air supply is related to the heat transfer in the servers of the data center, and the airflow rate affects its airflow circulation.According to the results of the CFD simulation, the  index at lower airflow rates has an ideal value, whereas the energy utilization index at a higher airflow rate is the best.Therefore, through the thermal and airflow simulations, we found the optimal operating conditions, suitable airflow temperatures, as well as the quantitative relationship between the room temperature and the room air distribution evaluation indices.The utility model implemented high-efficiency heat transfer in the data room, which improved the efficiency of the cooling process and the effective utilization of the air conditioner.
The outcome of this study has the potential to contribute to treating the problems of airflow and thermal distribution in data centers through simulation models and computational analysis involving detailed 3D CFD calculations.Optimal parameter settings for the tiles and severs can be generated and used to obtain the desired distribution of flow rates of cold air.This study can also be used as the foundation for further research on airflow distribution and thermal consumption by

Conclusions
The consumption of energy in data centers has consistently increased due to the rapid development of big data.A very important issue in the design of data centers is the balancing of the airflow distribution through the tiles to fulfill the cooling requirements of the adjacent computer equipment.By using CFD simulation software, a series of influence parameters for a confined data room were simulated in this paper.According to the thermal and the bypass phenomena, we used two kinds of evaluation indices, the beta index and the energy utilization index.By analyzing the relation between the airflow outlet angle with the tiles and airflow distribution, we found a minimum β index and a maximum energy utilization index at an angle of 60 • , which represented better, more uniform air distribution.The variation in air temperature affects the thermal environment and air distribution in the confined room, and the unit energy consumption also changed.The simulation results showed that an air supply temperature of 15-17 • C is most suitable for the full utilization of the cooling capacity.Moreover, floor air supply is related to the heat transfer in the servers of the data center, and the airflow rate affects its airflow circulation.According to the results of the CFD simulation, the β index at lower airflow rates has an ideal value, whereas the energy utilization index at a higher airflow rate is the best.Therefore, through the thermal and airflow simulations, we found the optimal operating conditions, suitable airflow temperatures, as well as the quantitative relationship between the room temperature and the room air distribution evaluation indices.The utility model implemented high-efficiency heat transfer in the data room, which improved the efficiency of the cooling process and the effective utilization of the air conditioner.
The outcome of this study has the potential to contribute to treating the problems of airflow and thermal distribution in data centers through simulation models and computational analysis involving detailed 3D CFD calculations.Optimal parameter settings for the tiles and severs can be generated and used to obtain the desired distribution of flow rates of cold air.This study can also be used as the foundation for further research on airflow distribution and thermal consumption by considering such parameters as equipment arrangement, rack loadings, airflow rate, airflow outlet angel, heat load distribution and other cases of cooling.

Figure 1 .
Figure 1.Energy consumption distribution of data centers.

Figure 1 .
Figure 1.Energy consumption distribution of data centers.

Figure 2 .
Figure 2. The small scale data center with meth.

Figure 2 .
Figure 2. The small scale data center with meth.

Figure 3 .
Figure 3.A typical temperature distribution layout.

Figure 3 .
Figure 3.A typical temperature distribution layout.

Figure 6 .
Figure 6.The velocity vectors and contours of local temperature in selected cross-sections with X = 2.95 m through the computational domain: (A) 60; and (B) 90.

Figure 7 .
Figure 7.The relationship between airflow outlet angle and:  (A); and r (B) (airflow rate 4.22 m 3 /s, the cooling air temperature 15 °C).

Figure 6 .
Figure 6.The velocity vectors and contours of local temperature in selected cross-sections with X = 2.95 m through the computational domain: (A) 60 • ; and (B) 90 • .

Figure 6 .
Figure 6.The velocity vectors and contours of local temperature in selected cross-sections with X = 2.95 m through the computational domain: (A) 60; and (B) 90.

Figure 7 .
Figure 7.The relationship between airflow outlet angle and:  (A); and r (B) (airflow rate 4.22 m 3 /s, the cooling air temperature 15 °C).

Figure 8 .
Figure 8.Effect of temperature of cooling airflow on: the  index (A); and the energy utilization index r (B) (airflow outlet angle 60, airflow rate 4.22 m 3 /s).

Figure 8 .
Figure 8.Effect of temperature of cooling airflow on: the β index (A); and the energy utilization index η r (B) (airflow outlet angle 60 • , airflow rate 4.22 m 3 /s).

Figure 10 .
Figure 10.The contours of local temperature in selected cross-sections with Z = 2.48 m through the computational domain (airflow outlet angle 60 • , the cooling air temperature 15 • C): (A) airflow rate 2.0 m 3 /s; (B) airflow rate 3.1 m 3 /s; (C) airflow rate 4.22 m 3 /s; and (D) airflow rate 5.64 m 3 /s.

Figure 11 .
Figure 11.The effect of airflow rate on: the β index (A); and the energy utilization index η r (B) (airflow outlet angle 45 • , the cooling air temperature 15 • C).

Table 1 .
Equipment parameters in numerical simulation.