Comparing Performance Metrics of Partial Aisle Containments in Hard Floor and Raised Floor Data Centers Using CFD

: In data centers, e ﬃ cient cooling systems are required to both keep the energy consumption as low as possible and to fulﬁll the temperature requirements. The aim of this work is to numerically investigate the e ﬀ ects of using partial aisle containment between the server racks for hard and raised ﬂoor conﬁgurations. The computational ﬂuid dynamics (CFD) software ANSYS CFX was used together with the Reynolds stress turbulence model to perform the simulations. Velocity measurements in a server room were used for validation. Boundary conditions and the load of each rack were also retrieved from the experimental facility, implying an uneven load between the racks. A combination of the performance metrics Rack Cooling Index (RCI), Return Temperature Index (RTI) and Capture Index (CI) were used to evaluate the performance of the cooling systems for two supply ﬂow rates at a 100% and 50% of operating condition. Based on the combination of performance metrics, the airﬂow management was improved in the raised ﬂoor conﬁgurations. With the supply ﬂow rate set to operating conditions, the RCI was 100% for both raised ﬂoor and hard ﬂoor setups. The top- or side-cover fully prevented recirculation for the raised ﬂoor conﬁguration, while it reduced the recirculation for the hard ﬂoor conﬁguration. However, the RTI was low, close to 40% in the hard ﬂoor case, indicating poor energy e ﬃ ciency. With the supply ﬂow rate decreasing with 50%, the RTI increased to above 80%. Recirculation of hot air was indicated for all the containments when the supply rate was 50%, but the values of RCI still indicated an acceptable performance of the cooling system.


Introduction
The worldwide demand for storage and processing of data has increased rapidly during the last several years and as a result, the quantity and the size of data centers are increasing [1]. The development of sustainable facilities is at the same time very important since their energy consumption is extremely large. In 2012, 1.4% of the total energy consumption in the world originated from data centers [2].
Uninterrupted operation is the most crucial requirement for data centers [3]. Adequate cooling of the servers is therefore very important, while over cooling is a waste of energy. In average 40% of the total energy consumption in data centers is related to the cooling process [4]. Energy efficient cooling systems are required in order to both keep the energy consumption as low as possible and to satisfy the temperature requirements in data centers. The general guidelines that specify allowable and recommended rack intake temperatures are at present 15-32 • C and 18-27 • C, respectively [5].
Two alternative setups are commonly used in air cooled data centers; hard floor and raised floor configurations. Cold air is supplied directly to the room from the Computer Room Air Conditioner (CRAC) units if a hard floor configuration is used. In a raised floor setup, the CRAC units deliver cold air into an under-floor space instead, whereas the air then enters the data center through perforated floor tiles. It is common practice to place the perforated tiles right beneath the rack intakes in order to make it more likely that the cold air finds its way to the servers. Cold air bypass is one problem that may occur as a result of poor airflow management [6]. The cold air then immediately returns to the outlet of the CRAC units without cooling the servers [7], leading to loss of cooling capacity. To compensate for this, the supply flow rate must be increased to make sure that enough cold air is available at the server racks resulting in over-dimensioned cooling systems. Another example of poor air management is when the exhausted hot air from a server enters the intake side of a server rack, rather than the outlet of the CRAC unit. This phenomenon is referred to as hot air recirculation.
In order to efficiently manage the airflow, it is common to form hot and cold aisles by placing the server racks in parallel rows. When the front sides are facing each other cold aisles are formed, while back sides facing each other form hot ailses. Aisle containment is a strategy to minimize hot air recirculation and cold air bypass. Hot aisle containment prevents the hot air from mixing with cold air by using panels as a physical barrier to enclose the hot aisles. A ceiling void space can be used to isolate the hot air when moving towards the CRAC units. Cold aisle containment prevents the cold air supplied by the perforated tiles to mix with hot air by using panels to enclose the cold aisles. For an ideal cold aisle containment, the total flow rate through the perforated tiles and the total flow rate demanded by the server racks would be equal. However, there is often air leakage either from or to the cold aisles in existing data centers. The air leakage makes up for the excess or deficit in the supply flow rate, compared with the flow rate through the server racks. The aisles can also be partially contained by using panels on either side or on top of it. The potential to increase the thermal performance of a server room by the use of partial or full aisle containment has been verified in several studies, but mainly for cold aisles only. Srinarayana et al. [8] numerically compared room and ceiling return strategies in both hard and raised floor configurations. Arghode et al. [9] compared numerical and experimental results of a contained cold aisle to an open cold aisle for both the under-and over-provisioning of supply air. Sundaralingam et al. [10] experimentally investigated the performance of the cooling system in different setups with either an open, partially contained or fully contained cold aisle. Alkharabsheh et al. [11] used an experimentally validated numerical model to study both open and contained cold aisle setups. Nada et al. [12] numerically studied partial cold aisle containment and also compared different arrangements of CRAC units and server racks with respect to airflow management. Values of flow rates through the perforated tiles and rack intake temperatures were used for validation. In several of the previous studies, computational fluid dynamics (CFD) was used to provide details of the airflow [8,9,11,12]. Further studies of CFD for airflow management can be found in the reviews by Chu & Wang [13] and Lu et al. [14]. In this context, it is crucial that the simulations are trustworthy. Boundary conditions, mesh and application of turbulence models should for example be carefully addressed to ensure reliable results [11,15]. Different levels of details are furthermore needed depending on whether the component level, rack level or room level is in focus [13].
Detailed flow analysis of hard floor and raised floor setups in the SICS ICE data center in Luleå, Sweden, has previously been performed with numerical simulations [7,15,16] and measurements [17,18]. The influence of a turbulence model on the fluid flow in a hard floor server room was investigated by Wibron et al. [15], recommending the Reynolds stress model (RSM) for modeling the flow on a server room scale. Local details of the steady state and transient flow in hard floor and raised floor configurations were examined by Wibron et al. [7]. The aim of this study is to investigate the effects of using partial aisle containment for the different floor types, i.e., considering both cool aisle and hot aisle containments. The software ANSYS CFX was used to model the flow and temperature fields inside the server room. Velocity measurements were used for validation of both floor types with open aisles. Uneven load distribution between servers and the RSM turbulence model were employed in all simulations. A combination of the performance metrics Rack Cooling Index (RCI), Return Temperature Index (RTI) and Capture Index (CI) were used to evaluate the performance of the cooling system for both hard-and raised floor configurations. The setups based on the conditions in the existing data center are very well cooled since the flow rate supplied by the computer room air handler (CRAH) units or perforated tiles exceeds the flow through the server racks. Therefore, the same setups with only 50% of the original supply flow rate are also investigated to study the possibility of increasing the energy efficiency through decreased supply flow rate.

Method
The geometry and boundary conditions retrieved from the experimental facility is first presented. Strategies for validation of the numerical simulations are then addressed, and finally the grid, simulation settings and performance metrics are discussed.

Geometry
A server room (Module 2) at RISE SICS North in Luleå, Sweden, was chosen for the study. In this data center it is possible to choose between hard or raised floor setups, which makes it suitable for the present study. In Figure 1, the geometries for both floor types with open aisles are illustrated. The dimensions of the data center were 6.484 × 7.000 × 3.150 m 3 . In Figure 2, the main components and measurement locations are illustrated. The measurement locations are the same for both configurations. The locations were chosen from flow field considerations and were intended to capture the different flow conditions in the server room. The velocity was measured at the heights of 0.6, 1.0, 1.4, 1.8 and 2.2 m for each location to enable validation of the digital model with open aisles. Partial aisle containment was then imposed in the simulations by using either a side or top cover to partially enclose the aisle between the two rows of server racks. The side cover was located at the end of the rows and at the end of the perforated tiles for the hard and raised floor configurations, respectively. The top cover was located on top of the server racks and reached from the wall out to the end of the rows for both configurations. model were employed in all simulations. A combination of the performance metrics Rack Cooling Index (RCI), Return Temperature Index (RTI) and Capture Index (CI) were used to evaluate the performance of the cooling system for both hard floor and raised floor configurations. The setups based on the conditions in the existing data center are very well cooled since the flow rate supplied by the computer room air handler (CRAH) units or perforated tiles exceed the flow rate demanded by the server racks. Therefore, the same setups with only 50% of the original supply flow rate are also investigated to study the possibility of increasing the energy efficiency through decreased supply flow rate.

Method
The geometry and boundary conditions retrieved from the experimental facility is first presented. Strategies for validation of the numerical simulations are then addressed, and finally the grid, simulation settings and performance metrics are discussed.

Geometry
A server room (Module 2) at the research facility SICS ICE in Luleå, Sweden, was chosen for the study. In this data center it is possible to choose whether the cooling system should be based on a hard or raised floor configuration, which makes it suitable for the present study. In Figure 1 Partial aisle containment was then imposed in the simulations by using either a side or top cover to partially enclose the aisle between the two rows of server racks. The side cover was located at the end of the rows and at the end of the perforated tiles for the hard and raised floor configurations, respectively. The top cover was located on top of the server racks and reached from the wall out to the end of the rows for both configurations. In the hard floor configuration, the main components were 10 server racks composed of 0.8 m wide cabinets from Siemon and four computer room air handler (CRAH) units of the label SEE Cooler HDZ-2. The servers inside the racks were not limited to one manufacturer, and temperatures of the inlets and outlets of the server racks were therefore measured for input to the numerical model. The location of the inlets and outlets of the servers were furthermore retrieved from measurements in the server room. In the raised floor configuration, there were fourteen perforated tiles located in the cold aisle. The front sides of the CRAH units were covered, and the cold air was supplied through an In the hard floor configuration, the main components were 10 server racks composed of 0.8 m wide cabinets from Siemon and four computer room air handler (CRAH) units of the label SEE Cooler HDZ-2. The servers inside the racks were not limited to one manufacturer, and temperatures of the inlets and outlets of the server racks were therefore measured for input to the numerical model. The location of the inlets and outlets of the servers were furthermore retrieved from measurements in the server room. In the raised floor configuration, there were fourteen perforated tiles located in the cold aisle. Covers on the front sides of the CRAH units were used to direct the cold air under the floor. In the experiments, there were two different kinds of perforated tiles with percentage open areas of 56% and 77%, respectively. The under-floor space was not taken into account in the simulations. An uninterruptible power supply (UPS) was also present in the data center. To include the effect of flow blockage the UPS was modeled without heat loads, while smaller obstructions, i.e., cables and pipes, were ignored. An uninterruptible power supply (UPS) was also present in the data center. To include the effect of flow blockage the UPS was modeled without heat loads, while smaller obstructions, i.e. cables and pipes, were ignored.

Boundary Conditions
Flow through the servers were not resolved in this study. Boundary conditions were instead applied to the inlet and outlets of the servers and the CRAH units, i.e. the so-called black box model [19] was used. The supply temperature and the flow rates from both the CRAH units and the perforated tiles were determined from measurements. The hotwire thermo-anemometer VTlO0 from Kimo was used to measure the temperatures, while flow rates were measured with the DP-CALC Micromanometer Model 8715 and the Velocity Matrix add-on kit from TSI. The Velocity Matrix provides area-averaged multi-point face velocity measurements. It calculates the average velocity based on the difference between total and static pressures [20]. The accuracy of the velocity and temperature measurements are ±3% of reading ±0.04 m/s for the velocity range 0.25-12.5 m/s and ±0.4% of reading ±0.3 °C for temperatures from −20 °C to 80 °C. The supply temperature was measured to 18.3 °C for both configurations and hence independent of the measurement location. For the hard floor configuration, the flow rates were measured through the front sides of all the CRAH units. Each front side was divided into 15 measurement zones and the inlet conditions were determined from average velocities. The measured values for the CRAH units are presented in Table  1, showing only a small difference in velocity between the units. For the raised floor configuration, the flow rates were measured through all the perforated tiles ( Table 2). Each perforated tile was divided into five regions and the average velocities were used to calculate the inlet boundary conditions. To resolve the perforated tiles a very fine mesh is required when using CFD. A fully open tile model is however regarded to be adequate for tiles if the percentage of open area is 50% or more [21]. Therefore, the perforated tiles were modeled as open in this work. As observed in Table 2, there was a variation in velocity through the individual tiles. A

Boundary Conditions
Flow through the servers were not resolved in this study. Boundary conditions were instead applied to the inlet and outlets of the servers and the CRAH units, i.e., the so-called black box model [19] was used. The supply temperature and the flow rates from both the CRAH units and the perforated tiles were determined from measurements. The hotwire thermo-anemometer VTlO0 from Kimo was used to measure the temperatures, while flow rates were measured with the DP-CALC Micromanometer Model 8715 and the Velocity Matrix add-on kit from TSI. The Velocity Matrix provides area-averaged multi-point face velocity measurements. It calculates the average velocity based on the difference between total and static pressures [20]. The accuracy of the velocity and temperature measurements are ±3% of reading ±0.04 m/s for the velocity range 0.25-12.5 m/s and ±0.4% of reading ±0.3 • C for temperatures from −20 • C to 80 • C. The supply temperature was measured to 18.3 • C for both configurations and hence independent of the measurement location. For the hard floor configuration, the flow rates were measured through the front sides of all the CRAH units. Each front side was divided into 15 measurement zones and the inlet conditions were determined from average velocities. The measured values for the CRAH units are presented in Table 1, showing only a small difference in velocity between the units. For the raised floor configuration, the flow rates were measured through all the perforated tiles ( Table 2). Each perforated tile was divided into five regions and the average velocities were used to calculate the inlet boundary conditions. To resolve the perforated tiles a very fine mesh is required when using CFD. A fully open tile model is however regarded to be adequate for tiles if the percentage of open area is 50% or more [21]. Therefore, the perforated tiles were modeled as open in this work. As observed in Table 2, there was a variation in velocity through the individual tiles. A possible reason for this was the different loads of the server racks presented in Table 3 and the geometrical effects of the underfloor spacing. It was furthermore assumed that the mass flow rates supplied into the room where equal to the mass flow rate out of the room. The outlets were located on top of the CRAH units. The heat load produced by the servers results in a temperature difference across the servers which can be obtained from: where q is the heat load, m is the mass flow rate and c p is the specific heat capacity. A constant value of c p = 1004.4 J/KgK was used in all simulations. The flow rates were estimated by measuring the temperature difference across each server rack with temperature sensors from Raritan called DPX2-T1H1. The sensor has an accuracy of ±1 • C for temperatures from −25 • C to 75 • C. The sensors were placed 1.08 m above the floor on both sides of all server racks. The temperature difference was used to compute the mass flow rate through each server rack. The load of each server rack as well as the corresponding mass flow rates are presented in Table 3 for the raised floor configuration, and in Table 4 for the hard floor configuration. The total heat load in the server room were thus similar for the two cases, while a distinct variation of heat load was present for the different racks. All walls were assumed adiabatic; hence, no heat losses from the server room were accounted for. A no-slip boundary condition implying zero velocity at all surfaces was furthermore applied. Leakage of air from the server room was not included.

Validation
Velocity was measured for both raised and hard floor configurations with open aisles to validate the simulations. The experimental approach was similar to the one presented in Wibron et al. [15]. The anemometer ComfortSense Mini from Dantec with the omnidirectional 54T33 Draught probe was used in the experiments. The accuracy is ±2% of reading ±0.02 m/s for the velocity range 0.05-1 m/s, and ±5% of reading for the velocity range 1-5 m/s. Velocity was measured in the locations L1-L5 presented in Figure 2, at five heights (0.6 m, 1.0 m, 1.4 m, 1.8 m and 2.2 m). The chosen duration, 60 s, and sampling frequency, 10 Hz, of the measurements was after evaluation considered sufficient for the time-averaged velocities.

Grid Study
The raised floor configuration with an open aisle was chosen for the grid study. Following the methods described in [15], Grid Convergence Index (GCI) and Richardson's extrapolation [22] was utilized to evaluate the discretization error. The solutions of three tetrahedral grids created in ANSYS Meshing were compared in the study. The refinement factor of the grids was determined from: The number of elements are represented by N and the number of dimensions by D. Error bars for the computed profiles was determined using the GCI with an average value of p = P ave as a measure of the global order of accuracy [23]. For a constant refinement factor, the order of p was calculated as: where ε 32 = φ 3 -φ 2 , ε 21 = φ 2 -φ 1 . φ 3 , φ 2 and φ 1 are the solutions on the coarse, medium and fine grid, respectively. The relative error was approximated from: The extrapolated values were:

Simulation Settings
Time-averaged flow fields from transient simulations were used in the analysis since steady-state simulations did not converge due to fluctuations in the flow field. Values of velocity and temperature were logged at locations L1-L5 to ascertain that the simulations reached a statistically steady state. After a start up simulation period of 60 s, transient averages where recorded for 360 s to obtain time-independent results. A constant time-step equal to 0.05 s was decided on the basis of convergence. The time-step corresponded to an RSM Courant number of around 2 at operating conditions. The convergence criteria were set to 10 −5 for each time-step.
A common choice of turbulence model for data center applications is the k-ε model. However, the k-ε model is less accurate in regions of low velocity and the Reynolds stress model (RSM) is therefore a better choice [15]. Therefore, the RSM was used to model turbulence in this study. The RSM does not assume isotropic turbulence. Instead, it solves additional transport equations for the six individual Reynolds stresses in order to obtain closure of the Reynolds-averaged Navier-Stokes (RANS) equations that govern the transport of the time-averaged flow quantities. A wall function approach was used in ANSYS CFX to model the boundary layer where the velocity approaches zero close to solid walls.
Buoyancy effects should also be taken into account when modeling the airflow in data centers [24]. The Archimedes number represents the ratio of natural and forced convection and is expressed by: where L is a vertical length scale, ∆T is the temperature difference between hot and cold regions and V is a characteristic velocity. An Archimedes number of order 1 or larger indicates that natural convection dominates. In this study, Ar ≈ 2 and buoyancy is therefore included with the full buoyancy model in ANSYS CFX.

Performance Metrics
The Rack Cooling Index (RCI) is a measure of how well the servers are cooled and maintained within thermal guidelines [25,26]. Over-temperature conditions exist when the intake temperature at some server racks exceed the maximum recommended temperature. A summation of the over-temperatures of each rack gives the total over-temperature. RCI can be used to evaluate under-temperature conditions as well, but only the high side of the temperature range is analyzed here. The RCI was defined as: where the maximum allowable over-temperature is a summation of the difference between the maximum allowable and maximum recommended temperature across all the server racks. The RCI reaches its ideal value of 100% if all rack intake temperatures are below the maximum recommended temperature. Values below 100% mean that the intake temperature for one or more server rack exceeds the maximum recommended temperature. The RCI can be improved by increasing the flow rate or decreasing the temperature. However, these actions would increase the energy consumption. The Return Temperature Index (RTI) is a measure of the energy performance of the cooling system [27]. It is defined as: where T return , T supply and ∆T equip are weighted averages of the return temperature, supply temperature and temperature difference across the server racks, respectively. Deviations from 100% indicate declining performance. Values above 100% indicate mainly hot air recirculation and values below 100% indicate mainly cold air bypass. The Capture Index (CI) is based on the airflow patterns associated with either the supply of cold air or the removal of hot air [28]. The CI based on the supply of cold air is considered here and it is defined as the fraction of air that originates from the CRAH units. The ideal value is 100% which means that there is no hot air recirculation. Otherwise, the value is lower.

Results and Discussion
Discretization and modelling errors are first addressed through grid study and comparison with experimental measurements. The performance metrics CI, RCI and RTI are then compared for different confinement strategies and supply flow rates.

Grid Study
Errors originating from discretization was assessed using the GCI and Richardson's extrapolation for the raised floor configuration with an open aisle. The number of elements were 430,316, 932,066 and 2,025,136 for the coarse, medium and fine grid, respectively, leading to r = 1.3. The maximum cell sizes were 0.140 m, 0.106 m and 0.080 m for the respective grids. Inflation layers were used to attain a better resolution close to the walls, leading to an average y + of around 25. The flow field in a data center is complex with large velocity gradients depending on the position. This makes it suitable for the grid convergence study and transient averages of the velocity profiles along L5-a position where the inlet supply air flow is redirected towards the servers and where experimental data also exists-to be compared. The velocity profiles of three grids are presented in Figure 3a together with an extrapolated curve based on Richardson's extrapolation. The local order of accuracy p ranges from 0.0232 to 26.86, with a global average of 3.640. Calculated error bars for the fine grid are presented in Figure 3b. Based on the grid study, the fine grid is considered to give acceptable results and all further work was carried out with this grid size.

Validation
Both configurations with open aisles have been used for validation. For this study, where one of the main focuses is on the airflow in terms of recirculation between aisles, velocity was chosen as the means of comparison. With the velocity measurements, the validity of the correlation between server

Validation
Both configurations with open aisles have been used for validation. For this study, where one of the main focuses is on the airflow in terms of recirculation between aisles, velocity was chosen as the means of comparison. With the velocity measurements, the validity of the correlation between server load and mass flow (Equation (1)) can also be analyzed. A comparison of transient averages of velocity profiles and experimental values is shown in Figure 4. In general, good agreement was found between simulations and experimental values even though the flow field is very complicated with large velocity gradients. For the hard floor configuration, L1 was located between the inlets of two of the CRAH units and L5 was located behind a server rack inside the hot aisle. The simplified boundary conditions managed to recreate the velocity profiles from the measurements. At L3, outside the end of the hot aisle, the velocity was under-estimated by the simulations. It indicated that the airflow was pushed upwards and stays close to the ceiling on its way back to the CRAH units to a higher extent in the simulations than in the existing data center. For the raised floor configuration, very good agreement was obtained for L2-L5, while deviations were found for L1. This measurement location was close to the cover in front of the CRAH units and large velocity gradients were found in this region. L5 was located above a perforated tile in the cold aisle, indicating that the fully open tile model gave an accurate velocity profile.

Flow Fields
To illustrate the flow field of the air, Figure 5 shows transient averages of velocity and temperature on a plane through C2 and C4 (see Figure 2) for the hard floor configuration with an open aisle. It is apparent that there are large velocity gradients. The CRAH units supplied cold air at a higher flow rate than demanded by the server racks, hence parts of the cold air were immediately returned to the units without entering the servers. Low temperatures were found in front of the server racks while the highest temperatures were found close to the ceiling. Figure 6 correspondingly shows transient averages of velocity and temperature on a plane through C2 and C4 (see Figure 2) for the raised floor configuration with open aisles. As well as this, part of the cold air returned directly to the CRAH unit without entering the servers. Low temperatures were found in front of the server racks while most of the heat was retrieved at the back of the server racks. Flow and temperature fields further indicated less mixing of hot and cold air returning to the CRAH unit for the raised floor configuration (compare Figures 5 and 6).

Flow Fields
Transient averages of velocity and temperature on a plane through C2 and C4 (see Figure 2) is presented in Figure 5 for the hard floor configuration with an open aisle. It is apparent that there are large velocity gradients. The CRAH units supplied cold air at a higher flow rate than required by the servers, hence parts of the cold air were immediately returned to the units without entering the servers. Low temperatures were found in front of the server racks while the highest temperatures were found close to the ceiling. Figure 6 correspondingly shows transient averages of velocity and temperature on a plane through C2 and C4 (see Figure 2) for the raised floor configuration with open aisles. As well as this, part of the cold air returned directly to the CRAH unit without entering the servers. Low temperatures were found in front of the server racks while most of the heat was retrieved at the back of the server racks. Flow and temperature fields further indicated less mixing of hot and cold air returning to the CRAH unit for the raised floor configuration (compare Figures 5 and 6).

Comparison of Setups at Operating Conditions
Side-and top-covers of the aisle between the server racks were studied to investigate methods to improve the airflow management. A top cover between the server racks in the hard floor configuration was not expected to have a positive influence on the air management, but was still included for the comparison. To get quantitative measures of the differences between setups, the performance metrics RCI, RTI and CI were used for evaluation. Since the RCI is based on the rack intake temperatures, these temperatures are also further investigated. Figures 7 and 8 illustrate the rack intake temperatures for the hard and raised floor configurations, respectively. The viewing direction was normal to the front sides of the server racks where outlet conditions were applied. For the hard floor configurations, there were hotspots close to the end of the rows in the setup with an open aisle. The side cover reduced the extent of the hotspots. As shown in Figure 8, the hotspots were eliminated in the raised floor configurations. The only tendency to hotspots were close to the end of the rows in the setup with an open aisle. However, the intake temperatures were kept well below the maximum recommended temperature in this setup, meaning that ideal values of the RCI were obtained (Figure 9a). This result was expected based on the rack intake temperatures and given that the flow rate supplied by the CRAH units or perforated tiles exceeded the flow rate required by the server racks. The servers were very well cooled but cold air bypass still existed. Therefore, poor values of the RTI were obtained for all setups (Figure 9b). Hence, the servers were cooled and maintained within the thermal guidelines but the energy performance was inadequate.

Comparison of Setups at Operating Conditions
Side-and top-covers of the aisle between the server racks were studied to investigate methods to improve the airflow management. A top cover between the server racks in the hard floor configuration was not expected to have a positive influence on the air management, but was still included for the comparison. To get quantitative measures of the differences between setups, a combination of the performance metrics RCI, RTI and CI were used for evaluation. Since the RCI is based on the rack intake temperatures, these temperatures are also further investigated. Figures 7  and 8 illustrate the rack intake temperatures for the hard and raised floor configurations, respectively. The viewing direction was normal to the front sides of the server racks where outlet conditions were applied. For the hard floor configurations, there were hotspots close to the end of the rows in the setup with an open aisle. The side cover reduced the extent of the hotspots. As shown in Figure 8, the hotspots were eliminated in the raised floor configurations. The only tendency to hotspots were close to the end of the rows in the setup with an open aisle. However, the intake temperatures were kept well below the maximum recommended temperature in this setup, meaning that ideal values of the RCI were obtained (Figure 9a). This result was expected based on the rack intake temperatures and given that the flow rate supplied by the CRAH units or perforated tiles exceeded the flow rate demanded by the server racks. The servers were very well cooled but cold air bypass still existed. Therefore, poor values of the RTI were obtained for all setups (Figure 9b). Hence, the servers were cooled and maintained within the thermal guidelines but the energy performance was inadequate.   Hotspots in a data center can be an effect of poor airflow management. CI is based on the airflow patterns, indicating areas where hot air from the servers was circulated to the server inlets, which might explain why some server racks were more exposed to hotspots than others. The CI for all the  Hotspots in a data center can be an effect of poor airflow management. CI is based on the airflow patterns, indicating areas where hot air from the servers was circulated to the server inlets, which might explain why some server racks were more exposed to hotspots than others. The CI for all the Hotspots in a data center can be an effect of poor airflow management. CI is based on the airflow patterns, indicating areas where hot air from the servers was circulated to the server inlets, which might explain why some server racks were more exposed to hotspots than others. The CI for all the server racks in the different setups are shown in Table 5. The values were close to ideal for all servers in all setups, except for R1 in the hard floor configurations. It was located at the end of a row, which increased the risk of hot air recirculation since it can occur not only over the server rack, but around it as well. The fact that the CI was worse for R1 than for R1O, which was located at the end of the other row, was because the airflow rates demanded by R1-R3 were so large. It can be concluded that the temperature distributions in Figures 8 and 9 were directly reflected in the values of CI, although the CI value is not based on temperature. It is also apparent that the CI value discloses local problems with the cooling that are smeared out in the RCI performance metrics.

Comparison of Setups with 50% Supply Flow Rate
The setups based on the conditions in the existing data center were cooled according to the thermal guidelines, but the energy performance was poor since the supply flow rate exceeded the flow rate demanded by the server racks. Therefore, the same setups with only 50% of the original supply flow rate were investigated. Figures 10 and 11 illustrate rack intake temperatures for the hard and raised floor configurations, respectively. Reducing the supply flow rate gave more apparent hotspots, especially close to areas of high server load, but the same tendencies for recirculation appeared for the setups with 100% supply flow rate. For the hard floor configurations, hotspots were found at the end of the rows. When the side cover is used, the performance of the cooling system was improved, but hotspots were still found high up on the server racks. This indicated hot air recirculation over the rows instead of around them. The hotspots were reduced in the raised floor configurations. The side cover showed the lowest intake temperatures, while the top cover did not show any apparent improvement when compared to a fully open aisle when the supply rate was reduced. Energies 2019, 12, x FOR PEER REVIEW 15 of 18 Figure 10. Rack intake temperatures for the hard floor configurations with 50% supply flow rate. Figure 11. Rack intake temperatures for the raised floor configurations with 50% supply flow rate.  . Rack intake temperatures for the raised floor configurations with 50% supply flow rate. Figure 11. Rack intake temperatures for the raised floor configurations with 50% supply flow rate.
Although there obviously were hot areas in several of these setups, ideal or close to ideal values of RCI were obtained, (Figure 12a). The values of RTI were considerably improved compared to an airflow of 100% (Figure 12b), since the extent of cold air bypass decreased when the supply flow rate was reduced. Although there obviously were hot areas in several of these setups, ideal or close to ideal values of RCI were obtained, (Figure 12a). The values of RTI were considerably improved compared to an airflow of 100% (Figure 12b), since the extent of cold air bypass decreased when the supply flow rate was reduced. The CI for all the server racks in the different setups with 50% supply flow rate are shown in Table 6. Lower values than for the setups with 100% supply flow rate were obtained for several server racks. A low value of CI does however not automatically imply that temperatures are above recommended values. If the temperature of the surrounding environment is sufficiently low, the server rack will still be adequately cooled. A cooling set up with low CI is however usually more complex and unpredictable, and high values of CI are therefore preferable. This is also an indication that a combination of performance metrics based on both rack intake temperatures (Figures 10-12) and airflow patterns provide a more comprehensive result than a single performance metric does.

Conclusions
In this study, the effects of using partial aisle containment in the design of data centers were investigated using an experimentally validated CFD model. In general, good agreement was found between simulations and experimental values. Hard and raised floor configurations with either open or partially contained aisles were compared with respect to airflow management. The setups based on the conditions in the existing data center were very well cooled since the supply flow rate exceeded the flow rate demanded by the server racks, and the values of RTI were relatively low, about 40% and 50% for the hard and raised floor configurations, respectively. Hence, the indices indicate that the servers are cooled and maintained within the thermal guidelines but the energy performance is poor. Therefore, the same setups with only 50% of the original supply flow rate were also The CI for all the server racks in the different setups with 50% supply flow rate are shown in Table 6. Lower values than for the setups with 100% supply flow rate were obtained for several server racks. A low value of CI does however not automatically imply that temperatures are above recommended values. If the temperature of the surrounding environment is sufficiently low, the server rack will still be adequately cooled. A cooling set up with low CI is however usually more complex and unpredictable, and high values of CI are therefore preferable. This is also an indication that a combination of performance metrics based on both rack intake temperatures (Figures 10-12) and airflow patterns provide a more comprehensive result than a single performance metric does.

Conclusions
The effects of using partial aisle containment in the design of data centers were investigated in this study using an experimentally validated CFD model. In general, good agreement was found between simulations and experimental values. Hard and raised floor configurations with either open or partially contained aisles were compared with respect to airflow management. The setups based on the conditions in the existing data center were very well cooled since the supply flow rate exceeded the flow rate required by the server racks, and the values of RTI were relatively low, about 40% and 50% for the hard and raised floor configurations, respectively. Hence, the indices indicate that the servers are cooled and maintained within the thermal guidelines but the energy performance is poor. Therefore, the same setups with only 50% of the original supply flow rate were also investigated. The supply flow rate was then approximately the same as the flow rate through the server racks. Reducing the supply flow rate gave more apparent hotspots and the values of RTI were raised to over 80% in all setups, but the same tendencies as in the setups with 100% supply flow rate were found.
Based on the combination of performance metrics where both rack intake temperatures and airflow patterns were taken into account, the airflow management was significantly improved in the raised floor configurations. For the hard floor configurations, there were hotspots close to the end of the rows. The side cover reduced the extent of these. However, using the top cover reinforced the hot air recirculation around the rows which declined the performance of the cooling system. For the raised floor configurations, partially contained aisles were preferred compared with open aisles, and the side cover performed better than the top cover, especially in the case of decreased flow rate.
In addition to hot spots at the end of the rows, potential hot spots were observed especially in the vicinity of the areas of high server load, hence individual supply flow rates between computer room air handling (CRAH) units should be further investigated.