BLOG | 31 August 2022
THERMAL RIDE THROUGH/SHORT BREAK STUDY OF A DATA CENTRE
Cloud-based data storing in the 21st century
According to the data centre decade report of 2020, the worldwide data centre market has doubled from 2010 to 2019, and it is forecasted to grow even more rapidly. This growth is due to people opting more and more to use cloud-based data storing methods rather than paper-based. Going cloud-based brings advantages like reduced paper trails, lifelong data storage, encrypted data, and easy collaboration on projects.
With the increasing demand for data centres, it has become ever more critical for data centre design engineers to analyse the thermal behaviour of their designs with optimal accuracy and ease. One might want to consider various design conditions when designing a data centre which will require numerous parametric studies or one would like to know if the data centre can withstand a loss of mains power trip. Doing the before-mentioned analyses without engineering software is next to impossible.
“Flownex uses fundamental physics, mathematics and thermo-fluid principles to solve even the most complex thermo-fluid and heat transfer problems.”
Flownex is the industry leader in 1D CFD thermal fluid software for both steady state and transient analysis of large integrated systems. Flownex uses fundamental physics, mathematics and thermo-fluid principles to solve even the most complex thermo-fluid and heat transfer problems. The versatility that Flownex offers means that an engineer can apply it within the data centre industry.
Some of the typical applications of Flownex in the data centre space include the following:
- Simulate large integrated systems to verify designs.
- Simulate transient thermal ride through/short break.
- Quantify expected excursion hours based on typical weather data.
- Quantify expected ATES/HDAC/Cooling tower water usage.
- Control philosophy testing and optimisation.
- Assess performance following design changes.
- Root cause failure analysis.
Thermal ride-through/short break study
Today I am summarising a thermal ride-through/short break study on a simplified data centre case study. The scenario compares the system’s reaction to a power mains failure at two different ambient temperatures, 39°C and 42°C. The chilled water (CHW) supply setpoint is 20°C, and the CRAH air supply setpoint is 25°C. The total IT load of the data centre is 1.2 MW with only the chillers not on UPS and with a generator start-up time of 20s.
“The few seconds between a power trip and generator start-up can be crucial in the uptime of a data centre.”
A short break study is one of the more complex transient analyses on data centres and crucial in understanding the design behaviour during a power loss. The few seconds between a power trip and generator start-up can be critical in the uptime of a data centre.
SYSTEM PIPING
The thermal inertia of the water in the piping system can be captured by importing a PCF file, which is a non-proprietary piping format, typically from Revit into Flownex. The image below displays the piping of the simplified data centre used for this study, imported from Revit into Flownex.
The simplified system consists of eight CRAHs per floor and two floors. The four chillers and HDACs are on the roof. A typical data centre would be more in the order of forty CRAHs per floor and around four to five floors.
CRAH MODELLING
At the heart of the CRAH is an incremented finned tube heat exchanger to characterise the heat transfer performance and pressure drop. This finned tube heat exchanger component calculates the change in water temperature through the tubes and the air temperature over the fins.
The CRAH fan is modelled with a variable speed component. The fan curves can be imported into Flownex and assigned to the variable speed component to capture the pressure rise at different fan speeds. The fan control system is also included to vary the fan speed to control the CRAH supply air temperature.
The pressure-independent control valve (PICV) is modelled on the water side. A valve opening or volume flow setpoint is provided to the component. The PICV is linked to a control loop that changes the valve opening or volume flow setpoint to control the CRAH air supply temperature.
WHITE SPACE MODELLING
For modelling the white space, two major approaches are possible when using Flownex. The first approach is a well-mixed approach, and the second is to couple the system modelled in Flownex with a full 3D CFD co-simulation.
The well-mixed approach assumes that the temperature in the hot and cold aisles is uniform throughout. Each CRAH component has one inlet temperature from the hot aisle and one outlet temperature for the cold aisle. This method substantially simplifies the system, leading to faster solving times. Still, it does, however, require a steady state CFD to calibrate white space recirculation and pressure drop.
For coupling the Flownex system model with a full 3D CFD co-simulation, Flownex can link with 6Sigma Room developed by Future Facilities. A dedicated component is available in Flownex to handle the data transfer between the two software packages and makes co-simulation convenient and hassle-free. This approach is well suited for data centres with a non-uniform load and has fundamental calculations for both white space recirculation and white space pressure drop. However, it comes with the penalty of longer solving times.
For this thermal ride-through study, the well-mixed approach is used. The image below displays the white space and CRAH modelling in Flownex.
ADDITIONAL MODELS
Other equipment typically used in data centres are also modelled in Flownex, such as air and water-cooled chillers, HDACs, cooling towers, etc. The image below displays the cooling equipment used during this study: the water-cooled chillers, HDACs, CHW pumps and cooling ring pumps.
Chillers
It can be pretty challenging to obtain the geometry of the internals of a chiller from the manufacturers. Hence, we have created a reduced-order model that captures the chiller’s performance using the steady state capacity data and chiller restart performance. The chiller’s reduced order model requires no internal geometry or compressor data. This method gives us a robust and accurate chiller model using data accessible from the manufacturer.
HDACs
The HDAC is a heat exchanger that transfers heat to ambient and uses wetted pads to cool the air flowing over the tubes adiabatically. An incremented finned tube heat exchanger is used to model the heat transfer in the chilled water ring. The spray water used for cooling is modelled using an adiabatic saturation model to account for the humidity increase and temperature decrease fundamentally. The fan speed is controlled with a control loop to manage the cooling.
Chilled water pump control logic
The chilled water pumps are controlled based on the pressure difference between the hot and cold rings. The pressure differences are measured between the risers on each floor on all four corners. The median pressure difference is then taken and fed to a master PID that calculates the flow setpoint needed to maintain the pressure difference setpoint. The flow setpoint from the master is provided to the slave PID of each pump to control the pump speed to achieve the required flow setpoint.
FLOWNEX SIMULATION RESULTS
Flownex has a steady state and a transient solver built into the software and applied to the same components. Therefore, there is no need to create separate models. Once the model is set up, you can run the model. Flownex uses a non-iterative transient solver, a very efficient solver for transient simulations. It is, therefore, possible to have quick turnarounds on studies for different scenarios using the fast-solving capabilities of Flownex.
“Flownex uses a non-iterative transient solver, which is a very efficient solver for transient simulations.”
The key results we will look at today for the thermal ride-through study are the steady state temperature distribution, chiller heat transfer and the cold aisle data hall temperatures.
Steady-state temperature distribution
The below image displays the 3D steady-state temperature distribution of the system.
Chiller heat transfer
On the left-hand side of the image below is the heat transfer of the chillers for 39°C ambient, and on the right-hand side for 42°C ambient. At the beginning of the transient, we can see that the chiller capacity is around 320 kW. After 10 seconds, the power trips and after 20 seconds, the generator starts up. One can see that the chiller capacity does not fall to zero after the trip because of how we captured the thermal inertia inside the chiller. Sixty seconds after power is restored, the compressors of the chillers are all online, and cooling can commence according to the restart characteristics of the chiller.
In the above images, one can see the difference that the ambient temperature has on the cooling capacity of the chillers. At a lower ambient temperature, the chillers can do more cooling. For an ambient of 39°C, the chillers max out at 500 kW, but at 42°C, the chillers max out at 400 kW. The constrain is due to the temperature difference between the HDAC and the environment (ambient) being smaller at a higher ambient temperature, limiting the cooling the HDAC can provide to the chiller condenser.
The constraint will have a domino effect on the rest of the system and eventually affect the maximum data hall temperatures experienced.
Data hall cold aisle temp
The average data hall cold aisle temperatures at 39°C ambient are displayed on the left-hand side of the image below. Similarly, on the right-hand side is the results of 42°C ambient.
In the above images, one can see that the maximum data hall temperature occurs for the first floor at both 39°C and 42°C ambient. However, due to the limited chiller capacity at the higher ambient temperature of 42°C, the maximum data hall temperature at this ambient is higher. For an ambient of 39°C, the maximum data hall temperature is around 26.8°C, but at an ambient of 42°C, the data hall temperature sees a maximum of around 27.3°C.
Also, notice the time it takes for the system to reach steady state conditions again. For an ambient of 39°C, steady-state conditions are reached at around 9 minutes, but for an ambient of 42°C steady-state conditions are only reached at approximately 13 minutes. The reason is due to the limited capacity of the chillers when the ambient conditions are 42°C.
“A complex system such as a data centre can be simulated in both steady state and transient using 1D CFD software.”
The above thermal ride-through/short break study showcases one of the many applications of Flownex within the data centre space. Flownex opens a world of possibilities for data centre design engineers. A complex system such as a data centre can be simulated in both steady state and transient using 1D CFD software. An engineer can now weigh up various scenarios and design conditions using Flownex. Flownex as a tool and the specialised consultation services is well equipped to be part of your journey.
If you are a data centre engineer looking to take your designs and analyses to the next level, look no further. Flownex is the tool for you!
Donovan hitchcock
Donovan is a consultant engineer at Flownex who specialises in thermo-fluid systems design.
Share:
