Journal of the Architectural Institute of Korea (대한건축학회논문집)

Architectural Institute of Korea (대한건축학회)
권호 표지
DOI QR코드

DOI QR Code

Assessing Design and Energy Efficiency of Liquid-based Cooling Systems for Data Centers

액체냉매기반 데이터센터 냉각시스템의 설계기준 및 에너지 효율평가에 관한 연구

  • Cho, Jinkyun (Dept. of Building and Plant Engineering, National Hanbat University)
  • 조진균 (국립한밭대학교 설비공학과 )
  • Received : 2024.03.17
  • Accepted : 2024.04.23
  • Published : 2024.05.30
⟨ Previous Next ⟩

Abstract

This study explores the design and energy assessment of liquid-based cooling systems to meet the growing demands and high power densities of IT equipment in data centers. It creates a baseline model for cooling systems to handle a 200 kW ITE equipment load and sets objective criteria for evaluating cooling energy efficiency. This study identified seven distinct data center workloads and points out the limitations of traditional air-cooling methods for IT power densities exceeding 50 kW/rack, suggesting a shift to liquid-based alternatives. Liquid-based cooling systems were classified into three main types: RDEX, plate cooling, and immersion cooling, with specific operational characteristics depending on the system configuration. To gauge system efficiency, this study introduces the TUE metric, correlating the total energy input to the data center with the energy used by computing components. An analysis of energy efficiency showed a significant reduction in the Power Usage Effectiveness (PUE), with immersion cooling achieving an outstanding PUE of 1.05. The TUE metric also revealed considerable efficiency gains across all liquid-based cooling systems compared to traditional air cooling. This study anticipates a rise in the adoption of liquid-based cooling systems and stresses the need for universal design criteria, taking into account ITE power density and energy efficiency.

Keywords

Acknowledgement

이 연구는 2024년도 한국연구재단 연구비 지원에 의한 결과의 일부임. 과제번호: 2022R1F1A1068262

References

  1. AFCOM. (2021). The 2021 state of the data center report: A look at the evolution of our industry 5th edition, The Association for Computer Operations Management.
  2. ASHRAE. (2014). Liquid Cooling Guidelines for Datacom Equipment Centers, 2nd Edition, American Society of Heating Refrigerating and Air-Conditioning Engineers.
  3. ASHRAE. (2018). Datacom equipment power trends and cooling applications, 3rd Edition, American Society of Heating Refrigerating and Air-Conditioning Engineers.
  4. ASHRAE. (2021). Thermal guidelines for data processing environments, 5th Edition, American Society of Heating Refrigerating and Air-Conditioning Engineers.
  5. ASHRAE TC 9.9. (2019). White Paper; Water-Cooled Servers-Common Designs, Components, and Processes, American Society of Heating Refrigerating and Air-Conditioning Engineers.
  6. ASHRAE TC 9.9. (2021). White Paper; Emergence and Expansion of Liquid Cooling in Mainstream Data Centers, American Society of Heating Refrigerating and Air-Conditioning Engineers.
  7. Avelar, V., Azevedo, D., & French, A. (2012). PUETM: a comprehensive examination of the metric, the Green Grid white paper #49. The Green Grid Technical Committee.
  8. Beaty, D., & Schmidt, R. (2004). Back to the future: liquid cooling data center considerations, ASHRAE Journal, 46(12), 42-46.
  9. Birbarah, P., Gebrael, T., Foulkes, T., Stillwell, A., Moore, A., Pilawa-Podgurski, R., & Miljkovic, N. (2020). Water immersion cooling of high power density electronics, International Journal of Heat and Mass Transfer, 147, 118918.
  10. Bizo, D., Ascierto, R., Lawrence, A., & Davis, J. (2021). Uptime institute global data center survey 2021; growth stretches an evolving sector, Uptime Institute.
  11. Cho, J. (2021). An analysis of the data center energy consumption structure for efficient energy utilization, Journal of the Architectural Institute of Korea, 37(8), 153-164. https://doi.org/10.5659/JAIK.2021.37.8.153
  12. Cho, J. (2023a). Operation reliability and energy optimization for a hot-standby cooling system of data center, Journal of the Architectural Institute of Korea, 39(3), 189-199. https://doi.org/10.5659/JAIK.2023.39.3.189
  13. Cho, J. (2023b, December). Issue Briefing 1: The impact of data centers on various industries, WIN CLAS(IBK webzine) http://ibkmagazine.co.kr/
  14. Cho, J., & Lim, S. B. (2023a). Data center inside+out; mission critical facilities, Munundang Press, Part 2.
  15. Cho, J., & Lim, S. B. (2023b). Balanced comparative assessment of thermal performance and energy efficiency for three cooling solutions in data centers, Energy, 285, 129370.
  16. Cho, J., Park, B., & Jang, S. (2022). Development of an independent modular air containment system for high-density data centers: Experimental investigation of row-based cooling performance and PUE, Energy, 258, 124787.
  17. Cho, J., & Park, W. (2022). A case study on remodeling strategies of mission critical facility for existing data centers based on IT power density, Journal of the Architectural Institute of Korea, 38(5), 147-158.
  18. Dunlap, K., & Rasmussen, N. (2012). Choosing between room, row, and rack-based cooling for data centers, APC white paper 130 rev02, Schneider Electric-Data Center Science Center.
  19. Han, X., Guo, Y., Wang, Q., & Phelan, P. (2018). Optical characterization and durability of immersion cooling liquids for high concentration III-V photovoltaic systems, Solar Energy Materials and Solar Cells, 174, 124-131. https://doi.org/10.1016/j.solmat.2017.08.034
  20. Heydari, A. (2021). High heat density single- and two-phase cooling of data centers, Distinguished data center engineer, NVIDIA.
  21. Hnayno, M., Chehade, A., Klaba H., Polidori, G., & Maalouf, C. (2023). Experimental investigation of a data-centre cooling system using a new single-phase immersion/liquid technique, Case Studies in Thermal Engineering, 45, 102925.
  22. Huang, Y., Ge, J., Chen, Y., & Zhang, C. (2023). Natural and forced convection heat transfer characteristics of single-phase immersion cooling systems for data centers, International Journal of Heat and Mass Transfer, 207, 124023.
  23. Kanbur, B. B., Wu, C., Fan, S., & Duan, F. (2021). System-level experimental investigations of the direct immersion cooling data center units with thermodynamic and thermoeconomic assessments, Energy, 217, 119373.
  24. Khalaj, A. H., & Halgamuge, S. K. (2017). A review on efficient thermal management of air- and liquid-cooled data centers: from chip to the cooling system, Applied Energy, 205, 1165-1188. https://doi.org/10.1016/j.apenergy.2017.08.037
  25. Kibushi, R., Yuki, K., Unno, N., Ogushi, T., Murakami, M., Numata, T., Ide, T., & Nomura, H. (2021). Enhancement of the critical heat flux of saturated pool boiling by the breathing phenomenon induced by lotus copper in combination with a grooved heat transfer surface, International Journal of Heat and Mass Transfer, 179, 121663.
  26. Li, X., Xu, Z., Liu, S., Zhang, X., & Sun, H. (2023). Server performance optimization for single-phase immersion cooling data center, Applied Thermal Engineering, 224, 120080.
  27. Lionello, M., Rampazzo, M., Beghi, A., Varagnolo, D., & Vesterlund, M. (2020). Graph-based modelling and simulation of liquid immersion cooling systems, Energy, 207, 118238.
  28. Luo, Q., Wang, C., Wen, H., & Liu, L. (2022). Research and optimization of thermophysical properties of sic oil-based nanofluids for data center immersion cooling, International Communications in Heat and Mass Transfer, 131, 105863.
  29. Masanet, E., Shehabi, A., Lei, N., Smith, S., & Koomey, J. (2020). Recalibrating global data center energy-use estimates, Science, 367(6481), 984-986. https://doi.org/10.1126/science.aba3758
  30. Nada, S. A., El-Zoheiry, R. M., Elsharnoby, M., & Osman, O. S. (2021). Experimental investigation of hydrothermal characteristics of data center servers' liquid cooling system for different flow configurations and geometric conditions, Case Study in Thermal Engineering, 27, 101276.
  31. Pambudi, N. A., Sarifudin, A., Firdaus, R. A., Ulfa, D. K., Gandidi, I. M. & Romadhon, R. (2022). The immersion cooling technology: current and future development in energy saving, Alexandria Engineering Journal, 61(12), 9509-9527. https://doi.org/10.1016/j.aej.2022.02.059
  32. Parida, P. R., David, M., Iyengar, M., Schultz, M., Gaynes, M., Kamath, V., Kochu-parambil, B., & Chainer, T. (2012, March 18-22). Experimental investigation of water cooled server micro-processors and memory devices in an energy efficient chiller-less data center. the 28th Annual IEEE Semiconductor Thermal Measurement and Management Symposium (SEMI-THERM), 224-231.
  33. Patterson, M. K., Poole, S. W., Hsu, C. H., Maxwell, D., Tschudi, W., Coles, H., Martinez, D. J., & Bates N. (2013). TUE, a New Energy-Efficiency Metric Applied at ORNL's Jaguar, Lecture Notes in Computer Science, 7905, 372-382.
  34. Pires, I. A., Silva, R. A., Pereira, I. T. O., Faria, O. A., Maia, T. A. C., & Filho, B. D. J. C. (2020). An assessment of immersion cooling for power electronics: an oil volume case study, IEEE Transactions on Industry Applications, 56(3), 3231-3237. https://doi.org/10.1109/TIA.2020.2975762
  35. Schmidt, R. Iyengar, M., & Chu, R. (2005). Data centers: meeting data center temperature requirements, ASHRAE Journal, 47(4), 44-48.
  36. Shah, J. M., Eiland, R., Rajmane, P., Siddarth, A., Agonafer, D., & Mulay, V. (2019). Reliability considerations for oil immersion-cooled data centers, Journal of Electronic Packaging, 141(2), 021007.
  37. Shia, D., Yang, J., Sivapalan, S., Soeung, R., & Amoah-Kusi, C. (2021). Corrosion study on single-phase liquid cooling cold plates with inhibited propylene glycol/water coolant for data centers, Journal of Manufacturing Science and Engineering, 143(11), 111012.
  38. Sohel Murshed, S. M. & Nieto de Castro, C. A. (2017). A critical review of traditional and emerging techniques and fluids for electronics cooling, Renewable and Sustainable Energy Reviews, 78, 821-833. https://doi.org/10.1016/j.rser.2017.04.112
  39. Sun, Y., Wang, Y., Zhu, L., Yin, B., Xiang, H., & Huang, Q. (2014). Direct liquid-immersion cooling of concentrator silicon solar cells in a linear concentrating photovoltaic receiver, Energy, 65, 264-271. https://doi.org/10.1016/j.energy.2013.11.063
  40. Taddeo, P., Romani, J., Summers, J., Gustafsson, J., Martorell, I., & Salom, J. (2023). Experimental and numerical analysis of the thermal behaviour of a single-phase immersion-cooled data centre, Applied Thermal Engineering, 234, 121260.
  41. Building Energy Efficiency Standards. (2016). Energy Code (Title 24), The California Energy Commission.
  42. USDD. (1991). Military Handbook: Reliability predicion of electronic equipment (MIL-HDBK-217F), Springfiled.
  43. Wang, Y., Ren, L., Yang, Z., Deng, Z., & Ding, W. (2022). Application of two-phase immersion cooling technique for performance improvement of high power and high repetition avalanche transistorized subnanosecond pulse generators, IEEE Transactions on Power Electronics, 37(3), 3024-3039. https://doi.org/10.1109/TPEL.2021.3111348
  44. Zhang, H., Shao, S., Xu, H., Zou, H., & Tian, C. (2014). Free cooling of data centers: A review, Renewable and Sustainable Energy Reviews, 35, 171-182.  https://doi.org/10.1016/j.rser.2014.04.017