The Sea:JOURNAL OF THE KOREAN SOCIETY OF OCEANOGRAPHY (한국해양학회지:바다)

The Korean Society of Oceanography (한국해양학회)
권호 표지
DOI QR코드

DOI QR Code

Performance Optimization of Numerical Ocean Modeling on Cloud Systems

클라우드 시스템에서 해양수치모델 성능 최적화

  • JUNG, KWANGWOOG (School of Earth and Environmental Sciences, Seoul National University) ;
  • CHO, YANG-KI (School of Earth and Environmental Sciences/ Research Institute of Oceanography, Seoul National University) ;
  • TAK, YONG-JIN (Department of Atmospheric sciences, Yonsei University)
  • 정광욱 (서울대학교 지구환경과학부) ;
  • 조양기 (서울대학교 지구환경과학부/해양연구소) ;
  • 탁용진 (연세대학교 대기과학과)
  • Received : 2022.04.26
  • Accepted : 2022.08.03
  • Published : 2022.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, many attempts to run numerical ocean models in cloud computing environments have been tried actively. A cloud computing environment can be an effective means to implement numerical ocean models requiring a large-scale resource or quickly preparing modeling environment for global or large-scale grids. Many commercial and private cloud computing systems provide technologies such as virtualization, high-performance CPUs and instances, ether-net based high-performance-networking, and remote direct memory access for High Performance Computing (HPC). These new features facilitate ocean modeling experimentation on commercial cloud computing systems. Many scientists and engineers expect cloud computing to become mainstream in the near future. Analysis of the performance and features of commercial cloud services for numerical modeling is essential in order to select appropriate systems as this can help to minimize execution time and the amount of resources utilized. The effect of cache memory is large in the processing structure of the ocean numerical model, which processes input/output of data in a multidimensional array structure, and the speed of the network is important due to the communication characteristics through which a large amount of data moves. In this study, the performance of the Regional Ocean Modeling System (ROMS), the High Performance Linpack (HPL) benchmarking software package, and STREAM, the memory benchmark were evaluated and compared on commercial cloud systems to provide information for the transition of other ocean models into cloud computing. Through analysis of actual performance data and configuration settings obtained from virtualization-based commercial clouds, we evaluated the efficiency of the computer resources for the various model grid sizes in the virtualization-based cloud systems. We found that cache hierarchy and capacity are crucial in the performance of ROMS using huge memory. The memory latency time is also important in the performance. Increasing the number of cores to reduce the running time for numerical modeling is more effective with large grid sizes than with small grid sizes. Our analysis results will be helpful as a reference for constructing the best computing system in the cloud to minimize time and cost for numerical ocean modeling.

최근 클라우드 컴퓨팅 환경에서 해양수치모델 실험을 수행하는 많은 연구가 활발하게 진행되고 있다. 클라우드 컴퓨팅 환경은 대규모 자원이 필요한 해양수치모델을 구현하는데 매우 효과적인 수단이 될 수 있다. 정보처리 기술의 발달로 클라우드 컴퓨팅 시스템은 가상화와 원격 고속 네트워크, 직접 메모리 액세스와 같은 수치모델의 병렬처리에 필요한 다양한 기술과 환경을 제공한다. 이러한 새로운 기능은 클라우드 컴퓨팅 시스템에서 해양수치모델링 실험을 용이하게 한다. 많은 과학자들과 엔지니어들은 해양수치모델 실험에 있어서 가까운 미래에 클라우드 컴퓨팅이 주류가 될 것으로 기대하고 있다. 해양수치모델링을 위한 클라우드 컴퓨팅의 처리성능 분석은 수치모델의 수행 시간과 리소스 활용량을 최소화하는 데 도움이 될 수 있으므로 최적의 시스템을 적용하는 데 필수적이다. 특히 모델 격자 내 다양한 변수들이 다차원 배열 구조로 되어 있기 때문에 대량의 입출력을 처리하는 해양수치모델의 구조는 캐시메모리의 효과가 크며, 대량의 자료가 이동하는 통신 특성으로 인해서 네트워크의 속도가 중요하다. 최근에 주요한 컴퓨팅환경으로 자리잡고 있는 클라우드 환경이 이러한 해양수치모델을 수행하기에 적합한지 실험을 통해서 검토할 필요가 있다. 본 연구에서는 상용 클라우드 시스템에서 해양수치모델로 대표적인 Regional Ocean Modeling System (ROMS)와 더불어 다른 해양모델의 클라우드 환경으로 전환에도 도움이 될 수 있게 병렬처리 시스템의 성능을 측정할 수 있는 표준 벤치마킹 소프트웨어 패키지인 High Performance Linpack을 활용하여 초당 부동소수점 연산횟수 처리능력과 및 STREAM 벤치마크를 활용하여 다중 노드들로 구성된 수치모델용 클러스터의 메모리처리성능을 평가하고 비교하였다. 이러한 평가내용은 클라우드 환경에서 해양수치모델을 어떻게 수행할 것인가에 대해 중요한 정보를 제공할 수 있다. 가상화 기반 상용 클라우드에서 얻은 실제 성능 자료와 구성 설정 분석을 통해 가상화 기반 클라우드 시스템에서 해양수치모델의 다양한 격자 크기에 대한 컴퓨터 리소스의 효율성을 평가했다. 본 연구를 통해서 캐시 계층과 용량이 큰 메모리를 사용하는 HPC 클러스터가 ROMS의 성능에 매우 중요하다는 것을 발견했다. 수치모델링의 실행 시간을 줄이기 위해 코어 수를 늘리는 것은 작은 격자 보다 큰 격자 모델에서 더 효과적이다. 이러한 처리 성능 분석 결과는 클라우드 컴퓨팅 시스템에서 해양수치모델을 효율적으로 구축하는 데 중요한 자료로 이용될 것이다.

Keywords

Acknowledgement

이 논문은 2022년도 정부(해양수산부)의 재원으로 해양수산과학기술진흥원-해양기후변화 통합관측·장기전망 기반 구축 사업 지원을 받아 수행된 연구임(KIMST-20220033).

References

  1. Amante, C. and B.W. Eakins, 2009. ETOP01 1 arc-minute global relief model: Procedures, data sources and analysis, NOAA Tech. Memo., NESDIS NGDC-24, 19 pp.
  2. Antonov, J.I., D. Seidov, T.P. Boyer, R.A. Locarnini, A.V. Mishonov, H.E. Garcia, O.K. Baranova, M.M. Zweng and D.R. Johnson, 2010. World Ocean Atlas 2009, Volume 2: Salinity. S. Levitus, Ed. NOAA Atlas NESDIS 69, U.S. Government Printing Office, Washington, D.C., 184 pp.
  3. AWS, 2017. Amazon Machine Images, Available at: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html (Accessed: 03 March 2022).
  4. AWS, 2018. Spot-Instance, Available at: https://aws.amazon.com/ec2/spot/?nc1=h_ls/ (Accessed: 03 Feb 2019).
  5. AWS, 2022. Instance types, Available at: http://aws.amazon.com/ec2/instance-types (Accessed: 03 March 2022).
  6. Carton, J.A. and B.S. Giese, 2008. A reanalysis of ocean climate using Simple Ocean Data Assimilation (SODA). Mon. Weather Rev., 136(8): 2999-3017. https://doi.org/10.1175/2007MWR1978.1
  7. Chen, X., X. Huang, C. Jiao, M. Flanner, T. Raeker and B. Palen, 2017. Running climate model on a commercial cloud computing environment: A case study using Community Earth System Model (CESM) on Amazon AWS. Computers & Geo., 98: 21-25. https://doi.org/10.1016/j.cageo.2016.09.014
  8. Cheng, K.-Y., L.M. Harris and Y.Q. Sun, 2022. Enhancing the accessibility of unified modeling systems: GFDL System for High-resolution prediction on Earth-to-Local Domains (SHiELD) v2021b in a container. Geosci. Model Dev., 15(3): 1097-1105. https://doi.org/10.5194/gmd-15-1097-2022
  9. Dee, D.P., S.M. Uppala, A.J. Simmons, P. Berrisford, P. Poli, S. Kobayashi, 2011. The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Q. J. R. Meteorol. Soc., 137(656): 553-597. https://doi.org/10.1002/qj.828
  10. Egbert, G.D. and S.Y. Erofeeva, 2002. Efficient inverse modeling of Barotropic Ocean Tides. J. Atmos. Oceanic Technol., 19(2): 183-204. https://doi.org/10.1175/1520-0426(2002)019<0183:EIMOBO>2.0.CO;2
  11. Fairall, C.W., E.F. Bradley, D.P. Rogers, J.B. Edson and G.S. Young, 1996. Bulk parameterization of air-sea fluxes for Tropical Ocean-Global Atmosphere Coupled-Ocean Atmosphere Response Experiment. J. Geophys. Res., 101(C2): 3747-3764. https://doi.org/10.1029/95JC03205
  12. Fox-Kemper, B., A. Adcroft, C.W. Boning, E.P. Chassignet, E. Curchitser, G. Danabasoglu, C. Eden, M.H. England, R. Gerdes, R.J. Greatbatch, S.M. Griffies, R.W. Hallberg, E. Hanert, P. Heimbach, H.T. Hewitt, C.N. Hill, Y. Komuro, S. Legg, J.L. Sommer, S. Masina, S.J. Marsland, S.G. Penny, F. Qiao, T.D. Ringler, A.M. Treguier, H. Tsujino, P. Uotila and S.G. Yeager, 2019. Challenges and Prospects in Ocean Circulation Models. Front. Mar. Sci., 6: 65. https://doi.org/10.3389/fmars.2019.00065
  13. Gartner, 2018. Public Cloud Service, Available at: https://www.gartner.com/en/newsroom/press-releases/2018-08-01-gartner-says-worldwide-iaas-public-cloud-services-market-grew-30-percent-in-2017 (Accessed: 01 March 2022).
  14. GFDL, 2022. Geophysical Fluid Dynamics Laboratory, https://www.gfdl.noaa.gov/climate-modeling/ (Accessed 01 2022).
  15. Google, 2019. Pre-emptible VM Instances, Available at: https://cloud.google.com/compute/docs/instances/preemptible (Accessed: 01 Jan 2022).
  16. Gupta, A., L.V. Kale, F. Gioachin, V. March, C.H. Suen, B.-S. Lee, P. Faraboschi, R. Kaufmann and D. Milojicic, 2013. The who, what, why and how of high performance computing in the cloud. 2013 IEEE International Conference on Cloud Computing Technology and Science, 306-314.
  17. HPL, 2016. High-performance Linpack Benchmark, Available at: http://www.netlib.org/benchmark/hpl/index.html (Accessed: 02 Mar 2022).
  18. Intel, 2013. Memory Latency Checker, Available at: https://www.intel.com/content/www/us/en/developer/articles/tool/intelr-memory-latency-checker.html (Accessed: 03 Feb 2019).
  19. Intel, 2017. Xeon Processor Scalable Family Technical Overview, Available at: https://software.intel.com/en-us/articles/intel-xeon-processor-scalable-family-technical-overview (Accessed: 02 Mar 2022).
  20. Intel, 2018. MPI Benchmark, Available at: https://software.intel.com/en-us/articles/intel-mpi-benchmarks (Accessed: 03 Mar 2018).
  21. Jung, K., Y.-K. Cho and Y.-J. Tak, 2021. Containers and orchestration of numerical ocean model for computational reproducibility and portability in public and private clouds: Application of ROMS 3.6. Simul. Model Pract. Theory, 109: 102305. https://doi.org/10.1016/j.simpat.2021.102305
  22. Large, W.G., J.C. McWilliams and S.C. Doney, 1994. Oceanic vertical mixing: A review and a model with a nonlocal boundary layer parameterization. Rev. Geophys., 32(4): 363-403. https://doi.org/10.1029/94RG01872
  23. Locarnini, R.A., A.V. Mishonov, J.I. Antonov, T.P. Boyer, H.E. Garcia, O.K. Baranova, M.M. Zweng and D.R. Johnson, 2010. World Ocean Atlas 2009, Volume 1: Temperature. S. Levitus, Ed. NOAA Atlas NESDIS 68, U.S. Government Printing Office, Washington, D.C., 184 pp.
  24. McCalpin, J.D., 1995. Memory bandwidth and machine balance in current high performance computers. IEEE Computer Society Technical Committee on Computer Architecture (TCCA) Newsletter, 2: 19-25.
  25. McCalpin, J.D., 2017. STREAM: Sustainable memory bandwidth in high performance computers, a continually updated technical report (1991-2007) (Available at: https://www.cs.virginia.edu/stream/) (Accessed: 02 Feb 2022).
  26. Mell, P. and T. Grance, 2011. The NIST Definition of Cloud Computing. National Institute of Standards and Technology Speical Publication 800-145, Gaithersburg, 3 pp.
  27. Microsoft, 2015. Azure support for Linux RDMA, Available at https://azure.microsoft.com/en-us/updates/azure-support-for-linux-rdma (Accessed: 03 Jan 2022).
  28. Montes, D., J.A. Anel, T.F. Pena, P. Uhe and D.C.H. Wallom, 2017. Enabling BOINC in infrastructure as a service cloud system. Geosci. Model Dev., 10(2): 811-826. https://doi.org/10.5194/gmd-10-811-2017
  29. Oesterle, F., S. Ostermann, R. Prodan and G.J. Mayr, 2015. Experiences with distributed computing for meteorological applications: Grid computing and cloud computing. Geosci. Model Dev., 8(7): 2067-2078. https://doi.org/10.5194/gmd-8-2067-2015
  30. Rajan, A., B.K. Joshi, A. Rawat, R. Jha and K. Bhachavat, 2012. Analysis of process distribution in HPC cluster using HPL: 2nd IEEE International Conference on Parallel, Distributed and Grid Computing, Solan, India, 85-88.
  31. ROMS, 2015. Regional Ocean Modeling System (ROMS), Available at: https://www.myroms.org/ (Accessed: 09 Mar 2022).
  32. Seo, G.-H., Y.-K. Cho, B.-J. Cho, K.-Y. Kim, B. Kim and Y.-J. Tak, 2014. Climate change projection in the Northwest Pacific marginal seas through dynamic downscaling. J. Geophys. Res., 119(6): 3497-3516. https://doi.org/10.1002/2013JC009646
  33. Shchepetkin, A.F. and J.C. McWilliams, 2005. The Regional Oceanic Modeling System (ROMS): A split-explicit, free-surface, topography-following-coordinate oceanic model. Ocean Modell., 9(4): 347-404. https://doi.org/10.1016/j.ocemod.2004.08.002
  34. Vorosmarty, C., B. Fekete and B. Tucker, 1996. River discharge database version 1.0 (RivDIS v1.0). Volumes 0 through 6. A contribution to IHP-V Theme 1. Technical Documents in Hydrology Series.
  35. Wang, Q., X. Guo and H. Takeoka, 2008. Seasonal variations of the Yellow River plume in the Bohai Sea: A model study. J. Geophys. Res., 113(C8): C08046.
  36. Zhuang, J., D.J. Jacob, H. Lin, E.W. Lundgren, R.M. Yantosca, J.F. Gaya, M.P. Sulprizio and S.D. Eastham, 2020. Enabling High-Performance Cloud Computing for Earth Science Modeling on Over a Thousand Cores: Application to the GEOS-Chem Atmospheric Chemistry Model. J. Adv. Model. Earth Syst., 12(5): e2020MS002064. https://doi.org/10.1029/2020MS002064
  37. Zhuang, J., D.J. Jacob, J.F. Gaya, R.M. Yantosca, E.W. Lundgren, M.P. Sulprizio and S.D. Eastham, 2019. Enabling Immediate Access to Earth Science Models through Cloud Computing: Application to the GEOS-Chem Model. Bull. Amer. Meteor. Soc., 100(10): 1943-1960. https://doi.org/10.1175/BAMS-D-18-0243.1