Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Development of scalable big data storage system using network computing technology

네트워크 컴퓨팅 기술을 활용한 확장 가능형 빅데이터 스토리지 시스템 개발

  • Park, Jung Kyu (Department of Computer Software Engineering, Changshin University) ;
  • Park, Eun Young (Department of Biomedical Laboratory Science, Shinhan University)
  • Received : 2019.07.21
  • Accepted : 2019.09.02
  • Published : 2019.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

As the Fourth Industrial Revolution era began, a variety of devices are running on the cloud. These various devices continue to generate various types of data or large amounts of multimedia data. To handle this situation, a large amount of storage is required, and big data technology is required to process stored data and obtain accurate information. NAS (Network Attached Storage) or SAN (Storage Area Network) technology is typically used to build high-speed, high-capacity storage in a network-based environment. In this paper, we propose a method to construct a mass storage device using Network-DAS which is an extension technology of DAS (Direct Attached Storage). Benchmark experiments were performed to verify the scalability of the storage system with 76 HDD. Experimental results show that the proposed high performance mass storage system is scalable and reliable.

4차 혁명시대가 시작됨에 따라 다양한 장비들이 클라우드 기반으로 동작하고 있다. 이와 기기들은 다양한 형태의 데이터 또는 대용량의 멀티미디어 데이터를 계속 생성하고 있다. 이런 상황을 처리하기 위해서는 대용량의 저장장치가 필요하며 또한 저장된 데이터를 처리하고 정확한 정보를 얻기 위해서 빅데이터 기술이 필요하다. 일반적으로 네트워크 환경에서 대용량의 저장장치를 구축하기 위해서는 SAN 또는 NAS 기술이 사용된다. 본 논문에서는 DAS(Direct Attached Storage)의 확장기술인 Network-DAS를 사용하여 대용량 저장장치를 구성하는 방안을 제시한다. 논문에서 제시하는 스토리지 시스템의 확장성과 성능을 검증하기 위해 76개의 하드 디스크를 이용하여 벤치마크 실험을 수행하였다. 실험 결과 제안하는 고성능 대용량 스토리지 시스템의 확장성과 신뢰성을 검증하였다.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (NRF-2018R1C1B5046282).

References

  1. IDC. DataAge 2025 whitepaper [Internet]. Available: https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf.
  2. J. Cha, and S. Kim, "Analysis of I/O Performance for Optimizing Software Defined Storage in Cloud Integration," in Proceeding of the IEEE 3rd International Conference on Communication and Information Systems (ICCIS), Singapore: Singapore, pp. 222-226, 2018.
  3. J. Zhou, D. Dai, Y. Mao, X. Chen, Y. Zhuang, and Y. Chen, "I/O Characteristics Discovery in Cloud Storage Systems," in Proceeding of the IEEE 11th International Conference on Cloud Computing (CLOUD), San Francisco: CA, pp. 170-177, 2018.
  4. J. Park, Y. Bae, and S. Jung, "Technical analysis of Cloud Storage for Cloud Computing," International Journal of Information and Communication Engineering, vol. 17, no. 5, pp. 1129-1137, May. 2013.
  5. S. M. Shin, and V. Roy, "Hybrid key-Based Encryption in Cloud Storage," Asia-pacific Journal of Convergent Research Interchange, vol. 2, no. 3, pp. 29-34, Sep. 2016. https://doi.org/10.21742/apjcri.2016.09.04
  6. E. Tomes, and N. Altiparmak, "A Comparative Study of HDD and SSD RAIDs' Impact on Server Energy Consumption," in Proceeding of the IEEE International Conference on Cluster Computing (CLUSTER), Honolulu: HI, pp. 625-626, 2017.
  7. T. Ariefianto, L. V. Yovita, and D. Olviovitha, "Performance analysis of AoE-SAN using bonding interface over RAID," in Proceeding of the 2nd International Conference on Information and Communication Technology (ICoICT), Bandung: Indonesia, pp. 106-110, 2014.
  8. D. Li, Q. Wang, C. Guyot, A. Narasimha, D. Vucinic, Z. Bandic, and Q. Yang, "Hardware accelerator for similarity based data dedupe," in Proceeding of the IEEE International Conference on Networking, Architecture and Storage (NAS), Boston: MA, pp. 224-232, 2015.
  9. A. Jaikar, S. A. R. Shah, S. Noh, and S, Bae, "Performance Analysis of NAS and SAN Storage for Scientific Workflow," in Proceeding of the International Conference on Platform Technology and Service (PlatCon), Jeju, pp. 1-4, 2016.
  10. T. K. Okada, A. Goldman, and G. G. H. Cavalheiro, "Using NAS Parallel Benchmarks to evaluate HPC performance in clouds," in Proceeding of the IEEE 15th International Symposium on Network Computing and Applications (NCA), Cambridge: MA, pp. 27-30, 2016.
  11. M. Selvaganesan, and M. A. Liazudeen, "An Insight about GlusterFS and Its Enforcement Techniques," in Proceeding of the 2016 International Conference on Cloud Computing Research and Innovations (ICCCRI), Singapore: Singapore, pp. 120-127, 2016.
  12. NDAS. Network Direct Attached Storage [Internet]. Available: https://en.wikipedia.org/wiki/Network_Direct_Attached_Storage.
  13. fio. Flexible I/O tester [Internet]. Available: https://fio.readthedocs.io/en/latest/fio_doc.html.
  14. trace-replay. trace-replay [Internet]. Available: https://github.com/yongseokoh/trace-replay.
  15. Gluster. Gluster scalable network filesystem. [Internet]. Available: https://www.gluster.org/