The Journal of the Korea institute of electronic communication sciences (한국전자통신학회논문지)

Korea Institute of Electronic Communication Science (한국전자통신학회)
권호 표지
DOI QR코드

DOI QR Code

Performance Evaluation of Big Stream based High Speed Data Storage

빅 스트림 기반 초고속 데이터 스토리지 성능 평가

  • 송민규 (한국천문연구원 전파천문본부) ;
  • 강용우 (한국천문연구원 전파천문본부) ;
  • 김효령 (한국천문연구원 전파천문본부)
  • Received : 2017.09.30
  • Accepted : 2017.10.18
  • Published : 2017.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

It is very hard to find the system which processes single 10Gbps stream, and the related application is also rare. But in the field of science such as physics and astronomy, these high speed systems have been widely used and now more upgraded performance is expected. For this reason, high speed network based storage which captures and records 10Gbps level of packets was developed for the support of small astronomical company in KASI. But for the use of the system in research, system performance should be not only evaluated but also optimized. In this paper, we first implement system environment for the performance evaluation and discuss the experiment procedure and solution to acquire numerical results.

단일 스트림 기준으로 10Gbps에 달하는 데이터 처리를 수행하는 시스템은 매우 제한적이고, 그에 관한 애플리케이션 역시 극소수에 불과한 실정이다. 하지만 이러한 시스템은 물리, 천문 등 과학연구 분야에서 더욱 그 중요성이 부각되고 있으며, 업그레이드 된 성능을 요구하고 있다. 이를 위해 해당 분야의 중소기업 지원을 위해 1년 전 10Gbps 수준으로 네트워크 입력되는 고속의 패킷 수신, 저장을 수행하는 초고속 스토리지가 개발되었다. 하지만 실제 연구 현장에서 사용되기 위해서는 객관화된 성능 평가가 수반되어야 하며 최적화가 이뤄져야 한다. 본 논문에서는 이에 필요한 시스템 환경을 구축한 것을 시작으로 객관화된 성능 지표를 얻기 위한 실험 과정 및 방법론에 대해 논의하였다.

Keywords

References

  1. J. Jang, D. Kim, and C. Choi, "Study on Hybrid Type Cloud System," J. of the Korea Institute of Electronic Communication Sciences, vol. 11, no. 6, 2016, pp. 611-618. https://doi.org/10.13067/JKIECS.2016.11.6.611
  2. H. Hinteregger, A. Rogers, and R. Cappallo, "A high data rate recorder for astronomy," IEEE Trans. Magnetics, vol. 27, no. 3, 1991, pp. 3450-3460.
  3. A. Whitney, M. Kettenis, C. Phillips, and M. Sekido, "VLBI Data Interchange Format(VDIF)," In Proc. IVS The 8th Int. e-VLBI Workshop, Madrid, Spain, June 2009, pp. 1-7.
  4. A. Krizhanovsky, Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer. Los Angeles: LINUX Journal, 2013.
  5. H. Kim and S. Ahn, "BPLRU: A Buffer Management Scheme for Improving Random Writes in Flash Storage," In Proc. USENIX Conf. on File and Storage Technologies, San Jose, 2008, pp. 239-252.
  6. M.-G. Song, Y.-W. Kang and H.-R. Kim, "The Study on the Design and Optimization of Storage for the Recording of High Speed Astronomical Data," J. of The Korea Institute of Electronic Communication Sciences, vol. 12, no. 1, 2017, pp. 75-84. https://doi.org/10.13067/JKIECS.2017.12.1.75
  7. M. Chen, S. Mao, Y. Zhang, and V. C. Leung, Big Data Related Technologies, Challenges and Future Prospects. Heidelberg: Springer, 2014.
  8. S. Kershaw, R. Spencer, R. H. Jones, R. Burgess, S. Casey, A. Rushton and P. Boven, Protocols Performance Report - Final Report on Protocols and Network Infrastructure. Manchester: EXPReS, 2009.
  9. R. Deri, PF_RING API Revision:7479API documentation for PF_RING: high-speed packet capture, filtering and analysis framework. Pisa: ntop, 2016.
  10. W. R. Stevens, TCP/IP Illustrated, Volume 1: The Protocols. Boston: Addison-Wesley, 2001.
  11. A. A. Mamun, G. Guo and C. Bi, Hard Disk Drive Mechatronics and Control. Boca Raton: CRC Press, 2007.
  12. W. Brisken, Walter Brisken's library for reading Mark5 and similar VLBI-format data. Socorro: NRAO, 2014.
  13. J. Wagner, INFORMATION ON WIDEBAND VLBI OPERATION Summaries for operating the Mark6 and FILA10G, DiFX correlation, and extracting Phase Cal. Daejeon: KASI, 2015.