International journal of advanced smart convergence

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Implementation of Light-weight I/O Stack for NVMe-over-Fabrics

  • Ahn, Sungyong (School of Computer Science and Engineering, Pusan National University)
  • Received : 2020.08.19
  • Accepted : 2020.08.29
  • Published : 2020.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Most of today's large-scale cloud systems and enterprise data centers are distributing resources to improve scalability and resource utilization. NVMe-over-Fabric protocol allows submitting NVMe commands to a remote NVMe SSD through RDMA (Remote Direct Memory Access) network. It is attracting attention recently because it is possible to construct a disaggregation storage system with low latency through the protocol. However, the current I/O stack of NVMe-over-Fabric has an inefficient structure for maintaining compatibility with the traditional I/O stack. Therefore, in this paper, we propose a new mechanism to reduce I/O latency and CPU overhead by modifying I/O path of NVMe-over-Fabric to pass through legacy block layer. According to the performance evaluation results, the proposed mechanism is able to reduce the I/O latency and CPU overhead by up to 22% and 24% compared to the existing NVMe-over-Fabrics protocol, respectively.

Keywords

References

  1. A. Klimovic, C. Kozyrakis, E. Thereska, B. John, and S. Kumar, "Flash storage disaggregation," in Proc. 11th European Conference on Computer Systems (EuroSys '16), pp. 1-15, April 18-21, 2016. DOI: https://doi.org/10.1145/2901318.2901337
  2. NVM Express Base Specification Revision 1.4a, https://nvmexpress.org/wp-content/uploads/NVM-Express-1_4a- 2020.03.09-Ratified.pdf.
  3. NVMe_Over_Fabrics, https://nvmexpress.org/wp-content/uploads/NVMe_Over_Fabrics.pdf
  4. Flexible I/O Tester, https://github.com/axboe/fio.
  5. NVM Express over Fabrics Specification Revision 1.1, https://nvmexpress.org/wp-content/uploads/NVMe-over- Fabrics-1.1-2019.10.22-Ratified.pdf.
  6. R. Mittal, A. Shpiner, A. Panda, E. Zahavi, A. Krishnamurthy, S. Ratnasamy, and S. Shenker, "Revisiting network support for RDMA," in Proc. the 2018 Conference of the ACM Special Interest Group on Data Communication (SIGCOMM '18), pp. 313-326, Aug. 20-25, 2018 DOI: https://doi.org/10.1145/3230543.3230557
  7. Z. Guz, H. Li, A. Shayesteh, and V. Balakrishnan, "NVMe-over-fabrics performance characterization and the path to low-overhead flash disaggregation," in Proc. 10th ACM International Systems and Storage Conference, pp. 1-9, May 22-24, 2017. DOI: https://doi.org/10.1145/3078468.3078483
  8. J. Choi, H. Eom, and H. Yeom, "An Evaluation of Shared-Disk File Systems Using NVMe-over-Fabrics," in Proc. Korea Software Congress 2017, pp. 85-87, Dec. 20-22, 2017.
  9. J. Choi, H. Yeom, and H. Han, “An Evaluation of NVMe-over-Fabrics on PCIe JBOF SSDs,” Journal of KIISE, Vol. 46, No. 6, pp. 499-505, June 2019. DOI: https://doi.org/10.5626/JOK.2019.46.6.499
  10. Y. Zhu, W. Yu, B. Jiao, K. Mohror, A. Moody and F. Chowdhury, "Efficient User-Level Storage Disaggregation for Deep Learning," in Proc. 2019 IEEE International Conference on Cluster Computing (CLUSTER), pp. 1-12, Sept. 23-26, 2019. DOI: https://doi.org/10.1109/CLUSTER.2019.8891023
  11. D. Han and B. Nam, "Improving Access to HDFS using NVMeoF," in Proc. 2019 IEEE International Conference on Cluster Computing (CLUSTER), pp. 1-2, Sep. 23-26, 2019. DOI: https://doi.org/10.1109/CLUSTER.2019.8890996