IEMEK Journal of Embedded Systems and Applications (대한임베디드공학회논문지)

Institute of Embedded Engineering of Korea (대한임베디드공학회)
권호 표지
DOI QR코드

DOI QR Code

Kubernetes-based Heterogeneous Computational and Accelerator Resource Management System for Various Image Inferences in Edge Computing Environments

HeteroAccel: 엣지 컴퓨팅 환경에서의 다양한 영상 추론을 위한 쿠버네티스 기반의 이종 연산·가속기 자원 관리 시스템

  • Received : 2021.08.31
  • Accepted : 2021.10.18
  • Published : 2021.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Edge Computing enables image-based inference in close proximity to end users and real-world objects. However, since edge servers have limited computational and accelerator resources, efficient resource management is essential. In this paper, we present HeteroAccel system that performs optimal scheduling in Kubernetes platform based on available node and accelerator information for various inference requests. Our experiments showed 25.3% improvement in overall inference performance over the default scheduling scheme in edge computing environment in which four types of inference services are requested.

Keywords

Acknowledgement

이 논문은 2021년도 정부 (과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임 (No. 2020-0-00844, 엣지 서버시스템 자원 관리 및 제어를 위한 경량 시스템 소프트웨어 기술 개발).

References

  1. J. Ren, D. Zhang, S. He, Y. Zhang, T. Li, "A Survey on End-Edge-Cloud Orchestrated Network Computing Paradigms: Transparent Computing, Mobile Edge Computing, Fog Computing, and Cloudlet," ACM Computing Surveys, Vol. 52, No. 6, pp.1-36, 2020.
  2. X. Wang, Y .Han, V.C.M. Leung, D. Niyato, X. Yan, X. Chen, "Convergence of Edge Computing and deep Learning: A Comprehensive Survey," IEEE Communications Surveys & Tutorials, Vol. 22, No. 2, pp. 869-904, 2020. https://doi.org/10.1109/COMST.2020.2970550
  3. Karbon 700, https://www.onlogic.com/computers/rugged/karbon700/
  4. MK400-70, https://www.onlogic.com/mk400-70/
  5. S. Kim, Y. Kim, "A design of GPU container co-execution framework measuring interference among applications," KNOM Review, Vol. 23, No. 1, pp. 43-50, 2020. https://doi.org/10.22670/KNOM.2020.23.1.43
  6. Q. Zeng, Y. Du, K. Huang, K.K. Leung, "Energy-Efficient Resource Management for Federated Edge Learning with CPU-GPU Heterogeneous Computing," arXiv:2007.07122v2 [cs.IT], 2020.
  7. G. Cho, "Hybrid Resource Scheduling Scheme for Video Surveillance in GPU-FPGA Accelerated Edge System," Master's thesis, KAIST, https://koasas.kaist.ac.kr/handle/10203/284802
  8. X. Liu, J. Yang, C. Zou, Q. Chen, "Collaborative Edge Computing With FPGA-Based CNN Accelerators for Energy-Efficient and Time-Aware Face Tracking System," IEEE Transactions on Computational Social Systems (Early Access), doi: 10.1109/TCSS.2021.3059318
  9. OpenVINO, https://docs.openvinotoolkit.org/
  10. Kubernetes, https://kubernetes.io/
  11. Z. Zhong, and R. Buyya, " A Cost-Efficient Container Orchestration Strategy in Kubernetes-Based Cloud Computing Infrastructures with Heterogeneous Resources," ACM Trans. on Internet Technology, Vol. 20, Issue 2, pp. 1-24, 2021 https://doi.org/10.1145/3378447
  12. Prometheus, https://prometheus.io/