Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

RAVIP: Real-Time AI Vision Platform for Heterogeneous Multi-Channel Video Stream

  • Lee, Jeonghun (Dept. of Embedded Systems Engineering, Incheon National University) ;
  • Hwang, Kwang-il (Dept. of Embedded Systems Engineering, Incheon National University)
  • Received : 2020.07.29
  • Accepted : 2021.03.05
  • Published : 2021.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Object detection techniques based on deep learning such as YOLO have high detection performance and precision in a single channel video stream. In order to expand to multiple channel object detection in real-time, however, high-performance hardware is required. In this paper, we propose a novel back-end server framework, a real-time AI vision platform (RAVIP), which can extend the object detection function from single channel to simultaneous multi-channels, which can work well even in low-end server hardware. RAVIP assembles appropriate component modules from the RODEM (real-time object detection module) Base to create per-channel instances for each channel, enabling efficient parallelization of object detection instances on limited hardware resources through continuous monitoring with respect to resource utilization. Through practical experiments, RAVIP shows that it is possible to optimize CPU, GPU, and memory utilization while performing object detection service in a multi-channel situation. In addition, it has been proven that RAVIP can provide object detection services with 25 FPS for all 16 channels at the same time.

Keywords

Acknowledgement

This research was supported by Incheon National University Research Grant in 2018.

References

  1. Wikipedia, "Digital video recorder," [Online]. Available: https://en.wikipedia.org/wiki/Digital_video_recorder.
  2. Wikipedia, "Network video recorder," [Online]. Available: https://en.wikipedia.org/wiki/Network_video_recorder.
  3. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, 2005, pp. 886-893.
  4. D. G. Lowe, "Object recognition from local scale-invariant features," in Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, Greece, 1999, pp. 1150-1157.
  5. C. P. Papageorgiou, M. Oren, and T. Poggio, "A general framework for object detection," in Proceedings of the 6th International Conference on Computer Vision (IEEE Cat. No. 98CH36271), Bombay, India, 1998, pp. 555-562.
  6. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, 2014, pp. 580-587.
  7. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: unified, real-time object detection," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, 2016, pp. 779-788.
  8. J. Redmon and A. Farhadi, "YOLO9000: better, faster, stronger," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, 2017, pp. 7263-7271.
  9. J. Redmon and A. Farhadi, "Yolov3: an incremental improvement," 2018 [Online]. Available: https://arxiv.org/abs/1804.02767.
  10. S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: towards real-time object detection with region proposal networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 2017. https://doi.org/10.1109/TPAMI.2016.2577031
  11. J. Parsola, D. Gangodkar, and A. Mittal, "Post event investigation of multi-stream video data utilizing Hadoop cluster," International Journal of Electrical & Computer Engineering, vol. 8, no. 6, pp. 5089-5097, 2018. https://doi.org/10.11591/ijece.v8i6.pp5089-5097
  12. S. Karimi-Mansoub, R. Abri, and A. Yarici, "Concurrent real-time object detection on multiple live streams using optimization CPU and GPU resources in YOLOv3," in Proceedings of the 4th International Conference on Advances in Signal, Image and Video Processing, Athens, Greece, 2019, pp. 23-28.
  13. L. Tan, X. Dong, Y. Ma, and C. Yu, "A multiple object tracking algorithm based on YOLO detection," in Proceedings of 2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Beijing, China, 2018, pp. 1-5.
  14. D. Kim and S. Park, "An intelligent collaboration framework between edge camera and video analysis system," in Proceedings of 2018 International Conference on Electronics, Information, and Communication (ICEIC), Honolulu, HI, 2018, pp. 1-3.
  15. G. Mattela, C. Pal, M. Tripathi, R. Gavval, and A. Acharyya, "Enterprise class deep neural network architecture for recognizing objects and faces for surveillance systems," in Proceedings of 2019 11th International Conference on Communication Systems & Networks (COMSNETS), Bengaluru, India, 2019, pp. 607-612. IEEE.
  16. J. S. Park, M. Wiranegara, and G. Y. Son, "Multi-channel video analysis based on deep learning for video surveillance," The Journal of the Korea Institute of Electronic Communication Sciences, vol. 13, no. 6, pp. 1263-1268, 2018. https://doi.org/10.13067/JKIECS.2018.13.6.1263
  17. Y. Wu, Z. Zhao, S. Zhang, L. Yao, Y. Yang, T. Z. Fu, and S. Winkler, "Interactive multi-camera soccer video analysis system," in Proceedings of the 27th ACM International Conference on Multimedia, Nice, France, 2019, pp. 1047-1049.
  18. S. Aslan and S. C. Ileri, "Performance analysis of ARM big.LITTLE architecture based mobile processor with multi-thread face detection," in Proceedings of 2019 4th International Conference on Computer Science and Engineering (UBMK), Samsun, Turkey, 2019, pp. 336-339.
  19. X. Wang, "An efficient end-to-end object detection pipeline on GPU using CUDA," Master's thesis, Department of Mathematics and Computer Science, Eindhoven University of Technology, Eindhoven, The Netherlands, 2019.
  20. S. Liu, G. Liu, and H. Zhou, "A robust parallel object tracking method for illumination variations," Mobile Networks and Applications, vol. 24, no. 1, pp. 5-17, 2019. https://doi.org/10.1007/s11036-018-1134-8
  21. NVIDIA, "DeepStream SDK 5.1," 2021 [Online]. Available: https://developer.nvidia.com/deepstream-getting-started.
  22. H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP: a transport protocol for real-time applications," Internet Engineering Task Force, Fremont, CA, RFC 3550, Standard 64, 2003.
  23. H. Schulzrinne, A. Rao, and R. Lanphier, "Real Time Streaming Protocol (RTSP)," Internet Engineering Task Force, Fremont, CA, RFC 2326, 1998.