Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis of Feature Map Compression Efficiency and Machine Task Performance According to Feature Frame Configuration Method

피처 프레임 구성 방안에 따른 피처 맵 압축 효율 및 머신 태스크 성능 분석

  • Rhee, Seongbae (Graduate School of Electronic Information Convergence Engineering) ;
  • Lee, Minseok (Graduate School of Electronic Information Convergence Engineering) ;
  • Kim, Kyuheon (Graduate School of Electronic Information Convergence Engineering)
  • 이성배 (경희대학교 전자정보융합공학과) ;
  • 이민석 (경희대학교 전자정보융합공학과) ;
  • 김규헌 (경희대학교 전자정보융합공학과)
  • Received : 2022.04.18
  • Accepted : 2022.05.13
  • Published : 2022.05.30
Download PDF
⟨ Previous Next ⟩

Abstract

With the recent development of hardware computing devices and software based frameworks, machine tasks using deep learning networks are expected to be utilized in various industrial fields and personal IoT devices. However, in order to overcome the limitations of high cost device for utilizing the deep learning network and that the user may not receive the results requested when only the machine task results are transmitted from the server, Collaborative Intelligence (CI) proposed the transmission of feature maps as a solution. In this paper, an efficient compression method for feature maps with vast data sizes to support the CI paradigm was analyzed and presented through experiments. This method increases redundancy by applying feature map reordering to improve compression efficiency in traditional video codecs, and proposes a feature map method that improves compression efficiency and maintains the performance of machine tasks by simultaneously utilizing image compression format and video compression format. As a result of the experiment, the proposed method shows 14.29% gain in BD-rate of BPP and mAP compared to the feature compression anchor of MPEG-VCM.

최근 하드웨어 연산 장치와 소프트웨어 기반 프레임워크의 발전으로 딥러닝 네트워크를 활용한 머신 태스크가 다양한 산업 분야 및 개인 IoT 장비에서의 활용이 기대되고 있다. 그러나 딥러닝 네트워크를 구동하기 위한 장치의 고비용 문제와 서버에서 머신 태스크 결과만을 전송받을 때 사용자가 요구하는 결과를 받지 못할 수 있다는 제한 사항을 극복하기 위하여 Collaborative Intelligence (CI)에서는 피처 맵의 전송을 그 해결 방법으로 제시하였다. 본 논문에서는 CI 패러다임을 지원하기 위하여 방대한 데이터 크기를 갖는 피처 맵의 효율적인 압축 방법을 실험을 통해 분석 및 제시하였다. 해당 방법은 전통적인 비디오 코덱에서의 압축 효율을 높이기 위하여 피처 맵의 재정렬을 적용하여 중복성을 높였으며, 정지 영상 압축 포맷과 동영상 압축 포맷을 동시에 활용하여 압축 효율을 높이고 머신 태스크의 성능을 유지하는 피처 맵 방법을 제시하였다. 본 논문에서는 이와 같은 방법의 분석을 통해 MPEG-VCM의 피처 압축 앵커 대비 BPP와 mAP의 BD-rate에서 14.29%의 성능이 향상됨을 검증하였다.

Keywords

Acknowledgement

이 논문은 2022년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임 (No. 2020-0-00011, (전문연구실)기계를 위한 영상부호화 기술).

References

  1. LECUN, Yann, et al. Backpropagation applied to handwritten zip code recognition. Neural computation, 1989, 1.4: 541-551 doi: https://doi.org/10.1162/neco.1989.1.4.541
  2. KANG, Yiping, et al. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. ACM SIGARCH Computer Architecture News, 2017, 45.1: 615-629. doi: https://doi.org/10.1145/3093337.3037698
  3. CHEN, Zhuo, et al. Toward intelligent sensing: Intermediate deep feature compression. IEEE Transactions on Image Processing, 2019, 29: 2230-2243. doi: https://doi.org/10.1109/TIP.2019.2941660
  4. M. Rafie, Y. Zhang, and S. Liu, "[VCM] Evaluation Framework for Video Coding for Machines," ISO/IEC JTC 1/SC 29/WG 2, m58385, Online, Oct. 2021.
  5. Minhun Lee, Hansol Choi el al. "[VCM track 1] EE1.2 P-layer feature map anchor generation for object detection on OpenImageV6 dataset", ISO/IEC JTC 1/SC 29/WG 2, m58786, Online, Jan. 2022.
  6. Minhun Lee, Hansol Choi el al. "[VCM track 1] Advanced feature map compression based on optimal transformation with VVC and DeepCABAC", ISO/IEC JTC 1/SC 29/WG 2, m58787, Online, Jan. 2022.
  7. Jung Heum Kang, Hye Won Jeong et al. "[VCM track 1] Feature Compression with resize in feature domain", ISO/IEC JTC 1/SC 29/WG 2, m58867, Online, Jan. 2022.
  8. C. Rosewarne , Y. Kim et al. "[VCM] EE1 summary report", ISO/IEC JTC 1/SC 29/WG 2, m58793, Online, Jan. 2022.
  9. VTM12.0, https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/tree/VTM-12.0 (accessed Jan. 11, 2022).
  10. CHOI, Hyomin; BAJIC, Ivan V. Deep feature compression for collaborative object detection. In: 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, 2018. p. 3743-3747. doi: https://doi.org/10.1109/ICIP.2018.8451100
  11. HAN, Heeji, et al. Feature map channel reordering and compression for Neural Network feature map coding. In: Proceedings of the Korean Society of Broadcast Engineers Conference. The Korean Institute of Broadcast and Media Engineers, 2021. p. 39-42.
  12. Heeji Han, Haechul Choi et al. "[VCM] Investigation on feature map channel reordering and compression for object detection", ISO/IEC JTC 1/SC 29/WG 2, m56653, Online, Apr. 2021.
  13. Yong-Uk Yoon, Dongha Kim et al. "[VCM] Compression of reordered feature sequences based on channel means for object detection", ISO/IEC JTC 1/SC 29/WG 2, m57497, Online, Jul. 2021.
  14. TUCHLER, Michael; SINGER, Andrew C.; KOETTER, Ralf. Minimum mean squared error equalization using a priori information. IEEE Transactions on Signal processing, 2002, 50.3: 673-683. doi: https://doi.org/10.1109/78.984761
  15. WANG, Zhou, et al. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 2004, 13.4: 600-612. doi: https://doi.org/10.1109/TIP.2003.819861
  16. SUZUKI, Satoshi, et al. Deep Feature Compression With Spatio-Temporal Arranging for Collaborative Intelligence. In: 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020. p. 3099-3103. doi: https://doi.org/10.1109/ICIP40778.2020.9190933
  17. SENGUPTA, Abhronil, et al. Going deeper in spiking neural networks: VGG and residual architectures. Frontiers in neuroscience, 2019, 13: 95. doi: https://doi.org/10.3389/fnins.2019.00095
  18. PHAM, Vung; PHAM, Chau; DANG, Tommy. Road damage detection and classification with detectron2 and faster r-cnn. In: 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020. p. 5592-5601. doi: https://doi.org/10.1109/BigData50022.2020.9378027
  19. COCO2017 validation set, https://cocodataset.org/#download (accessed Apr. 11, 2022)
  20. WIECKOWSKI, Adam, et al. VVenC: An open and optimized VVC encoder implementation. In: 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE, 2021. p. 1-2. doi: https://doi.org/10.1109/ICMEW53276.2021.9455944
  21. OpenImageV6, https://storage.googleapis.com/openimages/web/download.html (accessed Apr. 11, 2022).
  22. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in CVPR, 2016. doi: https://doi.org/10.1109/cvpr.2016.90
  23. G. Bjontegaard, "Calculation of average PSNR differences between RDcurves," Tech. Rep. VCEGM33, Video Coding Experts Group (VCEG), 2001.