Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

A Feature Map Compression Method for Multi-resolution Feature Map with PCA-based Transformation

PCA 기반 변환을 통한 다해상도 피처 맵 압축 방법

  • Park, Seungjin (Department of Computer Engineering, Kwangwoon University) ;
  • Lee, Minhun (Department of Computer Engineering, Kwangwoon University) ;
  • Choi, Hansol (Department of Computer Engineering, Kwangwoon University) ;
  • Kim, Minsub (Department of Computer Engineering, Kwangwoon University) ;
  • Oh, Seoung-Jun (Department of Electronic Engineering, Kwangwoon University) ;
  • Kim, Younhee (Electronics and Telecommunications Research Institute) ;
  • Do, Jihoon (Electronics and Telecommunications Research Institute) ;
  • Jeong, Se Yoon (Electronics and Telecommunications Research Institute) ;
  • Sim, Donggyu (Department of Computer Engineering, Kwangwoon University)
  • Received : 2021.12.02
  • Accepted : 2021.12.16
  • Published : 2022.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a compression method for multi-resolution feature maps for VCM. The proposed compression method removes the redundancy between the channels and resolution levels of the multi-resolution feature map through PCA-based transformation. According to each characteristic, the basis vectors and mean vector used for transformation, and the transformation coefficient obtained through the transformation are compressed using a VVC-based coder and DeepCABAC. In order to evaluate performance of the proposed method, the object detection performance was measured for the OpenImageV6 and COCO 2017 validation set, and the BD-rate of MPEG-VCM anchor and feature map compression anchor proposed in this paper was compared using bpp and mAP. As a result of the experiment, the proposed method shows a 25.71% BD-rate performance improvement compared to feature map compression anchor in OpenImageV6. Furthermore, for large objects of the COCO 2017 validation set, the BD-rate performance is improved by up to 43.72% compared to the MPEG-VCM anchor.

본 논문에서는 VCM을 위한 다해상도 피처 맵에 대한 압축 방법을 제안한다. 제안하는 압축 방법은 PCA 기반의 변환을 통해 다해상도 피처 맵의 채널 및 해상도 계층 간 중복성을 제거하며 변환에 사용된 기저 벡터와 평균 벡터 그리고 변환을 통해 얻어진 변환 계수를 각각의 특성에 따라 VVC 기반 부호화기와 DeepCABAC을 통하여 압축한다. 제안하는 방법의 성능을 측정하기 위하여 OpenImageV6와 COCO 2017 validation set에 대하여 객체 검출 성능을 평가하며, MPEG-VCM 앵커 및 본 논문에서 제안하는 피처 맵 압축 앵커 대비 bpp와 mAP를 BD-rate 관점에서 비교한다. 실험 결과, 제안하는 방법은 OpenImageV6에서 피처 맵 압축 앵커 대비 25.71%의 BD-rate 성능 향상을 보이며, 특히 COCO 2017 validation set의 크기가 큰 객체들에 대해서 MPEG-VCM 앵커 대비 최대 43.72%의 BD-rate 성능이 향상됨을 보인다.

Keywords

Acknowledgement

이 논문은 2021년도 광운대학교 우수연구자 지원 사업과 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임 (No. 2020-0-00011, (전문연구실)기계를 위한 영상부호화 기술).

References

  1. M. F. Mahmood, N. Hussin, "Information in Conversion Era: Impact and Influence from 4th Industrial Revolution," International Journal of Academic Research in Business and Social Sciences, Vol. 8, No. 9, pp. 320-328, 2018
  2. Y. LeCun, Y. Bengio, and G. E. Hinton, "Deep learning," Nature, vol. 512, pp. 436-444, 2015. https://doi.org/10.1038/nature13439
  3. G. Sullivan, J. Ohm, W. Han, and T. Wiegand, "Overview of the high efficiency video coding (HEVC) standard," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668, Dec. 2012. https://doi.org/10.1109/TCSVT.2012.2221191
  4. B. Bross, Y. K. Wang, Y. Ye, S. Liu, and J. Chen, Overview of the versatile video coding (VVC) standard and its applications," IEEE Transactions on Circuits and Systems for Video Technology, Vol 31, No 10, pp. 3736-3764, 2021. https://doi.org/10.1109/TCSVT.2021.3101953
  5. VTM12.0, https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/tree/VTM-12.0 (accessed Nov. 26, 2021).
  6. M. Rafie, Y. Zhang, and S. Liu, "[VCM] Evaluation Framework for Video Coding for Machines," ISO/IEC JTC 1/SC 29/WG 2, m58385, Online, Oct. 2021.
  7. W. Gao, X. Xu, and S. Liu, "[VCM] Response to CfE: Investigation of VVC Codec for Video Coding for Machine," ISO/IEC JTC 1/SC 29/WG 2, m56681, Online, Apr. 2021.
  8. S. Wang, C. Lin, C. Lin, T., and Y. Nie, "[VCM] Enable IBC in VTM8.2 for VCM," ISO/IEC JTC 1/SC 29/WG 2, m56792, Online, Apr. 2021.
  9. S. Wang, C. Lin, and C. Lin (ITRI), "[VCM] A study on impact of coding tools on machine vision performance and visual quality," ISO/IEC JTC 1/SC 29/WG 2, m56867, Online, Apr. 2021.
  10. B. Zhu, L. Yu, and D. Li, "[VCM] Deep learning-based compression for machine vision," ISO/IEC JTC 1/SC 29/WG 2, m57335, Online, Jul. 2021.
  11. Y. Lee, S., K. Yoon, H. Lim, H. Choo, W. Cheong, and J. Seo, "[VCM] Updated FLIR Anchor results for object detection," ISO/IEC JTC 1/SC 29/WG 2, m57375, Online, Jul. 2021.
  12. S. Wang, Z. Wang, Y. Ye, and S. Wang, "[VCM] End-to-end image compression towards machine vision for object detection," ISO/IEC JTC 1/SC 29/WG 2, m57500, Online, Jul. 2021.
  13. S. Kim, M. Jeong, H. Jin H. Lee, H. Choo, H. Lim, and J. Seo, "[VCM] A report on intermediate feature coding for object detection and segmentation," ISO/IEC JTC 1/SC 29/WG 2, m55243, Online, Oct. 2020.
  14. S Wang, Z. Wang, Y. Ye, and S Wang, "[VCM] Image or video format of feature map compression for object detection," ISO/IEC JTC 1/SC 29/WG 2, m55786, Online, Jan. 2021.
  15. J. Do, J. Lee, Y. Kim, S. Yoon J., and J. Choi, "[VCM] Experimental Results of Feature Compression using CompressAI," ISO/IEC JTC 1/SC 29/WG 2, m56716, Online, Apr. 2021.
  16. H. Han, H. Choi, S. Kwak, J. Yun, W. Cheong, and J. Seo, "[VCM] Investigation on feature map channel reordering and compression for object detection," ISO/IEC JTC 1/SC 29/WG 2, m56653, Online, Apr. 2021.
  17. S. Wang, Z. Wang, Y. Ye, and S. Wang, "[VCM] Investigation on feature map layer selection for object detection and compression," ISO/IEC JTC 1/SC 29/WG 2, m55787, Online, Dec. 2020.
  18. OpenImageV6, https://storage.googleapis.com/openimages/web/download.html (accessed Nov. 26, 2021)
  19. COCO2017 validation set, https://cocodataset.org/#download (accessed Nov. 26, 2021).
  20. G. Bjontegaard, "Calculation of average PSNR differences between RDcurves," Tech. Rep. VCEGM33, Video Coding Experts Group (VCEG), 2001.
  21. L. Jinhua, T. Zhang, and G. Feng. "Channel Compression: Rethinking Information Redundancy Among Channels in CNN Architecture," IEEE Access, Vol. 8, pp. 147265-147274, 2020. https://doi.org/10.1109/access.2020.3015714
  22. M. Rafie, Y. Zhang, and S. Liu, "[VCM] Call for Evidence for Video Coding for Machines," ISO/IEC JTC 1/SC 29/WG 2, m56995, Online, Apr. 2021.
  23. S. Xie, R. Girshick, P. Dollar, Z. Tu, and K. He, "Aggregated Residual Transformations for Deep Neural Networks," arXiv, 2017.
  24. S. Wiedemann et al., "DeepCABAC: A universal compression algorithm for deep neural networks," IEEE J. Sel. Topics Signal Process., Vol. 14, No. 4, pp. 700-714, May 2020. https://doi.org/10.1109/jstsp.2020.2969554