방송공학회논문지 (Journal of Broadcast Engineering)

  • 제19권5호
  • /
  • Pages.640-655
  • /
  • 2014
  • /
  • 1226-7953(pISSN)
  • /
  • 2287-9137(eISSN)
한국방송∙미디어공학회 (The Korean Institute of Broadcast and Media Engineers)
권호 표지
DOI QR코드

DOI QR Code

깊이정보를 이용한 HEVC의 인코더 고속화 방법

HEVC Encoder Optimization using Depth Information

  • 이윤진 (경희대학교 컴퓨터공학과) ;
  • 배동인 (경희대학교 컴퓨터공학과) ;
  • 박광훈 (경희대학교 컴퓨터공학과)
  • 투고 : 2014.06.20
  • 심사 : 2014.08.29
  • 발행 : 2014.09.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

최근 영상시스템 환경은 2D 비디오카메라에 깊이 카메라가 부착되어 2D 및 3D 어플리케이션을 지원하는 형태로 보편화 되고 있다. 이러한 3차원 멀티미디어 시스템 환경으로의 변화는 비디오 시스템에서 깊이정보 획득을 용이하게 만들었다. 깊이정보는 객체 구분, 배경영역 인지 등에 이용할 수 있는데, 2D 부호화에 이를 이용한다면 높은 부호화 효율을 얻을 수 있다. 따라서, 본 논문에서는 차세대 2D 비디오 코덱인 HEVC 인코더에 반영한 깊이정보 이용 비디오 부호화 방법을 제안한다. 제안방법으로, 현재 부호화하려는 CU가 배경영역에 위치할 경우 1) 주변블록의 SKIP 모드를 참조하여 결정하는 CU 분할 조기 결정, 2) 시간적 위치의 CU 정보를 이용하여 수행하는 CU 분할 구조 제한, 3) 배경영역에 따른 움직임 예측 탐색 범위 제한이 있다. 실험은 HEVC 참조 소프트웨어인 HM 12.0에 적용하였고, 실험결과 40% 이상의 부호화 복잡도가 감소했으며, BD-Bitrate는 0.5% 손실되었다. 특히, 마이크로소프트사에서 개발한 키넥트를 통해 획득한 영상을 이용한 실험 결과에서는 영상 품질의 큰 열화 없이 기존대비 최대 53%의 부호화 복잡도가 감소하는 결과를 나타내어, 향후 실시간 화상통신, 모바일 또는 핸드헬드 환경에서의 비디오 서비스 등에서 광범위하게 적용할 수 있을 것으로 기대된다.

Many of today's video systems have additional depth camera to provide extra features such as 3D support. Thanks to these changes made in multimedia system, it is now much easier to obtain depth information of the video. Depth information can be used in various areas such as object classification, background area recognition, and so on. With depth information, we can achieve even higher coding efficiency compared to only using conventional method. Thus, in this paper, we propose the 2D video coding algorithm which uses depth information on top of the next generation 2D video codec HEVC. Background area can be recognized with depth information and by performing HEVC with it, coding complexity can be reduced. If current CU is background area, we propose the following three methods, 1) Earlier stop split structure of CU with PU SKIP mode, 2) Limiting split structure of CU with CU information in temporal position, 3) Limiting the range of motion searching. We implement our proposal using HEVC HM 12.0 reference software. With these methods results shows that encoding complexity is reduced more than 40% with only 0.5% BD-Bitrate loss. Especially, in case of video acquired through the Kinect developed by Microsoft Corp., encoding complexity is reduced by max 53% without a loss of quality. So, it is expected that these techniques can apply real-time online communication, mobile or handheld video service and so on.

키워드

참고문헌

  1. Kinect camera, http://www.xbox.com/en-US/kinect/default.htm
  2. Project Tango, http://www.google.com/atap/projecttango/
  3. Intel RealSense 3D Camera, http://www.intel.com/content/www/us/ en/architecture-and-technology/realsense-depth-technologies.html
  4. Francois, E., Chupeau, B., "Depth-based segmentation", IEEE Transactions On CSVT, vol. 7, no. 1, pp.237-240, Feb. 1997.
  5. Jungong Han, Ling Shao, Dong Xu, Shotton. J., "Enhanced Computer Vision With Microsoft Kinect Sensor: A Review", IEEE Transactions on Cybernetices, vol. 43, no. 5, pp.1318-1334, Oct. 2013. https://doi.org/10.1109/TCYB.2013.2265378
  6. Suarez, J., Murphy, R.R., "Hand gesture recognition with depth images: A review", 2012 IEEE Conference on RO-MAIN, pp.411-417, Sep. 2012.
  7. Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, Andrew Fitzgibbon, "KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera", UIST'11, pp.559-568, Oct. 2011.
  8. Yan Cui, Schuon, S., Chan, D., Thrun, S., Theobalt, C., "3D Shape Scanning with a Time-of-Flight Camera", 2010 IEEE Conference on CVPR, pp.1173-1180, Jun. 2010.
  9. MPEG, http://mpeg,chiaiglione.org
  10. VCEG, http://www.itu.int/en/ITU-T/studygroups/com16/video/Pages/ default.aspx
  11. JCT-VC, http://www.itu.int/en/ITU-T/studygroups/com16/video/Pages/ jctvc.aspx
  12. JCT-3V, http://www.itu.int/en/ITU-T/studygroups/2013-2016/16/Pages/ video/jct3v.aspx
  13. Sullivan, G.J., Ohm, J., Woo-Jin Han, Wiegand, T, "Overview of the High Efficiency Video Coding (HEVC) Standard", IEEE Transactions on CSVT, vol. 22, no. 12, pp.1649-1668, Dec. 2012
  14. T. Wiegand, G. J. Sullivan, G. Bjontegaard, A. Luthra, "Overview of the H.264/AVC video coding standard", IEEE Transactions on CSVT, vol. 13, no. 7, pp.560-576, Jul. 2003.
  15. JCT-3V of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "CE1.h: Backward View Synthesis Prediction using Neighbouring Blocks", JCT3V-C0152, Geneva, CH, Jan. 2013.
  16. JCT-3V of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "CE3: Results on Depth-based Block Partitioning (DBBP)", JCT3V-G0106, San Jose, US, Jan. 2014.
  17. Jorn Ostermann, Euee S. Jang, Jae-Seob Shin, Tsuhan Chen, "Coding of Arbitrarily Shaped Video Objects in MPEG-4", International Conference of Image Processing, pp.496-499, 1997.
  18. Kaup, A., "Object-based texture coding of moving video in MPEG-4", IEEE Transactions on CSVT, vol. 9, no. 1, pp.5-15, 1999.
  19. ISO/IEC 14496-2 (MPEG-4 Visual Version 1), "Coding of audio-visual objects-Part 2: Visual", Apr. 1999.
  20. Rong Ding, Qionghai Dai, Wenli Xu, Dongdong Zhu, Hao Yin, "Background-frame based motion compensation for video compression", 2004 IEEE ICME, vol. 2, pp.1487-1490, 2014.
  21. Ohm, J., Sullivan, G.J., Schwarz, H., Thiow Keng Tan, Wiegand, T., "Comparison of the Coding Efficiency of Video Coding Standards-Including High Efficiency Video Coding (HEVC)", IEEE Transactions on CSVT, vol. 22, no. 12, pp.1669-1684, Dec. 2012.
  22. Il-Koo Kim, Junghye Min, Tammy Lee, Woo-Jin Han, JeongHoon Park, "Block Partitioning Structure int the HEVC Standard", IEEE Transactions on CSVT, vol. 22, no. 12, pp.1697-2012, Dec. 2012
  23. Woo-Jin Han, "Conditional Probability Based Early Termination of Recursive Coding Unit Structures in HEVC", Journal of Broadcast Engineering, vol. 17, no. 2, pp.354-362, Mar. 2012. https://doi.org/10.5909/JEB.2012.17.2.354
  24. Chansub Park, Byeonggyu Kim, Dongsan Jeon, Sunheung Jung, Yeonhui Kim, Jinuk Suk, Jinsun Choi, "Early Termination Algorithm of Merge Mode Search for Fast High Efficiency Video Coding (HEVC) Encoder", Journal of Broadcast Engineering, vol. 18, no. 5, pp.691-701, Sep. 2012. https://doi.org/10.5909/JBE.2013.18.5.691
  25. Hansu Lee, Kyeongyong Kim, Taeryong Kim, Gwanghoon Park, Hwiyong Kim, Seongchang Lim, Jinho Lee, "Improving Encoder Complexity and Coding Method of the Split Information in HEVC", Journal of Broadcast Engineering, vol. 17, no. 2, pp.325-343, Mar. 2012. https://doi.org/10.5909/JEB.2012.17.2.325
  26. JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "Early Termination of CU Encoding to Reduce HEVC Complexity", JCTVC-F045, Torino, IT, Jul. 2011.
  27. JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "Coding tree pruning based CU early termination", JCTVC-F092, Torino, IT, Jul. 2011.
  28. JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "Common HM test conditions and software reference configurations", JCTVC-I1100, Geneva, Switzerland, May. 2012.
  29. Gary J. Sullvan and Thowas Wiegand, "Rate-Distortion Optimization for Video Compression", IEEE Signal Processing Magazine, vol. 15, no. 6, pp.74-90, Nov. 1998. https://doi.org/10.1109/79.733497
  30. JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "Samsung Response to the Call for Proposals on Video Compression Technology", JCTVC-A124, Dresden, German, Apr. 2011.
  31. JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "Description of video coding technology proposal by Fraunhofer HHI", JCTVC-A116, Dresden, German, Apr. 2011.
  32. Zoran Zivkovic, "Improved adaptive Gaussian mixture model for background subtraction", International Conference Pattern Recognition, vol. 2, pp.28-31, 2004.
  33. M. Piccardi, "Background subtraction techniques: a review", IEEE International Conference on SMC, vol. 4, pp. 3099-3100, 2004.
  34. Yongjo Ahn, Taejin Hwang, Sungeun Yoo, Woo-jin Han, Donggyu Sim, "Statistical Characteristics and Complexity Analysis of HEVC Encoder Software", Journal of Broadcast Engineering, vol. 17, no. 6, pp.1091-1105, Nov. 2012. https://doi.org/10.5909/JBE.2012.17.6.1091
  35. Jens-Rainer Ohm, G. J. Sullivan, Heiko Schwarz, T. K. Tan, T. Wiegand, "Comparison of the Coding Efficiency of Video Coding Standards-Including High Efficiency Video Coding (HEVC)", IEEE Transactions on CSVT, vol. 22, no. 12, pp.1669-1684, Dec. 2012.
  36. Frank Bossen, Benjamin Bross, Karsten Suhring, David Flynn, "HEVC Complexity and Implementation Analysis", IEEE Transactions on CSVT, vol. 22, no. 12, pp.1685-1696, Dec. 2012.
  37. G. Bjontegaard, "Calculation of average PSNR differences between RD-curves," ITU-T SG16 Q.6, VCEG-M33, Texas, USA, Apr. 2001.