Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

A Deep Learning based Inter-Layer Reference Picture Generation Method for Improving SHVC Coding Performance

SHVC 부호화 성능 개선을 위한 딥러닝 기반 계층간 참조 픽처 생성 방법

  • Received : 2019.03.18
  • Accepted : 2019.05.04
  • Published : 2019.05.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a reference picture generation method for Inter-layer prediction based deep learning to improve the SHVC coding performance. A description will be given of a structure for performing filtering using a VDSR network on a DCT-IF based upsampled picture to generate a new reference picture and a training method for generating a reference picture between SHVC Inter-layer. The proposed method is implemented based on SHM 12.0. In order to evaluate the performance, we compare the method of generating Inter-layer predictor by applying dictionary learning. As a result, the coding performance of the enhancement layer showed a bitrate reduction of up to 13.14% compared to the method using dictionary learning, a bitrate reduction of up to 15.39% compared to SHM, and a bitrate reduction of 6.46% on average.

본 논문에서는 SHVC 부호화 성능 개선을 위하여 딥러닝 기반 계층간 예측을 위한 참조 픽처 생성 방법을 제안한다. 새로운 참조 픽처를 생성하기 위하여 DCT-IF기반 업샘플링 된 픽처를 VDSR 네트워크를 이용한 필터링을 진행하는 구조와 SHVC 계층간 참조 픽처를 생성하기 위한 트레이닝 방법에 대해 설명한다. 제안하는 방법은 SHM 12.0 기반으로 구현되어 있다. 성능 평가를 위하여 사전 학습을 이용하여 계층간 예측 픽처를 생성하는 방법과 비교를 진행하였다. 그 결과 상위 계층의 부호화 성능은 사전 학습을 이용한 방법 대비 최대 13.14%의 비트 감소, SHM 대비 최대 15.39%의 비트 감소율을 보였고, 평균 6.46%의 비트 감소율을 보였다.

Keywords

BSGHC3_2019_v24n3_401_f0001.png 이미지

그림 1. 기존 SHVC 부호화기 흐름도[13] Fig. 1. The conventional SHVC encoder flowchat[13]

BSGHC3_2019_v24n3_401_f0002.png 이미지

그림 2. SHVC의 계층간 참조 방법의 예시 Fig. 2. An example of SHVC Inter-Layer reference picture method

BSGHC3_2019_v24n3_401_f0003.png 이미지

그림 3. 제안하는 방법에서 사용한 VDSR 네트워크 구조[8] Fig. 3. The VDSR network architecture in the proposed method[8]

BSGHC3_2019_v24n3_401_f0005.png 이미지

그림 4. 제안하는 방법의 부호화기 및 복호화기 블록도 Fig. 4. The proposed codec flowchart

표 1. 휘도 성분에 대한 업샘플링 필터 계수[3] Table 1. Upsampling filter coefficients for luminance component[3]

BSGHC3_2019_v24n3_401_t0001.png 이미지

표 2. VDSR 학습에 사용한 시퀀스 Table 2. Video sequences for training VDSR

BSGHC3_2019_v24n3_401_t0002.png 이미지

표 3. MPEG 테스트 시퀀스에 대한 SHM 대비 사전학습 방법과 제안하는 방법의 BD-비트율 비교 Table 3. The average BD-rate reduction of the dictionary learning and the proposed methods compared to SHM in MPEG test sequences

BSGHC3_2019_v24n3_401_t0003.png 이미지

표 4. QP 별 테스트 시퀀스에 대한 SHM 대비 제안하는 방법의 성능비교 Table 4. Performance analysis between SHM and the proposed method in QP-based MPEG test sequences

BSGHC3_2019_v24n3_401_t0004.png 이미지

References

  1. H. Choi, K. Lee, J. Kang, S. Bae, and J. Yoo, "Overview and Performance Analysis of the Emerging Scalable Video Coding", Journal of Broadcast Engineering Vol.12, no. 6, pp. 542-554, November, 2007, http://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE00926642. https://doi.org/10.5909/JBE.2007.12.6.542
  2. J. M. Boyce. Y. Ye, J. Chen and A. K. Ramasubramonian, "Overview of SHVC: Scalable Extensions of the High Efficiency Video Coding Standard", IEEE Transactions on Circuits and Systems for Video Technology, Vol. 26, no. 1, pp. 20-34, July, 2016, https://doi.org/10.1109/TCSVT.2015.2461951.
  3. J. Lee, J. Kang, H. Lee and J. Choi, "SHVC Standard Technology Trend," TTA Journal, vol.152, pp. 60-65, March, 2015, https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE06391006.
  4. Y. Kim, J. Choi and H. Choi, "Performance Analysis of Scalable HEVC Coding Tools", JBE Vol.20, no. 4, pp. 497-508, July, 2015, https://doi.org/10.5909/JBE.2015.20.4.497.
  5. J. Yang, J. Wright, T. S. Huang, and Y. Ma, "Image super-resolution via sparse representation", IEEE Transactions on Image Processing, vol. 19, no. 11, pp. 2861-2873, November 2010, https://doi.org/10.1109/TIP.2010.2050625.
  6. J. Yang, Z. Wang, Z. Lin, S. Cohen, and T. Huang, "Coupled dictionary training for image super-resolution", IEEE Transactions on Image Processing, vol. 21, no. 8, pp. 3467-3478, Aug 2012, https://doi.org/10.1109/TIP.2012.2192127.
  7. D. Chao, L. Chen, H. Kaiming and T. Xiaoou, "Image super-resolution using deep convolutional networks", IEEE transactions on pattern analysis and machine intelligence, Vol. 38, No.2. pp. 295-307, February, 2016, https://doi.org/10.1109/TPAMI.2015.2439281.
  8. J. Kim, J. Lee and K. Lee, "Accurate Image Super-Resolution Using Very Deep Convolutional Networks", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, pp. 1646-1654, June, 2016, https://doi.org/10.1109/CVPR.2016.182.
  9. C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang and W. Shi, "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network", Proceedings of the the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Hawaii, USA, pp.4681-4690, July, 2017, https://arxiv.org/abs/1609.04802.
  10. B. Lim, S. Son, H. Kim, S. Nah, and K. Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolution", Proceedings of the the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Hawaii, USA, pp.136-144, July, 2017, https://arxiv.org/abs/1707.02921.
  11. W. Park and M. Kim, "CNN-based in-loop filtering for coding efficiency improvement", Proceedings of the IEEE Image, Video, and Multidimensional Signal Processing Workshop (IVMSP), IEEE, Bordeaux, France, July, 2016, https://doi.org/10.1109/IVMSPW.2016.7528223.
  12. J. Schneider, J. Sauer, and M. Wien. "Dictionary learning based high frequency inter-layer prediction for scalable HEVC", Proceeding of IEEE Visual Communications and Image Processing (VCIP). IEEE, St. Petersburg, USA, pp.1-4, December, 2017, https://doi.org/10.1109/VCIP.2017.8305019.
  13. J. Chen, J. Boyce, Y. Ye, M. Hannuksela and G. Barroux, "SHVC Test Model 11 (SHM 11)", Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 113th Meeting, JCTVC-N15778, Geneva, Switzerland, pp. 1-12, October, 2015, https://mpeg.chiariglione.org/standards/mpeg-h/high-efficiency-video-coding/n15778-scalable-hevc-shvc-test-model-11-shm-11.
  14. V. Seregin and Y. He, "Common SHM test conditions and software reference configuration", Joint Collaborative Team on Video Coding(JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 17th Meeting, JCTVC-Q1009, Valencia, Spain, pp. 1-4, March, 2014, http://phenix.it-sudparis.eu/jct/doc_end_user/current_document.php?id=9106.