KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Joint Spatial-Temporal Quality Improvement Scheme for H.264 Low Bit Rate Video Coding via Adaptive Frameskip

  • Cui, Ziguan (Image Processing and Image Communication Lab, Nanjing University of Posts and Telecommunications) ;
  • Gan, Zongliang (Image Processing and Image Communication Lab, Nanjing University of Posts and Telecommunications) ;
  • Zhu, Xiuchang (Image Processing and Image Communication Lab, Nanjing University of Posts and Telecommunications)
  • Received : 2011.09.24
  • Published : 2012.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

Conventional rate control (RC) schemes for H.264 video coding usually regulate output bit rate to match channel bandwidth by adjusting quantization parameter (QP) at fixed full frame rate, and the passive frame skipping to avoid buffer overflow usually occurs when scene changes or high motions exist in video sequences especially at low bit rate, which degrades spatial-temporal quality and causes jerky effect. In this paper, an active content adaptive frame skipping scheme is proposed instead of passive methods, which skips subjectively trivial frames by structural similarity (SSIM) measurement between the original frame and the interpolated frame via motion vector (MV) copy scheme. The saved bits from skipped frames are allocated to coded key ones to enhance their spatial quality, and the skipped frames are well recovered based on MV copy scheme from adjacent key ones at the decoder side to maintain constant frame rate. Experimental results show that the proposed active SSIM-based frameskip scheme acquires better and more consistent spatial-temporal quality both in objective (PSNR) and subjective (SSIM) sense with low complexity compared to classic fixed frame rate control method JVT-G012 and prior objective metric based frameskip method.

Keywords

References

  1. Z. G. Li, F. Pan, K. P. Lim, G. N. Feng, X. Lin and S Rahardja, "Adaptive basic unit layer rate control for JVT," Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG. JVT-G012, 7th Meeting, Mar. 2003.
  2. W. Yuan, S. Lin, Y. Zhang, H. Luo and W. Yuan, "Optimum bit allocation and rate control for H.264/AVC," Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6) 15th Meeting, JVT-O016, Apr. 2005.
  3. D. K. Kwon, M. Y. Shen and C. C. J. Kuo, "Rate control for H.264 video with enhanced rate and distortion models," IEEE Transactions on Circuits and Systems for Video Technology, vol. 17, no. 5, pp. 517-529, May. 2007.
  4. S. S. Rodriguez, O. Esteban, M. Lopez and F. Maria, "Cauchy-density-based basic unit layer rate controller for H.264/AVC," of IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 8, pp. 1139-1143, Aug. 2010.
  5. L. Xu, D. B. Zhao, X. Y. Ji, S. Kwong and W. Gao, "Window-level rate control for smooth picture quality and smooth buffer occupancy," IEEE Transactions on Image Processing, vol. 21, no. 3, pp. 723-723, Mar. 2011.
  6. T. S. Ou, Y. H. Huang and H. H. Chen, "SSIM-based perceptual rate control for video coding," IEEE Transactions on Circuits and Systems for Video Technology, vol. 21, no. 5, pp. 682-291, May. 2011.
  7. J. Chen and H. Hang, "Source model for transform video coder and its application-Part II: Variable frame rate coding," IEEE Transactions on Circuits and Systems for Video Technology, vol. 7, no. 2, pp. 299-311, Feb. 1997. https://doi.org/10.1109/76.564109
  8. E. C. Reed and F. Dufaux, "Constrained bit-rate control for very low bit-rate streaming-video applications," IEEE Transactions on Circuits and Systems for Video Technology, vol. 11, no. 7, pp. 882-889, Jul. 2001. https://doi.org/10.1109/76.931115
  9. E. Reed and J. Lim, "Optimal multidimensional bit-rate control for video communication," IEEE Transactions on Image Processing, vol. 11, no. 8, pp. 873-885, Oct. 2002. https://doi.org/10.1109/TIP.2002.801122
  10. F. Pan, X. Lin, S. Rahardja, K. P. Lim, Z. G. Li, D. J. Wu and S. Wu, "Proactive frame-skipping decision scheme for variable frame rate video coding," IEEE International Conference on Multimedia and Expo(ICME), pp. 1903-1906, Jun. 2004.
  11. F. Pan, Z. Lin, X. Lin, S. Rahardja, W. Juwono and F. Slamet, "Adaptive frame skipping based on spatio-temporal complexity for low bit-rate video coding," Journal of Visual Communication and Image Representation, vol. 17, no. 3, pp. 554-563, 2006. https://doi.org/10.1016/j.jvcir.2005.07.006
  12. A. Thammineni, A. Raman, S. C. Vadapalli and S. Sethuraman, "Dynamic frame-rate selection for live LBR video encoders using trial frames," IEEE International Conference on Multimedia and Expo (ICME), pp. 817-820, Jun. 2008.
  13. J. Jun, S. Lee, Z. He, M. Lee and E. S. Jang, "Adaptive key frame selection for efficient video coding," Lecture Notes in Computer Science on Advances in Image and Video Technology, vol. 4872, pp. 853-866, 2007.
  14. P. Usach, J. Sastre and J. M. Lopez, "Variable frame rate and gop size H.264 rate control for mobile communications," IEEE International Conference on Multimedia and Expo (ICME), pp. 1772-1775, Jul. 2009.
  15. S. Liu and C. C. J. Kuo, "Joint temporal-spatial bit allocation for video coding with dependency," IEEE Transactions on Circuits and Systems for Video Technology, vol. 15, no. 1, pp. 15-26, Jan. 2005.
  16. A. Vetro, Y. Wang and H. F. Sun, "Rate-distortion optimized video coding considering frameskip," International Conference on Image Processing (ICIP), pp. 534-537, Oct. 2001.
  17. J. Lee, A. Vetro, Y. Wang and Y. Ho, "Bit allocation for MPEG-4 video coding with spatio-temporal tradeoffs," IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 6, pp. 488-502, 2003. https://doi.org/10.1109/TCSVT.2003.813421
  18. S. Hwangjun and C. C. J. Kuo, "Rate control for low-bit-rate video via variable-encoding frame rates," IEEE Transactions on Circuits and Systems for Video Technology, vol. 11, no. 4, pp. 512-521, Apr. 2001. https://doi.org/10.1109/76.915357
  19. Y. T. Yang, Y. S. Tung and J. L. Wu, "Quality enhancement of frame rate up-converted video by adaptive frame skip and reliable motion extraction," IEEE Transactions on Circuits and Systems for Video Technology, vol. 17, no. 12, pp. 1700-1713, Dec. 2007.
  20. Y. Chung, "A novel selective frame discard method for 3D video over IP networks," KSII Transactions on Internet and Information Systems, vol. 4, no. 6, pp. 1209-1221, Dec. 2010.
  21. Y. S. Hong and H. Park, "A cost-effective rate control for streaming video for wireless portable devices," KSII Transactions on Internet and Information Systems, vol. 5, no. 6, pp. 1147-1165, Jun. 2011.
  22. Z. Wang, A. C. Bovik, H. R. Sheikh E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004. https://doi.org/10.1109/TIP.2003.819861
  23. http://iphome.hhi.de/suehring/tml.download/.
  24. Z. Gan, L. Qi and X. Zhu, "Motion compensated frame interpolation based on H.264 decoder," Electronics Letters, vol. 43, no. 2, pp. 96-98, Jan . 2007. https://doi.org/10.1049/el:20072810
  25. S. Bandyopadhyay, Z. Wu, P. Pandit and J. Boyce, "Frame loss error concealment for H.264/AVC," ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6) 16th Meeting, JVT-P072, Jul. 2005.