KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Disparity-based Error Concealment for Stereoscopic Images with Superpixel Segmentation

  • Zhang, Yizhang (Jiangsu Key Lab on Image Processing & Image Communication, Nanjing University of Posts and Telecommunications) ;
  • Tang, Guijin (Jiangsu Key Lab on Image Processing & Image Communication, Nanjing University of Posts and Telecommunications) ;
  • Liu, Xiaohua (Jiangsu Key Lab on Image Processing & Image Communication, Nanjing University of Posts and Telecommunications) ;
  • Sun, Changming (CSIRO Data61)
  • Received : 2017.10.11
  • Accepted : 2018.04.06
  • Published : 2018.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

To solve the problem of transmission errors in stereoscopic images, this paper proposes a novel error concealment (EC) method using superpixel segmentation and adaptive disparity selection (SSADS). Our algorithm consists of two steps. The first step is disparity estimation for each pixel in a reference image. In this step, the numbers of superpixel segmentation labels of stereoscopic images are used as a new constraint for disparity matching to reduce the effect of mismatching. The second step is disparity selection for a lost block. In this step, a strategy based on boundary smoothness is proposed to adaptively select the optimal disparity which is used for error concealment. Experimental results demonstrate that compared with other methods, the proposed method has significant advantages in both objective and subjective quality assessment.

Keywords

References

  1. Liang, Yi J., John G. Apostolopoulos, and Bernd Girod. "Analysis of packet loss for compressed video: Effect of burst losses and correlation between error frames," IEEE Transactions on Circuits and Systems for Video Technology 18.7, 861-874, 2008. https://doi.org/10.1109/TCSVT.2008.923139
  2. Marvasti-Zadeh, Seyed Mojtaba, Hossein Ghanei-Yakhdan, and Shohreh Kasaei. "A novel video temporal error concealment algorithm based on moment invariants," in Proc. of Machine Vision and Image Processing (MVIP), 2015 9th Iranian Conference on. IEEE, 2015.
  3. Ilcikova, Ivana, et al. "Texture aware image error concealment with fuzzy segmentation," Systems, Signals and Image Processing (IWSSIP), 2016 International Conference on. IEEE, 2016.
  4. Song, Kwanwoong, et al. "Error concealment of multi-view video sequences using inter-view and intra-view correlations," Journal of Visual Communication and Image Representation 20.4, 281-292, 2009. https://doi.org/10.1016/j.jvcir.2009.02.002
  5. Xiang, Xinguang, et al. "Auto-regressive model based error concealment scheme for stereoscopic video coding," in Proc. of Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on. IEEE, 2011.
  6. Khattak, Shadan, et al. "Temporal and inter-view consistent error concealment technique for multiview plus depth video," IEEE Transactions on Circuits and Systems for Video Technology 26.5, 829-840, 2016. https://doi.org/10.1109/TCSVT.2015.2418631
  7. Rongfu, Zhang, Zhou Yuanhua, and Huang Xiaodong. "Content-adaptive spatial error concealment for video communication," IEEE Transactions on Consumer Electronics 50.1, 335-341, 2004. https://doi.org/10.1109/TCE.2004.1277882
  8. Ostergaard, Jan, et al. "Sequential error concealment for video/images by weighted template matching," Data Compression Conference (DCC), IEEE, 2012.
  9. Hsia, Shih-Chang, and Cheng Hung Hsiao. "Fast-efficient shape error concealment technique based on block classification," IET Image Processing 10.10, 693-700, 2006. https://doi.org/10.1049/iet-ipr.2016.0043
  10. Liu, Jing, et al. "Spatial error concealment with an adaptive linear predictor," IEEE Transactions on Circuits and Systems for Video Technology, 25.3, 353-366, 2015. https://doi.org/10.1109/TCSVT.2014.2359145
  11. Fan, Wen, et al. "Sequential error concealment via canonical correlation analysis," in Proc. of Signal Processing, Communications and Computing (ICSPCC), 2015 IEEE International Conference on. IEEE, 2015.
  12. Yang, Yuhong, Meng Yang, and Pamela Cosman. "A hybrid error concealment for intra frame in stereoscopic video," Signal Processing Conference (EUSIPCO), 2012 Proceedings of the 20th European. IEEE, 2012.
  13. Hu, Yang, et al. "An improved spatio-temporal video error concealment algorithm using partial differential equation,"Proceedings of SPIE, the International Society for Optical Engineering. Society of Photo-Optical Instrumentation Engineers, 2005.
  14. Aldahdooh, Ahmed, et al. "Inpainting-based error concealment for low-delay video communication," in Proc. of Acoustics, Speech and Signal Processing (ICASSP), 2017 IEEE International Conference on. IEEE, 2017.
  15. Pathak, Ketki C., Shubhi Singh, and Jigisha N. Patel. "Error detection and concealment algorithm for compressed video transmission," in Proc. of Humanitarian Technology Conference (R10-HTC), 2016 IEEE Region 10. IEEE, 2016.
  16. Koloda, Jan, et al. "Scalable kernel-based minimum mean square error estimator for accelerated image error concealment,"IEEE Transactions on Broadcasting 63.1, 59-70, 2017. https://doi.org/10.1109/TBC.2016.2619581
  17. Koloda, Jan, et al. " Kernel-Based MMSE Multimedia Signal Reconstruction and Its Application to Spatial Error Concealment,"IEEE Transactions on Multimedia, 16.6, 1729-1738, 2014. https://doi.org/10.1109/TMM.2014.2330314
  18. Belyaev, Evgeny, Soren Forchhammer, and Marian Codreanu. "Error concealment for 3-D DWT based video codec using iterative thresholding," IEEE Communications Letters, 2017.
  19. Tang, Gui-Jin, Xiu-Chang Zhu, and Tian-Liang Liu. "An error concealment algorithm for stereoscopic images based on local reliable disparities," Dianzi Xuebao (Acta Electronica Sinica), 41.4, 781-786, 2013.
  20. Achanta, Radhakrishna, et al. "SLIC superpixels compared to state-of-the-art superpixel methods," IEEE transactions on pattern analysis and machine intelligence, 34.11, 2274-2282, 2012. https://doi.org/10.1109/TPAMI.2012.120
  21. Yoon, Kuk-Jin, and In So Kweon. "Adaptive support-weight approach for correspondence search," IEEE Transactions on Pattern Analysis and Machine Intelligence, 28.4, 650-656, 2006. https://doi.org/10.1109/TPAMI.2006.70
  22. http://vision.middlebury.edu/stereo/data/
  23. Koloda J, Seiler J, Kaup A, et al. "Frequency selective extrapolation with residual filtering for image error concealment," in Proc. of Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on. IEEE, 2014.