KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Stereo Image Quality Assessment Using Visual Attention and Distortion Predictors

  • Hwang, Jae-Jeong (Dept. of Radiocommunication Eng., Kunsan National University) ;
  • Wu, Hong Ren (School of Electrical and Computer Eng., RMIT University)
  • Received : 2011.05.30
  • Accepted : 2011.08.18
  • Published : 2011.09.29
Download PDF
⟨ Previous Next ⟩

Abstract

Several metrics have been reported in the literature to assess stereo image quality, mostly based on visual attention or human visual sensitivity based distortion prediction with the help of disparity information, which do not consider the combined aspects of human visual processing. In this paper, visual attention and depth assisted stereo image quality assessment model (VAD-SIQAM) is devised that consists of three main components, i.e., stereo attention predictor (SAP), depth variation (DV), and stereo distortion predictor (SDP). Visual attention is modeled based on entropy and inverse contrast to detect regions or objects of interest/attention. Depth variation is fused into the attention probability to account for the amount of changed depth in distorted stereo images. Finally, the stereo distortion predictor is designed by integrating distortion probability, which is based on low-level human visual system (HVS), responses into actual attention probabilities. The results show that regions of attention are detected among the visually significant distortions in the stereo image pair. Drawbacks of human visual sensitivity based picture quality metrics are alleviated by integrating visual attention and depth information. We also show that positive correlation with ground-truth attention and depth maps are increased by up to 0.949 and 0.936 in terms of the Pearson and the Spearman correlation coefficients, respectively.

Keywords

References

  1. A. Kubota, et al. "Multiview Imaging and 3DTV," IEEE Signal Processing Mag., vol. 24, no. 6, pp. 10-21, Nov. 2007.
  2. L.M.J. Meesters, W.A. IJsselsteijn, P.J.H. Seuntiens, "A Survey of Perceptual Evaluations and Requirements of Three-dimensional TV," IEEE Trans. on Circuits and Systems for Video Technol., vol. 14, no. 3, pp. 381-391, Mar. 2004. https://doi.org/10.1109/TCSVT.2004.823398
  3. ICIP2010 Special Session, WP.L1, 3D Video Quality Assessments, Hong Kong, Sept. 26-29, 2010.
  4. ITU-R, Recommendation BT.500-11, "Methodology for the Subjective Assessment of the Quality of Television Pictures," 2002.
  5. ITU-T, Recommendation P.910, "Subjective Video Quality Assessment Methods for Multimedia Applications," Apr. 2008.
  6. ITU-T, Recommendation J.144, "Objective Perceptual Video Quality Measurement Techniques for Digital Cable Television in the Presence of a Full Reference," Geneva, Mar. 2004.
  7. J. Caviedes, F. Oberti, "No-reference Quality Metric for Degraded and Enhanced Video," in Proc. SPIE, vol.5150, pp. 621-632, July 2003.
  8. M.H. Pinson, S. Wolf, "A New Standardized Method for Objectively Measuring Video Quality," IEEE Trans. on Broadcasting, vol. 50, no. 3, pp. 312-322, Sep. 2004. https://doi.org/10.1109/TBC.2004.834028
  9. F. Yang, S. Wan, Q. Xie, H.R. Wu, "No-reference Quality Assessment for Networked Video via Primary Analysis of Bit Stream," IEEE Trans. on Circuits and Sys. for Video Tech.., vol. 20, no. 11, pp. 1544-1554, Nov. 2010. https://doi.org/10.1109/TCSVT.2010.2087433
  10. S. Narkhede, F. Golshani, "Stereoscopic imaging: a real-time, in depth look," IEEE Potentials, vol. 23, no. 1, pp. 38-42, Feb.-Mar. 2004. https://doi.org/10.1109/MP.2004.1266940
  11. G. Sun, N.S. Holliman, "Evaluating Methods for Controlling Depth Perception in Stereoscopic Cinematography," Stereoscopic Displays and Virtual Reality Systems, SPIE, vol. 7237, Jan. 2009.
  12. A. Awawdeh, G. Fan, "Pseudocepstrum for Assessing Stereo Quality of Retinal Images," in Proc. of Asilomar Conf. on Signals, Systems and Computers, vol. 2, pp. 1953-1957, Nov. 2003.
  13. M. Ferre, R. Aracil, M. Sanchez-Uran, "Stereoscopic human interfaces," IEEE Robotics & Automation Mag., vol. 15, no. 4, pp. 50-57, Dec. 2008. https://doi.org/10.1109/MRA.2008.929929
  14. W.A. IJsselsteijn, H. de Ridder, J. Vliegen, "Subjective Evaluation of Stereoscopic Images: Effects of Camera Parameters and Display Duration," IEEE Trans. on Circuits and Systems for Video Technol., vol. 10, no. 2, pp. 225-233, Mar. 2000. https://doi.org/10.1109/76.825722
  15. J. You, L. Xing, A. Perkis, X. Wang, "Perceptual Quality Assessment for Stereoscopic Images based on 2D Image Quality Metrics and Disparity Analysis," in Proc. of 15th Int. Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), Jan. 13-15, 2010.
  16. A. Benoit, et al., "Quality Assessment of Stereoscopic Images," EURASIP J. on Image and Video Process., Special issue on 3D Image and Video Process., vol. 2008, pp. 1-13, 2008.
  17. Z. Wang, et al., "Image Quality Assessment: From Error Visibility to Structural Similarity," IEEE Trans. on Image Processing, vol. 13, no. 4, pp. 600-612, 2004. https://doi.org/10.1109/TIP.2003.819861
  18. M. Carnec, P. Le Callet, D. Barba, "An Image Quality Assessment Method based on Perception of Structural Information," in Proc. of the IEEE Int. Conf. on Image Processing (ICIP '03), vol. 2, pp. 185-188, Sept. 2003.
  19. C.T.E.R. Hewage, et al., "Quality Evaluation of Color Plus Depth Map-based Stereoscopic Video," IEEE J. of Sel. Topics in Signal Process., vol. 3, no. 2, pp. 304-318, Apr. 2009. https://doi.org/10.1109/JSTSP.2009.2014805
  20. L. Shen, J. Yang, Z. Zhang, "Quality Assessment of Stereo Images with Stereo Vision," Int. Congress on Image and Signal Processing, pp. 1-4, 2009.
  21. J. Yang et al., "Objective Quality Assessment Method of Stereo Images," in Proc. of 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, pp. 1-4, 2009.
  22. P. Campisi, P.L. Callet, E. Marini, "Stereoscopic Images Quality Assessment," in Proc. of 15th European Signal Process. Conf. (EUSIPCO), pp. 2110-2114, Sep., 2007.
  23. H. Shao, X. Cao, G. Er, "Objective Quality Assessment of Depth Image based Rendering in 3DTV System," in Proc. of 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, pp. 1-4, 2009.
  24. Z.M.P. Sazzad, et al., "Stereoscopic Image Quality Prediction," in Proc. of Int. Work. on Quality of Multimedia Experience, pp. 180-185, July 2009.
  25. D. Huang, M. Yu, Y. Yang, "Image Evaluation Algorithm for Right View of Stereoscopic Video," in Proc. of Int. Conf. on Signal Processing, pp. 1051-1054, Oct. 2008.
  26. A. Smolic, et al., "Coding algorithms for 3DTV- A Survey," IEEE Trans. on Circuits and Systems for Video Technol., vol. 17, no. 11, pp. 1606-1620, 2007. https://doi.org/10.1109/TCSVT.2007.909972
  27. F. Lu, et al., "Quality Assessment of 3D Asymmetric View Coding using Spatial Frequency Dominance Model," in Proc. of 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, pp. 1-4, 2009.
  28. P. Seuntiënsa, L. Meestersa, W. IJsselsteijna, "Perceptual Evaluation of JPEG Coded Stereoscopic Images," SPIE Stereoscopic displays and virtual reality systems, vol. 5006, pp. 215-226, Jan. 2003.
  29. M.G. Perkins, "Data Compression of Stereopairs," IEEE Trans. on Communication, vol. 40, no. 4, pp. 684-696, 1992. https://doi.org/10.1109/26.141424
  30. P.W. Gorley, N.S. Holliman, "Stereoscopic image quality metrics and compression," in Proc. of SPIE-IS&T Electronic Imaging, Stereoscopic Displays and Virtual Reality Systems, vol. 6803, Jan. 2008.
  31. E. Peli, "Contrast in complex images," J. Opt. Soc. Am. A, vol. 7, no. 10, pp. 2032-2040, Oct. 1990. https://doi.org/10.1364/JOSAA.7.002032
  32. D.G. Lowe, "Object Recognition from Local Scale-Invariant Features," in Proc. of Int. Conf. on Computer Vision, vol. 2, pp. 1150-1157, 1999.
  33. M.A. Fischler, R.C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Comm. of the ACM, vol. 24, pp. 381-395, June 1981. https://doi.org/10.1145/358669.358692
  34. A. Boev, et al., "Modelling of the stereoscopic HVS," MOBILE 3DTV Technical Report D5.3, Apr. 2009.
  35. Y. Zhang, et al., "Stereoscopic Visual Attention Model for 3D Video," Lecture Notes in Computer Sci., vol. 5916, pp. 314-324, Dec. 2009.
  36. L. Itti, C. Koch, "Computational Modeling of Visual Attention," Nature Rev. Neuroscience, vol. 2, no. 11, pp. 194-203, Mar. 2001. https://doi.org/10.1038/35058500
  37. A.M. Treisman, G. Gelade, "A feature-integration theory of attention," Cognitive Psychology., vol. 12, pp. 97-136, 1980. https://doi.org/10.1016/0010-0285(80)90005-5
  38. J.W. Crabtree, et al., "Contributions of Y- and W-cell Pathways to Response Properties of Cat Superior Colliculus Neurons: Comparison of Antibody- and Deprivation-induced Alterations," J. Neurophysiol., vol. 56, no. 4, pp. 1157-1173, 1986. https://doi.org/10.1152/jn.1986.56.4.1157
  39. M. Mancas, B. Gosselin, B. Macq, "A Three-level Computational Attention model," in Proc. of ICVS Workshop on Comput. Attention & Appl., 2007.
  40. A.K. Jain, Fundamentals of digital image processing, Prentice Hall, 1989.
  41. Y. Zhai, M. Shah, "Visual Attention Detection in Video Sequences using Spatiotemporal Cues," in Proc. the 14th ACM Int. Conf. on Multimedia, pp. 815-824, Dec. 2006.
  42. S. Daly, "The Visible Differences Predictor: An Algorithm for the Assessment of Image Fidelity," Digital Image and Human Vision, Cambridge, MIT press, pp. 179-206, 1993.
  43. B.A. Wandell, Foundations of vision, Sinauer Associates, Inc. Pub., 1995.
  44. L. Itti, C. Koch, E. Niebur, "A Model of Saliency-based Visual Attention for Rapid Scene Analysis", IEEE Trans. PAMI, vol. 20, no. 11, pp. 1254-1259, 1998. https://doi.org/10.1109/34.730558
  45. C. Zitnick, T. Kanade, "A Cooperative Algorithm for Stereo Matching and Occlusion Detection, Robotics Institute Tech. Report, CMU-RI-TR-99-35, Carnegie Mellon University, Oct. 1999.
  46. A.B. Watson, J. Hu, J.F. McGowan, "Digital Video Quality Metric based on Human Vision", J. of Electronic Imaging, vol. 10, no. 1, pp. 20-29, 2001. https://doi.org/10.1117/1.1329896
  47. Adobe Creative Team, "Adobe Photoshop CS4 classroom in a book," Adobe Press, 2008.
  48. A.B. Poirson, B.A. Wandell, "Appearance of Colored Patterns: Pattern-Color Separability," J. Opt. Soc. Am. A, vol. 10, no. 12, pp. 2458-2470, Dec. 1993. https://doi.org/10.1364/JOSAA.10.002458
  49. CCIR, "Encoding Parameters of Digital Television for Studios," CCIR Recommendation 601-2, Int. Radio Consult. Committee, Geneva, 1990.
  50. W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, "Numerical recipes : the art of scientific computing," Ch. 14, Cambridge University Press, 2007.

Cited by

  1. Visual Fatigue Reduction Based on Depth Adjustment for DIBR System vol.6, pp.4, 2011, https://doi.org/10.3837/tiis.2012.04.013
  2. Visible Distortion Predictors Based on Visual Attention in Color Images vol.10, pp.3, 2011, https://doi.org/10.6109/jicce.2012.10.3.300
  3. Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics vol.22, pp.5, 2011, https://doi.org/10.1109/tip.2013.2240003
  4. A Simple Quality Assessment Index for Stereoscopic Images Based on 3D Gradient Magnitude vol.2014, pp.None, 2014, https://doi.org/10.1155/2014/890562
  5. 3D Visual Attention for Stereoscopic Image Quality Assessment vol.9, pp.7, 2014, https://doi.org/10.4304/jsw.9.7.1841-1847
  6. Blind Image Quality Assessment for Stereoscopic Images Using Binocular Guided Quality Lookup and Visual Codebook vol.61, pp.2, 2011, https://doi.org/10.1109/tbc.2015.2402491
  7. Full-Reference Quality Assessment of Stereoscopic Images by Learning Binocular Receptive Field Properties vol.24, pp.10, 2011, https://doi.org/10.1109/tip.2015.2436332
  8. 3D-MAD: A Full Reference Stereoscopic Image Quality Estimator Based on Binocular Lightness and Contrast Perception vol.24, pp.11, 2011, https://doi.org/10.1109/tip.2015.2456414
  9. Image deblurring via adaptive proximal conjugate gradient method vol.9, pp.11, 2011, https://doi.org/10.3837/tiis.2015.11.020
  10. Toward a Blind Deep Quality Evaluator for Stereoscopic Images Based on Monocular and Binocular Interactions vol.25, pp.5, 2016, https://doi.org/10.1109/tip.2016.2538462
  11. Learning Blind Quality Evaluator for Stereoscopic Images Using Joint Sparse Representation vol.18, pp.10, 2011, https://doi.org/10.1109/tmm.2016.2594142