KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

3D View Synthesis with Feature-Based Warping

  • Hu, Ningning (Institute of Information Science, Beijing Jiaotong University) ;
  • Zhao, Yao (Institute of Information Science, Beijing Jiaotong University) ;
  • Bai, Huihui (Institute of Information Science, Beijing Jiaotong University)
  • Received : 2017.01.05
  • Accepted : 2017.06.01
  • Published : 2017.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Three-dimensional video (3DV), as the new generation of video format standard, can provide the viewers with a vivid screen sense and a realistic stereo impression. Meanwhile the view synthesis has become an important issue for 3DV application. Differently from the conventional methods based on depth, in this paper we propose a new view synthesis algorithm, which can employ the correlation among views and warp in the image domain only. There are mainly two contributions. One is the incorporation of sobel edge points into feature extraction and matching, which can obtain a better stable homography and then a visual comfortable synthesis view compared to SIFT points only. The other is a novel image blending method proposed to obtain a better synthesis image. Experimental results demonstrate that the proposed method can improve the synthesis quality both in subjectivity and objectivity.

Keywords

References

  1. C. Fehn, "Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV," in Proc. of SPIE, Stereoscopic Displays and Virtual Reality Systems XI,93, vol. 5291, May 2004.
  2. A. Smolic, K. Mueller, P. Merkle, N. Atzpadin, C. Fehn, M. Muller, O.Schreer, R. Tanger, P. Kauff, and T. Wiegand, "Multi-view video plus depth (MVD) format for advanced 3D video systems," JVT of ISO/IEC MPEG & ITU-T VCEG JVT-W100, Apr. 2007.
  3. T. Zarb, C.J. Debono, "Depth-based image processing for 3d video rendering applications," in Proc. of IWSSIP 2014, IEEE, pp.215-218, May 2014.
  4. N. Stefanoski, O. Wang, M. Lang, P. Greisen, S. Heinzle, and A. Smolic, "Automatic view synthesis by image-domain-warping," IEEE Transactions on Image Processing, vol. 22(9), pp. 3329-3341, Sep. 2013. https://doi.org/10.1109/TIP.2013.2264817
  5. M. Farre, O. Wang, M. Lang, N. Stefanoski, A. Hornung, and A. Smolic, "Automatic content creation for multi-view autostereoscopic displays using image domain warping," in Proc. of 2011 IEEE Intertional Conference on Multimedia and Expo, IEEE, pp. 1-6, Jul. 2011.
  6. A. Smolic, Y. Wang, N. Stefanoski, M. Lang, A. Hornung, and M. Gross, "Non-linear warping and warp coding for content-adaptive prediction in advanced video coding applications," in Proc. of 2010 IEEE Intertional Conference on Image Processing, IEEE, pp. 4225-4228, Sep. 2010.
  7. W. Tao, H. Bai, M. Liu and Y. Zhao, "Asymmetric multiview image coding based on feature matching," in Proc. of SPIE, Optoelectronic Imaging and Multimedia Technology III, 92732C, vol.9273, Oct.2014.
  8. R. Hartley, A. Zisserman , [Multiple view geometry in computer vision], Cambridge University Press, 2003.
  9. Z. Gang, Y. Ping, Y. He, "A new inter-view prediction method for multi-view video coding," in Proc. of 2007 IEEE Workshop on Signal Processing System, IEEE, pp. 227-340, Oct. 2007.
  10. B. Fan, F. Wu, Z. Hu, "Robust line matching through line-point invariants," Pattern Recogn, vol.45(2), pp. 794-805, 2012. https://doi.org/10.1016/j.patcog.2011.08.004
  11. Q. Jia, X. Gao, X. Fan, Z. Luo, H. Li, and Z. Chen, "Novel coplannar line-points invariants for robust line matching across views," European Conference on Computer Vision, Spinger International Publishing, pp. 599-611, 2016.
  12. D.G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, vol.60(2), pp. 91-110, Nov.2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  13. I. Sobel, G. Feldman, "A isotropic 3x3 gradient operator for image processing," a talk presented at the Standard Artificial Intelligence project, pp. 271-272, 1968.
  14. C.A. Glasbey, K.V. Mardia, "A review of image-warping methods," Journal of Applied Statistics,vol. 25(2), pp. 155-171, 1998. https://doi.org/10.1080/02664769823151
  15. H. Wang, X. Zhang, H. Xiong, "Spatio-temporal coherence for 3-D view synthesis with curve-based dispa rity warping," Visual Communications and Image Processing conference, IEEE, pp.177-180, Dec. 2014.
  16. X. Xiao, X. Wang, S. Wang, "Research on image matching based on edge point feature," International Conference on Romote Sensing, Oct. 2010.
  17. M.A. Fischler, R. C. Bolles, "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography," Communications of the ACM, vol.24(6), pp. 381-395, Jun.1981. https://doi.org/10.1145/358669.358692
  18. K. Wegner, O. Stankiewicz, M. Tanimoto, M. Domanski, "Enhanced view synthesis reference software (VSRS) for free-view television," ISO/IEC JTC1/SC29/WG11MPEG2013/M31520, Oct. 2013.