Real-time Virtual-viewpoint Image Synthesis Algorithm Using Kinect Camera

  • Lee, Gyu-Cheol ;
  • Yoo, Jisang
  • Received : 2013.06.09
  • Accepted : 2014.03.13
  • Published : 2014.05.01


Kinect is a motion sensing camera released by Microsoft in November 2010 for the Xbox360 that is used to produce depth and color images. Because Kinect uses an infrared pattern, it generates holes and noises around an object's boundaries in the obtained images. The flickering phenomenon and unmatched edges also occur. In this paper, we propose a real time virtual-view video synthesis algorithm which results in a high quality virtual view by solving these problems stated above. The experimental results show that the proposed algorithm performs much better than the conventional algorithms.


3D-warping;Kinect;Virtual-viewpoint;Occlusion region;Hole filling


  1. T.J. Kim and J.S. Yoo, "Hierarchical stereo matching with color information," The Journal of Korea Institute of Communications and Information Sciences, vol. 34, no. 3, pp. 279-287, Mar. 2009.
  2. Retrevo Corporation, Could low interest in 3DTV hurt the TV business?, Retrieved Nov., 2011, from
  3. G. M. Um, G. H. Cheong, W. S. Cheong and N. H. Hur, "Technical development and standardization trends of multi-view 3D and free-viewpoint video," The Magazine of the IEEK, vol. 38, no. 2, pp. 18-23, Feb. 2011.
  4. T. Leyvand, C. Meekhof, Y. C. Wei, J. Sun and B. Guo, "Kinect identity: technology and experience," Computer published by the IEEE Computer Society, vol. 44, no. 4, pp. 94-96, Apr. 2011.
  5. Y. S. Park, S. M. Yun and C. S. Won, "Hole filling for kinect depth image according to the causes of the holes," The Conference of Korean Society of Broadcast Engineers, Jeju univ, Korea, pp. 75-80, Jul. 2012.
  6. L. Chen and S. Li, "Depth image enhancement for Kinect using region growing and bilateral filter," ICPR 2012, pp. 3070-3073, japan, Nov. 2012
  7. A. Telea, "An image inpainting technique based on the fast marching method," J. Graphics Tools, vol. 9, no. 1, pp. 25-36, Dec. 2004.
  8. L. Zhao and H. Wang, "Image denoising using trivariate shrinkage filter in the wavelet domain and joint bilateral filter in the spatial Domain," IEEE Trans. Image Process., vol. 18, no. 10, pp. 2364-2369, Oct. 2009.
  9. G. C. Lee, Y. H. Seo and J. S. Yoo, "GPGPU-based multiview synthesis using kinect depth image," The Conference of Korea Institute of Communications and Information Sciences, Yongpyong, Korea, Jan. 2012.
  10. S. A. Hojjatoleslami and J. Kittler, "Region Growing: a New Approach," IEEE Transactions on Image Processing, vol. 7, no. 7, pp. 1079-1084, July. 1998.
  11. C. J. Park, J. H. Ko, and E. S. Kim, "A new intermediate view reconstruction scheme based-on stereo image rectification algorithm," J. KICS, vol. 29, no. 5C, pp. 632-641, May. 2004.
  12. A. Criminisi, P. Perez, and K. Toyama, "Region filling and object removal by exemplar-based image in-painting," IEEE Trans. Image Process., vol. 13, no. 9, pp. 1200-1212, Sep. 2004.
  13. M. S. Ko and J. S. Yoo, "Boundary noises removal and hole filling algorithm for virtual viewpoint image generation," J. KICS, vol. 37, no. 8, pp. 679-688, Aug. 2012.
  14. W. Sun, O. Au, L. Xu, Y.Li and W. Hu, "Novel temporal domain hole filling based on bakground modeling for view synthesis," IEEE International Conference on Image Processing 2012, Florida, USA, Oct. 2012
  15. K.J. Oh, S. Yea, and Y.S. Ho, "Hole filling method using depth based inpainting for view synthesis in free viewpoint television and 3-d video," Proc. of the 27th conference on Picture Coding Symposium (PCS'09), pp. 233-236, May. 2009.
  16. G. C. Lee and J. S. Yoo, "Real-time virtual-view image synthesis algorithm using Kinect camera," The Journal of Korea Institute of Communications and Information Sciences, vol.38, no.5, pp. 409-419, May. 2013.

Cited by

  1. Optimal depth recovery using image guided TGV with depth confidence for high-quality view synthesis vol.39, 2016,
  2. Stereo Matching with Confidence-Region Decomposition and Processing vol.14, pp.1, 2019,