Visual Quality Enhancement of Three-Dimensional Integral Imaging Reconstruction for Partially Occluded Objects Using Exemplar-Based Image Restoration

  • Received : 2015.12.30
  • Accepted : 2016.01.29
  • Published : 2016.03.31


In generally, the resolution of reconstructed three-dimensional images can be seriously degraded by undesired occlusions in the integral imaging system, because the undesired information of the occlusion overlap the three-dimensional images to be reconstructed. To solve the problem of the undesired occlusion, we present an exemplar-based image restoration method in integral imaging system. In the proposed method, a minimum spanning tree-based stereo matching method is used to remove the region of undesired occlusions in each elemental image. After that, the removed occlusion region of each elemental images are re-established by using the exemplar-based image restoration method. For further improve the performance of the image restoration, the structure tensor is used to solve the filling error cause by discontinuous structures. Finally, the resolution enhanced three-dimensional images are reconstructed by using the restored elemental images. The preliminary experiments are presented to demonstrate the feasibility of the proposed method.


Integral Imaging;Image restoration;Computational reconstruction


  1. G. Lippmann, “La photographic integrale,” Comptes Rendus de l'Académie des Sciences, vol. 146, pp. 446-451, 1908.
  2. J. J. Lee, B. G. Lee, and H. Yoo, “Depth extraction of three-dimensional objects using block matching for slice images in synthetic aperture integral imaging,” Applied Optics, vol. 50, no. 29, pp. 5624-5629, 2011.
  3. J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Optics Letters, vol. 27, no. 13, pp. 1144-1146, 2002.
  4. A. Stern and B. Javidi, “3-D computational synthetic aperture integral imaging (COMPSAII),” Optics Express, vol. 11, no. 19, pp. 2446-2451, 2003.
  5. S. H. Hong, J. S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Optics Express, vol. 12, no. 3, pp. 483-491, 2004.
  6. J. Y. Jang, J. I. Ser, S. Cha, and S. H. Shin, “Depth extraction by using the correlation of the periodic function with an elemental image in integral imaging,” Applied Optics, vol. 51, no. 16, pp. 3279-3286, 2012.
  7. H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Optics Letters, vol. 26, no. 3, pp. 157-159, 2001.
  8. A. Criminisi, P. Perez, and K. Toyama, “Region filling and object removal by exemplar-based image restoration,” IEEE Transactions on Image Processing, vol. 13, no. 9, pp. 1200-1212, 2004.
  9. N. Komodakis, "Image completion using global optimization," in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, NY, pp. 442-452, 2006.
  10. J. Sun, L. Yuan, J. Jia, and H. Y. Shum, “Image completion with structure propagation,” ACM Transactions on Graphics, vol. 24, no. 3, pp. 861-868, 2005.
  11. T. Huang, S. Chen, J. Liu, and X. Tang, "Image inpainting by global structure and texture propagation," in Proceedings of the 15th International Conference on Multimedia, Augsburg, Germany, pp. 517-520, 2007.
  12. H. Zhou and J. Zheng, "Adaptive patch size determination for patch based image completion," in Proceedings of 17th IEEE International Conference on Image Processing, Hong Kong, pp. 421-424, 2010.
  13. Y. Liu, X. J. Tian, Q. Wang, S. X. Shao, and X. L. Sun, "Image inpainting algorithm based on regional segmentation and adaptive window exemplar," in Proceedings of 2010 2nd International Conference on Advanced Computer Control (ICACC), Shenyang, China, pp. 656-659, 2010.
  14. J. Wu and Q. Ruan, "Object removal by cross isophotes exemplar-based inpainting," in Proceedings of 18th International Conference on Pattern Recognition (ICPR), Hong Kong, pp. 810-813, 2006.
  15. A. Telea, “An image inpainting technique based on the fast marching method,” Journal of Graphics Tool, vol. 9, no. 1, pp. 25-36, 2004.
  16. Y. Piao, M. Zhang, and E. S. Kim, “Effective reconstruction of a partially occluded 3-D target by using a pixel restoration scheme in computational integral-imaging,” Optics and Lasers in Engineering, vol. 50, no. 11, pp. 1602-1610, 2012.
  17. S. H. Hong and B. Javidi, “Three-dimensional visualization of partially occluded objects using integral imaging,” Journal of Display Technology, vol. 1, no. 2, pp. 354-359, 2005.
  18. Y. Piao and E. S. Kim, “Performance-enhanced recognition of a far and partially occluded 3-D object by use of direct pixel-mapping in computational curving-effective integral imaging,” Optics Communications, vol. 284, no. 3, pp. 747-755, 2011.
  19. Z. Zhong, Y. Piao, H. Qu, and M. Zhang, "MST-based occlusion detection in synthetic aperture integral imaging," in Proceedings of 2015 OSA Imaging and Applied Optics Congress: Imaging and Applied Optics, Arlington, VA, 2015.
  20. K. Liu, B. T. Su, and Y. B. Wang, “Improved algorithm of exemplar-based image inpainting,” Computer Engineering, vol. 38, no. 7, pp. 193-195, 2012.