KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Novel Image Completion Algorithm Based on Planar Features

  • Xiao, Mang (School of Computer Science and Information Engineering, Shanghai Institute of Technology) ;
  • Liu, Yunxiang (School of Computer Science and Information Engineering, Shanghai Institute of Technology) ;
  • Xie, Li (College of Electronics and Information Engineering, Tongji University) ;
  • Chen, Qiaochuan (College of Electronics and Information Engineering, Tongji University) ;
  • Li, Guangyao (College of Electronics and Information Engineering, Tongji University)
  • Received : 2016.11.18
  • Accepted : 2018.02.22
  • Published : 2018.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

A novel image completion method is proposed that uses the advantage of planar structural information to fill corrupted portions of an image. First, in estimating parameters of the projection plane, the image is divided into several planes, and their planar structural information is analyzed. Second, in calculating the a priori probability of patch and patch offset regularity, this information is converted into a constraint condition to guide the process of filling the hole. Experimental results show that the proposed algorithm is fast and effective, and ensures the structure continuity of the damaged region and smoothness of the texture.

Keywords

References

  1. M. Bertalmio, G. Sapiro, V. Caselles and C. Ballester, "Image inpainting," in Proc. of Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, Louisiana, USA, pp. 417-424, 2000.
  2. C. Guillemot, O. Le Meur, "Image inpainting: Overview and recent advances," Signal Processing Magazine, IEEE, vol. 31, no. 1, pp. 127-144, 2014. https://doi.org/10.1109/MSP.2013.2273004
  3. T. F. Chan, J. Shen, "Nontexture inpainting by curvature-driven diffusions," Journal of Visual Communication and Image Representation, vol. 12, no. 4, pp. 436-449, 2001. https://doi.org/10.1006/jvci.2001.0487
  4. M. Bertalmio, L. Vese, G. Sapiro, S. Osher, "Simultaneous structure and texture image inpainting," Image Processing, IEEE Transactions on, vol. 12 no. 8, pp. 882-889, 2003. https://doi.org/10.1109/TIP.2003.815261
  5. A. Efros and T. K. Leung, "Texture synthesis by non-parametric sampling," in Proc. of IEEE International Conference on Computer Vision, Greece, pp. 1033-1038, 1999.
  6. M. Xiao, G. Y. Li, L. Xie, Y. L. Tan, Y. H. Mao, "Contour-guided image completion using a sample image," Journal of Electronic Imaging, vol. 24, no. 2, pp. 023029-023029, 2015. https://doi.org/10.1117/1.JEI.24.2.023029
  7. P. P. Criminisi and K. Toyama, "Region filling and object removal by exemplar-based image in painting," IEEE Transactions on Image Processing, vol. 13, no. 9, pp. 1200-1212, 2004. https://doi.org/10.1109/TIP.2004.833105
  8. M. Xiao, G. Y. Li, Y. Y. Jiang, L. Xie, Y. He, "Image Completion using Belief Propagation Based on Planar Priorities," KSII Transactions on Internet and Information Systems, vol. 10, no. 9, pp. 4405-4418, 2016. https://doi.org/10.3837/tiis.2016.09.021
  9. U. A. Ignácio, C. R. Jung, "Block-based image inpainting in the wavelet domain," The Visual Computer, vol. 23, no. 9, pp. 733-741, 2007. https://doi.org/10.1007/s00371-007-0139-2
  10. C. Barnes, E. Shechtman, A. Finkelstein, D. Goldman, "PatchMatch: A randomized correspondence algorithm for structural image editing," ACM Transactions on Graphics-TOG, vol. 28, no. 3, pp. 24, 2009.
  11. C. Barnes, E. Shechtman, D. B. Goldman, A. Finkelstein, "The generalized patchmatch correspondence algorithm," Computer Vision-ECCV, Springer Berlin Heidelberg, pp. 29-43, 2010.
  12. K. He, J. Sun, "Computing nearest-neighbor fields via propagation-assisted kd-trees," in Proc. of Computer Vision and Pattern Recognition (CVPR), IEEE Conference on, pp. 111-118, 2012.
  13. J. B. Huang, S. B. Kang, N. Ahuja, "Image completion using planar structure guidance," ACM Transactions on Graphics (TOG), vol. 33, no. 4, pp. 129, 2014.
  14. O. Chum, J. Matas, "Planar affine rectification from change of scale," Computer Vision-ACCV, Springer Berlin Heidelberg. pp. 347-360, 2011.
  15. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," International journal of computer vision, vol. 60, no. 2, pp. 91-110, 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  16. D. Comaniciu, P. Meer, "Mean shift: A robust approach toward feature space analysis," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 24, no. 5, pp. 603-619, 2002. https://doi.org/10.1109/34.1000236
  17. P. Arias, G. Facciolo, V. Caselles, G. Sapiro, "A variational framework for exemplar-based image inpainting," International journal of computer vision, vol. 93, no. 3, pp. 319-347, 2011. https://doi.org/10.1007/s11263-010-0418-7
  18. S. Darabi, E. Shechtman, C. Barnes, D. B. Goldman, P. Sen, "Image melding: combining inconsistent images using patch-based synthesis," ACM Trans. Graph., vol. 31, no. 4, pp. 82, 2012.
  19. Z. Wang, A. C. Bovik, H. R. Sheikh, "Image quality assessment: from error visibility to structural similarity," Image Processing, IEEE Transactions on, vol.13 no. 4, pp. 600-612,2004. https://doi.org/10.1109/TIP.2003.819861
  20. A. Newson, A. Almansa, M. Fradet, Y. Gousseau, P. Prez, "Video inpainting of complex scenes," SIAM Journal on Imaging Sciences, vol. 7, no. 4, pp. 1993-2019, 2014. https://doi.org/10.1137/140954933
  21. S. Yang, J. Liu, S. Song, M. Li, Z. Guo, "Structure-guided image completion via regularity statistics," in Proc. of Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International Conference on. IEEE, pp. 1711-1715, 2016.
  22. M. Li, J. Liu, Z. Xiong, X. Sun, Z. Guo, "MARLow: A Joint Multiplanar Autoregressive and Low-Rank Approach for Image Completion," in Proc. of European Conference on Computer Vision, Springer International Publishing, pp. 819-834, 2016.
  23. S. H. Baek, I. Choi, M. H. Kim, "Multiview image completion with space structure propagation," in Proc. of Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 488-496, 2016.