KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Multi-view Super-Resolution Method with Joint-optimization of Image Fusion and Blind Deblurring

  • Fan, Jun (College of Information System and Management, National University of Defense Technology) ;
  • Wu, Yue (College of Information System and Management, National University of Defense Technology) ;
  • Zeng, Xiangrong (College of Information System and Management, National University of Defense Technology) ;
  • Huangpeng, Qizi (College of Information System and Management, National University of Defense Technology) ;
  • Liu, Yan (College of Information System and Management, National University of Defense Technology) ;
  • Long, Xin (College of Information System and Management, National University of Defense Technology) ;
  • Zhou, Jinglun (College of Information System and Management, National University of Defense Technology)
  • Received : 2017.04.17
  • Accepted : 2018.01.08
  • Published : 2018.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

Multi-view super-resolution (MVSR) refers to the process of reconstructing a high-resolution (HR) image from a set of low-resolution (LR) images captured from different viewpoints typically by different cameras. These multi-view images are usually obtained by a camera array. In our previous work [1], we super-resolved multi-view LR images via image fusion (IF) and blind deblurring (BD). In this paper, we present a new MVSR method that jointly realizes IF and BD based on an integrated energy function optimization. First, we reformulate the MVSR problem into a multi-channel blind deblurring (MCBD) problem which is easier to be solved than the former. Then the depth map of the desired HR image is calculated. Finally, we solve the MCBD problem, in which the optimization problems with respect to the desired HR image and with respect to the unknown blur are efficiently addressed by the alternating direction method of multipliers (ADMM). Experiments on the Multi-view Image Database of the University of Tsukuba and images captured by our own camera array system demonstrate the effectiveness of the proposed method.

Keywords

References

  1. Jun Fan, Xiangrong Zeng, et al. "A depth-based Multi-view Super-Resolution Method Using Image Fusion and Blind Deblurring," KSII Transactions on Internet & Information Systems, vol. 10, no.10, pp. 5129-5152, 2016. https://doi.org/10.3837/tiis.2016.10.027
  2. B. Wilburn, N. Joshi, V. Vaish, et al, "High performance imaging using large camera arrays," ACM Transactions on Graphics (TOG), vol.24, no.3, pp. 765-776, 2005. https://doi.org/10.1145/1073204.1073259
  3. K. Venkataraman, D. Lelescu, J. Duparre, et al, "PiCam: An ultra-thin high performance monolithic camera array," ACM Transactions on Graphics (TOG), vol.32, no.6, pp. 2504-2507, 2013.
  4. C. Guillem, D. James, A. R. Harvey, "Super-resolution imaging using a camera array," Optics letters, vol.39, no.7, pp.1889-1892, 2014. https://doi.org/10.1364/OL.39.001889
  5. C. Guillem, et al. "Compact multi-aperture imaging with high-angular-resolution," Journal of the Optical Society of America A Optics Image Science & Vision, vol.32, no. 3, pp. 411-419, 2015. https://doi.org/10.1364/JOSAA.32.000411
  6. U Mudenagudi, et al. "Super Resolution of Images of 3D scenecs," in Proc. of Computer Vision - Asian Conference on Computer Vision (ACCV) 2007, pp.85-95, 2007.
  7. S Najafi, "Single and Multi-view Video Super-resolution," Master's thesis, McMaster University, 2012.
  8. K. Takahashi and T. Naemura, "Super-resolved free-viewpoint image synthesis based on view-dependent depth estimation," IPSJ Transactions on Computer Vision and Applications, vol. 4, pp.134-148, 2012. https://doi.org/10.2197/ipsjtcva.4.134
  9. R. Nakashima, K. Takahashi, and T. Naemura, "Super-resolved free-viewpoint image synthesis combined with sparse-representation-based super-resolution," in Proc. of IEEE Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2013 Asia-Pacific, pp.1-6, 2013.
  10. J. Yang, J. Wright, T.S. Huang, and Y. Ma, "Image super-resolution via sparse representation," IEEE Transactions on Image Processing, vol.19, no.11, pp.2861-2873, 2010. https://doi.org/10.1109/TIP.2010.2050625
  11. F Sroubek, P Milanfar, et al. "Robust Multichannel Blind Deconvolution via Fast Alternating Minimization," IEEE Transactions on Image Processing, vol.21, no.4, pp.1687-1700, 2011. https://doi.org/10.1109/TIP.2011.2175740
  12. D. Gabay and B. Mercier, "A dual algorithm for the solution of nonlinear variational problems via finite element approximation," Computers and Mathematics with Applications, vol. 2, pp.17-40, 1976. https://doi.org/10.1016/0898-1221(76)90003-1
  13. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, "Distributed optimization and statistical learning via the alternating direction method of multipliers," Foundations and Trends in Machine Learning, vol.3, no.1, pp.1-122, 2011. https://doi.org/10.1561/2200000016
  14. M. V. Afonso, J. M. Bioucas-Dias, M. A. T. Figueiredo, "Fast Image Recovery Using Variable Splitting and Constrained Optimization," IEEE Transactions on Image Process, vol. 19, no.9, pp. 2345-2356, 2010. https://doi.org/10.1109/TIP.2010.2047910
  15. J. Kotera, F. Sroubek, P. Milanfar, "Blind deconvolution using alternating maximum a posteriori estimation with heavy-tailed priors," Computer Analysis of Images and Patterns 2013, pp.59-66, 2013.
  16. S. Lertrattanapanich, N. K. Bose, "High resolution image formation from low resolution frames using Delaunay triangulation," IEEE Trans. Image Process, vol.11, pp.1427-1441, 2002. https://doi.org/10.1109/TIP.2002.806234
  17. Z. Z. Wang, F. H. Qi, "On ambiguities in super-resolution modeling," IEEE Signal Process. Letters, vol.11, pp.678-681, 2004. https://doi.org/10.1109/LSP.2004.831674
  18. R. Hartley, A. Zisserman, "Multiple view geometry in computer vision," Cambridge University Press, 2000.
  19. Qizi Huangpeng, Xiangrong Zeng, Quan Sun and Jun Fan, "Super-resolving blurry multiframe images through multiframe blind deblurring using ADMM," Multimedia Tools and Applications.
  20. D. Krishnan, and R. Fergus, "Fast image deconvolution using hyper-laplacian priors. In NIPS," Proceedings of Neural Information Processing Systems Blurred Lut Nr, pp.1033-1041, 2009.
  21. M. S. Almeida, F. Mario and M. A. T. Figueiredo, "Deconvolving images with unknown boundaries using the alternating direction method of multipliers," IEEE Transactions on Image Processing, vol.22, no.22, pp.3074-3086, 2013. https://doi.org/10.1109/TIP.2013.2258354
  22. P. L. Combettes and V. R. Wajs, "Signal recovery by proximal forward-backward splitting," Multiscale Modeling & Simulation, vol. 4, pp.1168-1200, 2005. https://doi.org/10.1137/050626090
  23. J. Duchi, S. Shalev-Shwartz, Y. Singer, and T. Chandra, "Efficient projections onto the l1-ball for learning in high dimensions," in Proc. of 25th international conference on Machine learning, pp.272-279, 2008.
  24. Z. Wang, A. C. Bovik, H. R. Sheikh, et al, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol.13, no.4, pp.600-612, 2004. https://doi.org/10.1109/TIP.2003.819861