KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Dual Exposure Fusion with Entropy-based Residual Filtering

  • Heo, Yong Seok (Dept. of Electrical and Computer Engineering, Ajou University) ;
  • Lee, Soochahn (Dept. of Electrical Engineering, Soonchunhyang University) ;
  • Jung, Ho Yub (Dept. of Computer Engineering, Chosun University)
  • Received : 2016.05.09
  • Accepted : 2017.03.08
  • Published : 2017.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents a dual exposure fusion method for image enhancement. Images taken with a short exposure time usually contain a sharp structure, but they are dark and are prone to be contaminated by noise. In contrast, long-exposure images are bright and noise-free, but usually suffer from blurring artifacts. Thus, we fuse the dual exposures to generate an enhanced image that is well-exposed, noise-free, and blur-free. To this end, we present a new scale-space patch-match method to find correspondences between the short and long exposures so that proper color components can be combined within a proposed dual non-local (DNL) means framework. We also present a residual filtering method that eliminates the structure component in the estimated noise image in order to obtain a sharper and further enhanced image. To this end, the entropy is utilized to determine the proper size of the filtering window. Experimental results show that our method generates ghost-free, noise-free, and blur-free enhanced images from the short and long exposure pairs for various dynamic scenes.

Keywords

References

  1. U. S. Kim, J. M. Lee, Y. M. Kim, K. T. Park and Y. S. Moon, "Photographic Color Reproduction Based on Color Variation Characteristics of Digital Camera," KSII Transactions on Internet and Information Systems, vol. 5, no. 11, pp. 2160-2174, Nov. 2011. https://doi.org/10.3837/tiis.2011.11.016
  2. B. Jin, Z. Jing, and H. Pan, "Multi-modality image fusion via generalized Riesz-wavelet transformation," KSII Transactions on Internet and Information Systems, vol. 8, no. 11, pp. 4118-4136, Nov. 2014. https://doi.org/10.3837/tiis.2014.11.026
  3. F.-P. An, X.-W. Zhou, and D.-C. Lin, "Multiscale self-coordination of bidimensional empirical mode decomposition in image fusion," KSII Transactions on Internet and Information Systems, vol. 9, no. 4, pp. 1441-1456, Apr. 2015. https://doi.org/10.3837/tiis.2015.04.010
  4. P. Sen, N. K. Kalantari, M. Yaesoubi, S. Darabi, D. B Goldman, and E. Shechtman, "Robust Patch-Based HDR Reconstruction of Dynamic Scenes," in Proc. of SIGGRAPH Asia, pp. 203:1-203:11, 2012.
  5. E. Reinhard, G. Ward, S. Pattanaik, and P. Debevec, "High Dynamic Range Imaging: Acquisition, Display and Image-Based Lighting," Morgan Kauffman, 2005.
  6. P. Debevec and J. Malik, "Recovering high dynamic range radiance maps from photographs," in Proc. of SIGGRAPH, pp. 369-378, 1997.
  7. K. Jacobs and C. Loscos and G. Ward, "Automatic high-dynamic range image generation for dynamic scenes," IEEE Computer Graphics and Applications, vol. 28, no. 2, pp. 84-93, 2008. https://doi.org/10.1109/MCG.2008.23
  8. T. Grosch, "Fast and Robust High Dynamic Range Image Generation with Camera and Object Movement," in Proc. of Vision, modeling and visualization, 2006.
  9. O. Gallo, N. Gelfandz, W.C. Chenz, M. Tico, and K. Pulli, "Artifact-free High Dynamic Range Imaging," in Proc. of IEEE Int'l Conf. on Computational Photography, pp. 1-7, 2009.
  10. S. Raman, V. Kumar, and S. Chaudhuri, "Blind De-ghosting for Automatic Multi-Exposure Compositing," in Proc. of SIGGRAPH Asia Poster, 2009.
  11. E. Khan, A. Akyuz, and E. Reinhard, "Ghost removal in high dynamic range images," in Proc. of IEEE Int'l Conf. on Image Processing, pp.1-4, 2006.
  12. Y. S. Heo, K. M. Lee, S. U. Lee, Y. Moon, and J. Cha, "Ghost-Free High Dynamic Range Imaging," in Proc. of Asian Conf. Computer Vision, pp. 486-500, 2010.
  13. T.J. Park and I.K. Park, "High dynamic range image acquisition using multiple images with different apertures," Optical Engineering, vol. 51, no. 12, 127002, 2012. https://doi.org/10.1117/1.OE.51.12.127002
  14. C. Barnes, E. Shechtman, A. Finkelstein, and D. B. Goldman, "PatchMatch: A randomized correspondence algorithm for structural image editing," in Proc. of SIGGRAPH, pp. 24:1-24:12, 2009.
  15. T. Mertens, J. Kautz, and F. V. Reeth, "Exposure Fusion," in Proc. of Pacific Conference on Computer Graphics and Applications, pp. 1-9, 2007.
  16. M. Tico, N. Gelfand, and K. Pulli, "Motion-blur-free Exposure Fusion," in Proc. of IEEE Int'l Conf. on Image Processing, pp. 3321-3324, 2010.
  17. A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen, "Interactive Digital Photomontage," in Proc. of SIGGRAPH, pp. 294-302, 2004.
  18. G. Petschnigg, M. Agrawala, H. Hoppe, R. Szeliski, M. Cohen, and K. Toyama, "Digital Photography with Flash and No-Flash Image Pairs," in Proc. of SIGGRAPH, pp. 664-672, 2004.
  19. L. Yuan, J. Sun, L. Quan, and H.-Y. Shum, "Image Deblurring with Blurred/Noisy Image Pairs," in Proc. of SIGGRAPH, pp.1:1-1:10, 2007.
  20. L. Zhang, A. Deshpande, and X. Chen, "Denoising vs. Deblurring: HDR Imaging Techniques Using Moving Cameras," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 522-529, 2010.
  21. N. Joshi and M. F. Cohen, "Seeing Mt. Rainier: Lucky Imaging for Multi-Image Denoising, Sharpening, and Haze Removal," in Proc. of IEEE Int'l Conf. on Computational Photography, pp. 1-8, 2010.
  22. Y. HaCohen, E. Shechtman, D.B. Goldman, and D. Lischinski, "Non-Rigid Dense Correspondence with Applications for Image Enhancement," in Proc. of SIGGRAPH, pp. 70:1-70:10, 2011.
  23. R. Gonzalez and R. Woods, "Digital Image Processing," Second ed. Prentice Hall, 2002.
  24. A. Buades, B. Coll, and J.-M. Morel, "A non-local algorithm for image denoising," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1-6, 2005.
  25. J. Chen, C.-K. Tang and J. Wang, "Noise Brush: Interactive High Quality Image-Noise Separation," in Proc. of SIGGRAPH Asia, pp. 146:1-146:10, 2009.
  26. http://vision.middlebury.edu/stereo/data/, 2016.
  27. Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004. https://doi.org/10.1109/TIP.2003.819861