Journal of the Institute of Electronics and Information Engineers (전자공학회논문지)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

No-Reference Visibility Prediction Model of Foggy Images Using Perceptual Fog-Aware Statistical Features

시지각적 통계 특성을 활용한 안개 영상의 가시성 예측 모델

  • Choi, Lark Kwon (Department of Electrical and Computer Engineering, The University of Texas at Austin) ;
  • You, Jaehee (Department of Electronic and Electrical Engineering, Hongik University) ;
  • Bovik, Alan C. (Department of Electrical and Computer Engineering, The University of Texas at Austin)
  • Received : 2014.01.24
  • Accepted : 2014.04.02
  • Published : 2014.04.25
Download PDF
⟨ Previous Next ⟩

Abstract

We propose a no-reference perceptual fog density and visibility prediction model in a single foggy scene based on natural scene statistics (NSS) and perceptual "fog aware" statistical features. Unlike previous studies, the proposed model predicts fog density without multiple foggy images, without salient objects in a scene including lane markings or traffic signs, without supplementary geographical information using an onboard camera, and without training on human-rated judgments. The proposed fog density and visibility predictor makes use of only measurable deviations from statistical regularities observed in natural foggy and fog-free images. Perceptual "fog aware" statistical features are derived from a corpus of natural foggy and fog-free images by using a spatial NSS model and observed fog characteristics including low contrast, faint color, and shifted luminance. The proposed model not only predicts perceptual fog density for the entire image but also provides local fog density for each patch size. To evaluate the performance of the proposed model against human judgments regarding fog visibility, we executed a human subjective study using a variety of 100 foggy images. Results show that the predicted fog density of the model correlates well with human judgments. The proposed model is a new fog density assessment work based on human visual perceptions. We hope that the proposed model will provide fertile ground for future research not only to enhance the visibility of foggy scenes but also to accurately evaluate the performance of defog algorithms.

본 논문에서는 자연 이미지가 갖는 통계적 일관성과 안개를 인식하는 시지각적 통계 특성을 이용하여 단일 안개 영상에서, 안개가 없는 참조 영상과의 비교 없이, 시지각적으로 안개 영상의 가시성을 예측한다. 제안하는 모델은 기존 안개 영상의 가시성 예측 방법들이 불가피하게 사용했던 추가 정보들, 예를 들면, 다수의 다양한 안개 영상, 차량 탑재 카메라의 지리적 위치 정보, 사람의 가시성 평가에 대한 학습 결과, 도로 선 혹은 교통 신호와 같이 안개 영상의 돋보이는 특정 물체 정보 등을 사용하지 않는다. 본 논문의 모델은 오직 테스트 안개 영상이 자연 현상에 의한 안개 영상 혹은 안개가 전혀 없는 영상에서 일관적으로 발견되는 통계적 특성으로부터 얼마나 떨어져 있는지 측정함으로써 안개 영상의 가시성을 예측한다. 시지각적으로 안개를 인식하여 일관된 통계를 나타내는 특징 인자들은 공간상의 자연 이미지 통계 모델과 안개 영상의 특징 (명암대비의 감소, 색상과 채도의 감소, 밝기의 증가)으로부터 유도된다. 제안하는 모델은 안개 영상의 전체 영역에 대한 가시성뿐만 아니라 각 관심 영역에서 패치 크기에 따른 지역적 안개 영상의 가시성도 예측할 수 있다. 본 모델의 성능분석을 위하여 사람이 직접 인지하는 가시성 측정 실험을 100 장의 다양한 안개 영상에 대해 수행하였다. 본 논문에서 제시한 모델을 통해 예측된 안개 영상의 가시성과 사람이 체감한 안개 영상의 가시성을 비교한 결과, 둘 사이에 매우 높은 상관관계가 있는 것으로 평가되었다. 본 논문이 제안하는 무참조 안개 영상의 가시성 예측 모델은 사람의 시지각적 특성을 활용한 새로운 방법으로, 향후 안개 영상의 가시성 향상 알고리듬 개발과 선 개발된 안개 제거 및 가시성 향상 알고리듬들의 성능을 정확히 평가할 수 있는 새로운 측정방법 개발 등에 매우 유용할 것으로 기대된다.

Keywords

References

  1. S. K. Nayar, and S. G. Narasimhan, "Vision in Bad Weather," in Proc. IEEE Int. Conf. Comput. Vis., pp. 820-827, Sep. 1999.
  2. S. G. Narasimhan, and S. K. Nayar, "Contrast restoration of weather degraded images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 6, pp. 713-724, Jun. 2003. https://doi.org/10.1109/TPAMI.2003.1201821
  3. Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, "Instant dehazing of images using polarization," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 325-332, 2001.
  4. D. Pomerleau, "Visibility estimation from a moving vehicle using the RALPH vision system," in Proc. IEEE Intell. Transp. Syst., pp. 906-911, 1997.
  5. N. Hautiere, J. -P. Tarel, J. Lavenant, and D. Aubert. "Automatic fog detection and estimation of visibility distance through use of an onboard camera," Mach. Vis. Appl., vol. 17, no. 1, pp. 8-20, Apr. 2006. https://doi.org/10.1007/s00138-005-0011-1
  6. R. T. Tan, "Visibility in bad weather from a single image," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1-8, Jun. 2008.
  7. J. Kopf, B. Neubert, B. Chen, M. Cohen, D. Cohen-Or, O. Deussen, M. Uyttendaele, D. Lischinski, "Deep photo: Model-based photograph enhancement and viewing," ACM Trans. Graph., vol. 27, no. 5, pp.116:1-116:10, 2008.
  8. R. Fattal, "Single image dehazing," ACM Trans. Graph., SIGGRAPH, vol. 27, no. 3, p. 72, 2008.
  9. K. He, J. Sun, and X. Tang, "Single image haze removal using dark channel prior," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1956-1963, Jun. 2009.
  10. C. Ancuti, C. Hermans, and P. Bekaert, "A fast semi-inverse approach to detect and remove the haze from a single image," in Proc. Asian Conf. Comput. Vis., pp. 501-514, 2010.
  11. J. -P. Tarel and N. Hautiere, "Fast visibility restoration from a single color or gray level image," in Proc. IEEE Int. Conf. Comput. Vis., pp. 2201-2208, Sep. - Oct. 2009.
  12. N. Hautiere, J. -P. Tarel, D. Aubert, and E. Dumont, "Blind contrast enhancement assessment by gradient ratioing at visible edges," J. Image Anal. Stereol., vol. 27, no. 2, pp. 87-95, Jun. 2008. https://doi.org/10.5566/ias.v27.p87-95
  13. J. Yu and Q. Liao, "Fast single image fog removal using edge-preserving smoothing," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., pp. 1245-1248, May 2011.
  14. B. Qi, T. Wu, and H. He, "A new defogging method with nested windows," in Proc. IEEE Int. Conf. Inform. Eng. Comput. Sci., pp. 1-4, 2009.
  15. H. Koschmieder, "Theorie der horizontalen sichtweite," in Beitrage zur Physik der Freien Atmosphare. Munich, Germany: Keim & Nemnich, 1924.
  16. R. O. Duda and P.E. Hart, and D. G. Stork, Pattern classification. Wiley-Interscience, 2012.
  17. D. L. Ruderman, "The statistics of natural images," Netw. Comput. Neural Syst., vol. 5, no. 4, pp. 517-548, 1994. https://doi.org/10.1088/0954-898X/5/4/006
  18. M. Carandini, D. J. Heeger, and J. A. Movshon, "Linearity and normalization in simple cells of the macaque primary visual cortex," J. Neurosci., vol. 17, no. 21, pp. 8621-8644, 1997.
  19. M. J. Wainwright, O. Schwartz, and E. P. Simoncelli, "Natural image statistics and divisive normalization: Modeling nonlinearities and adaptation in cortical neurons," in Statistical Theories of the Brain. Cambridge, MA: MIT Press, 2002, pp. 203-222.
  20. A. Mittal, A. K. Moorthy, and A. C. Bovik, "No-reference image quality assessment in the spatial domain," IEEE Trans. Image Process., vol.21, no.12, pp.4695-4708, Dec. 2012. https://doi.org/10.1109/TIP.2012.2214050
  21. A. Mittal, R. Soundararajan, and A. C. Bovik, "Making a "Completely Blind" Image Quality Analyzer," Signal Processing Letters, IEEE, vol.20, no.3, pp.209-212, 2013. https://doi.org/10.1109/LSP.2012.2227726
  22. A. A. Michelson, Studies in Optics, University of Chicago Press, 1927.
  23. C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J., vol. 27, pp.379 -423, 1948. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x
  24. D. Hasler and S. Susstrunk, "Measuring colourfulness in natural images," in Proc. SPIE Human Vision and Electronic Imaging, vol. 5007, pp. 87-95, 2003.
  25. S. Winkler, "Analysis of public image and video databases for quality assessment," IEEE J. Sel. Topics Signal Process., vol. 6, no. 6, pp. 616-625, Oct. 2012. https://doi.org/10.1109/JSTSP.2012.2215007
  26. A. C. Bovik, "Perceptual image processing: Seeing the future," Proc. IEEE, vol. 98, no. 11, pp. 1799-1803, 2010. https://doi.org/10.1109/JPROC.2010.2068371
  27. H. R. Sheikh, M. F. Sabir, and A. C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms", IEEE Trans. Image Process., vol. 15, no. 11, pp. 3440-3451, Nov. 2006. https://doi.org/10.1109/TIP.2006.881959
  28. D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proc. IEEE Int. Conf. Comput. Vis., pp.416-423, Jul. 2001.
  29. E. C. Larson and D. M. Chandler, "Full-reference image quality assessment and the role of strategy," J. Electron. Imag., vol. 19, no. 1, 2010.
  30. ITU-R BT-500-11: Methodology for the subjective assessment of the quality of television pictures, Int. Telecommunication Union Std..
  31. D. H. Brainard, "The psychophysics toolbox," Spatial Vision, vol. 4, no. 4. pp. 433-436, 1997.