Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Non-Homogeneous Haze Synthesis for Hazy Image Depth Estimation Using Deep Learning

불균일 안개 영상 합성을 이용한 딥러닝 기반 안개 영상 깊이 추정

  • Received : 2022.06.11
  • Accepted : 2022.07.05
  • Published : 2022.07.26
Download PDF
⟨ Previous Next ⟩

Abstract

Image depth estimation is a technology that is the basis of various image analysis. As analysis methods using deep learning models emerge, studies using deep learning in image depth estimation are being actively conducted. Currently, most deep learning-based depth estimation models are being trained with clean and ideal images. However, due to the lack of data on adverse conditions such as haze or fog, the depth estimation may not work well in such an environment. It is hard to sufficiently secure an image in these environments, and in particular, obtaining non-homogeneous haze data is a very difficult problem. In order to solve this problem, in this study, we propose a method of synthesizing non-homogeneous haze images and a learning method for a monocular depth estimation deep learning model using this method. Considering that haze mainly occurs outdoors, datasets mainly containing outdoor images are constructed. Experiment results show that the model with the proposed method is good at estimating depth in both synthesized and real haze data.

영상의 깊이 추정은 다양한 영상 분석의 기반이 되는 기술이다. 딥러닝 모델을 활용한 분석 방법이 대두되면서, 영상의 깊이 추정 분야 또한 딥러닝을 활용하는 연구가 활발하게 이루어지고 있다. 현재 대부분의 딥러닝 영상 깊이 추정 모델들은 깨끗하고 이상적인 환경에서 학습되고 있다. 하지만 연무, 안개가 낀 열악한 환경에서도 깊이 추정 기술이 잘 동작할 수 있으려면 이러한 환경의 데이터를 포함하여야 한다. 하지만 열악한 환경의 영상을 충분히 확보하는 것이 어려운 실정이며, 불균일한 안개 데이터를 얻는 것은 특히 어려운 문제이다. 이를 해결하기 위해, 본 연구에서는 불균일 안개 영상 합성 방법과 이를 활용한 단안 기반의 깊이 추정 딥러닝 모델의 학습을 제안한다. 안개가 주로 실외에서 발생하는 것을 고려하여, 실외 위주의 데이터 세트를 구축한다. 그리고 실험을 통해 제안된 방법으로 학습된 모델이 합성 데이터와 실제 데이터에서 깊이를 잘 추정하는 것을 보인다.

Keywords

Acknowledgement

이 연구는 포스코 ICT 산학협력과제의 지원으로 수행되었습니다.

References

  1. Lee, Jonghyeop, et al., "Slow Sync Image Synthesis from Short Exposure Flash Smartphone Images," Journal of the Korea Computer Graphics Society, 27.3, 1-11, 2021.
  2. Ancuti, Codruta O., Cosmin Ancuti, and Radu Timofte., "NH-HAZE: An image dehazing benchmark with non-homogeneous hazy and haze-free images," Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2020.
  3. Miangoleh, S.M.H., et al., "Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive MultiResolution Merging," In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9685-9694, 2021.
  4. Yin, W., et al., "Learning to recover 3d scene shape from a single image," In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 204-213, 2021.
  5. Narasimhan, S.G., Nayar, S.K., "Vision and the atmosphere," International journal of computer vision, 48(3), pp. 233-254, 2002. https://doi.org/10.1023/A:1016328200723
  6. Ancuti, C., Ancuti, C.O., De Vleeschouwer, C., "D-hazy: A dataset to evaluate quantitatively dehazing algorithms," In: 2016 IEEE international conference on image processing (ICIP), pp. 2226-2230, 2016.
  7. Li, B., et al., "Benchmarking single-image dehazing and beyond," IEEE Transactions on Image Processing, 28(1), pp. 492-505, 2018. https://doi.org/10.1109/tip.2018.2867951
  8. He, K., Sun, J., Tang, X., "Single image haze removal using dark channel prior," IEEE transactions on pattern analysis and machine intelligence, 33(12), pp. 2341-2353, 2010. https://doi.org/10.1109/TPAMI.2010.168
  9. Ancuti, Codruta O., et al., "O-haze: a dehazing benchmark with real hazy and haze-free outdoor images," Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2018.
  10. Saxena, Ashutosh, Sung Chung, and Andrew Ng., "Learning depth from single monocular images," Advances in neural information processing systems, 18, 2005.
  11. Saxena, Ashutosh, Min Sun, and Andrew Y. Ng., "Make3d: Learning 3d scene structure from a single still image," IEEE transactions on pattern analysis and machine intelligence, 31.5, 824-840, 2008. https://doi.org/10.1109/TPAMI.2008.132
  12. Narasimhan, S.G., Nayar, S.K., "Chromatic framework for vision in bad weather," In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2000 (Cat. No. PR00662), pp. 598-605, 2000.
  13. FATTAL, R., "Single image dehazing," ACM Transactions on Graphics, vol. 27., 2008.
  14. Tankovich, V., et al., "Hitnet: Hierarchical iterative tile refinement network for real-time stereo matching," In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14362-14372, 2021.
  15. Shamsafar, F., et al., "MobileStereoNet: Towards Lightweight Deep Networks for Stereo Matching," In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2417-2426, 2022.
  16. Khamis, S., et al., "Stereonet: Guided hierarchical refinement for real-time edge-aware depth prediction," In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 573-590, 2018.
  17. Lee, Jonghyeop, et al., "SINGLE PANORAMA DEPTH ESTIMATION USING DOMAIN ADAPTATION," Journal of the Korea Computer Graphics Society, 26.3, 61-68, 2020. https://doi.org/10.15701/kcgs.2020.26.3.61
  18. Song, M., Lim, S., Kim, W., "Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals," IEEE Transactions on Circuits and Systems for Video Technology, 2021.
  19. PNVR, K., Zhou, H., Jacobs, D., "Sharingan: Combining synthetic and real data for unsupervised geometry estimation," In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13974-13983, 2020.
  20. Ranftl, R., Bochkovskiy, A., Koltun, V., "Vision transformers for dense prediction," In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12179-12188, 2021.
  21. Godard, C., et al., "Digging into self-supervised monocular depth estimation," In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3828-3838, 2019.
  22. Li, Boyi, et al., "Benchmarking single-image dehazing and beyond," IEEE Transactions on Image Processing, 28.1, 492-505, 2018. https://doi.org/10.1109/tip.2018.2867951
  23. Liu, Chunxiao, et al., "Non-homogeneous haze data synthesis based real-world image dehazing with enhancement-and-restoration fused CNNs," Computers & Graphics, 2022.
  24. Geiger, Andreas, et al., "Vision meets robotics: The kitti dataset," The International Journal of Robotics Research, 32.11, 1231-1237, 2013. https://doi.org/10.1177/0278364913491297