Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Real-time Defog Processing Using Cooperative Networks

  • Sanghyun Jung (ENERZAi)
  • Received : 2024.08.01
  • Accepted : 2024.09.25
  • Published : 2024.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a deep learning model and inference pipeline that can process high-resolution fog video in real-time, addressing limitations found in classical defogging algorithms and existing deep learning-based defogging models. The key idea is separating the tasks of inferring fog color and estimating the amount of fog into two distinct models, allowing for a more efficient, lightweight design that improves inference speed. While many deep defogging models perform well on synthetic fog images, they suffer from reduced effectiveness on real-world fog images with diverse fog colors and backgrounds. We solve this problem by introducing a synthetic fog dataset generation method tailored for real-world conditions. Through experiments, we demonstrate the increase in visible distance achieved by proposed model and compare its inference speed and defogging performance against pre-trained models on real-world CCTV fog images.

본 논문에서는 기존의 고전적인 안개 제거 알고리즘과 딥러닝 기반의 안개 제거 모델들의 문제점을 개선하고, 고해상도 안개 영상을 실시간으로 처리할 수 있는 딥러닝 모델과 추론 방식을 제안한다. 핵심 아이디어는 안개 영상에서 안개의 색상을 추론하는 모델과 안개량을 추론하는 모델을 분리하여 학습시켜 각각의 모델을 경량화함으로써 추론 속도를 향상시키는 것이다. 또한, 합성 안개 영상에 대해서는 잘 작동하지만 다양한 안개 색상과 배경을 갖는 실제 안개 영상에서는 성능이 떨어지는 문제점을 새로운 데이터셋 생성 방식을 이용하여 해결한다. 실험을 통해 우리의 안개 제거 모델로 안개 이미지를 처리한 후의 가시거리 증가량을 측정하고, 실제 CCTV 영상에 대하여 추론 속도와 안개 제거 성능을 사전 학습된 기존의 모델들과 비교한다.

Keywords

Acknowledgement

This work was supported by the Technology Innovation Program (20019466, Development of Main computing system dedicated to integrate video codec device, autonomous flight, object detect/recognition, communication system, FC) funded By the Ministry of Trade, Industry & Energy(MOTIE, Korea).

References

  1. K. He, J. Sun, and X. Tang, "Single image haze removal using dark channel prior," IEEE transactions on pattern analysis and machine intelligence, Vol. 33, No. 12, pp. 2341-2353, June 2009. DOI: 10.1109/cvpr.2009.5206515 
  2. Y. Xu, J. Wen, L. Fei, and Z. Zhang, "Review of video and image defogging algorithms and related studies on image restoration and enhancement," Ieee Access, Vol. 4, pp. 165-188, Dec. 2016. DOI: 10.1109/ACCESS.2015.2511558 
  3. H. Israel, and F. Kasten, "Koschmieders theorie der horizontalen sichtweite," Die Sichtweite im Nebel und die Moglichkeiten ihrer kunstlichen Beeinflussung, pp. 7-10, 1959. DOI: 10.1007/978-3-663-04661-5_2 
  4. C. L. Gui, Q. Yan, S. Anwar, R. Cong, W. Ren, and C. Li, "Image dehazing transformer with transmission-aware 3d position embedding," In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5812-5820, Jun. 2022. DOI: 10.1109/cvpr52688.2022.00572 
  5. Y. Song, Z. He, H. Qian, and X. Du, "Vision transformers for single image dehazing," IEEE Transactions on Image Processing, Vol. 32, pp. 1927-1941, Mar. 2023. DOI: 10.1109/tip.2023.3256763 
  6. Y. Cui, and A. Knoll, "Exploring the potential of channel interactions for image restoration," Knowledge-Based Systems, Vol. 282, pp. 111156, Dec. 2023. DOI: 10.1016/j.knosys.2023.111156 
  7. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial networks," Communications of the ACM, Vol. 63, No. 11, pp. 139-144, Oct. 2020. DOI: 10.1145/3422622 
  8. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, G. Sylvain, U. Jakob, and N. Houlsby, "An image is worth 16x16 words: Transformers for image recognition at scale," arXiv preprint arXiv:2010.11929, Oct. 2020. DOI: 10.48550/arxiv.2010.11929 
  9. L. Chen, X. Chu, X. Zhang, and J. Sun, "Simple baselines for image restoration," European conference on computer vision, pp. 17-33, 2022. DOI: 10.1007/978-3-031-20071-7_2 
  10. O. Ronneberger, Olaf, P. Fischer, and T, Brox, "U-net: Convolutional networks for biomedical image segmentation," In Proceedings of Medical image computing and computer-assisted intervention-MICCAI 2015: 18th international conference, part III 18, pp. 234-241, Nov. 2015. DOI: 10.1007/978-3-319-24574-4_28 
  11. T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Dollar, "Microsoft coco: Common objects in context," In Proceedings of the Computer Vision-ECCV 2014: 13th European Conference, Part V 13, pp. 740-755, Sep. 2014. DOI: 10.1007/978-3-319-10602-1_48 
  12. N. Carion, F. Massa, G. Synnaeve, N. Usunier, and A. Kirillov, "End-to-end object detection with transformers," In European conference on computer vision, pp. 213-229, Nov. 2020. DOI: 10.1007/978-3-030-58452-8_13 
  13. B. Li, W.Ren, D. Fu, D. Tao, D. Feng, W. Zeng, and Z. Wang, "Benchmarking Single-Image Dehazing and Beyond," IEEE Transactions on Image Processing, Vol. 28, No. 1, pp. 492-505, Aug. 2018. DOI: 10.1109/tip.2018.2867951