Journal of the Institute of Electronics and Information Engineers (전자공학회논문지)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

Rain Detection via Deep Convolutional Neural Networks

심층 컨볼루셔널 신경망 기반의 빗줄기 검출 기법

  • Son, Chang-Hwan (Department of Software Convergence Engineering, Kunsan National University)
  • 손창환 (군산대학교 소프트웨어융합공학과)
  • Received : 2017.04.25
  • Accepted : 2017.07.24
  • Published : 2017.08.25
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes a method of detecting rain regions from a single image. More specifically, a way of training the deep convolutional neural network based on the collected rain and non-rain patches is presented in a supervised manner. It is also shown that the proposed rain detection method based on deep convolutional neural network can provide better performance than the conventional rain detection method based on dictionary learning. Moreover, it is confirmed that the application of the proposed rain detection for rain removal can lead to some improvement in detail representation on the low-frequency regions of the rain-removed images. Additionally, this paper introduces the rain transfer method that inserts rain patterns into original images, thereby producing rain effects on the resulting images. The proposed rain transfer method could be used to augment rain patterns while constructing rain database.

본 논문에서는 단일 영상에서 빗줄기가 포함된 영역을 검출하기 위한 빗줄기 검출 기법을 제시하고자 한다. 특히 빗줄기가 포함된 패치와 그렇지 않은 패치들을 각각 수집한 후에 지도 학습 기반으로 심층 컨볼루셔널 신경망을 훈련시키고 빗줄기 영역을 검출하는 과정에 대해 자세히 소개하고자 한다. 또한 제안한 심층 컨볼루셔널 신경망 기반의 빗줄기 검출 기법이 기존의 사전 학습 기반의 빗줄기 검출 기법과 비교해서 저주파 영역에서 빗줄기 검출 성능이 더 우수함을 보이고자 한다. 그리고 제안한 빗줄기 검출 기법을 빗줄기 제거 분야에 적용해봄으로써 기존의 사전 학습 기반의 빗줄기 검출 기법보다 저주파 영역에서 디테일한 성분을 더 정확하게 묘사할 수 있음을 보여주고자 한다. 부가적으로 본 논문에서는 원본 영상에 빗줄기 패턴을 삽입하여 비가 내리는 시각적인 효과를 줄 수 있는 빗줄기 천이 기법에 대해서도 소개하고자 한다. 제안한 빗줄기 천이 기법은 빗줄기 영상 데이터베이스를 구축할 때 빗줄기의 다양한 패턴을 확보하는 데 유용하게 사용이 될 수 있다.

Keywords

References

  1. D.-Y. Choi, S.-J. Seo, and B.-C. Song, "DSP optimization for rain detection and removal algorithm," Journal of The Institute of Electronics and Information Engineers, vol. 52, no. 9, pp. 96-105, Sept. 2015. https://doi.org/10.5573/ieie.2015.52.9.096
  2. S.-K. Kim, K.-H. Choi, and S.-Y. Park, "A framework for object detection by haze removal," Journal of The Institute of Electronics and Information Engineers, vol. 51, no. 5, pp. 168-176, May 2014. https://doi.org/10.5573/ieie.2014.51.5.168
  3. A Krizhevsky, I Sutskever, GE Hinton, "Imagenet classification with deep convolutional neural networks," in Proc. Neural Information Processing Systems, 2012.
  4. T.-Y Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, P. Dollar, "Microsoft coco: Common objects in context," arXiv:1405.0312, 2014.
  5. C. Liu, J. Yuen and A. Torralba, "Nonparametric scene parsing via label transfer," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 33, no. 12, 2011.
  6. B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso and A. Torralba, "Semantic Understanding of Scenes through ADE20K Dataset,", arXiv:1608.05442.
  7. L.-W. Kang, C.-W. Lin, and Y.-H. Fu, "Automatic single-image-based rain steaks removal via image decomposition," IEEE Transactions on Image Processing, vo. 21, no. 4, pp. 1742-1755, Apr. 2012. https://doi.org/10.1109/TIP.2011.2179057
  8. D. G. Lowe, "Distinct image features from scale-invariant key points," International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, Nov. 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  9. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. IEEE International Conference on Computer Vision and Pattern Recognition, San Diego, CA, pp. 886-893, Jun. 2005.
  10. Li Y., Tan R., and Brown M. S. "Rain streak removal using layer priors", in Proc. IEEE Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 2736-2744, June 2016.
  11. J.-H. Kim, C. Lee, J.-Y. Sim and C.-S. Kim, "Single-image deraining using an adaptive nonlocal means filter," in Proc. IEEE International Conference on Image Processing, Melbourne, VIC, pp. 914-917, Sept. 2013.
  12. S.-C. Pei, Y.-T. Tsai, and C.-Y. Lee, "Removing rain and snow in a single image using saturation and visibility features," in Proc. IEEE International Conference on Multimedia and Expo Workshops, Chengdu, pp. 14-18, July 2014.
  13. C.-H. Son and X.-P. Zhang, "Rain removal via shrinkage of sparse codes and learned rain dictionary" in Proc. IEEE International Conference on Multimedia and Expo Workshop, Seattle, pp. 1-6, July 2016.
  14. A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, D. H. Salesin, "Image analogies," in Proc. 28th annual conference on Computer graphics and interactive techniques, Los Angeles, CA, USA, pp. 327-340, Aug. 2001.
  15. A. A. Efros and W. T. Freeman, "Image quilting for texture synthesis and transfer," in Proc. 28th annual conference on Computer graphics and interactive techniques, Los Angeles, CA, USA, pp. 341-346, Aug. 2001.
  16. http://www.vlfeat.org/matconvnet
  17. L. Zhang, L. Zhang, and A. C. Bovik, "A feature-enriched completely blind image quality evaluator," IEEE Transactions on Image Processing, vol. 24, no. 8, pp. 2579-2591, Aug. 2015. https://doi.org/10.1109/TIP.2015.2426416