The Journal of Korean Institute of Next Generation Computing (한국차세대컴퓨팅학회논문지)

Korean Institute of Next Generation Computing (한국차세대컴퓨팅학회)
권호 표지

TSSN: A Deep Learning Architecture for Rainfall Depth Recognition from Surveillance Videos

TSSN: 감시 영상의 강우량 인식을 위한 심층 신경망 구조

  • Received : 2018.12.03
  • Accepted : 2018.12.20
  • Published : 2018.12.31

⟨ Previous Next ⟩

Abstract

Rainfall depth is an important meteorological information. Generally, high spatial resolution rainfall data such as road-level rainfall data are more beneficial. However, it is expensive to set up sufficient Automatic Weather Systems to get the road-level rainfall data. In this paper, we proposed to use deep learning to recognize rainfall depth from road surveillance videos. To achieve this goal, we collected two new video datasets, and proposed a new deep learning architecture named Temporal and Spatial Segment Networks (TSSN) for rainfall depth recognition. Under TSSN, the experimental results show that the combination of the video frame and the differential frame is a superior solution for the rainfall depth recognition. Also, the proposed TSSN architecture outperforms other architectures implemented in this paper.

강우량은 매우 중요한 기상 정보이다. 일반적으로, 도로 수준과 같은 높은 공간 해상도의 강우량이 더 높은 가치를 가진다. 하지만, 도로 수준의 강우량을 측정하기 위해 충분한 수의 기상 관측 장비를 설치하는 것은 비용 관점에서 비효율적이다. 본 논문에서는 도로의 감시 카메라 영상으로부터 강우량을 인식하기 위해 심층 신경망을 활용하는 방법에 대해 제시한다. 해당 목표를 달성하기 위해, 본 논문에서는 교내 두 지역의 감시 카메라 영상과 강우량 데이터를 수집했으며, 새로운 심층 신경망 구조인 Temporal and Spatial Segment Networks(TSSN)를 제안한다. 본 논문에서 제시한 심층 신경망으로 강우량 인식을 수행한 결과, 프레임 RGB와 두 연속 프레임 RGB 차이를 입력으로 사용했을 때, 높은 성능으로 강우량 인식을 수행할 수 있었다. 또한, 기존의 심층 신경망 모델과 비교했을 때, 본 논문에서 제안하는 TSSN이 가장 높은 성능을 기록함을 확인할 수 있었다.

Keywords

Acknowledgement

Grant : 기상.지진See-At기술개발연구

Supported by : 기상청

References

  1. S. Kim, K.-H. Hong, H. Jun, Y.-J. Park, M. Park, and Y. Sunwoo, "Effect of precipitation on air pollutant concentration in seoul, korea," Asian Journal of Atmospheric Environment, vol. 8, no. 4, pp. 202-211, 2014. https://doi.org/10.5572/ajae.2014.8.4.202
  2. L.-C. Guo, Y. Zhang, H. Lin, W. Zeng, T. Liu, J. Xiao, S. Rutherford, J. You, and W. Ma, "The washout effects of rainfall on atmospheric particulate pollution in two chinese cities," Environmental Pollution, vol. 215, pp. 195-202, 2016. https://doi.org/10.1016/j.envpol.2016.05.003
  3. 성동수, "CCTV 카메라의 상태 정보에 근거하여 다수의 위치 정보를 처리하는 응급 상황 신고 서비스", 한국차세대컴퓨팅학회 논문지, 제11권, 제6호, pp.51-59, 2015. 12.
  4. 성동수, "CCTV 카메라 공유를 통한 안심 귀가 서비스의 개선", 한국차세대컴퓨팅학회 논문지, 제13권, 제1호, pp.65-73, 2017. 2.
  5. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998. https://doi.org/10.1109/5.726791
  6. 마누엘 에우제니오 모로초 카얌셀라, 임완수, "스마트디바이스에 적용한 기계학습기반 이미지 분류 어플리케이션", 한국차세대컴퓨팅학회 논문지, 제14권, 제1호, pp.16-26, 2018. 2.
  7. 박현철, 이은지, 김윤재, 이상웅, "합성곱 신경망을 이용한 대장내시경 영상 분류", 한국차세대컴퓨팅학회 논문지, 제14권, 제1호, pp.89-98, 2018. 2.
  8. 파튜 유 민 울라, 아민 울라, 칸 무함마드, 이미영, 백성욱, "스마트 감시 애플리케이션을 위해 Deep CNN을 이용한 폭력인식", 한국차세대컴퓨팅학회 논문지, 제14권, 제5호, pp.53-59, 2018. 10.
  9. K. Garg and S. K. Nayar, "Detection and removal of rain from videos," in Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, vol. 1. IEEE, 2004, pp. I-I.
  10. M. Shen and P. Xue, "A fast algorithm for rain detection and removal from videos," in Multimedia and Expo (ICME), 2011 IEEE International Conference on. IEEE, 2011, pp. 1-6.
  11. L.-W. Kang, C.-W. Lin, and Y.-H. Fu, "Automatic single-image-based rain streaks removal via image decomposition," IEEE Transactions on Image Processing, vol. 21, no. 4, p. 1742, 2012. https://doi.org/10.1109/TIP.2011.2179057
  12. X. Xue, X. Jin, C. Zhang, and S. Goto, "Motion robust rain detection and removal from videos," in Multimedia Signal Processing (MMSP), 2012 IEEE 14th International Workshop on. IEEE, 2012, pp. 170-174.
  13. S. Xingjian, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-c. Woo, "Convolutional lstm network: A machine learning approach for precipitation nowcasting," in Advances in neural information processing systems, 2015, pp. 802-810.
  14. Y.-D. Shin, B.-H. Hong, and J.-W. Lee, "Weather condition distinguishing method based data analysis using cctv video," in Proceedings of Korea Information Science Society Winter Conference, 2015, pp. 247-249.
  15. G. Bae, B. Hong, J. Kim, E. Kwon, J. Lee, and K. Kim, "A method for distinguish rainfall by using pixels of cctv images," in Proceedings of Korea Information Science Society Winter Conference 2016, 2016, pp. 1698-1700.
  16. R. Dong, J. Liao, B. Li, H. Zhou, and D. Crookes, "Measurements of rainfall rates from videos," in Image and Signal Processing, BioMedical Engineering and Informatics (CISPBMEI), 2017 10th International Congress on. IEEE, 2017, pp. 1-9.
  17. B. Ko, Z. Li, and H.-J. Choi, "Determination of precipitation from road cctv video by using cnnlstm," in Proceedings of Korea Software Congress 2017, 2017, pp. 820-822.
  18. J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, "Long-term recurrent convolutional networks for visual recognition and description," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 2625-2634.
  19. L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. Van Gool, "Temporal segment networks: Towards good practices for deep action recognition," in European Conference on Computer Vision. Springer, 2016, pp. 20-36.
  20. C. R. de Souza et al., "Action recognition in videos: data-efficient approaches for supervised learning of human action classification models for video," Ph.D. dissertation, Universitat Autonoma de Barcelona.
  21. Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." arXiv preprint arXiv:1502.03167 (2015).
  22. Bengio, Yoshua. "Deep learning of representations for unsupervised and transfer learning." In Proceedings of ICML Workshop on Unsupervised and Transfer Learning, pp. 17-36. 2012.
  23. Caruana, Rich. "Learning many related tasks at the same time with backpropagation." In Advances in neural information processing systems, pp. 657-664. 1995.
  24. J. Carreira and A. Zisserman, "Quo vadis, action recognition? a new model and the kinetics dataset," in Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on. IEEE, 2017, pp. 4724-4733.
  25. Z. Qiu, T. Yao, and T. Mei, "Learning spatiotemporal representation with pseudo-3d residual networks," in 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017, pp. 5534-5542.