Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Mask Wearing Detection System using Deep Learning

딥러닝을 이용한 마스크 착용 여부 검사 시스템

  • Nam, Chung-hyeon (Department of Computer Engineering, Korea University of Technology and Education) ;
  • Nam, Eun-jeong (Department of Computer Engineering, Korea University of Technology and Education) ;
  • Jang, Kyung-Sik (Department of Computer Engineering, Korea University of Technology and Education)
  • Received : 2020.10.05
  • Accepted : 2020.11.21
  • Published : 2021.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, due to COVID-19, studies have been popularly worked to apply neural network to mask wearing automatic detection system. For applying neural networks, the 1-stage detection or 2-stage detection methods are used, and if data are not sufficiently collected, the pretrained neural network models are studied by applying fine-tuning techniques. In this paper, the system is consisted of 2-stage detection method that contain MTCNN model for face recognition and ResNet model for mask detection. The mask detector was experimented by applying five ResNet models to improve accuracy and fps in various environments. Training data used 17,217 images that collected using web crawler, and for inference, we used 1,913 images and two one-minute videos respectively. The experiment showed a high accuracy of 96.39% for images and 92.98% for video, and the speed of inference for video was 10.78fps.

최근 COVID-19로 인해 마스크 착용 여부 자동 검사 시스템에 신경망 기술들을 적용하는 연구가 활발히 진행되고 있다. 신경망 적용 방식에 있어서 1단계 검출 방식 또는 2단계 검출 방식을 사용하며, 데이터를 충분히 확보할 수 없는 경우 사전 학습된 신경망에 대해 가중치 미세 조절 기법을 적용하여 학습한다. 본 논문에서는 얼굴 인식부와 마스크 검출부로 구성되는 2단계 검출 방식을 적용하였으며, 얼굴 인식부에는 MTCNN 모델, 마스크 검출부에는 ResNet 모델을 사용하였다. 마스크 검출부는 다양한 실 상황에서의 인식률과 추론 속도 향상을 위하여 5개의 ResNet모델을 적용하여 실험하였다. 학습 데이터는 웹 크롤러를 이용하여 수집한 17,219개의 정지 영상을 이용하였으며, 1,913개의 정지 영상과 1분 동영상 2개에 대해 각각 추론을 실시하였다. 실험 결과 정지 영상인 경우 96.39%, 동영상인 경우 92.98%의 높은 정확도를 보였고, 동영상 추론 속도는 10.78fps임을 확인하였다.

Keywords

References

  1. E. Blasch, S. Liu, Z. Liu, and Y. Zheng, "Deep Learning Measures of Effectiveness," in Proceeding of the 2018 IEEE National Aerospace and Electronics Conferences, Dayton, pp. 254-261, 2018.
  2. J. Talukdar, S. Gupta, P. S. Rajpura, and R. S. Hegde, "Transfer Learning for Object Detection using State-of-the-Art Deep Neural Networks," in Proceeding of the 5th International Conference on Signal Processing and Integrated Networks, Noida, pp. 78-83, 2018.
  3. S. S. Thomas, S. Gupta, and V. K. Subramanian, "Smart Surveillance Based On Video Summarization," in Proceeding of the 17th IEEE Region 10 Symposium, India, pp. 1-5, 2017.
  4. A. K. Diop, S. Meza, M. Gordan, and A. Vlaicu, "LDA based classification of video surveillance sequences using motion information," in Proceeding of the 20th International Conference on Adavanced Communication Technology, Korea, pp. 1-1, 2018.
  5. S. H. Lee, H. G. Kwon, Y. J. Kim, J. S. Jeong, and H. J. Seo, "Development of CCTV for Identification of Maskless Wearers based on Deep Learning," in Proceeding of the 28th Korea Society of Computer Information, Korea, pp. 317-318, 2020.
  6. W. Y. Cho, S. L. Park, H. S. Kim, and T. J. Yun, "Development of AI Systems for Counting Visitors and Check of Wearning Masks Using Deep Learning Algorithms," in Proceeding of the 28th Korea Society of Computer Information, Korea, pp. 285-286, 2020.
  7. P. Soviany and R. T. Ionescu, "Optimizing the Trade-off between Single-Stage and Two-Stage Deep Object Detectors using Image Difficulty Prediction," in Proceeding of the 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, Romania, pp. 209-214, 2018.
  8. K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks," IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503, Oct. 2016. https://doi.org/10.1109/LSP.2016.2603342
  9. K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in Proceeding of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, USA, pp. 770-778, 2015.
  10. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," in Proceeding of the 25th International Conference on Neural Information Processing Systems, Nevada, pp. 94-90, 2017.
  11. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke and A. Rabinovich, "Going deeper with convolutions," in Proceeding of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, USA, pp. 1-9, 2015.
  12. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proceeding of the 2015 International Conference on Learning Representations, CA, pp. 1-14, 2015.
  13. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proceeding of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Ohio, pp. 580-587, 2014.
  14. R. Girshick, "Fast R-CNN," in Proceeding of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, USA, pp. 1440-1448, 2015.
  15. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," in Proceeding of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, USA, pp. 779-788, 2015.
  16. R. U. Khan, X. Zhang, R. Kurnar, and E. O. Aboagye, "Evaluating the Performance of ResNet Model Based on Image Recognition," in Proceeding of the 2018 International Conference on Computing and Artifical Intelligence, Indonesia, pp. 86-90, 2018.