Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Determining Whether to Enter a Hazardous Area Using Pedestrian Trajectory Prediction Techniques and Improving the Training of Small Models with Knowledge Distillation

보행자 경로 예측 기법을 이용한 위험구역 진입 여부 결정과 Knowledge Distillation을 이용한 작은 모델 학습 개선

  • Choi, In-Kyu (Intelligent Image Processing Research Center, Korea Electronics Technology Institute) ;
  • Lee, Young Han (Intelligent Image Processing Research Center, Korea Electronics Technology Institute) ;
  • Song, Hyok (Intelligent Image Processing Research Center, Korea Electronics Technology Institute)
  • Received : 2021.07.07
  • Accepted : 2021.08.16
  • Published : 2021.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a method for predicting in advance whether pedestrians will enter the hazardous area after the current time using the pedestrian trajectory prediction method and an efficient simplification method of the trajectory prediction network. In addition, we propose a method to apply KD(Knowledge Distillation) to a small network for real-time operation in an embedded environment. Using the correlation between predicted future paths and hazard zones, we determined whether to enter or not, and applied efficient KD when learning small networks to minimize performance degradation. Experimentally, it was confirmed that the model applied with the simplification method proposed improved the speed by 37.49% compared to the existing model, but led to a slight decrease in accuracy. As a result of learning a small network with an initial accuracy of 91.43% using KD, It was confirmed that it has improved accuracy of 94.76%.

본 논문에서는 보행자 경로 예측 기법을 이용하여 보행자들이 현재 시점 이후로 위험구역으로 진입하는지 사전에 예측하는 방법과 경로 예측 네트워크의 효율적인 간소화 방법을 제안한다. 그리고 임베디드 환경에서 실시간 운용을 위해 작은 네트워크에 대하여 KD(Knowledge Distillation)을 적용하는 방법을 제안한다. 예측된 미래 경로와 위험구역 간의 상관관계를 이용하여 진입 여부를 판단하였으며 작은 네트워크를 학습할 때 효율적인 KD를 적용하여 성능저하를 최소화하였다. 실험을 통하여, 제안하는 간소화 기법을 적용한 모델이 기존 모델과 비교하여 37.49%의 속도향상 대비 미미한 정확도 저하를 이끌어 내는 것을 보여 주었다. 또한, 91.43%의 정확도를 가진 작은 네트워크를 KD를 이용하여 학습한 결과 94.76%의 향상된 정확도를 보임을 확인하였다.

Keywords

Acknowledgement

This work was supported by the Technology development Program(S2957951) funded by the Ministry of SMEs and Startups(MSS, Korea)

References

  1. K. Sharma and D. D. Londhe, "Human Safety Devices Using IoT and Machine Learning: A Review," in 2018 3rd International Conference for Convergence in Technology (I2CT), pp. 1-7, Apr. 2018.
  2. A. I. Maqueda, A. Loquercio, G. Gallego, N. Garcia, and D. Scaramuzza, "Event-based vision meets deep learning on steering prediction for self-driving cars," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5419-5427, 2018.
  3. Y. Huang and Y. Chen, "Survey of State-of-Art Autonomous Driving Technologies with Deep Learning," in 2020 IEEE 20th International Conference on Software Quality, Reliability and Security Companion (QRS-C), pp. 221-228, Dec. 2020.
  4. D. Tabernik, S. Sela, J. Skvarc, and D. Skocaj, "Segmentationbased deep-learning approach for surface-defect detection," Journal of Intelligent Manufacturing, vol. 31, no. 3, pp. 759-776, 2020. https://doi.org/10.1007/s10845-019-01476-x
  5. M. Rudolph, B. Wandt, and B. Rosenhahn, "Same same but differnet: Semi-supervised defect detection with normalizing flows," in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1907-1916, 2021.
  6. J. Yuan and C. Guo, "A deep learning method for detection of dangerous equipment," in 2018 Eighth International Conference on Information Science and Technology (ICIST), pp. 159-164, June. 2018.
  7. B. Cancela, A. Iglesias, M. Ortega, and M. G. Penedo, "Unsupervised trajectory modelling using temporal information via minimal paths," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2553-2560, 2014.
  8. S. Yi, H. Li, and X. Wang, "Understanding pedestrian behaviors from stationary crowd groups," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3488-3496, 2015.
  9. B. Zhou, X. Wang, and X. Tang, "Understanding collective crowd behaviors: Learning a mixture model of dynamic pedestrian-agents," in Computer Vision and Pattern Recognition (CVPR), pp. 2871-2878, 2012.
  10. C. Zhang, H. Li, X. Wang, and X. Yang, "Cross-scene crowd counting via deep convolutional neural networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 833-841, 2015.
  11. R. Mehran, A. Oyama, and M. Shah, "Abnormal crowd behavior detection using social force model," in Computer Vision and Pattern Recognition, pp. 935-942, 2009.
  12. X. Wang, X. Ma, and W. E. L. Grimson, "Unsupervised activity perception in crowded and complicated scenes using hierarchical bayesian models," IEEE Transactions on pattern analysis and machine intelligence, vol. 31, no. 3, pp. 539-555, 2009. https://doi.org/10.1109/TPAMI.2008.87
  13. S. Han, H. Mao, and W. J. Dally, "Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding," arXiv preprint arXiv: 1510.00149, 2015.
  14. D. Lin, S. Talathi, and S. Annapureddy, "Fixed point quantization of deep convolutional networks," in International conference on machine learning, pp. 2849-2858, June. 2016.
  15. A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, and H. Adam, "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017.
  16. S. Yi, H. Li, and X. Wang, "Pedestrian behavior understanding and prediction with deep neural networks," in European Conference on Computer Vision, pp. 263-279, Oct. 2016.
  17. A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese, "Social lstm: Human trajectory prediction in crowded spaces," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 961-971, 2016
  18. P. Zhang, W. Ouyang, P. hang, J. Xue, and N. Zheng, "Sr-lstm: State refinement for lstm towards pedestrian trajectory prediction," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12085-12094, 2019.
  19. J. Amirian, J. B. Hayet, and J. Pettre, "Social ways: Learning multi-modal distributions of pedestrian trajectories with gans," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019.
  20. M. R. U. Saputra, P. P. de Gusmao, Y. Almalioglu, A. Markham, and N. Trigoni, "Distilling knowledge from a deep pose regressor network," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 263-272, 2019.
  21. G. Hinton, O. Vinyals, and J. Dean, "Distilling the knowledge in a neural network," arXiv preprint arXiv: 1503.02531, 2015.
  22. A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, "Fitnets: Hints for thin deep nets," arXiv preprint arXiv:1412.6550, 2014.
  23. S. H. Lee, D. H. Kim, and B. C. Song, "Self-supervised knowledge distillation using singular value decomposition," in Proceedings of the European Conference on Computer Vision (ECCV), pp. 335-350, 2018.
  24. D. Lopez-Paz, L. Bottou, B. Scholkopf, and V. Vapnik, "Unifying distillation and privileged information," arXiv preprint arXiv:1511.03643, 2015.
  25. A. Polino, R. Pascanu, and D. Alistarh, "Model compression via distillation and quantization," arXiv preprint arXiv: 1802.05668, 2018.
  26. H. Wang, H. Zhao, X. Li and X. Tan, "Progressive Blockwise Knowledge Distillation for Neural Network Acceleration," in International Joint Conference on Artificial Intelligence, pp. 2769-2775, Jan. 2018.
  27. J. Yim, D. Joo, J. Bae, and J. Kim, "A gift from knowledge distillation: Fast optimization, network minimization and transfer learning," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4133-4141, 2017.
  28. M. Kang and S. Kang, "Data-free knowledge distillation in neural networks for regression," Expert Systems with Applications, vol. 175, no. 114813, 2021.
  29. M. Takamoto, Y. Morishita, and H. Imaoka, "An efficient method of training small models for regression problems with knowledge distillation," in 2020 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pp. 67-72, Aug. 2020.
  30. J. Yang, B. Marinez, S. A. Center, A. Bulat and G. Tzimiropoulos, "Knowledge distillation via softmax regression representation learning," International Conference on Learning Representations (ICLR), May. 2020.
  31. G. Chen, W. Choi, X. Yu, T. Han, and M. Chandraker, "Learning efficient object detection models with knowledge distillation," in Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 742-751, Dec. 2017.
  32. Y. Xu, Z. Piao, and S. Gao, "Encoding crowd interaction with deep neural network for pedestrian trajectory prediction," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5275-5284, 2018.