Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Deep learning based Person Re-identification with RGB-D sensors

  • Kim, Min (Dept. of Industrial & Management Engineering, Inha University) ;
  • Park, Dong-Hyun (Dept. of Industrial & Management Engineering, Inha University)
  • Received : 2021.01.13
  • Accepted : 2021.02.19
  • Published : 2021.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a deep learning-based person re-identification method using a three-dimensional RGB-Depth Xtion2 camera considering joint coordinates and dynamic features(velocity, acceleration). The main idea of the proposed identification methodology is to easily extract gait data such as joint coordinates, dynamic features with an RGB-D camera and automatically identify gait patterns through a self-designed one-dimensional convolutional neural network classifier(1D-ConvNet). The accuracy was measured based on the F1 Score, and the influence was measured by comparing the accuracy with the classifier model (JC) that did not consider dynamic characteristics. As a result, our proposed classifier model in the case of considering the dynamic characteristics(JCSpeed) showed about 8% higher F1-Score than JC.

본 연구에서는 3차원 RGB-D Xtion2 카메라를 이용하여 보행자의 골격좌표를 추출한 결과를 바탕으로 동적인 특성(속도, 가속도)을 함께 고려하여 딥러닝 모델을 통해 사람을 인식하는 방법을 제안한다. 본 논문의 핵심목표는 RGB-D 카메라로 손쉽게 좌표를 추출하고 새롭게 생성한 동적인 특성을 기반으로 자체 고안한 1차원 합성곱 신경망 분류기 모델(1D-ConvNet)을 통해 자동으로 보행 패턴을 파악하는 것이다. 1D-ConvNet의 인식 정확도와 동적인 특성이 정확도에 미치는 영향을 알아보기 위한 실험을 수행하였다. 정확도는 F1 Score를 기준으로 측정하였고, 동적인 특성을 고려한 분류기 모델(JCSpeed)과 고려하지 않은 분류기 모델(JC)의 정확도 비교를 통해 영향력을 측정하였다. 그 결과 동적인 특성을 고려한 경우의 분류기 모델이 그렇지 않은 경우보다 F1 Score가 약 8% 높게 나타났다.

Keywords

References

  1. R. Collins, A. Lipton, T. Kanade, "A System for Video Surveillance and Monitoring", Proc. Am. Nuclear Soc. (ANS) Eighth Int'l Topical Meeting Robotic and Remote Systems, April, 1999.
  2. B. Huang et al., "Sparsity-based occlusion handling method for person re-identification" in Multimedia Modeling, Cham, Switzerland:Springer, 2015.
  3. Y. Huang, S. Luo, E. Chen, "An Efficient Iris Recognition System", Proc. First Int'l Conf. Machine Learning and Cybernetics, pp. 450-454, Nov 2002.
  4. S. Modi, S. Elliott, J. Whetsone, and H. Kim, "Impact of age groups on fingerprint recognition performance," in IEEE Workshop on Automatic Identification Advanced Technologies, pp. 19-23, 2007.
  5. L. Deng, G. Hinton, B. Kingsbury, "New types of deep neural network learning for speech recognition and related applications: An overview", Proc. IEEE Int. Conf. Acoust. Speech Signal Process., pp. 8599-8603, 2013.
  6. G. Johansson, "Visual Motion Perception", Scientific Am., vol. 232, pp. 76-88, 1975. https://doi.org/10.1038/scientificamerican0675-76
  7. Z. Wu, Y. Li and R. J. Radke, "Viewpoint Invariant Human Re-Identification in Camera Networks Using Pose Priors and Subject-Discriminative Features," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 5, pp. 1095-1108, 1 May 2015, doi: 10.1109/TPAMI.2014.2360373.
  8. Shu, Guang. "Human detection, tracking and segmentation in surveillance video." (2014).
  9. S. Zhiyuan, M. Timothy, "Transferring a Semantic Representation for Person Re-Identification and Search," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 4184-4193
  10. T. Dutta, "Evaluation of the Kinect sensor for 3-D kinematic measurement in the workplace", Appl. Ergonom., vol. 43, no. 4, pp. 645-649, Jul. 2012. https://doi.org/10.1016/j.apergo.2011.09.011
  11. B. Bonnechère, B. Jansen, P. Salvia, H. Bouzahouene, L. Omelina, F. Moiseev, V. Sholukha, J. Cornelis, M. Rooze, S. Van Sint Jan, "Validity and reliability of the Kinect within functional assessment activities: Comparison with standard stereophotogram metry", Gait & Posture, vol. 39, no. 1, pp. 593-598, 2014. https://doi.org/10.1016/j.gaitpost.2013.09.018
  12. C. BenAbdelkader and R. Cutler, "View invariant estimation of height and stride for gait recognition", Workshop on Biometric Authentication ECCV, pp.15-167, May 2002.
  13. M. S. Islam, and M. R. Islam, "Window based clothing invariant gait recognition", International Conference on Advances in Electrical Enginering(ICAEE), pp. 411-414, 2013
  14. J. J. Litle and J. E. Boyd, "Recognition people by their gait: the shape of motion", Videre, vol. 1, no. 2, 1998.
  15. L. Le, and W. E. L Grimson, "Gait apearance for recognition", Workshop on Biometric Authentication ECCV, pp. 143-154, 2002.
  16. Q. -S. Li, Z. -T. Lu and D. -D. Zhang, "Integration of Gait and Side Face for Human Recognition in Video," 2009 Second International Symposium on Electronic Commerce and Security, Nanchang, 2009, pp. 65-69, doi: 10.1109/ISECS.2009.202.
  17. A. Kale, A. K. R. Chowdhury, R. Chelapa, "Towards a view invariant gait recognition algorithm", Advanced video and signal based surveilance IEEE Conference on, pp. 143-150, July 2003.
  18. D. Tan, K. Huang, S. Yu, and T. Tan, "Efficient night gait recognition based on template matching", The 18th International Conference on Patern Recognition(ICPR), vol. 3, pp. 100-103, August 2006.
  19. Z. Xue, D. Ming, W. Song, B Wan, and S. Jin, "Infrared gait recognition based on wavelet transform and suport vector machine", Patern Recognition, vol. 43, no. 8, pp. 2904-2910, August 2010. https://doi.org/10.1016/j.patcog.2010.03.011
  20. Xiaxi Huang, N. V. Boulgouris, "Gait recognition with shifted energy image and structural feature extraction", Image Procesing, IEEE Transaction on, Vol. 21, pp. 256-268, April 2012.
  21. S.D. Khan and H. Ullah, "A Survey of Advances in Vision-based Vehicle Re-identification," Computer Vision and Image Understanding, Vol. 182, pp. 50-63, 2019. https://doi.org/10.1016/j.cviu.2019.03.001
  22. Q. Leng, M. Ye, and Q. Tian, "A Survey of Open-world Person Re-identification," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 30, No. 4, pp. 1092-1108, 2019 https://doi.org/10.1109/tcsvt.2019.2898940
  23. E. Ahmed, M. Jones, and T. K. Marks, "An Improved Deep Learning Architecture for Person Re-Identification," Proceedings of the IEEE conference on computer vision and pattern recognition. pp.3908-3916, 2015.
  24. L. Zheng, Y. Huang, H. Lu and Y. Yang, "Pose Invariant Embedding for Deep Person Re-Identification," IEEE Transactions on Image Processing, pp.4500-4509, 2019. DOI: 10.1109/TIP.2019.2910414
  25. D. Wang, N. Canagarajah, D. Redmill and D. Bull, "Multiple description video coding based on zero padding," 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512), Vancouver, BC, 2004, pp. II-205, doi: 10.1109/ISCAS.2004.1329244.
  26. M. Kostinger, M. Hirzer, P. Wohlhart, P. M. Roth and H. Bischof, "Large scale metric learning from equivalence constraints," 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, 2012, pp. 2288-2295, doi: 10.1109/CVPR.2012.6247939.
  27. A. W. M. Smeulders, D. M. Chu, R. Cucchiara, S. Calderara, A. Dehghan and M. Shah, "Visual Tracking: An Experimental Survey," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 7, pp. 1442-1468, July 2014, doi: 10.11 09/TPAMI.2013.230. https://doi.org/10.1109/TPAMI.2013.230
  28. Manghisi, Vito Modesto, et al. "Real time RULA assessment using Kinect v2 sensor." Applied ergonomics 65 (2017): 481-491. https://doi.org/10.1016/j.apergo.2017.02.015