Journal of IKEEE (전기전자학회논문지)

Institute of Korean Electrical and Electronics Engineers (한국전기전자학회)
권호 표지
DOI QR코드

DOI QR Code

A Quantification Method of Human Body Motion Similarity using Dynamic Time Warping for Keypoints Extracted from Video Streams

동영상에서 추출한 키포인트 정보의 동적 시간워핑(DTW)을 이용한 인체 동작 유사도의 정량화 기법

  • Im, June-Seok (Dept. of Computer Engineering, Seokyeong University) ;
  • Kim, Jin-Heon (Dept. of Computer Engineering, Seokyeong University)
  • Received : 2020.11.26
  • Accepted : 2020.12.28
  • Published : 2020.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

The matching score evaluating human copying ability can be a good measure to check children's developmental stages, or sports movements like golf swing and dance, etc. It also can be used as HCI for AR, VR applications. This paper presents a method to evaluate the motion similarity between demonstrator who initiates movement and participant who follows the demonstrator action. We present a quantification method of the similarity which utilizes Euclidean L2 distance of Openpose keypoins vector similarity. The proposed method adapts DTW, thus can flexibly cope with the time delayed motions.

사람이 따라 하는 능력을 평가하는 스코어는 아동의 발달 단계 혹은 골프, 무용 동작 등을 점검하는 좋은 수단이 될 수 있다. 또한, 이는 AR, VR 응용에서 HCI로도 활용될 수 있다. 본 논문에서는 동작을 주도해서 수행하는 시범자와 그 동작을 따라 하는 참여자 간의 동작 유사도를 평가하는 방안을 제시하고, 여기서 우리는 Openpose의 키포인트 벡터 유사도의 유클리디안 L2 거리를 활용하는 동작 유사도를 제안한다. 제안된 기법은 DTW를 사용하기 때문에 시간 지연차가 있는 동작에 유연하게 대처할 수 있다.

Keywords

References

  1. S. H. Back and H. E. Lee, "Effect of motion factor-centered early childhood motion education program on basic motion ability and creativity," KOAECE, Vol.11, No.4, pp.37-60, 2006. DOI: 10.1080/03004430801931196
  2. Chen, Weiming, and Jiang, "Fall Detection Based on Key Points of Human-Skeleton Using OpenPose," Symmetry, Vol.12, No.5, pp.744, 2020. DOI: 10.3390/sym12050744
  3. Naidu, Chetna, and Archana Ghotkar, "Hand gesture recognition using leap motion controller," International Journal of Science and Research (IJSR) ISSN (Online): pp.2319-7064, 2006.
  4. Moon, Gyeongsik, Ju Yong Chang, and Kyoung Mu Lee, "V2v-posenet: Voxel-to-voxel prediction network for accurate 3d hand and human pose estimation from a single depth map," Proceedings of the IEEE conference on computer vision and pattern Recognition (CVPR), pp.5079-5088, 2018.
  5. Zhi-chao, Cao, and Lingling Zhang, "Key pose recognition toward sports scene using deeply-learned model," Journal of Visual Communication and Image Representation Vol.63, pp.102571, 2019. DOI: 10.1016/j.jvcir.2019.06.013
  6. Chen and Lin, "Hand gesture recognition using compact CNN via surface electromyography signals," Sensors, Vol.20, No.3 pp.672, 2020. DOI: 10.3390/s20030672
  7. Qiao, Sen, Yilin Wang, and Jian Li, "Real-time human gesture grading based on OpenPose," 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI). IEEE, pp.1-6, 2017. DOI: 10.1109/CISP-BMEI.2017.8301910
  8. Cao and Zhe, "OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields," arXiv preprint arXiv:1812.08008, 2018.
  9. Senin and Pavel, "Dynamic time warping algorithm review," Information and Computer Science Department University of Hawaii at Manoa Honolulu, Vol.40, No.855, pp.1-23, 2008.
  10. Schneider, Memmesheimer, Kramer, and Paulus, "Gesture recognition in RGB videos using human body keypoints and dynamic time warping," In Robot World Cup, pp.37-60, 2019.
  11. Muda, Lindasalwa, Mumtaj Begam, and Irraivan Elamvazuthi, "Voice recognition algorithms using mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques," arXiv preprint arXiv:1003.4083, 2010.
  12. Bala, Anjali, Abhijeet Kumar, and Nidhika Birla, "Voice command recognition system based on MFCC and DTW," International Journal of Engineering Science and Technology, Vol.2. No.12, pp.7335-7342, 2010.
  13. Sempena, Samsu, Nur Ulfa Maulidevi, and Peb Ruswono Aryan, "Human action recognition using dynamic time warping," Proceedings of the 2011 International Conference on Electrical Engineering and Informatics, IEEE, pp.1-5, 2011. DOI: 10.1109/ICEEI.2011.6021605
  14. Xiao, Bin, Haiping Wu, and Yichen Wei, "Simple baselines for human pose estimation and tracking," Proceedings of the European conference on computer vision (ECCV). pp.466-481, 2018.
  15. Mapari, Rajesh B., and Govind Kharat, "Real time human pose recognition using leap motion sensor," 2015 IEEE International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), IEEE, pp.323-328, 2015. DOI: 10.1109/ICRCICN.2015.7434258
  16. Doliotis and Paul, "Comparing gesture recognition accuracy using color and depth information," Proceedings of the 4th international conference on PErvasive technologies related to assistive environments, pp.1-7, 2011. DOI: 10.1145/2141622.2141647
  17. Alp Guler, Riza, Natalia Neverova, and Iasonas Kokkinos, "Densepose: Dense human pose estimation in the wild," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.7297-7306, 2018. DOI: 10.1109/CVPR.2018.00762
  18. JIAN and HO CHUN, Gesture recognition using windowed dynamic time warping, Diss, 2010.
  19. Muscillo and Rossana, "Early recognition of upper limb motor tasks through accelerometers: real-time implementation of a DTW-based algorithm," Computers in biology and medicine, Vol.41. No.3, pp.164-172, 2011. DOI: 10.1016/j.compbiomed.2011.01.007
  20. Shotton and Jamie, "Real-time human pose recognition in parts from single depth images," CVPR 2011, IEEE, pp.1297-1304, 2011. DOI: 10.1109/CVPR.2011.5995316