Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning Based 3D Gesture Recognition Using Spatio-Temporal Normalization

시 공간 정규화를 통한 딥 러닝 기반의 3D 제스처 인식

  • Chae, Ji Hun (Dept. of Computer Engineering, Graduate School, Keimyung University) ;
  • Gang, Su Myung (Dept. of Computer Engineering, Graduate School, Keimyung University) ;
  • Kim, Hae Sung (Faculty of Computer Engineering, Keimyung University) ;
  • Lee, Joon Jae (Faculty of Computer Engineering, Keimyung University)
  • Received : 2018.03.16
  • Accepted : 2018.04.23
  • Published : 2018.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

Human exchanges information not only through words, but also through body gesture or hand gesture. And they can be used to build effective interfaces in mobile, virtual reality, and augmented reality. The past 2D gesture recognition research had information loss caused by projecting 3D information in 2D. Since the recognition of the gesture in 3D is higher than 2D space in terms of recognition range, the complexity of gesture recognition increases. In this paper, we proposed a real-time gesture recognition deep learning model and application in 3D space using deep learning technique. First, in order to recognize the gesture in the 3D space, the data collection is performed using the unity game engine to construct and acquire data. Second, input vector normalization for learning 3D gesture recognition model is processed based on deep learning. Thirdly, the SELU(Scaled Exponential Linear Unit) function is applied to the neural network's active function for faster learning and better recognition performance. The proposed system is expected to be applicable to various fields such as rehabilitation cares, game applications, and virtual reality.

Keywords

References

  1. J.J. Chae, J.H. Lim, H.S. Kim, and J.J. Lee, "Study on Real-time Gesture Recognition Based on Convolutional Neural Network for Game Applications," Journal of Korea Multimedia Society, Vol. 20, No. 5, pp. 835-843, 2017. https://doi.org/10.9717/kmms.2017.20.5.835
  2. M.J. Kim, J. Heo, J.H. Kim, S.Y. Park, and J. Chang, "Development and Evaluation of Leapmotion-based Game Interface Considering Intuitive Hand Gestures," Journal of The Korean Society for Computer Game, Vol. 27, No. 4, pp. 69-75, 2014.
  3. J. Gu and D. Shin, "An Empirical Evaluation of the Representative Hand Gesture of Task from User Perspective," Archives of Design Research, Vol. 28, No. 1, pp. 133-145, 2015.
  4. K.S. Ham, D. Lee, H.J. Hong, S. Park, and J. Kim, "An Experimental Research on the Usability of Indirect Control Using Finger Gesture Interaction in Three Dimensional Space," Journal of The Korea Contents Association, Vol. 14, No. 11, pp. 519-532, 2014. https://doi.org/10.5392/JKCA.2014.14.11.519
  5. L. Rokach and O. Maimon, "Top-down Induction of Decision Trees Classifiers-A Survey," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) , Vol. 35, No. 4, pp. 476-487, 2005. https://doi.org/10.1109/TSMCC.2004.843247
  6. L. Breiman, "Random Forests," Machine Learning, Vol. 45, pp. 5-32, 2001. https://doi.org/10.1023/A:1010933404324
  7. C. Cortes and V. Vapnik, "Support-vector Networks," Machine Learning, Vol 20, No. 3, pp. 273-297, 1995. https://doi.org/10.1007/BF00994018
  8. H.C. Yoon and J.S. Cho, "Hand Feature Extraction Algorithm Using Curvature Analysis For Recognition of Various Hand Gestures," Journal of the Korea Society of Computer and Information, Vol. 20, No. 5, pp. 13-20, 2015. https://doi.org/10.9708/jksci.2015.20.5.013
  9. D.H. Song, D.M. Kim, and C.W. Lee, "The Chinese Characters Learning Contents Based on Gesture Recognition Using HMM Algorithm," Journal of Korea Multimedia Society, Vol. 15, No. 8, pp. 1067-1074, 2012. https://doi.org/10.9717/kmms.2012.15.8.1067
  10. A. Park, H. Jang, and K. Jung, "Fast and Efficient Implementation of Neural Networks using CUDA and OpenMP," Journal of KISS : Software and Applications, Vol. 36, No. 4, pp. 253-260, 2009.
  11. J. Schmidhuber, "Deep Learning in Neural Networks: An Overview," Neural Networks, Vol. 61, pp. 85-117, 2015. https://doi.org/10.1016/j.neunet.2014.09.003
  12. H.K. Lee and J.H. Kim, "An HMM-based Threshold Model Approach for Gesture Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 21, No. 10, pp. 961-973, 1999. https://doi.org/10.1109/34.799904
  13. J. Suarez and R.R. Murphy, "Hand Gesture Recognition with Depth Images: A Review," Proceeding of 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication, pp. 411-417, 2012.
  14. W. Xu and E.J. Lee, "Continuous Gesture Recognition System Using Improved HMM Algorithm Based on 2D and 3D Space," International Journal of Multimedia and Ubiquitous Engineering, Vol. 7, No. 2, pp. 335-340, 2012.
  15. S. Cho, H. Byun, H.K. Lee, and J. Cha, "Hand Gesture Recognition from Kinect Sensor Data," Journal of Broadcast Engineering, Vol. 17, No. 3, pp. 447-458, 2012. https://doi.org/10.5909/JBE.2012.17.3.447
  16. J. Oh, T. Kim, and H. Hong, "Gesture Recognition Method Using Tree Classification and Multiclass SVM," Journal of the Institute of Electronics and Information Engineers, Vol. 50, No. 6, pp. 238-245, 2013. https://doi.org/10.5573/ieek.2013.50.6.238
  17. M. Jang, H. Choi, J. Kim, and S. Lee, "Neural Network-based Hand Gesture Recognition System," Proceeding of Human Computer Interaction 2012, pp. 676-681, 2012.
  18. D. Xu, "A Neural Network Approach for Hand Gesture Recognition in Virtual Reality Driving Training System of SPG," Proceeding of The 18th International Conference on Pattern Recognition, Vol. 3, pp. 519-522, 2006.
  19. C.S. Lee, S.Y. Chun, M.G. Sohn, and S.H. Lee, "Hand Gesture Interface Using Mobile Camera Devices," Journal of the Korean Institute of Information Scientist and Engineers : Computing Practices and Letters, Vol. 16, No. 5, pp. 621-625, 2010.
  20. Self-Normalizing Neural Networks, https://arxiv.org/abs/1706.02515 (accessed Jun., 10, 2017).
  21. D.R. Cox, "The Regression Analysis of Binary Sequences," Journal of the Royal Statistical Society. Series B (Methodological) , Vol. 20, No. 2, pp. 215-242, 1958.
  22. Recurrent Neural Network, https://en.wikipedia.org/wiki/Recurrent_neural_network (accessed Jun., 10, 2017).
  23. S. Hochreiter and J. Schmidhuber, “Long Short-term Memory,” Neural Computation, Vol. 9, No. 8, pp. 1735-1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735
  24. K. Cho, B. Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk et al, "Learning Phrase Representations Using RNN Encoder-decoder for Statistical Machine Translation," Proceeding of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1724-1734, 2014.