A Hand Gesture Recognition Method using Inertial Sensor for Rapid Operation on Embedded Device

  • Lee, Sangyub (Embedded SW Research R&D Center, Korea Electronics Technology Institute) ;
  • Lee, Jaekyu (Embedded SW Research R&D Center, Korea Electronics Technology Institute) ;
  • Cho, Hyeonjoong (Department of Computer and Information Science, Korea University)
  • Received : 2019.09.02
  • Accepted : 2019.11.19
  • Published : 2020.02.29


We propose a hand gesture recognition method that is compatible with a head-up display (HUD) including small processing resource. For fast link adaptation with HUD, it is necessary to rapidly process gesture recognition and send the minimum amount of driver hand gesture data from the wearable device. Therefore, we use a method that recognizes each hand gesture with an inertial measurement unit (IMU) sensor based on revised correlation matching. The method of gesture recognition is executed by calculating the correlation between every axis of the acquired data set. By classifying pre-defined gesture values and actions, the proposed method enables rapid recognition. Furthermore, we evaluate the performance of the algorithm, which can be implanted within wearable bands, requiring a minimal process load. The experimental results evaluated the feasibility and effectiveness of our decomposed correlation matching method. Furthermore, we tested the proposed algorithm to confirm the effectiveness of the system using pre-defined gestures of specific motions with a wearable platform device. The experimental results validated the feasibility and effectiveness of the proposed hand gesture recognition system. Despite being based on a very simple concept, the proposed algorithm showed good performance in recognition accuracy.


Grant : Development wearable device and services for industrial convergence that support intelligent drivers's ADAS system

Supported by : KEIT


  1. C. Yoon, K. Kim, S. B, and S. Y. Park, "Development of Augmented In-Vehicle Navigation System for Head-Up Display," in Proc. of IEEE International conference ICT convergence, pp.601-602, 2014.
  2. Min Yuan, Heng Yao, Chuan Qin and Ying Tiann, "A daynamic hand gesture recognition system incorporating orientation based linear extrapolation predictor and velocity assisted longest common subsequence algorithm," KSII Transactions on internet and information systems, vol. 11, no.9, pp.4491- 4509, 2017.
  3. R. Xu, S. Zhou and W. J. Li., "MEMS accelerometer based nonspecific user hand gesture recognition," IEEE Sensors Journal, vol. 12, no. 5, pp.1166-1173, 2012.
  4. R. Xie, X. Sun, X. Xia and J. Cao., "Matching-Based Extensible Hand Gesture Recognition," IEEE Sensors Journal, vol. 15, no. 6, pp.3475-3483, 2015.
  5. T. Lu., "A motion control method of intelligent wheelchair based on hand gesture recognition," in Proc. of IEEE international conference industrial and electronics applications, pp.957-962, 2013.
  6. B. Zeng, G. Wang and X. Lin., "A hand gesture based interactive presentation system utilizing heterogeneous cameras," Tsinghua Science Technology, vol. 17, no. 3, pp.329-336, 2012.
  7. S. Lian, W. Hu, K. Wang, "Automatic user state recognition for hand gesture based low-cost television control system," IEEE Transaction on consumer electronics, vol. 60, no. 1, pp.107-115, 2014.
  8. C. Zhu and W. Sheng, "Wearable sensor-based hand gesture and daily activity recognition for robot-assisted living," IEEE Transaction on System and Humans, vol. 41, no. 3, pp.569-573, 2011.
  9. Yong-Suk Park, Se-Ho Park, Tae-Gon Kim and Jong-Moon Chung, "Implementation of Gesture Interface for Projected Surfaces," KSII Transactions on internet and information systems, vol. 9, no.1, pp.378- 290, 2015.
  10. Doyeob Lee, Dongkyoo Shin and Dognil Shin, "Real-Time Recognition Method of Counting Fingers for Natural User Interface," KSII Transactions on internet and information systems, vol. 10, no.5, pp.2363- 2373, 2016.
  11. D. Avola, L. Cinque, G. L. Foresti, and M. R. Marini, "An interactive and low-cost full body rehabilitation framework based on 3D immersive serious games," Journal of Biomedical Informatics, vol. 81, pp. 81-100, 2019.
  12. L. E. Sucar, R. Luis, R. Leder, J. Hernandez and I. Sanchez, "Gesture therapy: A vision-based system for upper extremity stroke rehabilitation," in Proc. of IEEE International Conference Engineering in Medicine and Biology, pp.3690-3693, 2010.
  13. D. Avola, L. Cinque, G. L. Foresti, M. R. Marini, D. Pannone, "VRheab: a fully immersive motor rehabilitation system based on recurrent neural network," Multimedia Tools Applications, vol. 77, pp. 24955-24982, 2018.
  14. J. Alon, V. Athitsos, Q. Yuan and S. Sclaroff, "A unified framework for gesture recognition and spatiotemporal gesture segmentation," IEEE Transaction on pattern analysis machine intelligence, vol. 31, no. 9, pp.1685-1699, 2009.
  15. J. K. Oh, "Inertial sensor based recognition of 3-D character gestures with an ensemble classifier," in Proc. of 9th International Workshop Frontiers Handwriting Recognition, pp.112-117, 2004.
  16. S. Zhou, Z. Dong, W. J. Li and C. P. Kwong, "Hand-written character recognition using MEMS motion sensing technology," in Proc. of IEEE/ASME International Conference Intelligent mechatronics, pp.1418-1423, 2008.
  17. A. Akl, C. Feng and S. Valaee, "A novel accelerometer-based gesture recognition system," IEEE Transaction on signal Processing, vol. 59, no. 12, pp.6197-6205, 2011.
  18. C. C. Yang, Y. L. Hsu, K. S. Shih and J. M. Lu, "Real-Time Gait Cycle Parameter Recognition Using a Wearable Accelerometry System," IEEE Sensors Journal, pp.7314-7326, 2011.
  19. J. S. Lipscomb, "A trainable gesture recognizer," Pattern and Recognize, vol. 24, no. 9, pp. 895-907, 1991.
  20. W. M. Newman and R. F. Sproull, Principles of Interactive Computer Graphics, McGraw-Hill: New York, 1979.
  21. D. H. Rubine, The Automatic Recognition of Gesture, Ph.D dissertation, Computer Science Department, Carnegie Mellon Univ., Pittsburgh, Dec. 1991.
  22. K. S. Fu, Syntactic Recognition in Character Recognition, Academic, New York, 1974.
  23. S. S. Fels and G. E. Hinton, "Glove-talk: A neural network interface between a data glove and a speech synthesizer," IEEE Transaction on Neural Network, vol. 4, no. l, pp.2-8, 1993.
  24. C. M. Bishop, Pattern Recognition and Machine Learning, 1st edition, Springer, New York, 2006.
  25. T. Schlomer, B. Poppinga, N. Henze and S. Boll, "Gesture recognition with a Wii controller," in Proc. of 2nd International Conference Tangible and Embedded Interaction, pp.11-14, 2008.
  26. G. Costante, L. Porzi, O. Lanz, P. Valigi and E. Ricci, "Personalizing a smartwatch-based gesture interface with transfer learning," in Proc. of Signal Processing Conference EUSIPCO, pp.2530-2534, 2014.
  27. S. Shin and W. Sung, "Dynamic Hand Gesture Recognition for Wearable Devices with Low Complexity Recurrent Neural Networks," in Proc. of IEEE International symposium on circuits and systems, pp. 2274-2277, 2016.
  28. G. Devineau, F. Moutarde, W. Xi and J. Yang, "Deep Learning for Hand Gesture Recognition on Skeletal Data," in Proc. of IEEE International Conference on Automatic Face and Gesture Recognition Proceedings, pp.106-113, 2018.
  29. Niclas Gyllsdorff, Distributed machine learning for embedded devices, Ph. D dissertation, Teknisk, UPPSALA Univ., Sweden, 2018.