Journal of Biomedical Engineering Research (대한의용생체공학회:의공학회지)

The Korean Society of Medical and Biological Engineering (대한의용생체공학회)
권호 표지
DOI QR코드

DOI QR Code

Hand Gesture Recognition with Convolution Neural Networks for Augmented Reality Cognitive Rehabilitation System Based on Leap Motion Controller

립모션 센서 기반 증강현실 인지재활 훈련시스템을 위한 합성곱신경망 손동작 인식

  • Song, Keun San (Department of Biomedical Engineering & Science, Graduate School of Konyang University) ;
  • Lee, Hyun Ju (Department of Physical Therapy, Konyang University) ;
  • Tae, Ki Sik (Department of Biomedical Engineering & Science, Graduate School of Konyang University)
  • 송근산 (건양대학교 대학원 의료공학과) ;
  • 이현주 (건양대학교 물리치료학과) ;
  • 태기식 (건양대학교 대학원 의료공학과)
  • Received : 2021.08.10
  • Accepted : 2021.08.24
  • Published : 2021.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we evaluated prediction accuracy of Euler angle spectrograph classification method using a convolutional neural networks (CNN) for hand gesture recognition in augmented reality (AR) cognitive rehabilitation system based on Leap Motion Controller (LMC). Hand gesture recognition methods using a conventional support vector machine (SVM) show 91.3% accuracy in multiple motions. In this paper, five hand gestures ("Promise", "Bunny", "Close", "Victory", and "Thumb") are selected and measured 100 times for testing the utility of spectral classification techniques. Validation results for the five hand gestures were able to be correctly predicted 100% of the time, indicating superior recognition accuracy than those of conventional SVM methods. The hand motion recognition using CNN meant to be applied more useful to AR cognitive rehabilitation training systems based on LMC than sign language recognition using SVM.

Keywords

References

  1. Panic AS. Addressing patient motivation in virtual reality based neurocognitive rehabilitation. Delft University of Technology, 2010.
  2. Doidge N. The Brain That Changes Itself: Stories of Personal Triumph from the Frontiers of Brain Science. Penguin Group. 2007.
  3. Burdea CG. Virtual Rehabilitation: benefits and Challenges. Methods of Information in Medicine. 2003;42(5):519-523. https://doi.org/10.1267/METH03050519
  4. Alankus G, Lazar A, May M, Kelleher C. Towards customizable games for stroke rehabilitation. Proceedings of the 28th Annual CHI Conference on Human Factors in Computing Systems (CHI '10). 2010;2113-2122.
  5. Burke JW, McNeill M, Charles D. Morrow P. Crosbie J. McDonough S. Designing engaging, playable games for rehabilitation. Proceedings of the 8th International Conference On Disability, 2010.
  6. Donker T, Cornelisz I, van Klaveren C, van Straten A, Carlbring P, Cuijpers P, van Gelder JL. Effectiveness of self-guided app-based virtual reality cognitive behavior therapy for acrophobia: A randomized clinical trial. JAMA Psychiatry. 2019;76:682-690. https://doi.org/10.1001/jamapsychiatry.2019.0219
  7. Tsao YC, Shu CC, Lan TS. Development of a Reminiscence Therapy System forthe Elderly Using the Integration of Virtual Realityand Augmented Reality. Sustainability. 2019;11:2-10. https://doi.org/10.3390/su11010002
  8. Annema JH, Verstraete M, Abeele VV, Desmet S. Video games in therapy: a therapist's perspective. International Journal of Arts and Technology. 2010;6(1):94-98.
  9. Andersson G, Carlbring P, Titov N, Lindefors N. Internet interventions for adults with anxiety and mood disorders: A narrative umbrella review of recent meta-analyses. Can. J. Psychiatry. 2019;64(7):465-470. https://doi.org/10.1177/0706743719839381
  10. Stinghen I, Chen E, da Silva JM. da Silva R. Gesture recognition using leap motion: a comparison between machine learning algorithms. ACM SIGGRAPH 2018 Posters, 2018.
  11. Khelil B, Amiri H, Hand Gesture Recognition Using leap motion controller for recognition of arabic sign language. Computer Science. 2016.
  12. Tran DS, Ho NH, Yang HJ, Baek ET. Real-Time Hand Gesture Spotting and Recognition Using RGB-D Camera and 3D Convolutional Neural Network. Applied Sciences. 2020;10(2):722. https://doi.org/10.3390/app10020722
  13. Huang DY, Hu WC, Chang SH. Vision-Based Hand Gesture Recognition Using PCA+Gabor Filters and SVM. 15th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, 2009.
  14. Cote-Allard U, Fall CL, Drouin A, Campeau-Lecours A, Gosselin C, Glette K, Laviolette F, Gosselin B. Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2019;27(4):760-771. https://doi.org/10.1109/tnsre.2019.2896269
  15. Radianti J, Majchrzak T, Fromm J, Wohlgenannt I. A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 2020;147:103778. https://doi.org/10.1016/j.compedu.2019.103778
  16. Huang DY, Hu WC, Chang SH. Gabor filter-based handpose angle estimation for hand gesture recognition under varying illumination. Expert Systems with Applications, 2011;38(5): 6031-6042. https://doi.org/10.1016/j.eswa.2010.11.016
  17. Sagayam KM, Hemanth DJ. Hand posture and gesture recognition techniques for virtual reality applications: a survey. Virtual Reality. 2017;21:91-107. https://doi.org/10.1007/s10055-016-0301-0
  18. Badi H. Recent methods in vision-based hand gesture recognition. International Journal of Data Science and Analytics. 2016;1:77-87. https://doi.org/10.1007/s41060-016-0008-z