Human Gesture Recognition Technology Based on User Experience for Multimedia Contents Control

멀티미디어 콘텐츠 제어를 위한 사용자 경험 기반 동작 인식 기술

  • 김윤식 (동명대학교 정보통신공학과) ;
  • 박상윤 (동명대학교 정보통신공학과) ;
  • 옥수열 (동명대학교 게임공학과) ;
  • 이석환 (동명대학교 정보보호학과) ;
  • 이응주 (동명대학교 정보통신공학과)
  • Received : 2011.12.29
  • Accepted : 2012.08.10
  • Published : 2012.10.31


In this paper, a series of algorithms are proposed for controlling different kinds of multimedia contents and realizing interact between human and computer by using single input device. Human gesture recognition based on NUI is presented firstly in my paper. Since the image information we get it from camera is not sensitive for further processing, we transform it to YCbCr color space, and then morphological processing algorithm is used to delete unuseful noise. Boundary Energy and depth information is extracted for hand detection. After we receive the image of hand detection, PCA algorithm is used to recognize hand posture, difference image and moment method are used to detect hand centroid and extract trajectory of hand movement. 8 direction codes are defined for quantifying gesture trajectory, so the symbol value will be affirmed. Furthermore, HMM algorithm is used for hand gesture recognition based on the symbol value. According to series of methods we presented, we can control multimedia contents by using human gesture recognition. Through large numbers of experiments, the algorithms we presented have satisfying performance, hand detection rate is up to 94.25%, gesture recognition rate exceed 92.6%, hand posture recognition rate can achieve 85.86%, and face detection rate is up to 89.58%. According to these experiment results, we can control many kinds of multimedia contents on computer effectively, such as video player, MP3, e-book and so on.


  1. Li Wei and E. J. Lee, "Human Head Mouse System Based on Facial Gesture Recognition," Journal of Korea Multimedia Society, Vol. 10, No. 12, pp. 1591-1600, 2007.
  2. Anh Viet Nguyen, A Development of Nontouch Human Computer Interface Technology using Gesture Information, Tongmyong University Master Thesis, 2008.
  3. Bai Yu and Eung-Joo Lee, "The Hand Mouse: Hand Detection and Hand Tracking," International Conference on Multimedia, Information Technology and its Application (MITA 2009) , pp. 244-245, 2009.
  4. Shang Yiting and Eung-Joo Lee, "The Human Facial Combined Expression Recognition System," Proceedings of the Korea Multimedia Society Conference, pp. 187, 2009.
  5. 박상윤, 휴먼 컴퓨터 인터랙션을 위한 신경회로망 기반 손 제스처 인식, 동명대학교 박사 학위논문, 2010.
  6. V. Athitsos and S. Sclaroff, "An Appearance -Based Framework for 3D Hand Shape Classification and Camera Viewpoint Estimation," IEEE Conference on Automatic Face and Gesture Recognition, pp. 45-52, 2002.
  7. K. Fukushima, "Neural Network Model for Selective Attention in Visual Pattern Recognition and Associative Recall," Applied Optics, Vol. 26, No. 23, pp. 4985-4992, 1987.
  8. P. Dayan, S. Kakade, and P.R. Montague, "Learning and Selective Attention," Nature Neuroscience, Vol. 3, No. 10, pp. 1218-1223, 2000.
  9. WenKai Xu and Eung-Joo Lee, "Hand Gesture Recognition using Improved Hidden Markov Models," Journal of Korea Multimedia Society, Vol. 14, No. 7, pp. 866-871, 2011.

Cited by

  1. Development of a Hand Gestures SDK for NUI-Based Applications vol.2015, 2015,
  2. Conceptual Metaphor based on Embodied Cognition vol.16, pp.7, 2013,