Journal of Institute of Control, Robotics and Systems (제어로봇시스템학회논문지)

Institute of Control, Robotics and Systems (제어로봇시스템학회)
권호 표지
DOI QR코드

DOI QR Code

A Robust Fingertip Extraction and Extended CAMSHIFT based Hand Gesture Recognition for Natural Human-like Human-Robot Interaction

강인한 손가락 끝 추출과 확장된 CAMSHIFT 알고리즘을 이용한 자연스러운 Human-Robot Interaction을 위한 손동작 인식

  • 이래경 (포항공과대학교 전자전기공학과) ;
  • 안수용 (포항공과대학교 전자전기공학과) ;
  • 오세영 (포항공과대학교 전자전기공학과)
  • Received : 2011.12.13
  • Accepted : 2012.02.11
  • Published : 2012.04.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a robust fingertip extraction and extended Continuously Adaptive Mean Shift (CAMSHIFT) based robust hand gesture recognition for natural human-like HRI (Human-Robot Interaction). Firstly, for efficient and rapid hand detection, the hand candidate regions are segmented by the combination with robust $YC_bC_r$ skin color model and haar-like features based adaboost. Using the extracted hand candidate regions, we estimate the palm region and fingertip position from distance transformation based voting and geometrical feature of hands. From the hand orientation and palm center position, we find the optimal fingertip position and its orientation. Then using extended CAMSHIFT, we reliably track the 2D hand gesture trajectory with extracted fingertip. Finally, we applied the conditional density propagation (CONDENSATION) to recognize the pre-defined temporal motion trajectories. Experimental results show that the proposed algorithm not only rapidly extracts the hand region with accurately extracted fingertip and its angle but also robustly tracks the hand under different illumination, size and rotation conditions. Using these results, we successfully recognize the multiple hand gestures.

Keywords

References

  1. D. D. Nguyen, T. C. Pham, and J. W. Jeon, "Fingertip detection with morphology and geometric calculation," IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1460-1465, St. Louis, USA, Oct. 11-15, 2009.
  2. L. K. Lee and S. S. Kim, "Recognition of hand gesture to human-computer interaction," Korean Institute of Intelligent Systems (in Korean), vol. 11, no. 1, pp. 28-32, 2001.
  3. L. K. Lee, S. Y. An, and S. Y. Oh, "A robust fingertip extraction and hand tracking for human-robot interaction," ICROS Annual conference, pp. 138-142, 2011.
  4. P. Viola and M. J. Jones, "Robust real-time face detection," International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, 2004. https://doi.org/10.1023/B:VISI.0000013087.49260.fb
  5. R. Lienhart and J. Maydt, "An extended set of haar-like features for rapid object detection," IEEE International Conference on Image Processing, pp. 900-903, 2002.
  6. R. L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, "Face detection in color images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 696-706, 2002. https://doi.org/10.1109/34.1000242
  7. M. J. Jones and J. M. Rehg, "Statistical color models with application to skin detection," Tech. Rep. Tech. Rep. CRL 98/11, Cambridge Research Laboratory, Dec. 1998.
  8. S. Malik, "Real-time hand tracking and finger tracking for interaction," CSC2503F Project Report, Dec. 2003.
  9. C. Davatzikos and J. L. Prince, "Convexity analysis of active contour problems," Image Vision Computing, vol. 17, pp. 27-36, 1999. https://doi.org/10.1016/S0262-8856(98)00087-0
  10. G. R. Bradski, "Computer vision face tracking for use in a perceptual user interface," Intel Technology Journal Q2, 1998.
  11. Applied Optimal Estimation, A. Gelb, ed. MIT Press, 1992.
  12. M. J. Black and A. Jepson, "A probabilistic framework for matching temporal trajectories: condensation-based recognition of gestures and expressions," Proc. of European Conference Computer Vision, vol. 1, pp. 909-924, 1998.
  13. S. Baker and I. Matthews, "Lucas-kande 20 years on: a unifying framework: Part 1," Technical Report CMU-RI-TR-02-16, Carnegie Mellon University Robotics Institute, 2002.
  14. S. W. Shon, J. Beh, C. J. Yang, H. Wang, and H. S. Ko, "Robot user control system using hand gesture recognizer," Journal of Institute of Control, Robotics and Systems (in Korean), vol. 17, no. 4, pp. 368-374, 2011. https://doi.org/10.5302/J.ICROS.2011.17.4.368
  15. K. E. Kom and K. B. Sim, "Study of emotion recognition based on facial image for emotional rehabilitation biofeedback," Journal of Institute of Control, Robotics and Systems (in Korean), vol. 16, pp. 957-962, 2010. https://doi.org/10.5302/J.ICROS.2010.16.10.957

Cited by

  1. The Development of a Real-Time Hand Gestures Recognition System Using Infrared Images vol.21, pp.12, 2015, https://doi.org/10.5302/J.ICROS.2015.15.0129