Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Intelligent Wheelchair System using Face and Mouth Recognition

얼굴과 입 모양 인식을 이용한 지능형 휠체어 시스템

  • 주진선 (건국대학교 대학원 신기술융합학과) ;
  • 신윤희 (건국대학교 대학원 신기술융합학과) ;
  • 김은이 (건국대학교 대학원 신기술융합학과)
  • Published : 2009.02.15
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we develop an Intelligent Wheelchair(IW) control system for the people with various disabilities. The aim of the proposed system is to increase the mobility of severely handicapped people by providing an adaptable and effective interface for a power wheelchair. To facilitate a wide variety of user abilities, the proposed system involves the use of face-inclination and mouth-shape information, where the direction of an Intelligent Wheelchair(IW) is determined by the inclination of the user's face, while proceeding and stopping are determined by the shape of the user's mouth. To analyze these gestures, our system consists of facial feature detector, facial feature recognizer, and converter. In the stage of facial feature detector, the facial region of the intended user is first obtained using Adaboost, thereafter the mouth region detected based on edge information. The extracted features are sent to the facial feature recognizer, which recognize the face inclination and mouth shape using statistical analysis and K-means clustering, respectively. These recognition results are then delivered to a converter to control the wheelchair. When assessing the effectiveness of the proposed system with 34 users unable to utilize a standard joystick, the results showed that the proposed system provided a friendly and convenient interface.

본 논문에서는 다양한 장애를 가진 사용자들을 위한 지능형 휠체어의 인터페이스를 제안한다. 제안된 시스템의 주된 목적은 전동휠체어의 조이스틱을 사용하기 힘든 장애인들에게 효율적인 인터페이스를 제공함으로써 그들의 안전한 이동성을 보장하여 독립적인 삶을 이끌어 나갈 수 있도록 하는 것이다. 이를 위해 제안된 시스템은 사용자의 얼굴 기울기를 인식하여 휠체어의 회전을 수행하고 입 모양을 인식하여 휠체어의 전진과 정지를 수행 한다. 이러한 얼굴 특징을 인식하기 위해 제안된 시스템은 얼굴 특징 검출기, 얼굴 특징 인식기, 전환기로 구성된다. 얼굴 특징 검출기는 Adaboost를 이용하여 얼굴 영역을 먼저 검출한 후 에지 정보를 이용하여 입 영역을 검출한다. 검출된 결과들은 얼굴 특징 인식기에저 statistical analysis와 K-means clustering을 이용하여 얼굴 각도와 입 모양을 인식한다. 전환기는 인식된 결과들을 휠체어의 모터를 제어하기 위한 명령어로 변환하여 사용자의 얼굴 및 입의 움직임으로 휠체어를 제어할 수 있도록 한다. 제안된 지능형 휠체어의 효율성을 증명하기 위하여 34명의 사용자를 대상으로 다양한 환경에서 실험한 결과 제안된 시스템은 전동 휠체어의 조이스틱을 사용 할 수 없는 장애인들에게 편리한 이동성을 제공하며, 보다 편리하고 친숙한 인터페이스로 사용 될 수 있음을 보여 주었다.

Keywords

References

  1. Y.-L.CHEN, S.-C.CHEN, W.-L.CHEN and J.-F.LIN, 'A head orientated wheelchair for people with disabilities,' DISABILITY AND REHABILITATION, Vol.25, No.6, pp. 249-253. 2003 https://doi.org/10.1080/0963828021000024979
  2. Rafael Barea, Luciano boquete, Elenalopez, Manuel Mazo. 'SIAMO Project(CICYT). Electronics Department. University of Alcala.Madrid.Spain
  3. H.A.Yanco, 'Shared user-computer control of a robotic wheelchair system,' Ph.D. dissertation, Department of Electrical Engineering and Computer Science, MIT, 2008
  4. Gabriel Pires, Rui Araujo, Urbano Nunes, A.T.de Almeida, 'ROBCHAIR A semi autonomouse wheelchair for disabled people,' 3rd IFAC Symposium on Intelligent Autonomous Vehicles(IAV'98), Madrid, pp. 648-652. 1998
  5. Simon P. Levine, David A. Bell, Lincoln A. Jaros, Richard C. Simpson, Yoram Koren, 'The Nav-Chair Assistive Wheelchair Navigation System,' IEEE Transaction on rehabilitation engineering, Vol.7, No.4.1999
  6. Tao Lu, Kui Yuan, Haibing Zhu, Huosheng Hu, 'An Embedded Control system for Intelligent Wheelchair,' IEEE engineering in Medicine & Biology Society. Proceedings of 27th pp. 1-4, 2005
  7. Pei JIA and Huosheng HU, 'Head gesture recognition for hands-free control of an Intelligent Wheelchair,' Industrial robot Vol.34, No.1, pp.60-68, 2007 https://doi.org/10.1108/01439910710718469
  8. Yoshinori Kuno, Nobutaka Shimada, And Yoshiaki Shirai, 'A robotic wheelchair based on the integration of human and environmental observations,' IEEE robotic & Automation Magazine, 2003
  9. Paul viola, Micheal Jones, 'Rapid Object Detection using a Boosted Cascade of Simple Features,' Conference on Computer Vision and Pattern Recognition, 2001
  10. Paul Viola, Michael J.Jones, Robust real-time face detection, International Journal of Computer Vision, pp.137-154, 2004
  11. Jin Sun Ju, Yunhee Shin, Eun Yi Kim, 'Intelligent Wheelchair using face and mouth shape recognition,' International Conference on Consumer Electronics. 2008 https://doi.org/10.1186/1743-0003-6-33
  12. Eun Yi Kim, Sin Kuk Kang.: Eye Tracking using Neural Network and Mean-shift. LNCS, volume 3982, pp. 1200-1209 (2006)
  13. R.Simpson, E.LoPresti S.Hayashi, I. Nourbakhsh and D.Miller, 'The smart wheelchair component system,' Journal of Rehabilitation Research and Development (JRRD), Vol.41, No.3B, pp.429-442, 2004 https://doi.org/10.1682/JRRD.2003.03.0032
  14. H.Mori, T. Nakata and Y.kamiyama, 'Next-Generation Intellignet wheelchair asdevelopmed by ROTA,' Proc. of 36th Int. Symposium on Robotics (ISR2005), pp.39, 2005
  15. A. Lankenau and T. Rofer, 'Smart Wheelchairs state o the art in an Emerging Market,' Kunstliche Intelligenz, Vol.14, No.4, pp. 37-39, 2000
  16. Y.Morere and A.Pruski, 'A multi-agent control structure for intelligent wheelchair and aided navigation for disabled people,' First French-Spanish Workshop on Assistive Technology, pp. 35-46, 2003
  17. Sarangi P.Parikh, Valdir Grassi Jr, Jun Okamoto Jr, 'Usability Study of a Control Framework for an Intelligent Wheelchair,' International Conference on Robotics and Automation Barcelona, 2005
  18. J.M.Park, J.Song, Y.J. Cho and S.R.Oh, 'A hybrid control architecture using a reactive sequencing startegy for mobile robot navigation,' proc. IROS 99, Vol.3, pp. 1279-1284, 1999