Journal of the Institute of Electronics Engineers of Korea CI (전자공학회논문지CI)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

Eye Tracking Using Neural Network and Mean-shift

신경망과 Mean-shift를 이용한 눈 추적

  • Kang, Sin-Kuk (School of Computer Science & Engineering, Seoul National University) ;
  • Kim, Kyung-Tai (Dept, of Computer Eng. Konkuk University) ;
  • Shin, Yun-Hee (Dept, of Computer Eng. Konkuk University) ;
  • Kim, Na-Yeon (Dept, of Computer Eng. Konkuk University) ;
  • Kim, Eun-Yi (Dept. of Internet and Multimedia Engineering, NITRI, Konkuk University)
  • 강신국 (서울대학교 컴퓨터공학부) ;
  • 김경태 (건국대학교 컴퓨터정보통신공학과) ;
  • 신윤희 (건국대학교 컴퓨터정보통신공학과) ;
  • 김나연 (건국대학교 컴퓨터정보통신공학과) ;
  • 김은이 (건국대학교 인터넷미디어공학부)
  • Published : 2007.01.25
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, an eye tracking method is presented using a neural network (NN) and mean-shift algorithm that can accurately detect and track user's eyes under the cluttered background. In the proposed method, to deal with the rigid head motion, the facial region is first obtained using skin-color model and con-nected-component analysis. Thereafter the eye regions are localized using neural network (NN)-based tex-ture classifier that discriminates the facial region into eye class and non-eye class, which enables our method to accurately detect users' eyes even if they put on glasses. Once the eye region is localized, they are continuously and correctly tracking by mean-shift algorithm. To assess the validity of the proposed method, it is applied to the interface system using eye movement and is tested with a group of 25 users through playing a 'aligns games.' The results show that the system process more than 30 frames/sec on PC for the $320{\times}240$ size input image and supply a user-friendly and convenient access to a computer in real-time operation.

본 논문은 신경망 (neural network: NN)과 mean-shift알고리즘을 이용하여 복잡한 배경에서 사용자의 눈을 정확히 추출하고 추적할 수 있는 눈 추적 시스템을 제안한다. 머리의 움직임에 강건한 시스템을 개발하기 위해서 먼저 피부색 모델과 연결 성분분석을 이용하여 얼굴영역을 추출한다. 그 다음 신경망기반의 텍스처 분류기를 이용하여 얼굴 영역(face region)을 눈 영역(eye region)과 비눈 영역(non-eye region)으로 구분함으로써 눈을 찾는다. 이러한 눈 검출 방법은 안경의 착용 유무에 상관없이 사용자의 눈 영역을 정확히 검출 할 수 있게 한다. 일단 눈 영역이 찾아지면 이후 프레임에서의 눈 영역은 mean-shift알고리즘에 의해 정확하게 추적된다. 제안된 시스템의 효율성을 검증하기 위해서 제안된 시스템은 눈의 움직임을 이용한 인터페이스 시스템에 적용되었고, 이 인터페이스를 이용한 'aliens game'이 구현되었다. 25명의 사용자에 대해 실험한 결과는 제안된 시스템이 보다 편리하고 친숙한 인터페이스로 활용될 수 있다는 것을 보여주었으며, 또한 $320{\times}240$ 크기의 영상을 초당 30프레임의 빠른 속도로 처리함으로써 실시간 시스템에 적용될 수 있음을 보여주었다.

Keywords

References

  1. William T. Freeman and C. D. Weissman. : Television control by hand gestures. IEEE international workshop on automatic face and gesture recognition, 1995
  2. Alois Ferscha, Stefan Resmerita, Clemens Holzmann and Martin Reicho, : Orientation sensing for gesture-based interaction with smart artifacts. Computer Communications, Volume 28, Issue 13, 2 August 2005, Pages 1552-1563 https://doi.org/10.1016/j.comcom.2004.12.046
  3. Hyun Kang, Chang Woo Lee and Keechul Jung, : Recognition-based gesture spotting in video games. Pattern Recognition Letters, Volume 25, Issue 15, November 2004, Pages 1701-1714 https://doi.org/10.1016/j.patrec.2004.06.016
  4. Jacob, Robert J. K.: Human-computer interaction: Input devices. ACM Computing Surveys, Vol. 28, No. 1 (1996) https://doi.org/10.1145/234313.234387
  5. Sharma, R., Pavlovic, V.I., Huang, T.S.: Toward multimodal human-computer interface. Proceedings of the IEEE , Volume: 86 , Issue: 5 (1998) 853-869 https://doi.org/10.1109/5.664275
  6. Kaufman, Arie E., Bandopadhay, Amit., Shaviv, Bernard D.: An Eye Tracking Computer User Interface. Virtual Reality, 1993. Proceedings., IEEE 1993 Symposium on Research Frontiers in , 25-26 (1993) https://doi.org/10.1109/VRAIS.1993.378254
  7. Scassellati, Brian.: Eye finding via face detection for a foveated, active vision system. American Association for Artificial Intelligence. (1998)
  8. Kurata, Takeshi., Okuma, Takashi., Kourogi, Masakatsu., Sakaue, Katsuhiko.: The Hand Mouse: GMM Hand-color Classification and Mean Shift Tracking. In Proc. Second International Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-time Systems (RATFG-RTS 2001) 119-124 https://doi.org/10.1109/RATFG.2001.938920
  9. Takami, N., Irie, Kang C., Ishimatsu, T., Ochiai, T.: Computer interface to use head movement for handicapped people. TENCON '96. Proceedings. 1996 IEEE TENCON. Digital Signal Processing Applications , Volume: 1 , 26-29 Nov. vol. 1 (1996) 468-472 https://doi.org/10.1109/TENCON.1996.608861
  10. Sako, H., Whitehouse, M., Smith, A., Sutherland, A.,: Real-time facial-feature tracking based on matching techniques and its applications. Pattern Recognition, 1994. Vol. 2-Conference B: Computer Vision & Image Processing., Proceedings of the 12th IAPR International. Conference on , Volume: 2 , 9-13 vol.2 (1994) 320-324 https://doi.org/10.1109/ICPR.1994.576928
  11. Kim, Sang-Hoon., Kim, Hyoung-Gon., Tchah, Kyun-Hyon.: Object oriented face detection using range and color information. Electronics Letters , Volume: 34 , Issue: 10, 14 (1998) 979-980 https://doi.org/10.1049/el:19980714
  12. Schiele, Bernet., Waibel, Alex.:Gaze Tracking Based on Face-Color. School of Computer Science, Carnegie Mello University (1995)
  13. Yang, Jie., A., Waibel.: A real-time face tracker.Applications of Computer Vision, 1996. WACV '96., Proceedings 3rd IEEE Workshop on , 2-4 (1996) 142-147 https://doi.org/10.1109/ACV.1996.572043
  14. Chan A. D. C., Englehart K., Hudgins B., and Lovely D. F. : Hidden markov model classification of myoelectric signals in speech. IEEE Engineering in Medicine and Biology Magazine, vol. 21, no. 4, pp. 143？146, 2002
  15. Betke, M., Gips, J., Fleming, P.: The camera mouse: visual tracking of body features to provide computeraccess for people with severe disabilities.Neural Systems and Rehabilitation Engineering, IEEE Transactions on [see also IEEE Trans. on Rehabilitation Engineering], Volume: 10 , Issue: 1. (2002) 1-10 https://doi.org/10.1109/TNSRE.2002.1021581