한국멀티미디어학회논문지 (Journal of Korea Multimedia Society)

  • 제12권8호
  • /
  • Pages.1089-1098
  • /
  • 2009
  • /
  • 1229-7771(pISSN)
  • /
  • 2384-0102(eISSN)
한국멀티미디어학회 (Korea Multimedia Society)
권호 표지

모바일 기기에서의 얼굴 특징점 및 선형 보간법 기반 시선 추적

Gaze Detection Based on Facial Features and Linear Interpolation on Mobile Devices

  • 발행 : 2009.08.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

최근에 인간컴퓨터 상호작용 분야에서 사용자의 시선 위치를 파악하여 더욱 편리한 입력 장치를 개발하고자 하는 연구가 많이 진행되고 있다. 기존의 대부분 연구들은 큰 모니터를 사용하는 컴퓨터 환경에서 시선 추적 시스템을 개발하였다. 최근 이동단말기의 사용 증대로 이동 중에 시선 추적에 의한 단말기 제어의 필요성이 증대되고 있다. 이에 본 연구에서는 이동형 컴퓨터 (Ultra-Mobile PC) 및 컴퓨터 내장 카메라를 이용하여 사용자의 얼굴을 추적하고, 얼굴내의 특징점의 위치를 능동외관모델 (Active Appearance Model)을 기반으로 추적하는 연구를 수행하였다. 본 논문의 독창성은 기존 연구와는 달리 소형 화면을 가지는 이동 단말기에서 사용자의 시선 위치를 추적할 수 있는 방법을 제안한 점과 정밀한 얼굴 특징점 검출을 위하여 능동외관모델을 사용한 점이다. 또한 사용자의 초기 캘리브레이션시 얻어진 특징값을 기반으로, 입력 특징값들을 정규화 함으로써, Z거리에 따라 시선 위치 정확도가 영향을 받지 않는다는 점이다. 실험결과, 약 1.77도의 시선 오차를 발생하였으나, 추가적인 얼굴 움직임에 의한 마우스 움직임 기능으로 이러한 시선 오차는 더욱 줄일 수 있음을 알 수 있었다.

Recently, many researches of making more comfortable input device based on gaze detection technology have been performed in human computer interface. Previous researches were performed on the computer environment with a large sized monitor. With recent increase of using mobile device, the necessities of interfacing by gaze detection on mobile environment were also increased. In this paper, we research about the gaze detection method by using UMPC (Ultra-Mobile PC) and an embedded camera of UMPC based on face and facial feature detection by AAM (Active Appearance Model). This paper has following three originalities. First, different from previous research, we propose a method for tracking user's gaze position in mobile device which has a small sized screen. Second, in order to detect facial feature points, we use AAM. Third, gaze detection accuracy is not degraded according to Z distance based on the normalization of input features by using the features which are obtained in an initial user calibration stage. Experimental results showed that gaze detection error was 1.77 degrees and it was reduced by mouse dragging based on the additional facial movement.

키워드

참고문헌

  1. Jacob, R. and J. K, "Eye Movement-Based Human-Computer Interaction Techniques : Toward Non-Command Interfaces," Advances in Human-Computer Interaction, H. R. Hartson and D. Hix, Editors., Ablex Publishing Co., pp. 151-190, 1993.
  2. L. E. Sibert, and R. J. K. Jacob, "Evaluation of Eye Gaze Interaction," Proceedings of the SIGCHI Conference, pp. 281-288, 2000.
  3. Ohno, T. "Quick Menu Selection Task with Eye Mark," Transactions of Information Processing Society of Japan, Vol.40, No.2, pp. 602-612, 1999.
  4. Yamato, M., Monden, A., Matsumoto, K., Inoue, K. and Torii, K, "Quick Button Selection with Eye Gazing for General GUI Environments," International Conference on Software: Theory and Practice, August 2000.
  5. Andrew T. Duchowski, Vinay Shiva shankariah, Tim Rawls, Anand K.Gramopadhye, Brian J. Melloy, and Barbara Kanki, "Binocular Eye Tracking in Virtual Reality for Inspection Training," Proceedings of the 2000 Symposium on Eye Tracking Research & Applications, pp. 89-96, 2000.
  6. Jeong Jun Lee, Kang Ryoung Park, and Jaihie Kim, "Gaze Detection System under HMD Environment for User Interface," Joint Conference of ICANN/ICONIP, June 2003.
  7. 이의철, 박강령, "HMD (Head Mounted Display)에서 시선 추적을 통한 3차원 게임 조작 방법 연구," 대한전자공학회 논문지, 제 45권 2호 sp편, pp. 49-64, 2008년 3월.
  8. Daniel Cheng and Roel Vertegaal, "An Eye for an Eye: A Performance Evaluation Comparison of the LC Technologies and Tobii Eye Trackers," Proceedings of ACM Eye Tracking Research and Applications Symposium, San Antonio, 2004.
  9. Shen-Wen Shih, and Jin Liu, "A Novel Approach to 3-D Gaze Tracking Using Stereo Cameras," IEEE Transactions on System, Man, and Cybernetics Part B, Vol.34, No.1, pp. 234-245, Feb. 2004. https://doi.org/10.1109/TSMCB.2003.811128
  10. Dong Hyun Yoo and Myung Jin Chung, "Non-Intrusive Eye Gaze Estimation without Knowledge of Eye Pose," Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004.
  11. 이의철, 박강령, “눈동자 시선 추적에 의한 3차원 1인칭 슈팅 게임,” 한국정보처리학회 논문지, 제12-B권 제4호, pp. 465-472, 2005년 8월.
  12. "Method and Apparatus for Tracking Gaze Position," 미국 특허 출원 중 (출원 번호 : 11/91813, 출원일 : 2007.12.06), 발명자 (이의철 외)
  13. Eui Chul Lee, and Kang Ryoung Park, "A Robust Eye Gaze Tracking Method based on Virtual Eyeball Model," Machine Vision and Applications, Vol.20, Issue 5, pp. 319-337, 2009. https://doi.org/10.1007/s00138-008-0129-z
  14. Eui Chul Lee, and Kang Ryoung Park, "A Study on Eye Gaze Estimation Method Based on Cornea Model of Human Eye," Lecture Notes in Computer Science (MIRAGE 2007), Vol.4418, pp. 307-317, INRIA Rocquencourt, France, Mar. 28-30, 2007.
  15. You Jin Ko, Eui Chul Lee, and Kang Ryoung Park, "A Robust Gaze Detection Method by Compensating for Facial Movements Based on Corneal Specularities,' Pattern Recognition Letters, Vol.29, Issue 10, pp. 1474-1485, 15 July 2008. https://doi.org/10.1016/j.patrec.2008.02.026
  16. You Jin Ko, Eui Chul Lee, and Kang Ryoung Park, "A Study on Robust Eye Gaze Tracking System Allowing the Natural Facial Movements," ACCV'07 Workshop on Multidimensional and Multi-view Image Processing, Tokyo, 19 Nov. 2007.
  17. 고유진, 이의철, 박강령, "적외선 환경에서 얼굴 및 눈동자 움직임이 자유로운 시선 위치 추적에 대한 연구," 2006년 한국 차세대컴퓨팅학회 논문 발표회, 일산 KINTEX, 2006.11.16-17.
  18. 이의철, 박강령, "사용자 시선 추적을 통한 3차원 FPS 게임 화면 조정," 2006년 한국 차세대컴퓨팅학회 논문 발표회, 일산 KINTEX, 2006. 11. 16-17.
  19. Eui Chul Lee, Kang Ryoung Park, Min Cheol Whang, and Junseok Park, "Robust Gaze Tracking Method for Stereoscopic Virtual Reality System," Lecture Notes in Computer Science (HCI International 2007), Vol.4552, pp. 700-709, Beijing, China, July 22-27, 2007.
  20. Eui Chul Lee, Kang Ryoung Park, Min Cheol Whang, and Joa Sang Lim, "Vision-based Gaze Estimation Method For Stereoscopic Display Systems," Proceedings of 10th International Federation of Automatic Control (IFAC), Ritz-Carlton Hotel, Seoul, Korea, 4-6 Sep., 2007.
  21. Eui Chul Lee, You Jin Ko and Kang Ryoung Park, "Gaze Tracking Based on AAM and Multiple SVR on Mobile Device," Optical Engineering, Vol. 48, No. 7, pp. 077002-1-077002-11, 2009. https://doi.org/10.1117/1.3158992
  22. P. Viola, M. J. Jones, "Robust Real-Time Face Detection," International Journal of Computer Vision, Vol.57, No.2, pp. 137-154, 2004. https://doi.org/10.1023/B:VISI.0000013087.49260.fb
  23. You Jin Ko, "A Study on Gaze Detection Technology," A Master Thesis of Sangmyung University, 2009. 2.
  24. Cootes, T. F., Edwards, G. J., and Taylor, C. J, "Active Appearance Models," IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.23, No.6, pp. 681-685, 2001. https://doi.org/10.1109/34.927467
  25. DTU(Technical university of Denmark) IMM face database, http://www2.imm.dtu.dk/~aam/(accessed on August 26, 2009)
  26. K. Ando et al., "Effects of Phencyclidine, Secobarbital and Diazepam on Eye Tracking in Rhesus Monkeys," Psychopharmacology, Vol.81, No.4, pp. 295-300, 1983. https://doi.org/10.1007/BF00427566
  27. Dal-ho Cho, Kang Ryoung Park, Dae Woong Rhee, Yanggon Kim, and Jonghoon Yang, "Pupil and Iris Localization for Iris Recognition in Mobile Phones," Proceedings of SNPD, Las Vegas Nevada, USA, June 19-20, 2006.
  28. 장영균, 강병준, 박강령, "홍채 인식을 위한 포물 허프 변환 기반 눈꺼풀 영역 검출 알고리즘," 대한전자공학회 논문지, 제 44권 SP편, 제01호, pp. 94-104, 2007년 1월
  29. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2/E, Prentice Hall, 2002
  30. Gregory A. Baxes, Digital Image Processing-Principles and Application, Wiley.
  31. http://www.nbinside.com/sony/UX17LP/UX17LP.htm (accessed on August 26, 2009).
  32. 이정준, 박강령, 김재희, "얼굴의 회전 및 이동분석에 의한 응시 위치파악," 대한전자공학회, 제 39권(SP편), 제5호, pp. 59-67, 2002년 9월.
  33. Kenichi Ohue, Yukinori Yamada, Shigeyasu Uozumi, Setsuo Tokoro, Akira Hattori, and Takeshi Hayashi, "Development of a New Pre-crash Safety System," Proceedings of the Society of Automotive Engineers, Detroit, MI, USA, April 3-6, 2006.
  34. Erik Murphy-Chutorian, Anup Doshi, and Mohan Manubhai Trivedi, "Head Pose Estimation for Driver Assistance Systems: A Robust Algorithm and Experimental Evaluation," Proceedings of the IEEE Intelligent Transportation Systems Conference, Seattle, WA, USA, Sept. 30 - Oct. 3, 2007.