DOI QR코드

DOI QR Code

Design of the Multimodal Input System using Image Processing and Speech Recognition

음성인식 및 영상처리 기반 멀티모달 입력장치의 설계

  • 최원석 (한국항공대학교 항공전자공학과) ;
  • 이동우 (한국항공대학교 항공전자공학과) ;
  • 김문식 (KT미래기술연구소) ;
  • 나종화 (한국항공대학교 항공전자공학과)
  • Published : 2007.08.01

Abstract

Recently, various types of camera mouse are developed using the image processing. The camera mouse showed limited performance compared to the traditional optical mouse in terms of the response time and the usability. These problems are caused by the mismatch between the size of the monitor and that of the active pixel area of the CMOS Image Sensor. To overcome these limitations, we designed a new input device that uses the face recognition as well as the speech recognition simultaneously. In the proposed system, the area of the monitor is partitioned into 'n' zones. The face recognition is performed using the web-camera, so that the mouse pointer follows the movement of the face of the user in a particular zone. The user can switch the zone by speaking the name of the zone. The multimodal mouse is analyzed using the Keystroke Level Model and the initial experiments was performed to evaluate the feasibility and the performance of the proposed system.

Keywords

References

  1. '컴퓨터 입력 장치의 장시간 사용으로 인한 직업병의 증가,' http://bric.postech.ac.kr/bbs/trend/0303/030306-10.html
  2. '눈동자 마우스'시대 곧 열린다,' htp://www.donga.com/docs/magazine/news_plus/news164jj030.html
  3. J. Na 'A novel camera based computer input device' ICMOCA 2006
  4. M. Betke, J. Gips, P. Fleming, 'The camera mouse: visual tracking of body features to provide computer access for people with severe disabilities,' Neural Systems and Rehabilitation Engineering, IEEE Transactions on, vol. 10, Issue 1 pp. 1-10, Mar. 2002 https://doi.org/10.1109/TNSRE.2002.1021581
  5. K. S. Pack, and K. T. Lee, 'Eye-controlled human /computer interface using the line of sight and the intentional blink,' Computers and ind. Engg. 30(3), 436-473, 1996
  6. D.O. Gorodnichy, 'On importance of nose for face tracking,' Fifth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 181-186, Washington, DC, USA, May 2002
  7. J. L. Tu, T. Huang, and H. Tao, 'Face as mouse through Vi-. sual Face Tracking,' CVIU special issue on V4HCI, 2006
  8. 'camera mouse,' http://www.cameramouse.com/
  9. S. K. Card, T. P. Moran, and A. Newell, 'The keystroke-level model for user performance time with interactive systems,' Communication of the ACM, 23(7), 396-410, 1980 https://doi.org/10.1145/358886.358895
  10. C. Fagiani, M. Betke, and J. Gips, 'Evaluation of tracking methods for human-computer interaction,' Applications of Computer Vision, 2002. (WACV 2002). Proceedings. Sixth IEEE Workshop on 3-4, pp. 121-126, Dec. 2002