Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Efficient Mobile Writing System with Korean Input Interface Based on Face Recognition

  • Received : 2020.04.20
  • Accepted : 2020.06.01
  • Published : 2020.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

The virtual Korean keyboard system is a method of inputting characters by touching a fixed position. This system is very inconvenient for people who have difficulty moving their fingers. To alleviate this problem, this paper proposes an efficient framework that enables keyboard input and handwriting through video and user motion obtained through the RGB camera of the mobile device. To develop this system, we use face recognition to calculate control coordinates from the input video, and develop an interface that can input and combine Hangul using this coordinate value. The control position calculated based on face recognition acts as a pointer to select and transfer the letters on the keyboard, and finally combines the transmitted letters to integrate them to perform the Hangul keyboard function. The result of this paper is an efficient writing system that utilizes face recognition technology, and using this system is expected to improve the communication and special education environment for people with physical disabilities as well as the general public.

가상 키보드방식의 문자 입력 시스템은 고정된 버튼 위치를 터치하여 입력하는 방식으로 손가락의 움직임이 불편한 사람들이나 노인들은 입력하기가 매우 불편하다. 이러한 문제를 완화시키고자, 본 논문에서는 모바일 디바이스의 RGB 카메라를 통해 얻은 영상과 사용자의 움직임을 통해 자판 입력과 필기 기능이 가능한 효율적인 프레임워크를 제안한다. 이 시스템을 개발하기 위해 얼굴인식을 활용하여 입력 영상으로부터 컨트롤 좌표를 계산하고, 이 좌표 값을 이용하여 한글을 입력하고 조합할 수 있는 인터페이스 개발한다. 얼굴인식을 기반으로 계산된 컨트롤 위치는 자판의 글자를 선택하고 전달하는 포인터 역할을 하며, 마지막으로 전달된 글자들을 조합하여 한글 자판 기능을 수행할 수 있도록 통합한다. 본 연구의 결과는 얼굴인식 기술을 활용한 효율적인 필기 시스템이며, 이 시스템을 사용하면 일반인 뿐 만 아니라 신체가 불편한 지체장애인의 의사소통 및 특수교육 환경도 개선시킬 수 있을 거라 기대한다.

Keywords

References

  1. S. E. Park, S. Y. Moon, H. S. Yoon, D. C. Shin, and H. S. Jeong, "An Input Method for Hangul Character and Implementation for Personal Digital Assistant," Proc. of Fall Conference of KIISE, vol. 21, no. 2, pp. 657-660, 1994.
  2. W. H. Choi, S. C. Kim, and K. R. Song, "Apparatus and Method for Inputting the Korean Alphabet Based on the Character-Generative Principles of HUNMINJUNGEUM," Patent no. 1999-052648, 1999.
  3. J. W. Kim, "Apparatus for Inputting Key and Method for Inputting Character by Direction Key," Patent no. 2001-0006208, 2001.
  4. Share-typing, https://www.sharetyping.com/.
  5. Kang, Seung-Shik, Hahn, Kwangsoo, "Hangul Vowel Input System for Hand-held Devices ", KIPS Transactions on Software and Data Engineering, pp. 507-512, 2005. DOI : 10.3745/KIPSTB.2005.12B.4.507.
  6. T. J. Eom, J. B. Lee, and B. K. Kim, "Design of Smart Phone-Based Braille Keyboard System for Visually Impaired People," Journal of Information Security, vol. 12, no. 1, pp. 63-70, 2012.
  7. Y. C. Byun, "2011 Survey on Impaired People," Research Report 2011-82, 2012.
  8. H. Y. Kim, Gesture Recognition Interface by Keyboard Insertion IR Module, Master Thesis, KAIST, 2011.
  9. D. H. Kim, Y. H. Kwon, and J. H. Kim, "A Hangul Input Interface Using Continuous Gestures and Language Models," Proc. of KCC2005, pp. 586-588, 2005.
  10. Won Il Lee, "Natural Language Processing for Korean Speech Recognition," Proc. of KCC2005, pp. 586-588, 2005.
  11. Sang Hun Jeong, "The Study on Phonetical Information for Speech Recognition", The Contents and Language Phenomena of Seokbosangjeol, vol. 49, pp.135-160, 2007.
  12. Keyboard for Android : http://swypeinc.com/.
  13. Jared Cechanowicz, Steven Dawson, Matt Victor, Sriram Subramanian, "Stylus based text input using expanding CIRRIN," AVI '06, May. 2006. DOI : 10.1145/1133265.1133299.
  14. Gennaro Costagliola, Vittorio Fuccella, Michele DiCapua, Giovanni Guardi, "Performances of Multiple-Selection Enabled Menus in Soft Keyboards", DMS'09 - The 15th International Conference on Distributed Multimedia Systems., pp. 359-364, 2009.
  15. Amberkar, Aditya, Parikshit Awasarmol, Gaurav Deshmukh, and Piyush Dave. "Speech Recognition using Recurrent Neural Networks." In International Conference on Current Trends towards Converging Technologies, pp. 1-4, 2018. DOI : 10.1109/ICCTCT.2018.8551185.
  16. Lim, Chee Peng, Siew Chan Woo, Aun Sim Loh, and Rohaizan Osman. "Speech recognition using artificial neural networks." In Proceedings of the First International Conference on Web Information Systems Engineering, vol. 1, pp. 419-423, 2000. DOI : 10.1109/ICCSP.2018.8524333.
  17. Graves, Alex, Navdeep Jaitly, and Abdel-rahman Mohamed. "Hybrid speech recognition with deep bidirectional LSTM." In IEEE workshop on automatic speech recognition and understanding, pp. 273-278, 2013. DOI : 10.1109/ASRU.2013.6707742.
  18. Sundermeyer, Martin, Ralf Schlüter, and Hermann Ney. "LSTM neural networks for language modeling." In Thirteenth annual conference of the international speech communication association. 2012.
  19. Wang-Jun Kyung, Dae-Chul Kim, Yeong-Ho Ha, "Enhancement of Faded Images Using Integrated Compensation Coefficients Based on Multi-Scale Gray World Algorithm", vol. 39, pp. 459-466, The Journal of Korea Information and Communications Society, 2012. DOI : 10.7840/kics.2014.39A.8.459
  20. Schneider, Steffen and Baevski, Alexei and Collobert, Ronan and Auli, Michael, "wav2vec: Unsupervised pre-training for speech recognition", arXiv preprint arXiv:1904.05862, 2019.
  21. Schneider, Steffen and Baevski, Alexei and Collobert, Ronan and Auli, Michael, "wav2vec: Unsupervised pre-training for speech recognition", arXiv preprint arXiv:1904.05862, 2019.
  22. Tian, Xu and Zhang, Jun and Ma, Zejun and He, Yi and Wei, Juan and Wu, Peihao and Situ, Wenchang and Li, Shuai and Zhang, Yang, "Deep LSTM for large vocabulary continuous speech recognition, arXiv preprint arXiv:1703.07090, 2017.
  23. Sriram, Anuroop and Jun, Heewoo and Gaur, Yashesh and Satheesh, Sanjeev, "Robust speech recognition using generative adversarial networks, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5639-5643. 2017. DOI : 10.1109/ICASSP.2018.8462456