Journal of Korea Society of Industrial Information Systems (한국산업정보학회논문지)

Korea Society of Industrial Information Systems (한국산업정보학회)
권호 표지
DOI QR코드

DOI QR Code

An User-Friendly Kiosk System Based on Deep Learning

딥러닝 기반 사용자 친화형 키오스크 시스템

  • 강수연 (덕성여자대학교 소프트웨어전공) ;
  • 이유진 (덕성여자대학교 소프트웨어전공) ;
  • 정현아 (덕성여자대학교 소프트웨어전공) ;
  • 조승아 (덕성여자대학교 소프트웨어전공) ;
  • 이형규 (덕성여자대학교 소프트웨어전공)
  • Received : 2023.12.31
  • Accepted : 2024.02.02
  • Published : 2024.02.29
Download PDF
⟨ Previous Next ⟩

Abstract

This study aims to provide a customized dynamic kiosk screen that considers user characteristics to cope with changes caused by increased use of kiosks. In order to optimize the screen composition according to the characteristics of the digital vulnerable group such as the visually impaired, the elderly, children, and wheelchair users, etc., users are classified into nine categories based on real-time analysis of user characteristics (wheelchair use, visual impairment, age, etc.). The kiosk screen is dynamically adjusted according to the characteristics of the user to provide efficient services. This study shows that the system communication and operation were performed in the embedded environment, and the used object detection, gait recognition, and speech recognition technologies showed accuracy of 74%, 98.9%, and 96%, respectively. The proposed technology was verified for its effectiveness by implementing a prototype, and through this, this study showed the possibility of reducing the digital gap and providing user-friendly "barrier-free kiosk" services.

본 연구는 키오스크 사용 증가로 인한 변화에 대응하기 위해 사용자 특성을 고려한 맞춤형 동적 키오스크 화면을 제공하는 것을 목표로 한다. 디지털 취약계층인 시각장애인, 노인, 어린이, 휠체어 사용자 등의 특성에 따른 화면 구성의 최적화를 위해 객체 탐지, 걸음걸이 인식, 음성발화 인식기술을 종합하여 사용자의 특성(휠체어 사용 여부, 시각 장애, 연령 등)을 실시간으로 분석하고, 이를 기반으로 9개의 카테고리로 사용자를 분류한다. 키오스크 화면은 사용자의 특성에 따라 동적으로 조정되어 효율적인 서비스 제공이 가능하다. 본 연구는 임베디드 환경에서 시스템 통신 및 운용이 이루어졌으며, 사용된 객체 탐지, 걸음걸이 인식, 음성발화 인식 기술은 각각 74%, 98.9%, 96%의 정확도를 보여준다. 제안된 기술은 프로토타입을 구현하여 그 효용성을 검증하였으며, 이를 통해 본 연구가 디지털 격차의 축소와 사용자 친화적인 "배리어 프리 키오스크" 서비스 제공의 가능성을 보였다.

Keywords

Acknowledgement

본 연구는 2023년도 덕성여자대학교 교내연구비 지원에 의해 이루어졌음

References

  1. AI-Hub, "The Open AI Dataset Project", https://www.aihub.or.kr (Accessed Oct. 28th, 2021)
  2. Hong, S. Y., and Choe, J. -H., A Study on the Kiosk UI Reflecting the Elderly"s Characteristics, JOURNAL OF THE KOREA CONTENTS ASSOCIATION, vol. 19, no. 4, pp. 556-563, 2019.
  3. Kim, S. C., Lee, J. W, Cho, K. O., Park, J. G., and Oh, Y. T. (2008) Study on the algorithm for speech recognition. Proceedings of KIEE summer conference 2008.
  4. Kim, J., Cho, G., and Hur, H. (2014). Gait Recognition using Joint Angle Pattern Analysis. Proceedings of KIIE fall conference 2014.
  5. Kim, J., Ahn H. J,, Baek, H. C., Yook, S. Kim, I. Y., So, S., and Sung, M. Y. (2018). Development of Age Classification Deep Learning Algorithm Using Korean Speech. Proceedings of Biomedical Engineering Research 2018, 39(2), 63-68.
  6. Kim, G. H., Korea Broadcasting News", http://www.ikbn.news/news/article.html?no=108791 (Accessed on Oct. 23th, 2020a)
  7. Kim, M., Yoo, C., Park, H., Ou, S., and Lee, J. (2020b). Implementation of Voice Recognizing KIOSK Application for the Visually Impaired. KIISE Transactions on Computing Practices, 26(7), 332-337. https://doi.org/10.5626/KTCP.2020.26.7.332
  8. Kim, Y., and Nam, Y., (2021a). A Study on the Elderly-friendly UX Design for Accessibility Improvement of Fast Food Kiosks - Focusing on KFC -. Journal of Integrated Design Research, 20(4), 39-57.
  9. Kim, S. (2021b). Recent trends in user authentication using AI. Communications of the Korean Institute of Information Scientists and Engineers, 39(8), 55-61.
  10. Kim, T., Kim, M., and Park, S. (2022). MediaPipe-Based Action Recognition For Embedded System. Proceedings of Symposium of the Korean Institute of communications and Information Sciences.
  11. Lee, H. Hong, C., Cho, S. H., Yoonm C. Kim, D. and Choi, S. (2021a). Ordering system for the disabled and the weak using a KIOSK with speech recognition technology. Proceedings of KIICE fall conference 2021.
  12. Lee, S., Shirong, Fu, and Park, J.-I (2021b). Detection of Helmet on Electric Scooter, Proceedings of the Korean Society of Broadcast Engineers Conference 2021.
  13. Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C., Yong, M.G., Lee, J., Chang, W., Hua, W., Georg, M., & Grundmann, M. (2019). MediaPipe: A Framework for Building Perception Pipelines.
  14. Pyeon, J.-H., Lee, S.-H., and Kwak, N,-J,, (2021). Detection of Wearing a Mask using YOLO V5. Porceedings of Korea Contents Association 2021.
  15. Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 779-788).
  16. Shin, J., Lee, M., and Lee, W. (2021). Kiosk Design Guideline for the Week: Focusing on the Case of Fast Food Restaurants. Proceedings of HCI Korea 2021.
  17. Seo, Y. J., and Shin, J. (2019). Speaker age estimation and acoustic characteristics: According to pitch and speech rate. Phonetics and Speech Sciences, 11(4), 9-18. https://doi.org/10.13064/KSSS.2019.11.4.009
  18. Yoo, H. W., and Kwon, K. Y. (2017). Method for Classification of Age and Gender Using Gait Recognition. Transactions of the Korean Society of Mechanical Engineers - A, 41(11), 1035-1045 https://doi.org/10.3795/KSME-A.2017.41.11.1035