The Journal of the Korea institute of electronic communication sciences (한국전자통신학회논문지)

Korea Institute of Electronic Communication Science (한국전자통신학회)
권호 표지
DOI QR코드

DOI QR Code

Expiration Date Notification System Based on YOLO and OCR algorithms for Visually Impaired Person

YOLO와 OCR 알고리즘에 기반한 시각 장애우를 위한 유통기한 알림 시스템

  • 김민수 (동서대학교 컴퓨터공학부) ;
  • 문미경 (동서대학교 소프트웨어학과) ;
  • 한창희 (동서대학교 소프트웨어학과)
  • Received : 2021.10.01
  • Accepted : 2021.12.17
  • Published : 2021.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

There are rarely effective methods to help visually impaired people when they want to know the expiration date of products excepted to only Braille. In this study, we developed an expiration date notification system based on YOLO and OCR for visually impaired people. The handicapped people can automatically know the expiration date of a specific product by using our system without the help of a caregiver, fast and accurately. The proposed system is worked by four different steps: (1) identification of a target product by scanning its barcode; (2) segmentation of an image area with the expiration date using YOLO; (3) classification of the expiration date by OCR: (4) notification of the expiration date by TTS. Our system showed an average classification accuracy of about 86.00% when blindfolded subjects used the proposed system in real-time. This result validates that the proposed system can be potentially used for visually impaired people.

점자를 제외한 시각 장애우들이 유통기한을 확인할 수 있는 효과적인 방법이 거의 개발되어 있지 않으며, 이로 인하여 시각 장애우들의 식품 안전성이 위협받고 있다. 본 연구에서는 시각 장애우의 식품 안전성 확보를 위해 실시간 객체 인식 알고리즘(you only look once, YOLO) 및 광학 문자 인식 (optical character recognition, OCR)에 기반한 유통기한 알림 시스템을 개발했다. 제안하는 시스템은 총 4가지 단계로 시각 장애우에게 유통기한 정보를 전달한다: (1) 표적 제품의 바코드 스캔을 통한 제품 확인 (2) 실시간으로 입력되는 제품 영상에서 YOLO 알고리즘을 활용하여 유통기한이 표기된 이미지 영역 검출; (3) 검출된 이미지 영역에서 OCR 알고리즘을 활용하여 유통기한 문자 인식; (4) Text to Speech (TTS) 기술을 활용하여 유통기한 정보를 사용자에게 전달. 성능 평가를 위한 온라인 실험 결과, 앞이 보이지 않는 피험자가 개발한 시스템을 사용해서 제품의 유통기한을 평균 86%의 높은 정확도로 확인할 수 있음이 검증되었다. 이러한 결과는 제안하는 시스템이 저시력자를 포함한 시각 장애우들의 식품 안전성 확보에 이바지할 수 있음을 보여준다.

Keywords

Acknowledgement

이 논문은 2021년도 동서대학교 "Dongseo Cluster Project"지원에 의하여 이루어진 것임(DSU-20210007)

References

  1. Alzanet Special Education Archives, "World Disability Report 2011," Report, 2011.
  2. Korea Consumer Agency, "The necessity of establishing the guideline of medicine braille," Report, 2019
  3. Ministry of Health and Welfare, "Survey on the Status of Persons with Disabilities," Technical survey, 2017.
  4. A. Koul, A. Li, E. Haroun, I. W. L. Chen, S. Sharma, C. Bianchet, S. Shaikh, S. Morichere - Matte, B. T. Lai, N. P. K. Lam, and W. Lu, "Augmented imaging assistance for visual impairment," United States Patent Application Publication, no. 15/242940, Aug. 2016.
  5. J. Shim, H. Lee, J. Shin, G. Kim, and Y. Han, "Walking Assistance Device for Prevention of Accidents of Visually Impaired People," J. of the Korea Institute of Electronic Communication Sciences, vol. 14, no. 06, Dec. 2019, pp. 1241-1248. https://doi.org/10.13067/JKIECS.2019.14.6.1241
  6. T. A. Qureshi, M. Rajbhar, Y. Pisat, and V. Bhosale, "AI Based App for Blind People," International Research Journal of Engineering and Technology Journal, vol. 08, no. 03, Mar. 2021, pp. 2883-2887.
  7. S. Kim, Y. Bang, and S. Cho, "Expiration Date Management Program," The Korean Institute of Information Scientists and Engineers, vol. 2017. no. 6, June. 2017, pp. 1712-1714.
  8. S. Choi, Y. Cho, and Y. Lee, "Automated management of food expiration date using text recognition," The Korean Institute of Information Scientists and Engineers, vol. 2019. no. 6, June. 2019, pp. 1660-1662.
  9. S. Oh and Y. Bae, "Braille Block Recognition Algorithm for the Visually Impaired Based on YOLO V3," Journal of Korean Institute of Intelligent Systems, vol. 31, no. 1, Feb. 2021, pp. 60-67. https://doi.org/10.5391/JKIIS.2021.31.1.060
  10. J. Yoo, D. Han, C. Hur, and U. Oh, "YOLO-based Walking Assistance Application for Blind People," The Korean Institute of Information Scientists and Engineers, vol. 2019. no. 6, June. 2019, pp. 1776-1778.
  11. J. Lee and H. Song, "An On-Line Barcode Verification System using Image Processing Technique," J. of the Korea Institute of Electronic Communication Sciences, vol. 07, no. 05, Oct. 2012, pp. 1053-1059. https://doi.org/10.13067/JKIECS.2012.7.5.1053
  12. S. Park, N. Kim, M. Jeong, D. Hwang, U. Enkhjargal, B. Kim, M. Park, H. Yoon, and W. Seo, "Study on Detection Technique for Coastal Debris by using Unmanned Aerial Vehicle Remote Sensing and Object Detection Algorithm based on Deep Learning," J. of the Korea Institute of Electronic Communication Sciences, vol. 15, no. 06, Dec. 2020, pp. 1209-1216. https://doi.org/10.13067/JKIECS.2020.15.6.1209