Journal of Korea Game Society (한국게임학회 논문지)

Korea Game Society (한국게임학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on subtitle synchronization calibration to enhance hearing-impaired persons' viewing convenience of e-sports contents or game streamer contents

청각장애인의 이스포츠 중계방송 및 게임 스트리머 콘텐츠 시청 편의성 증대를 위한 자막 동기화 보정 연구

  • Received : 2019.02.09
  • Accepted : 2019.02.15
  • Published : 2019.02.20
Download PDF
⟨ Previous Next ⟩

Abstract

This study is intended to suggest ways to improve the quality of the service of subtitles provided for the convenience of viewing for deaf people on e-sports broadcast content and game streamer content. Generally, subtitling files of broadcast content are manually written on air by stenographers, so a delay of 3 to 5 seconds is inevitable compared to the original content. Therefore, the present study proposed the formation of an automatic synchronization calibration system using speech recognition technology. In addition, a content application experiment using this system was conducted, and the final result confirmed that the time of synchronization error of subtitling data could be reduced to less than 1 second.

본 연구는 e-sports 중계 콘텐츠 및 게임 스트리머 콘텐츠에 대한 청각장애인들의 시청 편의성을 위해 제공되는 자막의 서비스의 품질을 높이는 방안을 제시하기 위한 연구이다. 일반적으로 방송 콘텐츠의 자막 파일은 속기사에 의해 방송 중에 수동 작성되므로 원본 콘텐츠 대비 3~5초의 자막표시 지연이 필연적이다. 이에, 본 연구에서는, 음성인식 기술을 활용한 동기화 자동 보정 시스템의 구성을 제안하였다. 또한 이 시스템을 활용한 콘텐츠 적용실험을 진행하였으며 최종 결과로 자막 데이터의 동기화 오차 시간을 1초 이내로 줄일 수 있음을 확인 하였다.

Keywords

KGOHCL_2019_v19n1_73_f0001.png 이미지

[Fig. 1] Voice recognition process[7]

KGOHCL_2019_v19n1_73_f0002.png 이미지

[Fig. 2] Production work-flow of closed- caption broadcasting

KGOHCL_2019_v19n1_73_f0003.png 이미지

[Fig. 3] Calibration System for closed caption Synchronization

KGOHCL_2019_v19n1_73_f0004.png 이미지

[Fig. 4] Character-based location search

KGOHCL_2019_v19n1_73_f0005.png 이미지

[Fig. 5] Word-based location search

KGOHCL_2019_v19n1_73_f0006.png 이미지

[Fig. 6] Synchronization calibration process

KGOHCL_2019_v19n1_73_f0007.png 이미지

[Fig. 7] Original stenograph subtitle

KGOHCL_2019_v19n1_73_f0008.png 이미지

[Fig. 8] Google API voice recognition result

KGOHCL_2019_v19n1_73_f0009.png 이미지

[Fig. 9] Experiment result : Subtitle synchronization calibration system

KGOHCL_2019_v19n1_73_f0010.png 이미지

[Fig. 10] Matched and unmatched area description

KGOHCL_2019_v19n1_73_f0011.png 이미지

[Fig. 11] Matching timeline prediction based on linear estimation method

[Table 1] Summary of Related Study

KGOHCL_2019_v19n1_73_t0001.png 이미지

[Table 2] Audio Recognition Rate

KGOHCL_2019_v19n1_73_t0002.png 이미지

[Table 3] Summary of Overalll experiments

KGOHCL_2019_v19n1_73_t0003.png 이미지

References

  1. Dong hwan Shin, Jeong Soo Kim, Chang won Kim, "A Study for Utilization of Smart Device and Audio Fingerprinting Technologies to Help the Vision and Hearing Impaired People Consuming Broadcasting Contents Conveniently", Journal of Information Technology and Architecture, Vol. 13, No.3, pp 457-466, 2016.
  2. Hye-Won Han, Seo-Yeon Kim, "Analysis of Storytelling in On-line Personal Game Broadcasting", Journal of Korea Game Society, Vol 14 No 2, pp 85-96, 2014 https://doi.org/10.7583/JKGS.2014.14.2.85
  3. Min-Ji Choe, Jeong-Min Park, Ghee-Young Noh, "A Study of Factors Influencing on Watching Personal Game Webcasting", Journal of Korea Game Society, Vol.16, No 6, pp 39-48, 2016. https://doi.org/10.7583/JKGS.2016.16.6.39
  4. Korea Communications Commission (KCC) Notification, No. 2011-53, 2011.
  5. Action on Hearing Loss, "Access to TV and video on demand for people with hearing loss", https://www.actiononhearingloss.org.uk/, 2015.
  6. ATVOD, "Video on demand access service best practice guidelines for service providers", https://www.atvod.co.uk/, 2012.
  7. Jin Tai Kim, Hun Jeong, "Speech Recognition Technology trend and Applying Model to Navy Info-Comm area", Defense Technology, pp 120-127, 2017.
  8. Young Jun Kim, "Evolution of Speech interface", Tech Planet 2016, SK Telecom, 2016
  9. Jung Youn Kim, Jeho Nam, "A Study on Multimedia Application Service using DTV Closed Caption Data", Journal of broadcast engineering, Vol 14, No.4, pp 488-500, 2009 https://doi.org/10.5909/JBE.2009.14.4.488
  10. Hyon Gun Park, Hee Suk Lee, Sang Moon Lee, "A Study on The Automatic Caption System for Hearing Impaired Person", Korea Society of Computer Information, Vol 18, No.2, pp 335-336, 2010.
  11. Minho Kim, Hyosoon Kang, "Implement closed captioning systems for the deaf", Journal of Korea Game Society, Vol.16, No.1, pp 103-110, 2016 https://doi.org/10.7583/JKGS.2016.16.1.103
  12. Chung Hyun Ahn, In Sun Jang, "Subtitle generation using Speech recognition", The Korean Institute of Broadcast and Media Engineers, pp 60-61, 2016
  13. Seong Joon Chu, Min Gyu Lee, Jin Gyeong Jeong, Chi wan Song, Han Jong Ko, Han Sung Kim, Won-Gil Hong, "A Study on the Similarity Measure between Speech to Text Result and Real Text Using Smith -Waterman Algorithm", The Korean Institute of Communications and Information Sciences, Summer Conference, 2017
  14. Seung Joo Choi, Jong-Bae Kim "Comparison Analysis of Speech Recognition Open APIs' Accurac", Asia-pacific Journal of Multimedia Services Convergent with Art, Humanities, and Sociology, Vol. 7, No. 8, 2017