Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Video Content Editing System for Senior Video Creator based on Video Analysis Techniques

영상분석 기술을 활용한 시니어용 동영상 편집 시스템

  • 장달원 (한국전자기술연구원 정보미디어연구센터) ;
  • 이재원 (한국전자기술연구원 정보미디어연구센터) ;
  • 이종설 (한국전자기술연구원 정보미디어연구센터)
  • Received : 2022.05.16
  • Accepted : 2022.07.05
  • Published : 2022.07.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper introduces a video editing system for senior creator who is not familiar to video editing. Based on video analysis techniques, it provide various information and delete unwanted shot. The system detects shot boundaries based on RNN(Recurrent Neural Network), and it determines the deletion of video shots. The shots can be deleted using shot-level significance, which is computed by detecting focused area. It is possible to delete unfocused shots or motion-blurred shots using the significance. The system detects object and face, and extract the information of emotion, age, and gender from face image. Users can create video contents using the information. Decorating tools are also prepared, and in the tools, the preferred design, which is determined from user history, places in the front of the design element list. With the video editing system, senior creators can make their own video contents easily and quickly.

본 논문에서는 영상 편집이 익숙하지 않은 시니어 동영상 크리에이터를 위한 동영상 편집 시스템을 설명한다. 영상분석 기술을 이용하여 편집소스 동영상을 분석하여 각종 정보를 제공하고, 자동으로 일부 장면을 삭제한다. 사용자가 다수의 소스 콘텐츠를 입력하였을 때, RNN(Recurrent Neural Network) 기술을 기반으로 샷 단위로 분할하고, 이 중 동영상 편집에서 배제할 부분을 구분한다. 각 샷 별로 중요도를 계산하여 샷 단위로 자동 삭제가 가능하도록 한다. 중요도 계산을 위해서 동영상 초점 정보를 추출하여 활용하는데, 이는 초점이 맞지 않는 영상 또는 흔들린 영상을 배제할 수 있도록 한다. 이후 시스템은 객체 인식을 수행하고, 얼굴이 나온 영상에 대해서 감정, 나이, 성별 등의 정보를 추출하여 사용자에게 제공한다. 사용자는 이런 정보를 활용하여 동영상을 제작한다. 동영상에 자막을 삽입하는 등 동영상을 꾸미기 위한 기능들도 포함되어 있으며, 이런 기능들을 활용할 시, 사용자의 과거 정보를 이용해서 선호 디자인을 쉽게 찾을 수 있도록 앞서 배치하고 있다. 시니어 동영상 크리에이터들이 본 시스템을 통해서 쉽고 빠르게 동영상 콘텐츠를 제작할 수 있다.

Keywords

Acknowledgement

This research was supported by Culture, Sports and Tourism R&D Program through the Korea Creative Content Agency grant funded by the Ministry of Culture, Sports and Tourism in 2022 (Project Name: Development of intelligent authoring tools for senior creators, Project Number: R2020040068).

References

  1. J. Lee, "A Study on Types of Short-form Video Contents," Humanities Contents, Vol. 58, pp.121-139, 2020. doi: https://doi.org/10.18658/humancon.2020.09.121
  2. J.-H. Kwon, "A Study on the Planning of a Space for Senior Citizens Using Digital Contents," Journal of Digital Convergence, Vol. 18, No. 5, pp. 257-267, 2020. doi: https://doi.org/10.14400/JDC.2020.18.5.257
  3. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," Proceedings of the IEEE International conference on computer vision and pattern recognition(CVPR), Las Vegas, NV, USA, pp. 779-788, 2016. doi: https://doi.org/10.48550/arXiv.1506.02640
  4. P. Viola and M. J. Jones, "Robust real-time face detection," International Journal of Computer Vision. Vol. 57, pp. 137-154, 2004 doi: https://doi.org/10.1023/B:VISI.0000013087.49260.fb
  5. X. Yi and M. Eramian, "LBP-based segmentation of defocus blur," IEEE Trans. Image Process, Vol. 25, no. 4, pp. 1626-1638, Apr. 2016. doi: https://doi.org/10.1109/TIP.2016.2528042
  6. W. Zhao, F. Zhao, D. Wang, and H. Lu., "Defocus blur detection via multi-stream bottom-top-bottom fully convolutional network," Proceeding of IEEE International Conference on Computer Vision and Pattern Recognition(CVPR), Salt Lake City, UT, USA, pp 3080- 3088, 2018. doi: https://doi.org/10.1109/CVPR.2018.00325
  7. C. Tang, X. Zhu, X. Liu, L. Wang, and A. Zomaya, "DeFusionNET: Defocus blur detection via recurrently fusing and refining multi-scale deep features," Proceeding of IEEE International Conference on Computer Vision and Pattern Recognition(CVPR), Long Beach, CA, USA, pp. 2700-2709, 2019. doi: https://doi.org/10.1109/CVPR.2019.00281
  8. J. Shi, L. Xu, and J. Jia, "Discriminative blur detection features," Proceeding of IEEE International Conference on Computer Vision and Pattern Recognition(CVPR), Columbus, OH, USA, pp. 2965-2972, 2014. doi: https://doi.org/10.1109/CVPR.2014.379
  9. Y. Gao, Y. Lai and Y.Liu, "Fast Video Shot Boundary Detection Based on Visual Perception", Proceeding of IEEE International Conference on Consumer Electronics(ICCE), Las Vegas, NV, USA, pp.1-4, 2019. doi: https://doi.org/10.1109/ICCE.2019.8662083
  10. M. Gygli, "Ridiculously fast shot boundary detection with fully convolutional neural networks," Proceeding of International Conference on Content-Based Multimedia Indexing(CBMI), La Rochelle, France, Sep. 2018, pp. 1-4. doi: https://doi.org/10.1109/CBMI.2018.8516556
  11. M. Brindha and R. Amsaveni, "Shot change detection on news videos using color histogram and edge based approaches," Proceeding of IEEE International Conference on Advances in Computer Applications(ICACA), Coimbatore, India, pp.50-54, 2016. doi: https://doi.org/10.1109/ICACA.2016.7887922
  12. G. Kwon and J. Kwon, "A Study on the Usability of Video Authoring Tool Application for Active Senior," Journal of Next-generation Convergence Information Services Technology, 9, no.4 (2020) : 351-361. doi: http://doi.org/10.29056/jncist.2020.12.03
  13. S.-D. Park, "Education of media by production of image contents - Focusing on Non-Linear Editing," Journal of the Korea Institute of Information and Communication Engineering, Vol. 23, No. 9, 1096~1103. doi: https://doi.org/10.6109/jkiice.2019.23.9.1096