Journal of IKEEE (전기전자학회논문지)

Institute of Korean Electrical and Electronics Engineers (한국전기전자학회)
권호 표지
DOI QR코드

DOI QR Code

A Reference Frame Selection Method Using RGB Vector and Object Feature Information of Immersive 360° Media

실감형 360도 미디어의 RGB 벡터 및 객체 특징정보를 이용한 대표 프레임 선정 방법

  • Park, Byeongchan (Dept. of Computer Science and Engineering, Soongsil University) ;
  • Yoo, Injae (Research Institute, Beyondtech Inc.) ;
  • Lee, Jaechung (Research Institute, Beyondtech Inc.) ;
  • Jang, Seyoung (Dept. of Computer Science and Engineering, Soongsil University) ;
  • Kim, Seok-Yoon (Dept. of Computer Science and Engineering, Soongsil University) ;
  • Kim, Youngmo (Dept. of Computer Science and Engineering, Soongsil University)
  • Received : 2020.11.25
  • Accepted : 2020.12.23
  • Published : 2020.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Immersive 360-degree media has a problem of slowing down the video recognition speed when the video is processed by the conventional method using a variety of rendering methods, and the video size becomes larger with higher quality and extra-large volume than the existing video. In addition, in most cases, only one scene is captured by fixing the camera in a specific place due to the characteristics of the immersive 360-degree media, it is not necessary to extract feature information from all scenes. In this paper, we propose a reference frame selection method for immersive 360-degree media and describe its application process to copyright protection technology. In the proposed method, three pre-processing processes such as frame extraction of immersive 360 media, frame downsizing, and spherical form rendering are performed. In the rendering process, the video is divided into 16 frames and captured. In the central part where there is much object information, an object is extracted using an RGB vector per pixel and deep learning, and a reference frame is selected using object feature information.

실감형 360도 미디어는 기존 영상보다 고품질, 초대용량으로 영상의 크기가 크며, 다양한 렌더링 방식을 사용하여 기존방식으로 이미지 처리할 경우 영상인식 속도가 느려지는 문제가 있다. 또한, 실감형 360도 미디어의 특성상 특정 장소에서 카메라를 고정시켜 한 장면만 촬영하는 경우가 대부분이기 때문에, 모든 영상에서 특징정보를 추출할 필요가 없다. 본 논문에서는 실감형 360 미디어의 프레임 추출과정, 프레임 다운사이징, 구형 형태의 렌더링 과정을 거치고, 렌더링 과정에서 영상을 16개 프레임으로 분할 캡처하여 캡처된 프레임에서 객체 정보가 많은 중앙 부분에서 픽셀당 RGB 벡터와 딥 러닝을 이용하여 객체를 추출한 뒤, 객체 특징정보를 이용하여 대표 프레임을 선정하는 방법을 제안한다.

Keywords

References

  1. J. S. Lee, "Changes in realistic media content distribution environment and production technology in the 5G era," NIPA Issue Report, Vol.22, 2019.
  2. S. W. Son, "5G era, single media copyright issues and trends," VideoPlus, 2019.
  3. M. G. Kim, "Korean 5G-based VR.AR development urgentThere is no "law/policy," EYENEWS 24, 2019.
  4. Y. Kim and D. Sin, "Feature-based filtering technology performance evaluation trend," Korean Institute of Information Technology, Vol.11, No.2, pp.1-7, 2013.
  5. Y. S. Ho, "VR/AR content creation using 360-degree video," The Magazine of the IEEE, Vol.45, No.8, pp. 33-40, 2018.
  6. S. Mun, M. Whang, S. Park, D. W. Lee and H. I. Kim, "Overview of VR Media Technology and Methods to Reduce Cybersickness," Journal of Broadcast Engineering, Vol.23. No.6, pp.800-812, 2018. https://doi.org/10.5909/JBE.2018.23.6.800
  7. G. S. Lee, J. Y. Jeong, H. C. Shin and J. I. Seo, "Standardization Trend of 3DoF+ Video for Immersive Media," Electronics and Telecommunications Trends, Vol.34, No.6, pp.156-163, 2019. DOI: 10.22648/ETRI.2019.J.340614
  8. Y. Kim, B. Park, S. Jang, I. Yoo, J. Lee and S. Y. Kim, "A Study on Fingerprinting Robustness Indicators for Immersive 360-degree Video," Journal of IKEEE, Vol.24, No.3, pp.743-753, 2020. DOI: 10.7471/ikeee.2020.24.3.743
  9. S. Ren, K. He, R. Girshick and J. Sun "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.39, No.6, pp.1137-1149, 2017. https://doi.org/10.1109/TPAMI.2016.2577031
  10. D. Belur V, "Nearest neighbor (NN) norms: NN pattern classification technique," EEE Computer Society Press, 1991.
  11. J. Lee, "An Efficient kNN Algorithm," KIPS Transactions on Software and Data Engineering (KTSDE), Vol.11, No.7, pp.849-854, 2004.