Journal of Convergence for Information Technology (융합정보논문지)

Convergence Society for SMB (중소기업융합학회)
권호 표지
DOI QR코드

DOI QR Code

Design of Real-time MR Contents using Substitute Videos of Vehicles and Background based on Black Box Video

블랙박스 영상 기반 차량 및 배경 대체 영상을 이용한 실시간 MR 콘텐츠의 설계

  • Kim, Sung-Ho (Department of Computer Science and Engineering, Sangji University)
  • Received : 2021.04.13
  • Accepted : 2021.06.20
  • Published : 2021.06.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we detect and track vehicles by type based on highway daytime driving videos taken with black boxes for vehicles. In addition, we design a real-time MR contents production method that can be newly created by placing substitute videos of each type of detected vehicles in the same location as the new background video. To detect and track vehicles by type, we use the YOLO algorithm. And we also use the mask technique based on RGB color for substitute videos of each type of vehicles detected. The size of the vehicle substitute videos to be used for MR content are substituted by the same size as the area size of the detected vehicles. In this paper, we confirm that real-time MR contents design is possible as a result of experiments and simulations and believe that It will be usefully utilized in the field of VR contents.

본 논문에서는 차량용 블랙박스로 촬영된 고속도로 주간 주행 영상을 기반으로 차량을 종류별로 검출하고 추적한다. 그리고 검출된 차량의 종류별 대체 영상을 새로운 배경 영상의 같은 위치에 올려놓음으로써 새롭게 창조될 수 있는 실시간 MR 콘텐츠 제작 방안을 설계한다. 차량을 종류별로 검출하고 추적하기 위해서는 딥러닝의 객체 검출 분야에서 가장 잘 알려지고 유명한 YOLO 알고리즘을 사용한다. 또한, 검출된 차량의 종류별 대체 영상을 위해서는 RGB 색상을 기반으로 하는 Mask 기법을 사용한다. 실시간 MR 콘텐츠를 위해 사용될 차량 대체 영상의 크기는 원본 영상에서 검출된 차량의 영역 크기와 같은 크기로 대체된다. 본 논문에서는 실시간 MR 콘텐츠 설계가 가능함을 실험 및 시뮬레이션으로 확인하였으며 VR 콘텐츠 분야에서 유용하게 활용할 수 있을 것으로 판단한다.

Keywords

Acknowledgement

This research was supported by Sangji University Research Fund, 2019.

References

  1. https://docs.openvinotoolkit.org/latest/omz_models_intel_semantic_segmentation_adas_0001_description_semantic_segmentation_adas_0001.html
  2. O. Ronneberger, P. Fischer & T. Brox. (2015, October). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234-241). Springer, Cham.
  3. V. Badrinarayanan, A. Kendall & R. Cipolla. (2017). Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 39(12), 2481-2495. DOI : 10.1109/TPAMI.2016.2644615
  4. A. Kendall, V. Badrinarayanan & R. Cipolla. (2018). SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding. arXiv preprint arXiv:1511.02680
  5. H. Zhao, J. Shi, X. Qi, X. Wang & J. Jia. (2017). Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2881-2890).
  6. G. Lin, A. Milan, C. Shen & I. Reid. (2017). Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1925-1934).
  7. H. LI et al. (2018). Pyramid attention network for semantic segmentation. arXiv preprint arXiv:1805.10180.
  8. L. C. Chen, G. Papandreou, F. Schroff & H. Adam. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587.
  9. L. C. Chen et al. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV) (pp. 801-818).
  10. M. Yang, K. Yu, C. Zhang, Z. Li & K. Yang. (2018). Denseaspp for semantic segmentation in street scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3684-3692).
  11. C. Yu et al. (2018). Bisenet: Bilateral segmentation network for real-time semantic segmentation. In Proceedings of the European conference on computer vision (ECCV) (pp. 325-341).
  12. https://github.com/guptavasu1213/Yolo-Vehicle-Counter
  13. J. Redmon & A. Farhadi. (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.
  14. T. N. Doan & M. T. Truong. (2020, November). Real-time vehicle detection and counting based on YOLO and DeepSORT. In 2020 12th International Conference on Knowledge and Systems Engineering (KSE) (pp. 67-72). IEEE.. DOI : 10.1109/KSE50997.2020.9287483
  15. D. Bolya, C. Zhou, F. Xiao & Y. J. Lee. (2019). Yolact: Real-time instance segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 9157-9166).
  16. Y. H. Lee & Y. Kim. (2020). Comparison of CNN and YOLO for Object Detection. Journal of the semiconductor & display technology, 19(1), 85-92.