방송공학회논문지 (Journal of Broadcast Engineering)

  • 제25권5호
  • /
  • Pages.685-697
  • /
  • 2020
  • /
  • 1226-7953(pISSN)
  • /
  • 2287-9137(eISSN)
한국방송∙미디어공학회 (The Korean Institute of Broadcast and Media Engineers)
권호 표지
DOI QR코드

DOI QR Code

LDPM 영상 평가를 활용한 동영상 스티칭의 시차 왜곡 검출 및 정정 방법

Parallax Distortion Detection and Correction Method for Video Stitching by using LDPM Image Assessment

  • Rhee, Seongbae (Department of Electronic Engineering, Kyung Hee University) ;
  • Kang, Jeonho (Department of Electronic Engineering, Kyung Hee University) ;
  • Kim, Kyuheon (Department of Electronic Engineering, Kyung Hee University)
  • 투고 : 2020.07.13
  • 심사 : 2020.09.03
  • 발행 : 2020.09.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

파노라마(Panorama) 및 360도 영상과 같은 몰입형(Immersive) 미디어 영상은 영상 내 공간을 사용자가 직접 방문한 것 같은 현장감을 제공해야하므로 실제 세계의 모습을 사실 그대로 나타낼 수 있어야 한다. 그러나 파노라마 및 360도 영상에서는 촬영 카메라들간의 시차(Parallax)로 인해 사물이 사라지거나 중복해서 나타나는 현상이 나타나며, 이와 같은 시차 왜곡은 사용자의 콘텐츠 몰입을 방해할 수 있다. 이에 따라서, 시차 왜곡을 극복하기 위한 많은 동영상 스티칭 알고리즘이 제안되고 있지만, Object detection 모듈의 낮은 성능과 Seam 생성 방식 등의 제한으로 여전히 시차 왜곡이 발생하고 있다. 이에 본 논문에서는 기존 동영상 스티칭 기술의 제한 사항을 분석하고, 해당 동영상 스티칭 기술의 제한을 극복하기 위하여 LDPM(Local Differential Pixel Mean) 영상 평가를 활용한 동영상 스티칭의 시차 왜곡 검출 및 정정 방법을 제안한다.

Immersive media videos, such as panorama and 360-degree videos, must provide a sense of realism as if the user visited the space in the video, so they should be able to represent the reality of the real world. However, in panorama and 360-degree videos, objects appear to overlap or disappear due to parallax between cameras, and such parallax distortion may interfere with immersion of the user's content. Accordingly, although many video stitching algorithms have been proposed to overcome parallax distortion, parallax distortion still occurs due to the low performance of the Object detection module and limitations of the Seam generation method. Therefore, this paper analyzes the limitations of the existing video stitching technology and proposes a method for detecting and correcting parallax distortion of video stitching using the LDPM (Local Differential Pixel Mean) image evaluation method that overcomes the limitations of the video stitching technique.

키워드

참고문헌

  1. Jeonho Kang, Junsik Kim, SangIL Kim, and Kyuheon Kim, "Method of Video Stitching based on Minimal Error Seam", The Korean Institute of Broadcast and Media Engineers, Vol.24, No.1, pp.142-152, January, 2019.
  2. R. Szeliski, "Image Alignment and Stitching: A Tutorial." Foundations and Trends in Computer Graphics and Computer Vision, Vol. 2, No.1, 2006.
  3. Kang, Jeonho, et al. "Minimum Error Seam-Based Efficient Panorama Video Stitching Method Robust to Parallax." IEEE Access 7 (2019): 167127-167140. https://doi.org/10.1109/ACCESS.2019.2953705
  4. Wei, L. Y. U., et al. "A survey on image and video stitching." Virtual Reality & Intelligent Hardware Vol. 1, No.1 pp.55-83, 2019. https://doi.org/10.3724/SP.J.2096-5796.2018.0008
  5. Levin, Anat, et al. "Seamless image stitching in the gradient domain." European Conference on Computer Vision. Springer, Berlin, Heidelberg, 2004.
  6. Bujnak, Martin, and Radim Sara. "A robust graph-based method for the general correspondence problem demonstrated on image stitching." 2007 IEEE 11th International Conference on Computer Vision. IEEE, 2007.
  7. Zhi, Qi, and Jeremy R. Cooperstock. "Toward dynamic image mosaic generation with robustness to parallax." IEEE Transactions on Image Processing, pp.366-378, Vol.21, No.1, 2011. https://doi.org/10.1109/TIP.2011.2162743
  8. Qureshi, H. S., et al. "Quantitative quality assessment of stitched panoramic images." IET image processing, Vol.6, no.9, pp.1348-1358, 2012. https://doi.org/10.1049/iet-ipr.2011.0641
  9. Zaragoza, Julio, et al. "As-projective-as-possible image stitching with moving DLT." Proceedings of the IEEE conference on computer vision and pattern recognition. 2013.
  10. Zhang, Guofeng, et al. "Multi-viewpoint panorama construction with wide-baseline images." IEEE Transactions on Image Processing, Vol.25, No.7, pp.3099-3111, 2016. https://doi.org/10.1109/TIP.2016.2535225
  11. Lin, Kaimo, et al. "Seagull: Seam-guided local alignment for parallax-tolerant image stitching." European conference on computer vision. Springer, Cham, 2016.
  12. Gao, Junhong, et al. "Seam-Driven Image Stitching." Eurographics (Short Papers). 2013.
  13. Abdukholikov, Murodjon, and Taegkeun Whangbo. "Fast image stitching method for handling dynamic object problems in Panoramic Images." KSII Transactions on Internet & Information Systems, Vol.11, No.11, 2017.
  14. Kwatra, Vivek, et al. "Graphcut textures: image and video synthesis using graph cuts." ACM Transactions on Graphics (ToG), Vol.22, No.3, pp.277-286. 2003. https://doi.org/10.1145/882262.882264
  15. Rhee, Seongbae, Jeonho Kang, and Kyuheon Kim. "Local Differential Pixel Assessment Method for Image Stitching." Journal of Broadcast Engineering, Vol.24, No.5, pp.775-784, 2019 https://doi.org/10.5909/JBE.2019.24.5.775
  16. Wang, Zhou, et al. "Image quality assessment: from error visibility to structural similarity." IEEE transactions on image processing, Vol.13, No.4, pp.600-612, 2004. https://doi.org/10.1109/TIP.2003.819861