Broadcasting and Media Magazine (방송과미디어)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지

인공지능 기반 3차원 공간 복원 최신 기술 동향

  • Published : 2020.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

최근 스마트폰에서의 증강현실, 미적 효과의 증대(예, 라이브 포커싱) 등의 어플리케이션을 제공하기 위해 모바일 기기에서의 3차원 공간 복원 기술에 대한 관심이 증가하고 있다. 소비자들의 요구에 발 맞춰 최근 스마트폰 제조사는 모든 플래그십 모델에 다중 카메라 및 뎁스 센서(거리 측정 센서)를 탑재하는 추세이다. 본 고에서는 모바일 폰에 탑재되고 있는 대표적인 세 축의 뎁스 추정(공간 복원) 방식에 대해 간단히 살펴보고, 최근 심층학습(Deep learning)의 등장으로 기술 발전의 새로운 국면에 접어 든 다중 시점 매칭(Multi-view stereo) 방법에 대해 소개하고자 한다. 심층 신경망이 재조명 받은 2012년 전까지 주류 연구 방향이었던 전통 기하학 기반의 방법에 대한 소개를 시작으로 심층 신경망기반의 방법론으로의 발전된 형태를 살펴본다. 또한, 신경망기반의 방법론은 크게 3 세대로 나누어 각 세대별 특징에 대해 자세히 살펴보고, 다양한 데이터에 대한 실험 결과를 통해 세대별 공간 복원 결과를 비교 분석한다.

Keywords

References

  1. Google photos: One year, 200 million users, and a whole lot of self-ies. https://blog.google/products/photos/googlephotos-one-year-200-million/, accessed: 2016-05-27
  2. Augmented faces. https://developers.google.com/ar/develop/java/augmented-faces, accessed: 2019-12-183.
  3. Scharstein, Daniel, and Richard Szeliski. "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms." International journal of computer vision 47.1-3 (2002): 7-42. https://doi.org/10.1023/A:1014573219977
  4. Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light.In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition(CVPR). vol. 1 (2003)
  5. Foix, Sergi, Guillem Alenya, and Carme Torras. "Lock-in time-of-flight (ToF) cameras: A survey."?IEEE Sensors Journal 11.9 (2011): 1917-1926. https://doi.org/10.1109/JSEN.2010.2101060
  6. Hartley, Richard, and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge university press, 2003.
  7. Geiger, Andreas, Philip Lenz, and Raquel Urtasun. "Are we ready for autonomous driving? the kitti vision benchmark suite." 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2012.
  8. Zbontar, Jure, and Yann LeCun. "Stereo matching by training a convolutional neural network to compare image patches." The journal of machine learning research 17.1 (2016): 2287-2318.
  9. Mayer, Nikolaus, et al. "A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
  10. Kendall, Alex, et al. "End-to-end learning of geometry and context for deep stereo regression." Proceedings of the IEEE International Conference on Computer Vision (CVPR). 2017.
  11. Chang, Jia-Ren, and Yong-Sheng Chen. "Pyramid stereo matching network." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2018.
  12. Im, Sunghoon, et al. "Dpsnet: End-to-end deep plane sweep stereo." International Conference on Learning Representations (ICLR) 2019.
  13. Schonberger, Johannes L., et al. "Pixelwise view selection for unstructured multi-view stereo." European Conference on Computer Vision. Springer, Cham, 2016.
  14. Ummenhofer, Benjamin, et al. "Demon: Depth and motion network for learning monocular stereo." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017.
  15. Huang, Po-Han, et al. "Deepmvs: Learning multi-view stereopsis." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2018.
  16. Yao, Yao, et al. "Mvsnet: Depth inference for unstructured multi-view stereo." Proceedings of the European Conference on Computer Vision (ECCV). 2018.
  17. Schops, Thomas, et al. "A multi-view stereo benchmark with high-resolution images and multi-camera videos." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.
  18. Cheng, Xinjing, Peng Wang, and Ruigang Yang. "Learning Depth with Convolutional Spatial Propagation Network." IEEE transactions on pattern analysis and machine intelligence (2019).