방송과미디어 (Broadcasting and Media Magazine)

한국방송∙미디어공학회 (The Korean Institute of Broadcast and Media Engineers)
권호 표지

메타버스 서비스를 위한 휴먼 모델링 기술 동향

  • 발행 : 2021.10.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

영상, 비디오 정보로부터 3D 휴먼 모델을 생성하고 실제 사람 같이 움직이게 변형하는 기술은 다가오는 메타버스 시대의 핵심 기술이며, 이와 같은 기술이 갖는 잠재적인 파급력 때문에 많은 글로벌 IT 기업에서 앞다투어 관련 기술에 대한 투자를 진행하고 있다. 본 기고문에서는 지난 10년간 휴먼 모델링 기술이 어떻게 발전되어 왔는지 살펴보고 현재 기술의 특징 및 한계점과 함께 앞으로의 기술 전망에 대해 살펴본다.

키워드

과제정보

이 논문은 2021년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임 (No.2020-0-00103, 가상공간구성을 위한 5G 기반 3D 공간 스캔 디바이스 기술 개발)

참고문헌

  1. LG유플러스, 세계 최초 4K 화질 3D AR 콘텐츠 제작, http://www.uplus.co.kr, 2019.
  2. SKT 점프 스튜디오, 혼합현실 기술로 한층 더 진화하는 K팝 열풍, https://www.jumpstudio.co.kr/news-detail?newsSeq=11, 2021.
  3. Wave XR, https://wavexr.com/
  4. NETFLIX, Army of the dead, https://www.netflix.com/kr/title/81046394
  5. Pinscreen, https://www.pinscreen.com/
  6. RealityCapture, https://www.capturingreality.com
  7. 조슈아 산토소, 전성호, 장주용, 박인규, 영상과 비디오로부터 3차원 휴먼 자세 및 형상 복원 기술, 한국방송.미디어공학회, Volume 26 Issue 3, 2021.
  8. A. Collet, M. Chuang, P. Sweeney, D. Gillett, D. Evseev, D. Calabrese, H. Hoppe, A. Kirk, and S. Sullivan, High-Quality Streamable Free-Viewpoint Video, in SIGGRAPH, 2015.
  9. G. Varol, D. Ceylan, B. Russell, J. Yang, E. Yumer, I. Laptev, and C. Schmid, BodyNet: Volumetric Inference of 3D Body Shapes, in ECCV, 2018.
  10. S. Saito, Z. Huang, R. Natsume, S. Morishima, A. Kanazawa, and Hao Li. PIFu: Pixel-aligned implicit function for high-resolution clothed human digitization. In Proceedings of the International Conference on Computer Vision (ICCV), 2019.
  11. S. Saito, T. Simon, J. Saragih, and H. Joo. PIFuHD: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  12. T. He, J. Collomosse, H. Jin, and S. Soatto, Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction, in NIPS, 2020.
  13. Y. Hong, J. Zhang, B. Jiang, Y. Guo, L. Liu, and H. Bao, StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
  14. Y. Lizhen, Z. Xiaochen, Y. Tao, W. Songtao, and L. Yebin, Z, NormalGAN: Learning Detailed 3D Human from a Single RGB-D Image, in ECCV, 2020.
  15. V. Gabeur, J.-S. Franco, X. Martin, C. Schmid, and G. Rogez, Moulding Humans: Non-parametric 3D Human Shape Estimation from a Single Image", in ICCV, 2019.
  16. Y. Jafarian and H. S. Park, Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos, in CVPR, 2021.
  17. M. Dou, S. Khamis, Y. Degtyarev, P. Davidson, and S. R. Fanello, Fusion4D: Real-time Performance Capture of Challenging Scenes, in ACM Trans. on Graphics (ToG), 2016.
  18. R. A. Newcombe, D. Fox, and S. M. Seitz, DynamicFusion: Reconstruction and Tracking of Non-rigid Scenes in Real-Time, in CVPR, 2015.
  19. S. Lee, M. Park, K. Lee, and J. Lee, Scalable Muscle-Actuated Human Simulation and Control, in ACM Trans. on Graphics (ToG), 2019.
  20. H. Joo, T. Simon, and Y. Sheikh, Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies, in CVPR, 2018.
  21. R. M.-Brualla, P. Lincoln, A. Kowdle, C. Rhemann, D. B Goldman, C. Keskin, S. Seitz, S. Izadi, S. Fanello, R. Pandey, S. Yang, P. Pidlypenskyi, J. Taylor, J. Valentin, S. Khamis, P. Davidson, and A. Tkach, LookinGood: Enhancing Performance Capture with Real-time Neural Re-Rendering, in ACM Trans. on Graphics (ToG), 2018.
  22. R. Kellnhofer, L. C. Jebe, A. Jones, R. Spicer, K. Pulli, and G. Wetzstein, Neural Lumigraph Rendering, in CVPR, 2021.
  23. Z. Zheng, T. Yu, Y. Wei, Q. Dai, and Y. Liu, DeepHuman: 3D Human Reconstruction from a Single Image, in ICCV, 2019.
  24. Z. Zheng, T. Yu, H. Li, K. Guo, Q. Dai, L. Fang, and Y. Liu, HybridFusion: Real-Time Performance Capture using a Single Depth Sensor and Sparse IMUs, in ECCV, 2018.
  25. T. Yu, J. Zhao, Z. Zheng, K. Guo, Q. Dai, H. Li, G. Pons-Moll, and Y. Liu, DoubleFusion: Real-Time Capture of Human Performance with Inner Body Shapes from a Single Depth Sensor, in Trans. on PAMI, 2020.
  26. T. Yu, K. Guo, F. Xu, Y. Dong, Z. Su, J. Zhao, J. Li, Q. Dai, and Y. Liu, BodyFusion: Real-time Capture of Human Motion and Surface Geometry using a Single Depth Camera, in ICCV, 2017.
  27. 8i Studio, https://8i.com/
  28. Intel Studios, https://newsroom.intel.com/press-kits/intel-studios/
  29. Canon Volumetric Video Studio, https://global.canon/en/vvs/
  30. Intel RealSense, https://www.intelrealsense.com/
  31. Microsoft Kinect, https://azure.microsoft.com/ko-kr/services/kinect-dk/
  32. StereoLabs, https://www.stereolabs.com/zed/