Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Stereoscopic Free-viewpoint Tour-Into-Picture Generation from a Single Image

단안 영상의 입체 자유시점 Tour-Into-Picture

  • Kim, Je-Dong (Kangwon National University, Dept. of Computer and Communications) ;
  • Lee, Kwang-Hoon (Kangwon National University, Dept. of Computer and Communications) ;
  • Kim, Man-Bae (Kangwon National University, Dept. of Computer and Communications)
  • 김제동 (강원대학교 컴퓨터정보통신공학과) ;
  • 이광훈 (강원대학교 컴퓨터정보통신공학과) ;
  • 김만배 (강원대학교 컴퓨터정보통신공학과)
  • Received : 2009.12.28
  • Accepted : 2010.02.16
  • Published : 2010.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

The free viewpoint video delivers an active contents where users can see the images rendered from the viewpoints chosen by them. Its applications are found in broad areas, especially museum tour, entertainment and so forth. As a new free-viewpoint application, this paper presents a stereoscopic free-viewpoint TIP (Tour Into Picture) where users can navigate the inside of a single image controlling a virtual camera and utilizing depth data. Unlike conventional TIP methods providing 2D image or video, our proposed method can provide users with 3D stereoscopic and free-viewpoint contents. Navigating a picture with stereoscopic viewing can deliver more realistic and immersive perception. The method uses semi-automatic processing to make foreground mask, background image, and depth map. The second step is to navigate the single picture and to obtain rendered images by perspective projection. For the free-viewpoint viewing, a virtual camera whose operations include translation, rotation, look-around, and zooming is operated. In experiments, the proposed method was tested eth 'Danopungjun' that is one of famous paintings made in Chosun Dynasty. The free-viewpoint software is developed based on MFC Visual C++ and OpenGL libraries.

자유시점 비디오는 원하는 시점을 자유로이 선택하여 보는 능동형 비디오이다. 이 기술은 박물관 투어, 엔터테인먼트 등의 다양한 분야에서 활용된다. 본 논문에서는 자유시점 비디오의 새로운 분야로 가상 카메라와 깊이맵을 이용하여 한 장의 영상 내부를 항해하는 입체 자유시점 Tour-Into-Picture (TIP)을 제안한다. 오래전부터 TIP가 연구되어 왔는데, 이 분야는 한 장의 단안 사진 내부를 항해하면서 애니메이션으로 볼 수 있게 하는 기술이다. 제안 방법은 전경 마스크, 배경영상, 및 깊이맵을 반자동 방법으로 구한다. 다음에는 영상 내부를 항해하면서 입체 원근투영 영상들을 획득한다. 배경영상과 전경객체의 3D 데이터를 기반으로 가상 카메라의 3차원 공간이동, 요/피치/롤링 등의 회전, 룩어라운드, 줌 등의 다양한 카메라 기능을 활용하여 입체 자유시점 비디오를 구현한다. 원근투영은 직교투형보다 우수한 입체감을 전달하며, 기존 방법과 비교하여 텍스쳐의 3D 데이터를 직접 원근투영하여 처리속도를 향상시켰다. 소프트웨어는 MFC Visual C++ 및 OpenGL 기반으로 구축되었으며, 실험영상으로 신윤복의 단오풍정을 사용하여 고전화의 입체 자유시점 비디오를 시청이 가능하다.

Keywords

References

  1. A. Smolic, and P. Kauff, "Interactive 3D Video Representation and Coding Technologies", Proceedings of the IEEE, Special Issue on Advances in Video Coding and Delivery, vol. 93, no. 1, Jan. 2005.
  2. Masayuki Tanimoto, "Free Viewpoint Television - FTV", Proc. PCS 2004, Picture Coding Symposium, San Francisco, CA, USA, December 15.-17. 2004.
  3. A. Smolic, and D. McCutchen, "3DAV Exploration of Video-Based Rendering Technology in MPEG", IEEE Trans. on Circuits and Systems for Video Technology, Vol. 14, No. 3, pp. 348-356, March 2004. https://doi.org/10.1109/TCSVT.2004.823395
  4. Y. Horry, K. Anjyo, and K. Arai, "Tour Into the Picture: Using a Spidery Mesh Interface to Make Animation from a Single Image," Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, pp. 225-232, 1997.
  5. K. Boulanger, K. Bouatouch, S. Pattanaik, "ATIP: A Tool for 3D Navigation inside a Single Image with Automatic Camera Calibration," EG UK Theory and Practice of Computer Graphics 2006.
  6. N. S. Chu and C. Tai, "Animating Chinese landscape paintings and panorama using multi-perspective modeling," Proceedings of International Conference on Computer Graphics, IEEE, pp. 107-113, 2001.
  7. 조철용, 김만배, "3DTIP: 한국 고전화의 3차원 입체 Tour-Into- Picture," 방송공학회논문지, 제14권 제5호, pp. 616-624, 2009년 9월. https://doi.org/10.5909/JBE.2009.14.5.616
  8. A. Criminisi, P. Perez and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE Trans. Image Processing, Vol. 13, No. 9, pp. 1200-1212, 2004. https://doi.org/10.1109/TIP.2004.833105
  9. R. Seidel, "A simple and fast randomized algorithm for computing trapezoidal decompositions and for triangulating polygons," Computational Geometry Theory & Application, Vol. 1, pp. 51-64, 1991. https://doi.org/10.1016/0925-7721(91)90012-4
  10. H. Yamanoue, M. Okui and F. Okano, "Geometrical analysis of puppet-theater and cardboard effects in stereoscopic HDTV Images," IEEE Trans. On Circuits and Systems for Video Technology, Vol. 16, No. 6, June 2006.
  11. H. Yamanoue, M. Okui and I. Yuyama, "A study on the relationship between shooting conditions and cardboard effect of stereoscopic Images," IEEE Trans. On Circuits and Systems for Video Technology, Vol. 10, No. 3, Apr. 2000.
  12. D. F. McAllister (editor), Stereo computer graphics and other true 3D technologies, Princeton, NJ:Princeton University Press, 1993.
  13. E. R. Davies, Laws' texture energy in Texture. In Machine Vision: Theory, Algorithms, Practicalities, 2nd Ed., Academic Press, San Diego, 1997.
  14. J. Michels, A. Saxena, and A. Y. Ng, "High speed obstacle avoidance using monocular vision and reinforcement learning," Proc. of International Conference on Machine Learning, ACM, 2005.
  15. F. S. Hill, Jr, Computer graphics using OpenGL, 2nd Ed. Prentice-Hall, 1990.