The Journal of the Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회논문지)

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

2D to 3D Conversion Using The Machine Learning-Based Segmentation And Optical Flow

학습기반의 객체분할과 Optical Flow를 활용한 2D 동영상의 3D 변환

  • Received : 2011.05.09
  • Accepted : 2011.06.10
  • Published : 2011.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose the algorithm using optical flow and machine learning-based segmentation for the 3D conversion of 2D video. For the segmentation allowing the successful 3D conversion, we design a new energy function, where color/texture features are included through machine learning method and the optical flow is also introduced in order to focus on the regions with the motion. The depth map are then calculated according to the optical flow of segmented regions, and left/right images for the 3D conversion are produced. Experiment on various video shows that the proposed method yields the reliable segmentation result and depth map for the 3D conversion of 2D video.

본 논문에서는 2D 동영상을 3D 입체영상으로 변환하기 위해서 머신러닝에 의한 학습기반의 객체분할과 객체의 optical flow를 활용하는 방법을 제안한다. 성공적인 3D 변환을 가능하게 하는 객체분할을 위해서, 객체의 칼라 및 텍스쳐 정보는 학습을 통해 반영하고 움직임이 있는 영역 위주로 객체분할을 수행할 수 있도록 optical flow를 도입한 새로운 에너지함수를 설계하도록 한다. 분할된 객체들에 대해 optical flow 크기에 따른 깊이맵을 추출하여 입체영상에 필요한 좌우 영상을 합성하여 생성하도록 한다. 제안한 기법으로 인해 효과적인 객체분할과 깊이맵을 생성하여 2D 동영상에서 3D 입체동영상으로 변환됨을 실험결과들이 보여준다.

Keywords

References

  1. Y. Matsumoto, H. Terasaki. K. Sugimoto, and T. Arakawa, "Conversion system of monocular image sequence to stereo using motion parallax," Proc. of SPIE Stereoscopic Displays and Virtual Reality Systems vol. 3012, pp.108-115, 1997.
  2. 홍호기, 백윤기, 이승현, 김동욱, 유지상, "2D H.264 동영상의 3D 입체변환," 한국통신학회, 제31권, 제12C호, 1208-1215쪽, 2005년 12월.
  3. S. Battiato, S. Curti, M. LaCascia, E. Scordato, and M. Tortora, "Depth-Map generation by image classification," Proc. of SPIE Electronic Imaging2004, Three-Dimensional Image Capture and Application VI, vol. 5302, pp.95-104, 2004.
  4. Ross, J., "Stereopsis by binocular delay," Nature, vol. 248, pp.354-364, 1974. https://doi.org/10.1038/248354a0
  5. 이요섭, "2D-3D 변환 기술의 동향 및 전망," 전자공학회, 제38권, 제2호, 37-43쪽, 2011년 2월.
  6. J. Friedman, T. Hastie, and R. Tibshirani, "Additive logistic regression: a statistica view of boosting," The Annual of Statistics, vol. 28, pp.377-386, 2000.
  7. R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, and C. Rother, "A comparative study of energy minimization methods for markov random fields with smoothness-based priors," IEEE Trans. PAMI, vol. 30, no. 6, pp.1068-1080, 2008. https://doi.org/10.1109/TPAMI.2007.70844
  8. J. Lafferty, A. McCallumm, and F. Pereira, "Conditional random fields: Probabilistic models and for segmenting and labeling sequence data," Proc. of ICML, 2001.
  9. C. Zach, T. Pock, and H. Bischof, "A duality based approach for realtime tv-l1 optical flow," Proc. of the 29th DAGM Conference on Pattern Recognition, 2007.
  10. Y. Boykov, O. Veksler, and R. Zabih, "Fast approximate energy minimization via graph cuts," IEEE Trans. PAMI, vol. 23, no. 11, pp.1222-1239, 2001. https://doi.org/10.1109/34.969114