Foreground Segmentation and High-Resolution Depth Map Generation Using a Time-of-Flight Depth Camera

깊이 카메라를 이용한 객체 분리 및 고해상도 깊이 맵 생성 방법

  • 강윤석 (광주과학기술원 정보통신공학부) ;
  • 호요성 (광주과학기술원 정보통신공학부)
  • Received : 2012.07.04
  • Accepted : 2012.09.05
  • Published : 2012.09.30


In this paper, we propose a foreground extraction and depth map generation method using a time-of-flight (TOF) depth camera. Although, the TOF depth camera captures the scene's depth information in real-time, it has a built-in noise and distortion. Therefore, we perform several preprocessing steps such as image enhancement, segmentation, and 3D warping, and then use the TOF depth data to generate the depth-discontinuity regions. Then, we extract the foreground object and generate the depth map as of the color image. The experimental results show that the proposed method efficiently generates the depth map even for the object boundary and textureless regions.


  1. C. Fehn, R. Barre, and S. Pastoor, "Interactive 3-DTV- Concepts and Key Technologies," Proc. of the IEEE, vol. 94, no. 3, pp. 524-538, Mar. 2006.
  2. J. Sun, N.N. Zheng, and H.Y. Shum, "Stereo Matching Using Belief Propagation," IEEE Trans. of Pattern Analysis and Machine Intelligence (PAMI), vol. 25, no. 5, pp. 787-800, Jul. 2003.
  3. S. A. Gudmundsson, H. Aanaes, and R. Larsen, "Fusion of Stereo Vision and Time-of-Filght Imaging for Improved 3D Estimation," International Journal of Intelligent Sys. Technologies and Applications, vol. 5, no. 3, pp. 425-433, Nov. 2008.
  4. B. Bartczak and R. Koch, "Dense Depth Maps from Low Resolution Time-of-Flight Depth and High Resolution Color Views," Proc. of 5th International Symposium on Visual Computing, pp. 1-12, Nov. 2009.
  5. J. Zhu, L. Wang, R. Yang, and J. Davis, "Fusion of Time-of-Flight Depth and Stereo for High Accuracy Depth Maps," Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 231-236, June 2008.
  6. K. D. Kuhnert and M. Stommel, "Fusion of Stereo-Camera and PMD-Camera Data for Real-time Suited Precise 3D Environment Reconstruction," Proc. of the IEEE/RSJ International Conf. on Intelligent Robots and Syst. (IROS), pp. 4780-4785, Mar. 2006.
  7. A. Wang, T. Qiu, and L. Shao, "A Simple Method of Radial Distortion Correction with Centre of Distortion Estimation," Journal of Mathematical Imaging and Vision, vol. 35, no. 3, pp. 165-172, Nov. 2009.
  8. Z. Zhang, "A Flexible New Technique for Camera Calibration," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330-1334, Oct. 2000.
  9. C. Rother, V. Kolmogorov, and A. Blake, "GrabCut - Interactive Foreground Extraction using Iterated Graph Cuts," ACM Trans. on Graphics, vol. 23, no. 1, pp. 309-314, Jan. 2004.
  10. P.F. Felzenszwalb and D.P. Huttenlocher, "Efficient Belief Propagation for Early Vision," International Journal of Computer Vision, vol. 70, no. 1, pp. 41-54, Oct. 2006.
  11. Y. S. Kang and Y. S. Ho, "Multi-view Image Enhancement by Compensating Geometric Error," Telecomm. Review, vol. 20, no. 3, pp. 507-518, 2010.
  12. A. Telea, "An Image Inpainting Technique based on The Fast Marching Method," Journal Graphics Tools, vol. 9, pp. 25-36, May 2004.