Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Joint Segmentation of Multi-View Images by Region Correspondence

영역 대응을 이용한 다시점 영상 집합의 통합 영역화

  • Lee, Soo-Chahn (School of Electrical Engineering and Computer Science, Seoul National Univ.) ;
  • Kwon, Dong-Jin (School of Electrical Engineering and Computer Science, Seoul National Univ.) ;
  • Yun, Il-Dong (School of Digital Information Engineering, Hankuk University of Foreign Studies) ;
  • Lee, Sang-Uk (School of Electrical Engineering and Computer Science, Seoul National Univ.)
  • 이수찬 (서울대학교 전기.컴퓨터공학부) ;
  • 권동진 (서울대학교 전기.컴퓨터공학부) ;
  • 윤일동 (한국외국어대학교 용인캠퍼스 디지털정보공학과) ;
  • 이상욱 (서울대학교 전기.컴퓨터공학부)
  • Published : 2008.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents a method to segment the object of interest from a set of multi-view images with minimal user interaction. Specifically, after the user segments an initial image, we first estimate the transformations between foreground and background of the segmented image and the neighboring image, respectively. From these transformations, we obtain regions in the neighboring image that respectively correspond to the foreground and the background of the segmented image. We are then able to segment the neighboring image based on these regions, and iterate this process to segment the whole image set. Transformation of foregrounds are estimated by feature-based registration with free-form deformation, while transformation of backgrounds are estimated by homography constrained to affine transformation. Here, both are based on correspondence point pairs. Segmentation is done by estimating pixel color distributions and defining a shape prior based on the obtained foreground and background regions and applying them to a Markov random field (MRF) energy minimization framework for image segmentation. Experimental results demonstrate the effectiveness of the proposed method.

본 논문은 다시점에서 물체를 촬영한 영상들의 집합, 즉, 다시점 영상 집합(multi-view image set)이 주어진 경우, 적은 사용자 입력을 통해 효율적으로 영상 집합 내 관심 물체의 영역을 추출하는 기법을 제안한다. 제안하는 기법은 사용자가 직접 입력을 통해 영역화한 하나의 영상을 바탕으로, 그 영상의 배경 및 전경과 인접 영상 간의 변형을 각각 근사하여 전경 및 배경에 대응되는 인접 영상의 영역을 파악하고, 이 영역들을 통해 인접 영상을 영역화한 후, 영역화된 영상을 바탕으로 다음 인접 영상을 영역화하는 과정을 순차적으로 반복하여 영상 집합 전체를 영역화한다. 이때 전경 및 배경의 변형은 각각 특징점 기반 레지스트레이션(registration) 기법과 선형성 거리비율 보존(affine) 변형을 가정한 대응점 기반 변형행렬(homography)을 통해 근사되며, 각 대응 영역을 기반으로 하는 화소 색 분포 및 형상 정보(shape prior)를 마르코프 랜덤 장(Markov random field)에서의 에너지 최소화에 기반을 둔 영역화 기법에 적용하여 영역화를 수행한다. 제시하는 실험 결과는 제안하는 기법이 적은 사용자 입력으로 다시점 영상 집합 전체를 효과적으로 영역화한다는 것을 뒷받침한다.

Keywords

References

  1. Y. Boykov, M. P. Jolly, "Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D images," In Proceedings of International Conference on Computer Vision (ICCV), vol. I, pp. 105-112, July 2001
  2. Y. Boykov, V. Kolmogorov, "An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision," In IEEE transactions on Pattern Analysis and Machine Intelligence (PAMI), vol. 26, no. 9, pp. 1124-1137, Sept 2004 https://doi.org/10.1109/TPAMI.2004.60
  3. N. Campbell, G. Vogiatzis, C. Hernandez, R. Cipolla, "Automatic 3D Object Segmentation in Multiple Views Using Volumetric Graph-Cuts," In Proceedings of British Machine Vision Conference, vol. 1, pp. 530-539, September 2007
  4. D. Comaniciu, P, Meer, "Mean Shift: A Robust Approach Toward Feature Space Analysis," In IEEE Transactions on Pattern Analysis and Machine Intelligence, (PAMI) vol. 24, no. 5, pp. 603-619, May 2002 https://doi.org/10.1109/34.1000236
  5. A. Criminisi, G. Cross, A. Blake, V. Kolmogorov, "Bilayer Segmentation of Live Video," In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol 1. pp 53-60, June 2006
  6. P. F. Felzenswab, D. P. Huttenlocker, Distance "Transforms of Sampled Functions," Cornell University Technical Report, 2004
  7. R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press; Second Ed., 2004
  8. M. P. Kumar, P. H. S Torr, "Obj Cut," In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol 1. pp 18-25, June 2005
  9. D. J. Kwon, I. D. Yun, K. H. Lee, S. U. Lee, "An Efficient Feature-Based Nonrigid Registration Using Free-Form Deformations: Application to Multiphase Liver CT Images," Submitted To Pattern Recognition
  10. Y. Li, J. Sun, H.-Y. Shum, "Video object cut and paste," In ACM Transactions on Graphics (SIGGRAPH), vol. 24, no. 3, pp. 595-600, August 2005 https://doi.org/10.1145/1073204.1073234
  11. D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," In International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, November 2004 https://doi.org/10.1023/B:VISI.0000029664.99615.94
  12. J. Pilet, V. Lepetit, P. Fua, "Real-Time Non-Rigid Surface Detection," In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 822-828, June 2005
  13. C. Rother, V. Kolmogorov, A. Blake, "GrabCut - Interactive Foreground Extraction using Iterated Graph Cuts," In ACM Transactions on Graphics (SIGGRAPH), vol. 23, no. 3, pp. 309-314, August 2004 https://doi.org/10.1145/1015706.1015720
  14. J. R. Shewchuk, An Introduction to the Conjugate Gradient Method Without the Agonizing Pain, Carnegie Mellon University, 1994
  15. J. Shi, J. Malik, "Normalized Cuts and Image Segmentation," In IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), vol. 22, no. 8, pp 885-905, August 2000
  16. M. Sormann, C. Zach, J. Bauer, K. F. Karner, H. Bischof, "Automatic foreground propagation in image sequences for 3d reconstruction," In Pattern Recognition, 27th DAGM Symposium, pp. 93-100, August 2005
  17. M. Sormann, C. Zach, K. F. Karner, "Graph Cut Based Multiple View Segmentation for 3D Reconstruction," In Proceedings of 3rd International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT 2006), pp. 1085-1092, June 2006
  18. J. Sun, W. Zhang, X. Tang, H. Y. Shum, "Background Cut," In Proceedings of European Conference on Computer Vision, vol. 2, pp. 628-641, May 2006
  19. L. Vincent, P. Soille, "Watersheds in Digital Spaces: an Efficient Algorithm Based On Immersion Simulations," IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), vol. 13, no. 6, pp. 583-598, 1991 https://doi.org/10.1109/34.87344
  20. J. Wang, P. Bhat, A. Colburn, M. Agrawala, M. F. Cohen, "Interactive video cutout," In ACM Transactions on Graphics (SIGGRAPH), vol. 24, no. 3, pp. 585-594, August 2005 https://doi.org/10.1145/1073204.1073233
  21. P. Yin, A. Criminisi, J. Winn, I. A. Essa, "Tree-Based Classifiers for Bilayer Video Segmentation," In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1-8, June 2007
  22. 윤일동, 이경준, 이상욱, "카메라 보정 기법의 성능향상에 관한 연구,"