Journal of the Institute of Electronics Engineers of Korea SP (대한전자공학회논문지SP)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

Computation ally Efficient Video Object Segmentation using SOM-Based Hierarchical Clustering

SOM 기반의 계층적 군집 방법을 이용한 계산 효율적 비디오 객체 분할

  • Published : 2006.07.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes a robust and computationally efficient algorithm for automatic video object segmentation. For implementing the spatio-temporal segmentation, which aims for efficient combination of the motion segmentation and the color segmentation, an SOM-based hierarchical clustering method in which the segmentation process is regarded as clustering of feature vectors is employed. As results, problems of high computational complexity which required for obtaining exact segmentation results in conventional video object segmentation methods, and the performance degradation due to noise are significantly reduced. A measure of motion vector reliability which employs MRF-based MAP estimation scheme has been introduced to minimize the influence from the motion estimation error. In addition, a noise elimination scheme based on the motion reliability histogram and a clustering validity index for automatically identifying the number of objects in the scene have been applied. A cross projection method for effective object tracking and a dynamic memory to maintain temporal coherency have been introduced as well. A set of experiments has been conducted over several video sequences to evaluate the proposed algorithm, and the efficiency in terms of computational complexity, robustness from noise, and higher segmentation accuracy of the proposed algorithm have been proved.

본 논문에서는 계산 효율적이고 노이즈에 강건한 비디오 객체 분할 알고리즘을 제안한다. 움직임 분할과 색 분할을 효율적으로 결합한 시공간 분할 방법의 구현을 위해 SOM 기반의 계층적 군집 방법을 도입하여 특징 벡터들의 군집 관점에서 분할 과정을 해석함으로써 기존의 객체 분할 방법에서 정확한 분할 결과를 얻기 위해서 요구되어지는 많은 연산량과 노이즈에 의한 시스템의 성능 저하 문제를 최소화한다. 움직임 분할 과정에서는 움직임 추정 에러에 의한 영향을 최소화하기 위해서 MRF 기반의 MAP 추정 방법을 이용하여 계산한 움직임 벡터의 신뢰도를 이용한다. 또한 움직임 분할의 성능 향상을 위해서 움직임 신뢰도 히스토그램을 이용한 노이즈 제거 과정을 거칠 뿐만 아니라 자동으로 장면 내에 존재하는 객체의 수를 구하기 위해서 군집 유효성 지표를 이용한다. 객체 추적의 성능 향상을 위해 교차 투영 기법을 이용하며, 분할 결과의 시간적 일관성 유지를 위해 동적 메모리를 이용한다. 다양한 특성을 가지는 비디오 시퀀스들을 이용한 실험을 통해 제안하는 방법이 계산 효율적이고 노이즈에 강건하게 비디오 객체 분할을 수행함은 물론 기존의 구현 방법에 비해 정확한 분할 결과를 얻을 수 있음을 확인하였다.

Keywords

References

  1. Y. Liu and Y. F. Zheng , 'Video object segmentation and tracking using ${\psi}-learning$ classification,' IEEE Trans. on Cir. and Sys. for Video Tech, vol. 15, pp. 885-899, 2005 https://doi.org/10.1109/TCSVT.2005.848346
  2. H. Xu, A. Younix, and M. Kabuka, 'Automatic moving object extraction for content-based applications,' IEEE Trans. on Cir. and Sys. for Video Tech, vol. 14, pp. 796-812, 2004 https://doi.org/10.1109/TCSVT.2004.828338
  3. T. Meier and K. N. Ngan, 'Video segmentation for content-based coding,' IEEE Trans. on Cir. and Sys. for Video Tech, vol. 9, pp. 1190-1203, 1999 https://doi.org/10.1109/76.809155
  4. M. Kim, J. G. Choi, D. Kim, H. Lee, M. H. Lee, C. Ahn, and Y-S. Ho, 'A VOP generation tool: automatic segmentation of moving objects in image sequences based on spatio-ternporal information,' IEEE Trans. on Cir. and Sys. for Video Tech, vol. 9, pp. 1216-1226, 1999 https://doi.org/10.1109/76.809157
  5. A. Doulamis, N. Doulamis, K. Ntalianis, and S. Kollias, 'An efficient fully unsupervised video object segmentation scheme using an adaptive neural-network classifier architecture,' IEEE Trans. on Neu Net, vol. 14, pp. 616-630, 2003 https://doi.org/10.1109/TNN.2003.810605
  6. V. Mezaris, I. Kompatsiaris, and M. G. Strintzis, 'Video object segmentation using Bayes-based temporal tracking and trajectory-based region merging,' IEEE Trans. on Cir. and Sys. for Video Tech, vol. 14, pp. 782-795, 2004 https://doi.org/10.1109/TCSVT.2004.828341
  7. J. Y. A. Wang and E H Adelson, 'Representing moving images with layers,' IEEE Trans. on Image. Proc, vol. 3, pp. 625-638, 1994 https://doi.org/10.1109/83.334981
  8. D. W. Murray and B. F. Buxton, 'Scene segmentation from visual motion using global optimization,' IEEE Trans. on Pat Anal. and Mach Intel., vol. 9, pp. 220-228, 1987 https://doi.org/10.1109/TPAMI.1987.4767896
  9. A. A. Alatan, L. Onural, M. Wollborn, R. Mech, E Tuncel, and T. Sikora, 'Image sequence analysis for emerging interactive multimedia services-the European COST 211 framework,' IEEE Trans. on Cir. and Sys. for Video Tech, vol. 8, pp. 802-813, 1998 https://doi.org/10.1109/76.735378
  10. E. Tuncel and L. Onural, 'Utilization of the recursive shortest spanning tree algorithm for video-object segmentation by 2- D affine motion modeling,' IEEE Trans. on Cir. and Sys. for Video Tech, vol. 10, pp. 776-781, 2000 https://doi.org/10.1109/76.856454
  11. D. Wang, 'Unsupervised video segmentation based on watersheds and temporal tracking,' IEEE Trans. on Cir. and Sys. for Video Tech, vol. 8, pp. 539-546, 1998 https://doi.org/10.1109/76.718501
  12. A. D. Doulamis, N. Doulamis, and S. Kallas, 'Non-sequential video content representation using temporal variation of feature vectors,' IEEE Trans. on Cons. Elec., vol. 46, pp. 758-768, 2000 https://doi.org/10.1109/30.883444
  13. J. Kim and T. Chen, 'Multiple feature clustering for image sequence segmentation,' Pat. Rec. Let., vol. 22, pp. 1207-1217, 2001 https://doi.org/10.1016/S0167-8655(01)00053-8
  14. J. Vesanto and E. Alhoniemi, 'Clustering of the self-organizing map,' IEEE Trans. on Neu. Net. vol. 11, pp. 586-600, 2000 https://doi.org/10.1109/72.846731
  15. S. Wu and T. W. S. Chow, 'Clustering of the self-organizing map using a clustering validity index based on inter-cluster and intra-cluster density,' Pat Rec., vol. 37, pp. 175-188, 2004 https://doi.org/10.1016/S0031-3203(03)00237-1
  16. R. Cucchiara, A Prati, and R Vezzani, 'Real-time motion segmentation from moving cameras,' Real-Time Imag., vol. 10, pp. 127-143, 2004 https://doi.org/10.1016/j.rti.2004.03.002
  17. U. Maulik and S. Bandyopadhyay, 'Performance evaluation of some clustering algorithms and validity indices,' IEEE Trans. on Pat. Anal. and Mach Intel., vol. 24, pp. 1650-1654, 2002 https://doi.org/10.1109/TPAMI.2002.1114856
  18. F. Dufaux and J. Konrad, 'Efficient, robust, and fast global motion estimation for video coding,' IEEE Trans. on Imag. Proc., vol. 9, pp. 497-501, 2000 https://doi.org/10.1109/83.826785
  19. Y. Su, M.-T. Sun, and V. Hsu, 'Global motion estimation from coarsely sampled motion vector field and the applications,' IEEE Trans. on Cir. and Sys. for Video Tech, vol. 15, pp. 232-242, 2005 https://doi.org/10.1109/TCSVT.2004.841656
  20. Y. Tsaig and A Averbuch, 'Automatic segmentation of moving objects in video sequences: a region labeling approach,' IEEE Trans. on Cir. and Sys. for Video Tech, vol. 12, pp. 597-612, 2002 https://doi.org/10.1109/TCSVT.2002.800513
  21. I. Patras, E. A. Hendriks, and R. L. Lagendijk, 'Confidence measures for block matching motion estimation,' IEEE Int. Conf. on Imag. Proc, vol. 2, pp. 277-280, 2002 https://doi.org/10.1109/ICIP.2002.1039941
  22. R. Neher and A. Srivastava, 'A Bayesian MRF framework for labeling terrain using hyperspectral imaging,' IEEE Trans. on Ceo. and Rem Sen, vol. 43, pp. 1363-1374, 2005 https://doi.org/10.1109/TGRS.2005.846865
  23. O. J. Morris, J. M. Lee, A. G. Constantinides, 'A unified method for segmentation and edge detection using graph theory,' IEEE Int Conf. on Acou, Spe., and Sig. Proc., vol. 11, pp. 2051-2055, 1986 https://doi.org/10.1109/ICASSP.1986.1168866
  24. M. Y. Kiang, 'Extending the Kohonen selforganizing map networks for clustering analysis,' Comp. Stat. & Data Anal., vol. 38, pp. 161-180, 2001 https://doi.org/10.1016/S0167-9473(01)00040-8
  25. P. Xu, C.-H. Chang, and A. Paplinski, 'Self-organizing topological tree for online vector quantization and data clustering,' IEEE Trans. on Sys., Man and Cyb., vol. 35, pp. 515-526, 2005 https://doi.org/10.1109/TSMCB.2005.846651
  26. S. H. Kwok, A. G. Constantinides, and W.-c. Siu, 'An efficient recursive shortest spanning tree algorithm using linking properties,' IEEE Trans. on Cir. and Sys. for Video Tech, vol. 14, pp. 852-863, 2004 https://doi.org/10.1109/TCSVT.2004.828334