Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Salient Object Extraction from Video Sequences using Contrast Map and Motion Information

대비 지도와 움직임 정보를 이용한 동영상으로부터 중요 객체 추출

  • 곽수영 (연세대학교 컴퓨터과학과) ;
  • 고병철 (계명대학교 정보통신학부) ;
  • 변혜란 (연세대학교 컴퓨터과학과)
  • Published : 2005.11.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes a moving object extraction method using the contrast map and salient points. In order to make the contrast map, we generate three-feature maps such as luminance map, color map and directional map and extract salient points from an image. By using these features, we can decide the Attention Window(AW) location easily The purpose of the AW is to remove the useless regions in the image such as background as well as to reduce the amount of image processing. To create the exact location and flexible size of the AW, we use motion feature instead of pre-assumptions or heuristic parameters. After determining of the AW, we find the difference of edge to inner area from the AW. Then, we can extract horizontal candidate region and vortical candidate region. After finding both horizontal and vertical candidates, intersection regions through logical AND operation are further processed by morphological operations. The proposed algorithm has been applied to many video sequences which have static background like surveillance type of video sequences. The moving object was quite well segmented with accurate boundaries.

본 논문에서는 시공간 정보를 이용하여 동영상에서 움직이는 객체를 자동으로 추출하는 방법을 제안한다. 본 논문에서 제안하는 방법은 다른 영역과 구별되는 현저한 장소에 무의식적으로 집중되는 시각주의 특성을 컴퓨터 시스템에 도입한 대비 지도(contrast map)와 중요 특징점(salient point)을 적용한 것이 큰 특징이라고 할 수 있다. 대비 지도는 밝기(luminance), 색상(color) 그리고 방향성(direction) 3가지의 특징 정보 중 자기와 방향성의 특징을 나타내는 자기 지도(luminance map)와 방향성 지도(directional map)를 결합하여 대비 지도를 생성한다. 또한, 사람이 시각적으로 볼 때 의미 있다고 생각하는 중요 특징점을 웨이블릿 변환을 이용하여 찾아낸다. 이렇게 생성된 대비 지도와 중요 특징점을 이용하여 대략적인 집중윈도우(AW:Attention Window)의 위치와 크기를 결정한다. 다음으로, 동영상의 가장 큰 특징인 움직임 정보를 추정하여 집중윈도우를 객체에 가장 근사하게 축소시키고, 윤곽선 정보를 이용하여 객체를 추출한다. 윤곽선을 추출하기 위해 캐니에지(canny edge)를 사용하였으며, 배경의 윤곽선 제거를 위하여 윤곽선의 차이(DE:Difference of Edge)를 이용하여 가로 후보영역과 세로 후보영역을 추출한다. 추출된 2개의 후보영역을 AND연산과 모폴로지 연산을 이용하여 객체를 자동으로 추출하는 방법을 제안한다. 실험은 카메라가 고정된 상태에서 촬영한 동영상에 대해 이루어 졌으며, 객체와 배경이 효과적으로 분리되는 것을 확인하였다.

Keywords

References

  1. Q. Tian, Y. Wu, and T.S. Huang. 'Combine user defined region-of-interest and spatial layout for image retrieval,' Proceedings of International Conference on Image Processing, Vol.3. pp.746-749, 2000 https://doi.org/10.1109/ICIP.2000.899562
  2. S. Kim, S. Park, and M. Kim, 'Central Object Extraction for Object-based Image Retrieval,' Proceedings of International Conference on Image and Video Retrieval, pp. 39-49, 2003 https://doi.org/10.1007/3-540-45113-7_5
  3. T. Meier and K. N. Ngan, 'Video Segmentation for Content-based Coding,' IEEE Transaction on Circuits and Systems Video Technology, Vol. 9, No. 8, pp. 1190-1203, 1999 https://doi.org/10.1109/76.809155
  4. Y. Tsaig and A. Averbuch, 'Automatic Segmentation of Moving Objects in Video Sequences: A Region Labeling Approach,' IEEE Transaction on Circuits and Systems Video Technology, Vol. 12, No. 7, pp. 597-612, 2002 https://doi.org/10.1109/TCSVT.2002.800513
  5. J. Shi and J. Malik, 'Motion Segmentation and Tracking Using Normalized Cuts,' Proceedings of International Conference on Computer Vision, pp. 1154-1160, 1998 https://doi.org/10.1109/ICCV.1998.710861
  6. G. Adiv, 'Inherent Ambiguities in Recovering 3-D Motion and Structure from a Noisy Flow Field,' IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 11, pp. 477-489, 1989 https://doi.org/10.1109/34.24780
  7. G. D. Borshukov,G. Bozdagi,Y. Altunbasak, and A. M. Tekalp, 'Motion Segmentation by Multistage Affine Classification,' IEEE Transaction on Image Processing, Vol. 6, No. 11, pp. 1591-1594, 1997 https://doi.org/10.1109/83.641420
  8. P. Bouthemy and E. Francois, 'Motion Segmentation and Qualitative Dynamic Scene Analysis from an Image Sequence,' International Journal of Computer Vision, Vol. 10, No. 2, pp. 157-182, 1993 https://doi.org/10.1007/BF01420735
  9. J. Y. A. Wang and E. H. Adelson, 'Representing Moving Images with Layers,' IEEE Transaction on Image Processing, Vol. 3, No, 5, pp. 625-638, 1994 https://doi.org/10.1109/83.334981
  10. M. M. Chang, A. M. Tekalp, and M. I. Sezan, 'Simultaneous Motion Estimation and Segmentation,' IEEE Transaction on Image Processing, Vol. 6, No. 9, pp. 1326-1333, 1997 https://doi.org/10.1109/83.623196
  11. C. Stiller, 'Object-based Estimation of Dense Motion Fields,' IEEE Transaction on Image Processing, Vol. 6, No. 2, pp. 234-250, 1997 https://doi.org/10.1109/83.551695
  12. E. Tuncel and L. Onural, 'Utilization of the Recursive Shortest Spanning Tree Algorithm for Video Object Segmentation by 2-D Affine Motion Modeling,' IEEE Transaction on Circuits and Systems Video Technology, Vol. 10, No. 5, pp. 776-781, 2000 https://doi.org/10.1109/76.856454
  13. S. R. Sternberg, 'Grayscale Morphology,' Computer Vision, Graphics, and Image Processing, Vol. 35, No. 3, pp. 333-355, 1996 https://doi.org/10.1016/0734-189X(86)90004-6
  14. D. Wang, 'Unsupervised Video Segmentation based on Watersheds and Temporal Tracking,' IEEE Transaction on Circuits and Systems Video Technology, Vol. 8, No. 5, pp. 539-546, 1998 https://doi.org/10.1109/76.718501
  15. J. G. Choi, S.W. Lee, and S. D. Kim, 'Spatiotemporal Video Segmentation Using a Joint Similarity Measure,' IEEE Transaction on Circuits and Systems Video Technology, Vol. 7, No 2, pp. 279-286, 1997 https://doi.org/10.1109/76.564107
  16. P. Salembier, P. Brigger, J. R. Casas, and M. Pardas, 'Morphological Operators for Image and Video Compression,' IEEE Transaction on Image Processing, Vol. 5, No. 6, pp. 881-898, 1996 https://doi.org/10.1109/83.503906
  17. L. Itti, C. Koch, and E. Niebur, 'A Model of Saliency-based Visual Attention for Rapid Scene Analysis,' IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol.20, No. 11, pp.1254-1259, 1998 https://doi.org/10.1109/34.730558
  18. E. Loupias, and N. Sebe, 'Wavelet-based Salient Points for Image Retrieval,' Research Report RR 99.11, RFV-INSA Lyon, 1999
  19. B. C. Ko, and H. Byun, 'Region-based Image Retrieval: A New Method for Extraction of Salient Regions and Learning of Importance Scores,' International Journal of Pattern Recognition and Artificial Intelligence, Vol. 17, No. 8, pp. 1349-1367, 2003 https://doi.org/10.1142/S0218001403002939
  20. http://www.cipr.rpi.edu/resource/sequences/index.html
  21. J. Z. Wang, J. Li, and G. Wiederhold, 'SIMPLicity: Semantics-Sensitive Integrated Matching for Picture Libraries,' IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 9, pp. 947-963, 2001 https://doi.org/10.1109/34.955109