Human Tracking and Body Silhouette Extraction System for Humanoid Robot

휴머노이드 로봇을 위한 사람 검출, 추적 및 실루엣 추출 시스템

  • 곽수영 (연세대학교 컴퓨터과학과) ;
  • 변혜란 (연세대학교 컴퓨터과학과)
  • Published : 2009.06.30


In this paper, we propose a new integrated computer vision system designed to track multiple human beings and extract their silhouette with an active stereo camera. The proposed system consists of three modules: detection, tracking and silhouette extraction. Detection was performed by camera ego-motion compensation and disparity segmentation. For tracking, we present an efficient mean shift based tracking method in which the tracking objects are characterized as disparity weighted color histograms. The silhouette was obtained by two-step segmentation. A trimap is estimated in advance and then this was effectively incorporated into the graph cut framework for fine segmentation. The proposed system was evaluated with respect to ground truth data and it was shown to detect and track multiple people very well and also produce high quality silhouettes. The proposed system can assist in gesture and gait recognition in field of Human-Robot Interaction (HRI).


  1. I. Haritaoglu, D. Harwood, and L.S. Davis,'W4: Real-Time Surveillance of People and Their Activities', IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.22, No.8, pp. 809-830, 2000
  2. A. Yilmaz, X. Li and M. Shah, 'Object ContourTracking Using Level Sets', Asian Conferenceon Computer Vision, 2004
  3. L. Davis, V. Philomin and R. Duraiswami 'Tracking humans from a moving platform',The 15th International Conference on Pattern Recognition. Vol. 4, pp. 171-78, 2000
  4. C. Stauffer and W. E. L. Grimson, 'Adaptive background mixture models for real-time tracking', Proceeding IEEE International Conference on Computer Vision and Pattern Recognition, pp. 246-252, 1999
  5. S. Yu, R. Gross and J. Shi, 'Concurrent object recognition and segmentation by graph partitioning', Proceeding Neural InformationProcessing Systems, pp. 1383-1390. 2002
  6. Y. Boykov, and M. Jolly, 'Iterative graph cuts for optimal boundary and region segmentation of objects in N-D Images,' Proceeding IEEE 8th International Conference on Computer Vision, Canada, 2001
  7. Y. Li, J. Sun, C.-K. Tang and H.-Y. Shum,'Lazy Snapping', Proceeding ACM SIGRAPH,Vol 23, No. 3, 2004
  8. A. Blake, C. Rother, M. Brown, P. Perez, and P. Torr. 'Interactive image segmentation using an adaptive GMMRF model,' Proceeding European Conference on Computer Vision, pp.428-442, 2004
  9. C. Harris and M. J. Stephens, 'A combined corner and edge detector,' In Alvey Vision Conference, pp. 147–152, 1988
  10. D. Comaniciu, and V. Ramesh.: Kernel-Based Object Tracking. IEEE Transaction on Pattern Analysis and Machine Intelligence Vol. 25. pp.564-577, 2003
  11. M. P. Wand and M. C. Jones.: KernelSmoothing. Chapman & Hall. 1995
  12. D. Comaniciu and P. Meer, 'Mean shift: arobust approach toward feature space analysis,' IEEE Transaction on Pattern Analysis and Machine Intelligence, pp. 603-619, 2002
  13. Y. Boykov and V. Kolmogorov, 'An experimental comparison of min-cut/ max-flow algorithms for energy minimization in vision', IEEE Transaction on Pattern Analysis andMachine Intelligence, Vol. 26, pp. 1124-1137, 2004