Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

A Fast and Accurate Face Detection and Tracking Method by using Depth Information and color information

깊이정보와 컬러정보를 이용한 고속 고정밀 얼굴검출 및 추적 방법

  • Received : 2012.04.17
  • Accepted : 2012.05.16
  • Published : 2012.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes a fast face detection and tracking method which uses depth images as well as RGB images. It consists of the face detection procedure and the face tracking procedure. The face detection method basically uses an existing method, Adaboost, but it reduces the size of the search area by using the depth information and skin color. The proposed face tracking method uses a template matching technique and incorporates an early-termination scheme to reduce the execution time further. The results from implementing and experimenting the proposed methods showed that the proposed face detection method takes only about 39% of the execution time of the existing method. The proposed tracking method takes only 2.48ms per frame. For the exactness, the proposed detection method and previous method showed a same detection ratio but in the error ratio, which is about 0.66%, the proposed method showed considerably improved performance. In all the cases except a special one, the tracking error ratio is as low as about 1%. Therefore, we expect the proposed face detection and tracking methods can be used individually or in combined for many applications that need fast execution and exact detection or tracking.

본 논문에서는 RGB영상과 깊이영상을 사용하여 얼굴검출 및 추적을 고속으로 수행할 수 있는 방법을 제안한다. 이 방법은 얼굴검출 과정과 얼굴추적 과정으로 구성되며, 얼굴검출 과정은 기본적으로 기존의 Adaboost 방법을 사용하나, 깊이정보와 피부색을 사용하여 탐색영역을 축소한다. 얼굴추적은 템플릿 매칭방법을 사용하며, 조기종료 기법을 사용하여 수행시간을 줄였다. 이 방법들을 구현하여 실험한 결과, 얼굴검출 방법은 기존의 방법에 비해 약 39%의 수행시간을 보였으며, 얼굴추적 방법은 프레임 당 2.48ms의 추적시간을 보였다. 또한 검출율에 있어서도 제안한 얼굴검출 방법은 기존방법과 비슷한 검출률을 보였지만, 오검출률에 있어서는 0.66%로 기존방법보다 상당히 향상된 성능을 보였다. 또한 얼굴추적 방법은 특별한 경우를 제외한 모든 경우에서 약 1%의 낮은 추적오차율을 보였다. 따라서 제안한 얼굴검출 및 추적방법은 각각 또는 결합하여 고속 동작과 높은 정확도를 필요로 하는 응용분야에 사용될 수 있을 것으로 기대된다.

Keywords

References

  1. G, Q, Zhao, et al., "A Simple 3D face Tracking Method based on Depth Information," Int'l Conf. on Machine Learning and Cybernetics, pp. 5022-5027, Aug. 2005.
  2. C. X. Wang and Z. Y. Li, "A New Face Tracking Algorithm Based on Local Binary Pattern and Skin Color Information," ISCSCT, Vol. 2, pp. 20-22, Dec. 2008.
  3. M. H. Yang, et al., "Detecting Faces in Images; A Survey," IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 24, No. 1, pp. 34-58, Jan. 2002. https://doi.org/10.1109/34.982883
  4. G. Z. Yang and T. S. Huang, "Human Face Detection in Complex Background," Pattern Recognition, Vol. 27, No. 1, pp. 53-63, Jan. 1994. https://doi.org/10.1016/0031-3203(94)90017-5
  5. K. C. Yow, R. Cipolla, "Feature-Based Human Face Detection," Image and Vision Computing, Vol. 15, No. 9, pp. 713-735, Sept. 1997. https://doi.org/10.1016/S0262-8856(97)00003-6
  6. Y. Dai and Y. Nakano, "Face-texture Model based on SGLD and its Application in Face Detection in a Color Scene," Pattern Recognition, Vol. 29, No. 6, pp. 1007-1017, June 1996. https://doi.org/10.1016/0031-3203(95)00139-5
  7. J. Yang and A. Waibel; "A Real-Time Face Tracker," WACV'96, pp. 142-147, 1996.
  8. S. J. McKenna, S. Gong, and Y. Raja; "Modelling Facial Colour and Identity with Gaussian Mixtures," Pattern Recognition, Vol. 31, No. 12, pp. 1883-1892, 1998. https://doi.org/10.1016/S0031-3203(98)00066-1
  9. P. Kakumanu, S. Makrogiannis, and N. Bourbakis, "A Survey of Skin-color Modeling and Detection Methods," Pattern Recognition, Vol. 40, pp. 1106-1122, Mar. 2007. https://doi.org/10.1016/j.patcog.2006.06.010
  10. R. Kjeldsen and J. Kender, "Finding Skin in Color Images," Proc. Second Int'l Conf. Automatic Face and Gesture Recognition, pp. 312-917, 1996.
  11. L. Craw, D. Tock, and A. Bennett, "Finding Face Features," Proc. Second European Conf. Computer Vision, pp. 92-96, 1992.
  12. M. Turk and A. Pentland, "Eigenfaces for Recognition," Journal of Cognitive Neuroscience, Vol. 3, pp. 71-86, 1991. https://doi.org/10.1162/jocn.1991.3.1.71
  13. P. N. Belhumeur, et al., " Eigenfaces vs. Fisherfaces: Recognition using Class Specific Linear Projection," IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, pp. 711-720, 1997. https://doi.org/10.1109/34.598228
  14. E. Osuna, R. Freund, and F. Girosi, "Training Support Vector Machines: an Application to Face Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 130-136, 1997.
  15. P. Viola and M. J. Jones, "Robust Real-Time Face Detection," Computer Vision, Vol. 52, No. 2, pp. 137-154, 2004.
  16. C. P. Papageorgiou, M. Oren, and T. Poggio, "A General Framework for Object Detection," IEEE Int'l Conf. Computer Vision, pp. 555-562, 1998.
  17. R. Lienhart and J. Maydt, "An Extended Set of Haar-like Features for Rapid Object Detection," Int'l Conf. Image Processing, Vol. 1, pp. 22-25, Sept, 2002.
  18. K. An and M. Chung, "Cognitive face analysis system for future interactive TV," IEEE Trans. Consumer Electronics, Vol. 55, No. 4, pp. 2271-2275, Nov. 2009. https://doi.org/10.1109/TCE.2009.5373798
  19. K. Hariharakrishnan and D. Schonfeld, "Fast object tracking using adaptive block matching," IEEE Trans. Multimedia, vol. 7, no. 5, 2005.
  20. M. Lievin and F. Luthon; "Nonlinear Color Space and Spatiotemporal MRF for Hierarchical Segmentation of Face Features in Video," Proc. IEEE Int'l Conf. Image Processing, pp. 63-71, 2004.
  21. Y. Lin et al., "Real-time Tracking and Pose Estimation with Partitioned Sampling and Relevance Vector Machine," IEEE Intl. Conf. Robotics and Automation, pp. 453-458, 2009.
  22. A. An and M. Chung, "Robust Real-time 3D Head Tracking based on Online Illumination Modeling and its Application to Face Recognition," IEEE Intl. Conf. Intelligent Robots and Systems, pp. 14661471, 2009.
  23. R. Okada, Y. Shirai, and J. Miura, "Tracking a Person with 3-D Motion by Integrating Optical Flow and Depth," Proc. Fourth IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 336-341, 2000.
  24. G. Zhao, et al., "A Simple 3D Face Tracking Method based on Depth Information," Intl Conf. Machine Learning and Cybernetics, pp. 5022-5027, 2005.
  25. Y. H. Lee et al., "A Robust Face Tracking using Stereo Camera," SICE Annual Conf., pp. 1985-1989, Sept. 2007.
  26. S. Kosov et al., "Rapid Stereo-vision Enhanced Face Recognition," IEEE Intl. Conf. Image Processing, pp. 2437-2440, Sept. 2010.
  27. Mesa Imaging, SR4000 user manual v2.0, May 2011.
  28. J. L. Wilson, Microsoft kinect for Xbox 360, PC Mag. Com, Nov. 10, 2010.
  29. M. Hacker, et al., "Geometric Invariants for Facial Feature Tracking with 3D TOF Cameras," Int'l Symposium on Signals, Circuits and Systems, Vol. 1, pp. 1-4, 2007.
  30. Douglas Chai, et al., "Locating Facial Region of a Head-and -Shoulders Color Image," Int'l Conf. Automatic Face and Gesture Recognition, pp. 124-129, April 1998.
  31. Y. Freund and R. E. Schapire, "A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting," J. Computer and System Sciences, Vol. 55, pp. 119-139, 1997. https://doi.org/10.1006/jcss.1997.1504
  32. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd Ed., Pearson Prentice Hall, Upper Saddle River, NJ, 2008.

Cited by

  1. 얼굴 검증을 이용한 개선된 얼굴 검출 vol.22, pp.10, 2018, https://doi.org/10.6109/jkiice.2018.22.10.1334