The KIPS Transactions:PartB (정보처리학회논문지B)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Robust AAM-based Face Tracking with Occlusion Using SIFT Features

SIFT 특징을 이용하여 중첩상황에 강인한 AAM 기반 얼굴 추적

  • Received : 2010.07.02
  • Accepted : 2010.08.16
  • Published : 2010.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Face tracking is to estimate the motion of a non-rigid face together with a rigid head in 3D, and plays important roles in higher levels such as face/facial expression/emotion recognition. In this paper, we propose an AAM-based face tracking algorithm. AAM has been widely used to segment and track deformable objects, but there are still many difficulties. Particularly, it often tends to diverge or converge into local minima when a target object is self-occluded, partially or completely occluded. To address this problem, we utilize the scale invariant feature transform (SIFT). SIFT is an effective method for self and partial occlusion because it is able to find correspondence between feature points under partial loss. And it enables an AAM to continue to track without re-initialization in complete occlusions thanks to the good performance of global matching. We also register and use the SIFT features extracted from multi-view face images during tracking to effectively track a face across large pose changes. Our proposed algorithm is validated by comparing other algorithms under the above 3 kinds of occlusions.

얼굴추적은 3차원 공간상에서 머리(head)와 안면(face)의 움직임을 추정하는 기술로, 얼굴 표정 감정인식과 같은 상위 분석단계의 중요한 기반기술이다. 본 논문에서는 AAM 기반의 얼굴추적 알고리즘을 제안한다. AAM은 변형되는 대상을 분할하고 추적하는데 광범위하게 적용되고 있다. 그러나 여전히 여러 가지 해결해야할 제약사항들이 있다. 특히 자체중첩(self-occlusion)과 부분적인 중첩, 그리고 일시적으로 완전히 가려지는 완전중첩 상황에서 보통 국부해에 수렴(local convergence)하거나 발산하기 쉽다. 본 논문에서는 이러한 중첩상황에 대한 AAM의 강인성을 향상시키기 위해서 SIFT 특징을 이용하고 있다. SIFT는 일부 영상의 특징점으로 안정적인 추적이 가능하기 때문에 자체와 부분중첩에 효과적이며, 완전중첩의 상황에도 SIFT의 전역적인 매칭성능으로 별도의 재초기화 없이 연속적인 추적이 가능하다. 또한 추적과정에서 큰 자세변화에 따른 움직임을 효과적으로 추정하기 위해서 다시점(multi-view) 얼굴영상의 SIFT 특징을 온라인으로 등록하여 활용하고 있다. 제안한 알고리즘의 이러한 강인성은 위 세 가지 중첩상황에 대해서 기존 알고리즘들과의 비교실험을 통해서 보여준다.

Keywords

References

  1. X. Gao, Y. Su, X. Li, and D. Tao, “A review of active appearance models,” IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.40, No.2, Dec. 2010.
  2. A. Samal and P. Iyengar, "Automatic Recognition and Analysis of Human Faces and Facial Expression: A Survey," Pattern Recognition, vol. 25(1), pp. 65-77, 1992. https://doi.org/10.1016/0031-3203(92)90007-6
  3. B. Fasel and J. Luettin, “Automatic Facial Expression Analysis: A Survey,” Pattern Recognition, Vol.36, pp. 259-275, 2003. https://doi.org/10.1016/S0031-3203(02)00052-3
  4. Y. Du, X. Lin, “Mapping emotional status to facial expressions,” Proceedings of 16th International Conference on Pattern Recognition, pp.524-527, August 2002.
  5. X. Li, C. Chang, S. Chang, “Face Alive Icons,” Journal of Visual Languages and Computing, Vol.18 , No.4, pp.440-453, 2007. https://doi.org/10.1016/j.jvlc.2007.02.008
  6. C. H. Lee, J. Wetzel, C. Y. Jang, Y. T, Shen, T. H. Chen, T. Selker, “Attention Meter: A Vision-based Input Toolkit for Interation Designers,” Conference on Human Factors in Computing Systems (CHI), pp.1007-1012, Montreal, Quebec, Canada, 2006.
  7. P. Ekman, “Facial expressions of emotion: an old controversy and new findings,” Philosophical Transactions: Biological Sciences, Vol.335, No.1273, pp.63-69, 1992. https://doi.org/10.1098/rstb.1992.0008
  8. P. Ekman, W. Friesen, and J. Hager, “Facial Action Coding System,” Tech. Report, Research Nexus, Network Research Information, Salt Lake City, UT, 2002.
  9. J. Cohn, T. Kanade, T. Moriyama, Z. Ambadar, J. Xiao, J. Gao, and H. Imamura, “A Comparative Study of Alternative Faces Coding Algorithms,” Tech. Report CMU-RI-TR-02-06, Robotics Institute, Carnegie Mellon University, Nov. 2001.
  10. T. F. Cootes, G. J. Edwards, and C. J. Taylor, “Active appearance models,” IEEE Trans. Pattern Anal. Mach. Intell., Vol.23, No.6, pp.681-685, Jun. 2001. https://doi.org/10.1109/34.927467
  11. I. Matthews and S. Baker, “Active appearance models revisited,” Int. J. Comput. Vis., Vol.60, No.2, pp.135–164, 2004. https://doi.org/10.1023/B:VISI.0000029666.37597.d3
  12. J. Xiao, S. Baker, I. Matthews, and T. Kanade, “Real-time combined 2D+3D active appearance models,” CVPR, 2004.
  13. R. Gross, I. Matthews, and S. Baker, “Constructing and fitting active appearance models with occlusion,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshops, Vol.5, pp.72, 2004.
  14. B. Theobald, I. Matthews, and S. Baker, “Evaluating error functions for robust active appearance model,” Proc. International Conference on Automatic Face and Gesture Recognition, pp.149-154, 2006.
  15. P. Mittrapiyanuruk, G. N. EdSouza, A. C. Kak, “Accurate 3D Tracking of Rigid Objects with Occlusion Using Active Appearance Models,” Proc. of the IEEE Workshop on Motion and Video Computing, pp.90-95, 2005.
  16. J. Sung, T. Kanade, D. Kim, “Pose robust face tracking by combining active appearance models and cylinder head models,” Int. J. Comput. Vis., Vol.80, No.2, pp.260-274, 2008. https://doi.org/10.1007/s11263-007-0125-1
  17. J. Xiao, T. Kanade and J. Cohn, “Robust full-motion recovery of head by dynamic templates and re-registration techniques,” Proc. International Conference on Automatic Face and Gesture Recognition, pp.156-162, 2002.
  18. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” IJCV, Vol.2, No.60, pp.91-110, 2004.
  19. J. Jang and T. Kanade, “Robust 3D Head Tracking by Online Feature Registration,” Proc. International Conference on Automatic Face and Gesture Recognition, 2008.
  20. M. Black and Y. Yacoob, “Recognizing facial expressions in image sequences using local parameterized models of image motion,” IJCV, Vol.25, No.1, pp.23-48, 1997. https://doi.org/10.1023/A:1007977618277
  21. S. Basu, I. Essa and A. Pentland, “Motion regularization for model-based head tracking,” in ICPR, pp. 611-616, 1996.
  22. M. La Cascia, S. Sclaroff and V. Athitsos, “Fast, reliable head tracking under varying illumination: An approach based on robust registration of texture-mapped 3D models,” IEEE Trans. PAMI, 2000.
  23. L. Lu, X.-T. Dai, G. Hager, “A particle filter without dynamics for robust 3D face tracking,” in CVPRW, pp.70, 2004.
  24. R. M. Murray, Z. Li, and S. S. Sastry, A Mathematical introduction to robotic manipulation, CRC Press, 1994.
  25. G. Aggarwal, A. Veeraraghavan, and R. Chellappa. “3D facial pose tracking in uncalibrated videos,” in PRMI, pp. 515-520, 2005.
  26. CMU Graphics Lab Motion Capture, http://mocap.cs.cmu.edu.
  27. Vicon, http://www.vicon.com