The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

Robust Real-time Tracking of Facial Features with Application to Emotion Recognition

안정적인 실시간 얼굴 특징점 추적과 감정인식 응용

  • Received : 2013.05.11
  • Accepted : 2013.07.25
  • Published : 2013.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Facial feature extraction and tracking are essential steps in human-robot-interaction (HRI) field such as face recognition, gaze estimation, and emotion recognition. Active shape model (ASM) is one of the successful generative models that extract the facial features. However, applying only ASM is not adequate for modeling a face in actual applications, because positions of facial features are unstably extracted due to limitation of the number of iterations in the ASM fitting algorithm. The unaccurate positions of facial features decrease the performance of the emotion recognition. In this paper, we propose real-time facial feature extraction and tracking framework using ASM and LK optical flow for emotion recognition. LK optical flow is desirable to estimate time-varying geometric parameters in sequential face images. In addition, we introduce a straightforward method to avoid tracking failure caused by partial occlusions that can be a serious problem for tracking based algorithm. Emotion recognition experiments with k-NN and SVM classifier shows over 95% classification accuracy for three emotions: "joy", "anger", and "disgust".

Keywords

Acknowledgement

Supported by : Ministry of Education, Science and Technology

References

  1. T. F. Cootes, C. J. Taylor, D. H. Cooper, J. Graham, "Active Shape Models-Their Training and Application", Computer Vision Image Understand, Vol. 61, No. 9, pp. 38-59, 1995. https://doi.org/10.1006/cviu.1995.1004
  2. T. F. Cootes, G. Edwards, C. Taylor. "Active appearance models", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 6, pp. 681-685, 2001. https://doi.org/10.1109/34.927467
  3. M. Zhou, L. Liang, J. Sun, Y. Wang, "AAM based Face Tracking with Temporal Matching and Face Segmentation", IEEE Conference on Computer Vision and Pattern Recognition, pp. 701-708, 2010.
  4. J. K. Kearney, W. B. Thompson, "Optical Flow Estimation: An Error Analysis of Gradient-based Methods with Local Optimization", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 9, No. 2, pp. 229-244, 1987.
  5. M. Stephen, N. Fred, "Locating facial features with an extended active shape model", Proceedings of the 10th European Conference on Computer Vision. Vol. 4, pp. 504-513, 2008.
  6. M. Kass, A. Witkin, D. Terzopoulos, "Snakes: Active Contour Models", International Journal of Computer Vision, Vol. 1, No. 4, pp. 321-331, 1988. https://doi.org/10.1007/BF00133570
  7. R. Gross, I. Matthews, S. Baker, "Active Appearance Models with Occlusion", Image and Vision Computing, Vol. 24, No. 6, pp. 593-604, 2006. https://doi.org/10.1016/j.imavis.2005.08.001
  8. Antonio G., Sorci M., Bierlaire M., Thiran J., "Discrete choice models for static facial expression recognition", Proceedings of the 8th International Conference on Advanced Concepts for Intelligent Vision Systems, Vol. 4179, pp. 710-721, 2006.
  9. Zhang Y., Ji Q., "Facial expression understanding in image sequences using dynamic and active visual information fusion", Proceedings of the 9th International Conference on Computer Vision, Vol. 2, pp. 1297-1304, 2003.
  10. Bimler D., Paramei G., "Facial-expression affective attributes and their configural correlates: components and categories", Spanish Journal of Psychology, Vol. 9, pp. 19-31, 2006. https://doi.org/10.1017/S113874160000593X
  11. M. Pantic and L. J. M. Rothkrantz, "Case-based reasoning for user-profiled recognition of emotions from face images," Proceedings of the International Conference on Multimedia, pp. 391-394, 2004.
  12. M. Valstar, M. Pantic, Z. Ambadar and J. F. Cohn, "Spontaneous versus posed facial behavior: Automatic analysis of brow actions", Proceedings of the International Conference on Multimodal Interfaces, pp. 162-170, 2006.

Cited by

  1. Path Estimation Method in Shadow Area Using Underwater Positioning System and SVR vol.12, pp.2, 2017, https://doi.org/10.7746/jkros.2017.12.2.173
  2. 윤곽선의 선분 근사화를 활용한 양팔 화가 로봇의 개발 vol.9, pp.3, 2013, https://doi.org/10.7746/jkros.2014.9.3.140
  3. 다시점 객체 공분할을 이용한 2D-3D 물체 자세 추정 vol.12, pp.1, 2013, https://doi.org/10.7746/jkros.2017.12.1.033