3D Facial Animation with Head Motion Estimation and Facial Expression Cloning

얼굴 모션 추정과 표정 복제에 의한 3차원 얼굴 애니메이션

  • Published : 2007.08.31


This paper presents vision-based 3D facial expression animation technique and system which provide the robust 3D head pose estimation and real-time facial expression control. Many researches of 3D face animation have been done for the facial expression control itself rather than focusing on 3D head motion tracking. However, the head motion tracking is one of critical issues to be solved for developing realistic facial animation. In this research, we developed an integrated animation system that includes 3D head motion tracking and facial expression control at the same time. The proposed system consists of three major phases: face detection, 3D head motion tracking, and facial expression control. For face detection, with the non-parametric HT skin color model and template matching, we can detect the facial region efficiently from video frame. For 3D head motion tracking, we exploit the cylindrical head model that is projected to the initial head motion template. Given an initial reference template of the face image and the corresponding head motion, the cylindrical head model is created and the foil head motion is traced based on the optical flow method. For the facial expression cloning we utilize the feature-based method, The major facial feature points are detected by the geometry of information of the face with template matching and traced by optical flow. Since the locations of varying feature points are composed of head motion and facial expression information, the animation parameters which describe the variation of the facial features are acquired from geometrically transformed frontal head pose image. Finally, the facial expression cloning is done by two fitting process. The control points of the 3D model are varied applying the animation parameters to the face model, and the non-feature points around the control points are changed by use of Radial Basis Function(RBF). From the experiment, we can prove that the developed vision-based animation system can create realistic facial animation with robust head pose estimation and facial variation from input video image.


  1. J. Chai, J. Xiao, and J. Hodgins, 'Vision-based Control of 3D Facial Animation', In Proc. ACM/Eurographics Symposium on Computer Animation, pp. 193-206, 2003
  2. X. Wei, Z. Zhu, L. Yin, and Q. Ji, 'A Real Time Face Tracking and Animation System', IEEE Conference on CVPR 2004, Vol. 5, pp. 71-78, 2004
  3. J.Y. Noh and U. Neumann, 'Expression Cloning', Computer Graphics, Proceedings of ACM SIGGRAPH, pp. 277-288, 2001
  4. Z. Liu and Z. Ahang, 'Robust Head Motion Computation by Taking Advantage of Physical Properties', Proc. HUMO2000, 2000
  5. D. DeCarlo and D. Metaxas, 'The Integration of Optical Flow and Deformable Models with Applications to Human Face Shape and Motion Estimation', CVPR96, pp. 231-238, 1996
  6. G.D. Hager and P.N. Belhumeur, 'Efficient Region Tracking with Parametric Models of Geometry and Illumination', PAMI, vol. 20, no. 10, pp. 1025-1039, 1998
  7. M.L. Cascia and S. Sclaroff, 'Fast, Reliable Head Tracking under Varying Illumination', CVPR99, pp. 604-610, 1999
  8. J. Ostermann, 'Animation of Synthetic Faces in MPEG-4', Computer Animation, pp. 49-51, 1998
  9. C. Chien, Y. Chang, and Y. Chen, 'Facial Expression Analysis under Various Head Poses', In Proc. the Third IEEE Pacific Rim Conference on Multimedia, Vol. 2532, pp. 1193-1200, 2002
  10. A. Wojdel, L. J. M. Rothkrantx, 'Parametric Generation of Facial Expressions Based on FACS', Computer Graphic Forum, Vol. 24, pp. 743-757, 2005
  11. F. Pighin, J. heccker, D. Lischinski, R. Szeliski, and D. Salesin, 'Synthesizing Realistic Facial Expression from Photographs', Proc. Computer Graphics SIGGRAPH, pp. 231-242, 1998
  12. D. Terzopoulos and K. Waters, 'Physically-based Facial Modeling, Analysis, and Animation', J. Visualization and Computer Animation, pp. 73-80, 1990
  13. J. Chun, K Min, 'A Realistic Human Face Modeling from Photographs by Use of Skin Color and Model Deformation,'LNCS 3480, pp 1135-1143, 2005
  14. J.L, Barron and N.A. Thacker, 'Tutorial: Computing 2D and 3D Optical Flow', Tina Memo No. 2004-012, 2005