The Journal of the Korea Contents Association (한국콘텐츠학회논문지)

The Korea Contents Association (한국콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

3D Face Modeling based on 3D Morphable Shape Model

3D 변형가능 형상 모델 기반 3D 얼굴 모델링

  • 장용석 (숭실대학교 정보통신전자공학부) ;
  • 김부균 (숭실대학교 정보통신전자공학부) ;
  • 조성원 (홍익대학교 지능정보처리 연구실) ;
  • 정선태 (숭실대학교 정보통신전자공학부)
  • Published : 2008.01.28
Download PDF
⟨ Previous Next ⟩

Abstract

Since 3D face can be rotated freely in 3D space and illumination effects can be modeled properly, 3D face modeling Is more precise and realistic in face pose, illumination, and expression than 2D face modeling. Thus, 3D modeling is necessitated much in face recognition, game, avatar, and etc. In this paper, we propose a 3D face modeling method based on 3D morphable shape modeling. The proposed 3D modeling method first constructs a 3D morphable shape model out of 3D face scan data obtained using a 3D scanner Next, the proposed method extracts and matches feature points of the face from 2D image sequence containing a face to be modeled, and then estimates 3D vertex coordinates of the feature points using a factorization based SfM technique. Then, the proposed method obtains a 3D shape model of the face to be modeled by fitting the 3D vertices to the constructed 3D morphable shape model. Also, the proposed method makes a cylindrical texture map using 2D face image sequence. Finally, the proposed method builds a 3D face model by rendering the 3D face shape model with the cylindrical texture map. Through building processes of 3D face model by the proposed method, it is shown that the proposed method is relatively easy, fast and precise than the previous 3D face model methods.

3D 얼굴 모델링은 33공간에서 얼굴을 자유롭게 회전 시켜 다양한 얼굴 자세를 표현하고 조명 효과도 적절하게 모델링 할 수 있으므로, 얼굴 자세, 조명, 표정 등의 표현에 있어서 2D 얼굴 모델링에 비해 보다 정교하며 사실감이 뛰어나 얼굴 인식, 게임, 아바타 등에서 많은 요구가 존재한다. 본 논문에서는 3D 변형 가능 형상 모델에 기반을 둔 3D 얼굴 모델링 방법을 제안한다. 제안된 3D 얼굴 모델링 방법은 먼저 3D 스캐너를 통하여 획득한 3D 얼굴 스캔 데이터를 이용하여 3D 얼굴 변형 가능 형상 모델을 구축한다. 다음, 3D 얼굴 모델링을 하고자 하는 얼굴의 2D 이미지 시퀀스로부터, 해당 얼굴의 특징점들을 검출하고 이들을 매칭하여, 매칭된 특징점들로부터 인수분해 기반 SfM 기법을 이용하여 해당 특징점의 3D 버텍스 좌표 값을 구한다. 이후, 구한 3D 버텍스들을 3D 변형 가능 형상 모델에 정합하여 해당 얼굴의 3D 형상 모델을 얻는다. 또한, 2D 얼굴 이미지 시퀀스들로부터 뷰 독립적인 2D 원통 좌표 텍스쳐 맵을 구하고 이를 이용하여 3D 형상 모델을 렌더링 함으로써, 최종적으로 3B 얼굴 모델을 완성한다. 제안된 3D 얼굴 모델링 방법에 의한 3D 얼굴 모델 생성 과정을 통해서, 본 논문에서 제안한 3D 얼굴 모델링 방법이 기존의 얼굴 모델링 방법들에 비해 상대적으로 빠르고 비교적 정교하게 수행됨을 볼 수 있었다.

Keywords

References

  1. W. Zhao and R. Chellappa, Face Processing: Advanced Modeling and Methods, Elsevier, 2005.
  2. Y. Lee, D. Terzopuolos, and K. Waters, "Realistic Modeling for Facial Animation," Proc. SIGGRAPH, Los Angeles, pp.55-61, 1995(8).
  3. F. I. Parke and K. Waters, "Appendix 1: Three-dimensional muscle model facial animation," Computer Facial Animation, A.K. Peters, 1996(9).
  4. F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. H. Salesin, "Synthesizing realistic facial expressions from photographs," In Computer Graphics, Annual Conference Series, SIGGRAPH, pp.75-84, 1998(7).
  5. V. Blanz and T. Vetter, "A Morphable Model for the Synthesis of 3D Faces," Proc. of the SIGGRAPH'99, Los Angeles, USA, pp.187-194, 1999(8).
  6. V. Blanz and T. Vetter, "Face Recognition Based on Fitting a 3D Morphable Model," IEEE Pattern Anal. Mach. Intell, Vol.25, No.9, pp. 1063-1074, 2003(9). https://doi.org/10.1109/TPAMI.2003.1227983
  7. J. Ahlberg, "CANDIDE-3 -- an updated parameterized face," Report No. LiTH-ISY -R-2326, Dept. of Electrical Engineering, $Link\ddot{o}ping$ University, Sweden, 2001.
  8. R. L. Hsu and A. K. Jain, "Face Modeling for Recognition," Proc. Int'l Conf. Image Processing(ICIP), Vol.2, pp.693-696, 2001.
  9. A. Ansari and M. Abdel-Mottaleb, "3-D Face Modeling Using Two Views and a Generic Face Model with Application to 3-D Face Recognition," IEEE Conf. on Advanced Video and Signal Based Surveillance, pp.203-222, 2003. https://doi.org/10.1109/AVSS.2003.1217899
  10. Y. Hu, D. Jiang, S. Yan, L. Zhang, and H. zhang, "Automatic 3D reconstruction for face recognition," Proc. 6th IEEE Int'l Conf. on Automatic Face and Gesture Recognition, pp.843-848, 2004. https://doi.org/10.1109/AFGR.2004.1301639
  11. Z. Mandun, M. Linna, X. Y. Zeng, and Y. S. Wang, "Image-Based 3D Face Modeling," Proc. of Int'l Conf. on Computer Graphics, Imaging and Visualization, pp.165-168, 200(7).
  12. H. Guo, J. Jiang, and L. Zhang, "Building a 3D morphable face model by using thin plate splines for face reconstruction," LNCS Vol. 3338, pp.258-267, 2004.
  13. Z. Zhang, Z. Liu, D. Adler, M. F. Cohen, E. Hanson, and Y. Shan, "Robust and Rapid Generation of Animated Faces from Video Images: A Model-Based Modeling Approach," International Journal of Computer Vision, Vol.58, No.2, pp.93-119, 2004(6).
  14. T. Russ, C. Boehnen, and T. Peters, "3D Face Recognition Using 3D Alignment for PCA," IEEE Conf. on Computer Vision and Pattern Recognition, Vol.2, pp.1391-1398, 2006. https://doi.org/10.1109/CVPR.2006.13
  15. http://www.cyberware.com/
  16. M. B. Stegmann, B. Mikkel, Gomez, and D. David : A Brief Introduction to Statistical Shape Analysis Technical University of Denmark, Lyngby, 2002.
  17. T. F. Cootes, D. J. Edwards, and S. J. Taylor, "Active Appearance Models," IEEE Trans. Pattern Anal. Mach. Intell, Vol.23, No.6, pp.681-685, 2001(6). https://doi.org/10.1109/34.927467
  18. 정선태, "점진적 AAM을 이용한 강인한 얼굴 윤곽 검출", 한국컨텐츠학회논문지, pp.11-20, 2007(2). https://doi.org/10.5392/JKCA.2007.7.2.011
  19. P. Besl and McKay, N. "A Method for Registration of 3-D Shapes," Trans. PAMI, Vol.14, No.2, 1992.
  20. S. Rusinkewicz and M. Levoy, "Efficient Variants of the ICP Algorithm," Third International Conference on 3D Digital Imaging and Modeling, pp.145-152, 2001(6).
  21. B. Brown and S. Rusinkiewicz, "Non-Rigid Range-Scan Alignment Using Thin-Plate Splines," Symposium on 3D Data Processing, Visualization and Transmission, Vol.6, No.9, pp.759-765, 2004(9).
  22. F. L. Bookstein, "Principal warps: Thin-plate splines and the decomposition of deforma tions," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.11, No.6, pp.567-585, 1989(6). https://doi.org/10.1109/34.24792
  23. C. Brechbuhler, G. Gerig, and O. Kubler, O.: "Parameterization of Closed Surfaces for 3-D Shape Description," Comp. Vision and Image Understanding, Vol.61, Issue.2, pp.154-170. 1995. https://doi.org/10.1006/cviu.1995.1013
  24. R. Davies, C. Twining, T. Cootes, J. Waterton, and C. Taylor. "A minimum description length approach to statistical shape modeling," IEEE Transactions on Medical Imaging, Vol.5, Issue.5, pp.525-537, 2002(5). https://doi.org/10.1109/TMI.2002.1009388
  25. R. Hartley and A. Zisserman, Multiple Geometry in Compution Vision, 2nd ed. Cambridge University Press, 2003.
  26. C. Tomasi and T. Kanade, "Shape and motion from image streams under orthography: A factorization method," International Journal of Computer Vision, Vol.9, No.2, pp.137-154, 1992. https://doi.org/10.1007/BF00129684
  27. C. J. Poelman and T. Kanade, "A Paraperspective Factorization Method for Shape and Motion Recovery," IEEE PAMI, Vol.19, No.3, 1997(3). https://doi.org/10.1109/34.584098
  28. S. Mahamud and M. Hebert, "Iterative projective reconstruction from multiple views," Proc. IEEE Conf. Computer Vision and Pattern Recognition, Vol.2, pp.430-437, 2000. https://doi.org/10.1109/CVPR.2000.854872
  29. T. Jebara, A. Azarbayejani, and A. Pentland, "3D structure from 2D motion," IEEE Signal Processing Magazine, Vol.16, Issue.3, pp.66-84, 1999(5). https://doi.org/10.1109/79.768574
  30. L. Torresani, A. Hertzmann, and C. Bregler, "Learning Non-Rigid 3D Shape from 2D Motion," Proc. Of Neural Information Processing Systems, 2003.