Journal of International Society for Simulation Surgery

International Society for Simulation Surgery (국제컴퓨터가상수술학회)
권호 표지
DOI QR코드

DOI QR Code

Personalized Face Modeling for Photorealistic Synthesis

  • Kim, Kyungmin (School of Integrated Technology, Yonsei Institute Convergence Technology, Yonsei University) ;
  • Shim, Hyunjung (School of Integrated Technology, Yonsei Institute Convergence Technology, Yonsei University)
  • Received : 2015.10.30
  • Accepted : 2015.11.12
  • Published : 2015.12.19
Download PDF
⟨ Previous Next ⟩

Abstract

Faces play a key role in revealing the personalized attributes such as the identity, emotion, health condition, etc. Due to the importance of faces, computer-assisted face modeling and reconstruction have been actively studied both in computer vision and graphics community. Especially, face reconstruction and realistic face synthesis are well-grounded research problems and various approaches have been proposed during the last decade. In this paper, we discuss a wide range of existing work in face modeling by introducing their target applications, categorizing them upon their methodology and addressing their strength and weakness on performance. Finally, we introduce remaining research issues and suggest the future research direction in face modeling. We believe that this paper provides a high-level overview on face modeling techniques and helps understand the major research issues and the trends of methodology.

Keywords

References

  1. https://developer.nvidia.com/faceworks. 2013;
  2. Alexander O, Fyffe G, Busch J, Yu X, Ichikari R, Jones A, et al. Digital Ira: Creating a Real-Time Photoreal Digital Actor. ACM SIGGRAPH Posters;2013
  3. Basri R, Jacobs D, Kemelmacher I. Photometric Stereo with General, Unknown Lighting. International Journal of Computer Vision 2007;72:239-257 https://doi.org/10.1007/s11263-006-8815-7
  4. Belhumeur PN, Hespanha JP, Kriegman DJ. Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 1991;19:711-720
  5. Blanz V, Vetter T. A morphable model for the synthesis of 3D faces. ACM SIGGRAPH 1999:187-194
  6. Bottino A, De Simone M, Laurentini A, Sforza C. A New 3-D Tool for Planning Plastic Surgery. Biomedical Engineering, IEEE Transactions on 2012;59:3439-3449 https://doi.org/10.1109/TBME.2012.2217496
  7. Debevec P, Hawkins T, Tchou C, Duiker HP, Sarokin W, Sagar M. Acquiring the Reflectance Field of a Human Face. Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques 2000:145-156
  8. Erdogmus N, Marcel S. Spoofing in 2D face recognition with 3D masks and anti-spoofing with Kinect. Biometrics: Theory, Applications and Systems (BTAS), 2013 IEEE Sixth International Conference on 2013:1-6
  9. Fyffe G, Jones A, Alexander O, Ichikari R, Debevec P. Driving High-Resolution Facial Scans with Video Performance Capture. ACM Trans Graph 2014;34:8:1-14
  10. Gao W, Cao B, Shan S, Chen X, Zhou D, Zhang X, Zhao D. The CASPEAL Large-Scale Chinese Face Database and Baseline Evaluations. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on 2008;38:149-161 https://doi.org/10.1109/TSMCA.2007.909557
  11. Georghiades AS, Belhumeur PN, Kriegman D. From few to many: illumination cone models for face recognition under variable lighting and pose. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2001;23:643-660 https://doi.org/10.1109/34.927464
  12. Graham DB, Allinson NM. Characterising Virtual Eigensignatures for General Purpose Face Recognition. Face Recognition. Springer Berlin Heidelberg 1998:446-456
  13. Gross R, Matthews I, Cohn J, Kanade T, Baker S. Multi-PIE. Image and Vision Computing 2010;28:807-813 https://doi.org/10.1016/j.imavis.2009.08.002
  14. Gupta S, Markey M, Bovic A. Anthropometric 3D Face Recognition. International Journal of Computer Vision 2010;90:331-349 https://doi.org/10.1007/s11263-010-0360-8
  15. Hammond P. The use of 3D face shape modelling in dysmorphology. Archives of Disease in Childhood 2007;92:1120-1126 https://doi.org/10.1136/adc.2006.103507
  16. Hassner T. Viewing Real-World Faces in 3D. Computer Vision (ICCV), 2013 IEEE International Conference on. 2013:
  17. Hwang BW, Roh MC, Lee SW. Performance evaluation of face recognition algorithms on Asian face database. Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on 2004:278-283
  18. Jafri R, Arabnia HR. A Survey of Face Recognition Techniques. Journal of Information Processing Systems 2009;5:41-68 https://doi.org/10.3745/JIPS.2009.5.2.041
  19. Jesorsky O, Kirchberg KJ, Frischholz R. Robust Face Detection Using the Hausdorff Distance. Proceedings of the Third International Conference on Audio- and Video-Based Biometric Person Authentication. Springer-Verlag 2001:90-95
  20. Kanade T, Cohn JF, Tian YL. Comprehensive database for facial expression analysis. Automatic Face and Gesture Recognition, 2000. Proceedings. Fourth IEEE International Conference on. 2000:46-53
  21. Kemelamacher I, Seitz SM. Face reconstruction in the wild. Computer Vision (ICCV), 2011 IEEE International Conference on 2011:1746-1753
  22. Koch RM, Gross MH, Carls FR, von Buren DF, Fankhauser G, Parish YIH. Simulating Facial Surgery Using Finite Element Models. Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques 1996:421-428
  23. Kovacs L, Zimmermann A, Wawrzyn H, Schwenzer K, Seitz H, Tille C, et al. Computer aided surgical reconstruction after complex facial burn injuries - opportunities and limitations. Burns 2005;31:85-91 https://doi.org/10.1016/j.burns.2004.07.010
  24. Lee TY, Sum YN, Lin YC, Lin LW, Lee CN. Three-dimensional facial model reconstruction and plastic surgery simulation. Information Technology in Biomedicine, IEEE Transactions on 1999;3:214-220 https://doi.org/10.1109/4233.788583
  25. Marszalec E, Martinkauppi B, Soriano M, Pietikainen M. Physicsbased face database for color research. Journal of Electronic Imaging 2000;9:32-38 https://doi.org/10.1117/1.482722
  26. Martinez A, Benavente R. The AR face database. Rapport Technique 24 1998.
  27. Mavadati SM, Mahoor MH, Bartlett K, Trinh P, Cohn JF. DISFA: A Spontaneous Facial Action Intensity Database. Affective Computing, IEEE Transactions on 2013;4:151-160 https://doi.org/10.1109/T-AFFC.2013.4
  28. Messer K, Matas J, Kittler J, Jonsson K. XM2VTSDB: The Extended M2VTS Database. In Second International Conference on Audio and Video-based Biometric Person Authentication 1999:72-77
  29. Peer P, Batagelj B, Kovac J, Solina F. Color-Based Face Detection in the "15 Seconds of Fame" Art Installation 2003
  30. Phillips PJ, Moon HJ, Rizvi SA, Rauss PJ. The FERET evaluation methodology for face-recognition algorithms. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2000;22:1090-1104 https://doi.org/10.1109/34.879790
  31. Reichardt J. Robots: Fact, Fiction, and Prediction. Penguin Books 1978
  32. Roth J, Tong Y, Liu X. Unconstrained 3D face reconstruction. Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on 2015:2606-2615
  33. Shim H. Probabilistic Approach to Realistic Face Synthesis With a Single Uncalibrated Image. Image Processing, IEEE Transactions on 2012;21:3784-3793 https://doi.org/10.1109/TIP.2012.2192125
  34. Sim T, Baker S, Bsat M. The CMU pose, illumination, and expression database. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2003;25:1615-1618 https://doi.org/10.1109/TPAMI.2003.1251154
  35. Taigman Y, Ming Y, Ranzato M, Wolf L. DeepFace: Closing the Gap to Human-Level Performance in Face Verification 2014:1701-1708
  36. Turk M, Pentland A. Eigenfaces for Recognition. Journal of Cognitive Neuroscience 1991:71-86
  37. Vasilescu MAO, Terzopoulos D. Multilinear analysis of image ensembles: TensorFaces. IEEE Euroupean Conference on Computer Vision (ECCV) 2002:447-460
  38. Wang Y, Liu Z, Hua G, Wen Z, Zhang Z, Samaras D. Face re-lighting from a single image under harsh lighting conditions. IEEE CVPR 2007:1-8
  39. Wenger A, Gardner A, Tchou C, Unger J, Hawkins T, Debevec P. Performance Relighting and Reflectance Transformation with Timemultiplexed Illumination. ACM Trans. Graph 2005;24:756-764 https://doi.org/10.1145/1073204.1073258
  40. Weyrich T, Matusik W, Pfister H, Bickel B, Donner C, Tu Chien, et al. Analysis of Human Faces Using a Measurement-based Skin Reflectance Model. ACM Trans. Graph 2006;25:1013-1024 https://doi.org/10.1145/1141911.1141987
  41. Wu L, Ganesh A, Shi B, Matsushita Y, Wang Y, Ma Y. Robust Photometric Stereo via Low-Rank Matrix Completion and Recovery. Computer Vision ACCV 2010. Springer Berlin Heidelberg 2011:703-717
  42. Yang C, Chen J, Su N, Su G. Improving 3D Face Details Based on Normal Map of Hetero-source Images. Computer Vision and Pattern Recognition Workshops (CVPRW), 2014 IEEE Conference on 2014:9-14
  43. Yin L, Chen X, Sun Y, Worm T, Reale M. A high-resolution 3D dynamic facial expression database. Automatic Face Gesture Recognition, 2008. FG '08. 8th IEEE International Conference on 2008:1-6
  44. Yin L, Wei X, Sun Y, Wang J, Rosato MJ. A 3D facial expression database for facial behavior research. Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference on 2006:211-216
  45. Zhang J, Luo S. Image-Based Augmented Reality Model for Image- Guided Surgical Simulation. Medical Imaging and Informatics. Springer Berlin Heidelberg 2008:330-339
  46. Zhang L, Wang S, Samaras D. Face synthesis and recognition under arbitrary unknown lighting using a spherical harmonic basis morphable model. IEEE CVPR 2005:209-216