한국정보과학회논문지:소프트웨어및응용 (Journal of KIISE:Software and Applications)

한국정보과학회 (Korean Institute of Information Scientists and Engineers)
권호 표지

역전가능 메쉬워프 알고리즘에 의한 정면 얼굴 영상의 포즈 변형

Pose Transformation of a Frontal Face Image by Invertible Meshwarp Algorithm

  • 발행 : 2003.02.01
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

본 논문에서는 기하학적인 3차원 모델을 사용하지 않고 정면이 얼굴 영상 및 2차원 메쉬만으로 얼굴의 포즈 변형을 수행하는 영상기반 렌더링(Image Based Rendering; IBR) 기법을 제안한다. 3차원 기하학적 모델을 대신하기 위해, 먼저 표준 인물의 정면, 좌우 반측면, 좌우 측면의 얼굴 영상에 대한 표준 메쉬를 작성한다. 합성하고자 하는 임의의 인물에 대해서는 주어진 정면 얼굴 영상의 메쉬만을 작성하고, 그 밖의 메쉬는 표준 메쉬 집합을 근거로 자동 생성된다. 그런 다음, 메쉬 제어점들의 중첩 및 역전을 허용하도록 개선한 역전가능 메쉬워프 알고리즘(invertible meshwarp algorithm)을 이용하여 얼굴의 입체적인 회전 변형을 수행한다. 또한, 눈이나 입의 개폐 변형도 동일한 워핑 알고리즘으로 구현한다. 얼굴 변형 성능을 평가하기 위해, 총 10명으로부터 머리를 수평으로 회전하면서 동영상을 취득한 후, 실제 영상과 변형 영상마다 양 눈의 중간 위치인 기준점에서 각 특징점까지의 거리를 계산하여 평균 차이를 구하였다. 그 결과, 기준점에서 입의 중간 위치까지의 거리에 비해 약 7.0%의 평균 위치 오차만이 발생하였다.

In this paper, we propose a new technique of image based rendering(IBR) for the pose transformation of a face by using only a frontal face image and its mesh without a three-dimensional model. To substitute the 3D geometric model, first, we make up a standard mesh set of a certain person for several face sides ; front. left, right, half-left and half-right sides. For the given person, we compose only the frontal mesh of the frontal face image to be transformed. The other mesh is automatically generated based on the standard mesh set. And then, the frontal face image is geometrically transformed to give different view by using Invertible Meshwarp Algorithm, which is improved to tolerate the overlap or inversion of neighbor vertexes in the mesh. The same warping algorithm is used to generate the opening or closing effect of both eyes and a mouth. To evaluate the transformation performance, we capture dynamic images from 10 persons rotating their heads horizontally. And we measure the location error of 14 main features between the corresponding original and transformed facial images. That is, the average difference is calculated between the distances from the center of both eyes to each feature point for the corresponding original and transformed images. As a result, the average error in feature location is about 7.0% of the distance from the center of both eyes to the center of a mouth.

키워드

참고문헌

  1. F. I. Parke and K. Waters, Computer Facial Animation, A. K. Peters, Ltd., 1996
  2. P. Ekman and W. V. Friesen, Manual for the Facial Action Coding System, Consulting Psychologist Press, Inc., 1978
  3. Y. Lee, D. Terzopoulos, and K. Waters, 'Realistic Modeling for Facial Animation,' Proc. of SIGGRAPH'95, pp.55-62, Aug. 1995 https://doi.org/10.1145/218380.218407
  4. Z. Liu, Z. Zhang, C. Jacobs, and M. Cohen, 'Raped Modeling of Animated Faces from Video Images,' Proc. of ACM Multimedia 2000, pp.475-476, 2000
  5. F. Pighin, J. Auslander, D. Lischinski, and D. Salesin, Realistic Facial Animation Using Image-Based 3D Morphing, Technical Reprot UW-CSE-97-01-03,1997
  6. S. B. Kang, 'A Survey of Image-based Rendering Techniques', Videometrics VI(SPIE Int'l Symp. on Electronic Imaging: Science and Technology), Vol. 3641, pp.2-16, Jan. 1999 https://doi.org/10.1117/12.333774
  7. T. Vetter and T. Poggio, 'Linear Object Classes and Image Synthesis from a Single Example Image,' IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.19, No.7, pp.733-742, Jul. 1997 https://doi.org/10.1109/34.598230
  8. L. McMillan and G. Bishop, 'Plenoptic Modeling:An Image-Based Rendering System,' Proc. of SIGGRAPH 95, pp.39-46, Aug. 1995
  9. R. Szelicki and H.-Y. Shum, 'Creating Full View Panoramic Image Mosaics and Environment Maps,' Porc. of SIGGRAPH'97, pp.251-258, Aug. 1997 https://doi.org/10.1145/258734.258861
  10. M. Levoy and P. Harahan, 'Light Field Rendering,' Proc. of SIGGRAPH'96, pp.31-42, Aug. 1996 https://doi.org/10.1145/237170.237199
  11. S. J. Gortler, R. Grzezczuk, R. Szeliski and M. F. Cohen, 'The Lumigraph,' Proc. of SIGGRAPH'96, pp.43-54, Aug. 1996 https://doi.org/10.1145/237170.237200
  12. S. M. Seitz and C. R. Dyer, 'View Morphing,' Proc. of SIGGRAPH'96, pp.21-30, Aug. 1996 https://doi.org/10.1145/237170.237196
  13. S. Avidan and A. Shashua, 'Novel View Synthesis in Tensor Space,' Proc. of Int'l Coft. on Computer Vision and Pattern Recognition, pp.1034-1040, Jun, 1997 https://doi.org/10.1109/CVPR.1997.609457
  14. P. E. Debevec, C. J. Taylor, and J. Malik, 'Modeling and Rendering Architecture from Photographs: A Hybrid Geometry and Image-Based Approach,' Proc. of SIGGRAPH'96, pp.11-20, Aug. 1996 https://doi.org/10.1145/237170.237191
  15. Jonas Gomes, Lucia Darsa, Bruno Costa and Luiz Velho, Warping and Morphing of Graphical Objects, Morgan Kaufmann Pub., 1998
  16. Randy Crane, Simplified Approach ro Image Processing, Prentice-Hall PTR, 1997