Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Neural network for automatic skinning weight painting using SDF

SDF를 이용한 자동 스키닝 웨이트 페인팅 신경망

  • Hyoseok Seol (Dept. of Computer and Software, Hanyang University) ;
  • Taesoo Kwon (Dept. of Computer and Software, Hanyang University)
  • 설효석 (한양대학교 일반대학원 컴퓨터소프트웨어학과) ;
  • 권태수 (한양대학교 일반대학원 컴퓨터소프트웨어학과)
  • Received : 2023.06.29
  • Accepted : 2023.08.16
  • Published : 2023.09.01
Download PDF
⟨ Previous Next ⟩

Abstract

In computer graphics and computer vision research and its applications, various representations of 3D objects, such as point clouds, voxels, or triangular meshes, are used depending on the purpose. The need for animating characters using these representations is also growing. In a typical animation pipeline called skeletal animation, "skinning weight painting" is required to determine how joints influence a vertex on the character's skin. In this paper, we introduce a neural network for automatically performing skinning weight painting for characters represented in various formats. We utilize signed distance fields (SDF) to handle different representations and employ graph neural networks and multi-layer perceptrons to predict the skinning weights for a given point.

컴퓨터 그래픽스 및 컴퓨터 비전 분야의 발전에 따라 삼차원 물체를 다양한 표현 방식으로 나타내고 있다. 이에 따라 여러 표현 방식을 사용하는 캐릭터의 애니메이션 제작에 대한 수요 또한 증가하고 있다. 캐릭터 애니메이션 제작에 주로 사용되는 스켈레탈 애니메이션의 경우 캐릭터 표면이 어느 관절로부터 영향을 받는지를 정하는 스키닝 웨이트 페인팅 작업이 필요하다. 본 논문은 삼각형 메시를 비롯한 여러 표현방식으로 나타난 캐릭터에 대한 스키닝 웨이트 페인팅 과정을 자동화하는 방법을 제안한다. 우선 다양한 표현 방식을 사용한 삼차원 캐릭터에 대해 일반적으로 사용할 수 있도록 Signed Distance Field(SDF)를 이용한다. 이후 그래프 신경망과 다층 퍼셉트론 계층 구조를 활용하여 캐릭터 표면 상에 주어진 위치에서의 스키닝 웨이트를 예측할 수 있다.

Keywords

Acknowledgement

이 성과는 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임(NRF-2020R1A2C1012847).

References

  1. T. Magnenat, R. Laperriere, and D. Thalmann, "Joint dependent local deformations for hand animation and object grasping," Canadian Inf. Process. Soc," Report, 1988. 
  2. L. Kavan, S. Collins, J. Zara, and C. O'Sullivan, "Skinning with dual quaternions," in Proceedings of the 2007 symposium on Interactive 3D graphics and games, 2007, Conference Proceedings, pp. 39-46. 
  3. I. Baran and J. Popovi'c, "Automatic rigging and animation of 3d characters," ACM Transactions on graphics (TOG), vol. 26, no. 3, pp. 72-es, 2007. 
  4. R. Wareham and J. Lasenby, "Bone glow: An improved method for the assignment of weights for mesh deformation," in Articulated Motion and Deformable Objects: 5th International Conference, AMDO 2008, Port d'Andratx, Mallorca, Spain, July 9-11, 2008. Proceedings 5. Springer, 2008, Conference Proceedings, pp. 63-71. 
  5. A. Jacobson, I. Baran, J. Popovic, and O. Sorkine, "Bounded biharmonic weights for real-time deformation," ACM Trans. Graph., vol. 30, no. 4, p. 78, 2011. 
  6. O. Dionne and M. de Lasa, "Geodesic voxel binding for production character meshes," in Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 2013, Conference Proceedings, pp. 173-180. 
  7. L. Liu, Y. Zheng, D. Tang, Y. Yuan, C. Fan, and K. Zhou, "Neuroskinning: Automatic skin binding for production characters with deep graph networks," ACM Transactions on Graphics (TOG), vol. 38, no. 4, pp. 1-12, 2019.  https://doi.org/10.1145/3306346.3322969
  8. Z. Xu, Y. Zhou, E. Kalogerakis, C. Landreth, and K. Singh, "Rignet: Neural rigging for articulated characters," arXiv preprint arXiv: 2005.00559, 2020. 
  9. A. Mosella-Montoro and J. Ruiz-Hidalgo, "Skinningnet: Two-stream graph convolutional neural network for skinning prediction of synthetic characters," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, Conference Proceedings, pp. 18 593-18 602. 
  10. X. Chen, Y. Zheng, M. J. Black, O. Hilliges, and A. Geiger, "Snarf: Differentiable forward skinning for animating non-rigid neural implicit shapes," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, Conference Proceedings, pp. 11 594-11 604. 
  11. P. Li, K. Aberman, R. Hanocka, L. Liu, O. Sorkine-Hornung, and B. Chen, "Learning skeletal articulations with neural blend shapes," ACM Transactions on Graphics (TOG), vol. 40, no. 4, pp. 1-15, 2021.  https://doi.org/10.1145/3476576.3476702
  12. F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, "The graph neural network model," IEEE transactions on neural networks, vol. 20, no. 1, pp. 61-80, 2008.  https://doi.org/10.1109/TNN.2008.2005605
  13. J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun, "Spectral networks and locally connected networks on graphs," arXiv preprint arXiv:1312.6203, 2013. 
  14. M. Defferrard, X. Bresson, and P. Vandergheynst, "Convolutional neural networks on graphs with fast localized spectral filtering," Advances in neural information processing systems, vol. 29, 2016. 
  15. D. K. Duvenaud, D. Maclaurin, J. Iparraguirre, R. Bombarell, T. Hirzel, A. Aspuru-Guzik, and R. P. Adams, "Convolutional networks on graphs for learning molecular fingerprints," Advancesin neural information processing systems, vol. 28, 2015. 
  16. T. N. Kipf and M. Welling, "Semi-supervised classification with graph convolutional networks," arXiv preprint arXiv:1609.02907, 2016. 
  17. F. Monti, D. Boscaini, J. Masci, E. Rodola, J. Svoboda, and M. M. Bronstein, "Geometric deep learning on graphs and manifolds using mixture model cnns," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, Conference Proceedings, pp. 5115-5124. 
  18. J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, "Neural message passing for quantum chemistry," in International conference on machine learning. PMLR, 2017, Conference Proceedings, pp. 1263-1272. 
  19. Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, "Dynamic graph cnn for learning on point clouds," Acm Transactions On Graphics (tog), vol. 38, no. 5, pp. 1-12, 2019.  https://doi.org/10.1145/3326362
  20. A. Zeng, S. Song, M. Niessner, M. Fisher, J. Xiao, and T. Funkhouser, "3dmatch: Learning local geometric descriptors from rgb-d reconstructions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, Conference Proceedings, pp. 1802-1811. 
  21. A. Dai, C. Ruizhongtai Qi, and M. Niessner, "Shape completion using 3d-encoder-predictor cnns and shape synthesis," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, Conference Proceedings, pp. 5868-5877. 
  22. D. Stutz and A. Geiger, "Learning 3d shape completion from laser scan data with weak supervision," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, Conference Proceedings, pp. 1955-1964. 
  23. A. Bozic, P. Palafox, M. Zollhofer, J. Thies, A. Dai, and M. Niessner, "Neural deformation graphs for globallyconsistent nonrigid reconstruction," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, Conference Proceedings, pp. 1450-1459. 
  24. B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, "Nerf: Representing scenes as neural radiance fields for view synthesis," Communications of the ACM, vol. 65, no. 1, pp. 99-106, 2021.  https://doi.org/10.1145/3503250
  25. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, and L. Antiga, "Pytorch: An imperative style, high-performance deep learning library," Advances in neural information processing systems, vol. 32, 2019. 
  26. M. Fey and J. E. Lenssen, "Fast graph representation learning with pytorch geometric," arXiv preprint arXiv:1903.02428, 2019. 
  27. D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.