Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Motion Style Transfer using Variational Autoencoder

변형 자동 인코더를 활용한 모션 스타일 이전

  • Ahn, Jewon (Dept. of Intelligence Convergence, Hanyang University) ;
  • Kwon, Taesoo (Dept. of Computer and Software, Hanyang University)
  • 안제원 (한양대학교 지능융합학과) ;
  • 권태수 (한양대학교 컴퓨터 소프트웨어학과)
  • Received : 2021.11.06
  • Accepted : 2021.11.26
  • Published : 2021.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a framework that transfers the information of style motions to content motions based on a variational autoencoder network combined with a style encoding in the latent space. Because we transfer a style to a content motion that is sampled from a variational autoencoder, we can increase the diversity of existing motion data. In addition, we can improve the unnatural motions caused by decoding a new latent variable from style transfer. That improvement was achieved by additionally using the velocity information of motions when generating next frames.

본 논문에서는 변형 자동 인코더 네트워크(variational autoencoder network)의 잠재 공간 내에 스타일 자동 인코더 네트워크를 적용하여 컨텐츠 캐릭터의 모션에 스타일 캐릭터 모션의 스타일 정보를 이전하는 프레임워크를 제안한다. 이 프레임워크를 사용하면 기존의 변형 자동 인코더를 통해 얻은 모션의 다양성을 스타일 캐릭터 모션의 스타일 정보를 이전하여 증가시킬 수 있다. 또한 입력 데이터 및 출력 데이터에 모션의 속도 정보를 포함시켜 이전 프레임의 모션에 속도를 적분하여 모션을 계산함으로써, 변형 자동 인코더로 인한 샘플링과 잠재 공간 내에서 스타일 정보가 이전된 새로운 잠재 변수의 디코더 네트워크를 통한 확장으로 발생할 수 있는 부자연스러운 동작을 개선할 수 있다.

Keywords

References

  1. Unuma, Munetoshi, Ken Anjyo, and Ryozo Takeuchi. "Fourier principles for emotion-based human figure animation." Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, pp. 91-96, 1995.
  2. Bruderlin, Armin, and Lance Williams. "Motion signal processing." Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, pp. 97-104, 1995.
  3. Pullen, Katherine, and Christoph Bregler. "Motion capture assisted animation: Texturing and synthesis." Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pp. 501-508, 2002.
  4. Yumer, M. Ersin, and Niloy J. Mitra. "Spectral style transfer for human motion between independent actions." ACM Transactions on Graphics (TOG), vol. 35, no. 4, pp. 1-8, 2016.
  5. Brand, Matthew, and Aaron Hertzmann. "Style machines." Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp. 183-192, 2000.
  6. Min, Jianyuan, Huajun Liu, and Jinxiang Chai. "Synthesis and editing of personalized stylistic human motion." Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, pp. 39-46, 2010.
  7. Holden, Daniel, et al. "Fast neural style transfer for motion data." IEEE computer graphics and applications, vol. 37, no. 4, pp. 42-49, 2017. https://doi.org/10.1109/MCG.2017.3271464
  8. Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge. "A neural algorithm of artistic style." arXiv preprint arXiv:1508.06576, 2015.
  9. Du, Han, et al. "Stylistic Locomotion Modeling with Conditional Variational Autoencoder." Eurographics (Short Papers), pp. 9-12, 2019.
  10. Bowden, Richard. "Learning statistical models of human motion." IEEE Workshop on Human Modeling, Analysis and Synthesis, CVPR, vol. 2000, 2000.
  11. Min, Jianyuan, and Jinxiang Chai. "Motion graphs++ a compact generative model for semantic motion analysis and synthesis." ACM Transactions on Graphics (TOG), vol. 31, no. 6, pp. 1-12, 2012.
  12. Holden, Daniel, et al. "Learning motion manifolds with convolutional autoencoders." SIGGRAPH Asia 2015 Technical Briefs, pp. 1-4, 2015.
  13. Motegi, Yuichiro, Yuma Hijioka, and Makoto Murakami. "Human motion generative model using variational autoencoder." International Journal of Modeling and Optimization, vol. 8, no. 1, 2018.
  14. Habibie, Ikhsanul, et al. "A recurrent variational autoencoder for human motion synthesis." 28th British Machine Vision Conference, 2017.
  15. Fragkiadaki, Katerina, et al. "Recurrent network models for human dynamics." Proceedings of the IEEE International Conference on Computer Vision, pp. 4346-4354, 2015.
  16. Kingma, Diederik P., and Max Welling. "Auto-encoding variational bayes." arXiv preprint arXiv:1312.6114, 2013.