Journal of Korea Society of Industrial Information Systems (한국산업정보학회논문지)

Korea Society of Industrial Information Systems (한국산업정보학회)
권호 표지
DOI QR코드

DOI QR Code

3D Reconstruction of a Single Clothing Image and Its Application to Image-based Virtual Try-On

의상 이미지의 3차원 의상 복원 방법과 가상착용 응용

  • Received : 2020.07.14
  • Accepted : 2020.08.19
  • Published : 2020.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Image-based virtual try-on (VTON) is becoming popular for online apparel shopping, mainly because of not requiring 3D information for try-on clothes and target humans. However, existing 2D algorithms, even when utilizing advanced non-rigid deformation algorithms, cannot handle large spatial transformations for complex target human poses. In this study, we propose a 3D clothing reconstruction method using a 3D human body model. The resulting 3D models of try-on clothes can be more easily deformed when applied to rest posed standard human models. Then, the poses and shapes of 3D clothing models can be transferred to the target human models estimated from 2D images. Finally, the deformed clothing models can be rendered and blended with target human representations. Experimental results with the VITON dataset used in the previous works show that the shapes of reconstructed clothing are significantly more natural, compared to the 2D image-based deformation results when human poses and shapes are estimated accurately.

가상착용기술은 온라인 의류 쇼핑 활성화를 위해 중요한 기술이다. 최근 이미지 기반 가상착용기술은 의상과 착용 대상 신체의 3차원 정보가 필요하지 않다는 실용성 때문에 큰 관심을 받고 있다. 그러나 기존의 이미지 기반 알고리즘의 2차원 기하변형 방식의 한계로 인하여 대상 인물의 포즈와 의상 이미지의 형태가 큰 차이가 있는 경우 자연스러운 의상변형을 하지 못한다. 본 논문에서는 이러한 문제를 해결하기 위해 3차원 인체 모델을 이용하여 2차원 의상 사진으로 부터 의상의 3차원 모델을 생성하고, 대상 인물의 자세와 체형에 맞게 3차원 변형 후 렌더링하고 대상 인간 이미지와 혼합을 통하여 가상착용 이미지를 생성할 수 있다. 기존 연구에서 사용된 VITON 데이터 세트를 사용한 실험 결과는 3차원 변형이 요구되는 경우에 2차원 이미지 기반 가상착용 결과들에 비교했을 때 자연스러운 결과를 보인다.

Keywords

References

  1. Ahn H., (2018). Image-based Virtual Try-On System: Full Automatic System Design and its Performance, J ournal of Korean Computer Game Society, 31(3), 37-45.
  2. Belongie, S., Malik, J., and Puzicha, J. (2002). Shape Matching and Object Recognition using Shape Contexts, IEEE Trans. on PAMI 24(4), 509-522, https://doi.org/10.1109/34.993558
  3. Bhatnagar, B. L., Tiwari, G., Theobalt, C., and Pons-Moll, G. (2019). Multi-garment Net: Learning to Dress 3d People from Images, Proc. of the IEEE International Conference on Computer Vision. pp. 5420-5430.
  4. Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., and Black, M. J. (2016). Keep It SMPL: Automatic Estimation of 3d Human Pose and Shape from a Single Omage, European Conference on Computer Vision, pp. 561-578
  5. Bookstein, F. L. (1989). Principal Warps: Thin-plate Splines and the Decomposition of Deformations, IEEE Trans. on PAMI 11(6), 567-585, DOI: 10.1109/34.24792
  6. Cao, Z., Martinez, G. H., Simon, T., Wei, S. E., and Sheikh, Y. (2018). Realtime Multi-person 2d Pose Estimation using Part Affinity Fields, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291-7299.
  7. Han, X., Wu, Z., Wu, Z., Yu, R., and Davis, L. S. (2018). VITON: An Image-based Virtual Try-on Network, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7543-7552.
  8. Liang, X., Gong, K., Shen, X., and Lin, L. (2018). Look into Person: Joint Body Parsing and Pose Estimation Network and a New Benchmark, IEEE Trans on PAMI. 41, 871-885. https://doi.org/10.1109/TPAMI.2018.2820063
  9. Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., and Black, M. J. (2015). Smpl: A Skinned Multi-person Linear Model, ACM Trans on Graphics, 34, 248-248. https://doi.org/10.1145/2816795.2818013
  10. Mir, A., Alldieck, T., and Pons-Moll, G. (2020). Learning to Transfer Texture from Clothing Images to 3d Humans, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7023-7034.
  11. Patel, C., Liao, Z., and Pons-Moll, G. (2020). Tailornet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7365-7375.
  12. Pishchulin, L., Insafutdinov, E., Tang, S., Andres, B., Andriluka, M., Gehler, P. V., and Schiele, B. (2016). Deepcut Joint Subset Partition and Labeling for Multi Person pose Estimation, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4929-4937
  13. Pons-Moll, G., Pujades, S., Hu, S., and Black, M. J. (2017). Clothcap: Seamless 4d Clothing Capture and Retargeting, ACM Trans. on Graphics. 36(4), 1-15 https://doi.org/10.1145/3072959.3073711
  14. Sun, F.D., Guo, J., Su, Z., and Gao, C. Y. (2019). Image-based Virtual Try-on Network with Structural Coherence, Proc. of the IEEE International Conference on Image Processing, pp. 519-523
  15. Wang, B., Zhang, H., Liang, X., Chen, Y., Lin, L., and Yang, M. (2018). Toward Characteristic Preserving Image-based Virtual Try-on Network, European Conference on Computer Vision, pp. 589-604.
  16. Weng, C. Y., Curless, B., and Kemelmacher-Shlizerman, I. (2019). Photo Wake-up: 3d Character Animation from a Single Photo, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5908-5917
  17. Yu, R., Wang, X., and Xie, X. (2019). VTNFP: An Image-based Virtual Try-on Network with Body and Clothing Feature Preservation, Proc. of the IEEE International Conference on Computer Vision, pp. 10511-10520.
  18. Zanfir, M., Popa, A. I., Zanfir, A., and Sminchisescu, C. (2018). Human Appearance Transfer, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5391-599.