Journal of Korea Society of Industrial Information Systems (한국산업정보학회논문지)

Korea Society of Industrial Information Systems (한국산업정보학회)
권호 표지
DOI QR코드

DOI QR Code

Performance Evaluation of VTON (Virtual-Try-On) Algorithms using a Pair of Cloth and Human Image

이미지를 사용한 가상의상착용 알고리즘들의 성능 분석

  • Received : 2019.10.02
  • Accepted : 2019.10.29
  • Published : 2019.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

VTON (Virtual try-on) is a key technology that can activate the online commerce of fashion items. However, the early 3D graphics-based methods require the 3D information of the clothing or the human body, which is difficult to secure realistically. In order to overcome this problem, Image-based deep-learning algorithms such as VITON (Virtual image try-on) and CP-VTON (Characteristic preserving-virtual try-on) has been published, but only a sampled results on performance is presented. In order to examine the strength and weakness for their commercialization, the performance analysis is needed according to the complexity of the clothes, the object posture and body shape, and the degree of occlusion of the clothes. In this paper, IoU and SSIM were evaluated for the performance of transformation and synthesis stages, together with non-DL SCM based method. As a result, CP-VTON shows the best performance, but its performance varies significantly according to posture and complexity of clothes. The reasons for this were attributed to the limitations of secondary geometric deformation and the limitations of the synthesis technology through GAN.

가상착용기술(VTON: Virtual try-on)은 의상의 온라인 유통을 활성화를 위하여 중요한 기술이다. 그러나 3차원 그래픽스기반 방식은 의상과 인체의 3차원 정보의 확보가 필요하여 범용화에 어려움이 있고, 이러한 제약을 해소하기 위해 개발되는 이미지 기반 방식들의 연구들은 그 기술적 한계가 불명확하다. 구체적으로 VITON (Virtual image try-on) 과 CP-VTON (Content preserving VTON)등은 가능성 위주의 매우 단편적인 결과만을 제시하고 있다. 본 논문은 이미지기반 기술의 상용화의 한계를 파악하기 위해, 세 가지 대표적 방식(SCMM 기반의 비-딥러닝 방식, 딥러닝기반 VITON 과 CP-VTON에 대하여 인물의 자세 및 체형, 의상의 가려짐 정도, 의상의 특성 등에 따라 분석을 하였다. 객관적인 평가를 위하여 변형단계와 합성단계의 성능을 각각 IoU와 SSIM로 평가하였고, 상대적인 비교 분석을 하였다. 그 결과, CP-VTON이 가장 좋은 성능을 보이지만, 자세와 의상의 복잡도에 따라 성능의 한계가 크게 차이가 남을 보였다. 그 주 원인은 2차 기하변형의 한계와 GAN을 통한 합성 기술의 한계로 파악되었다.

Keywords

References

  1. Ahn H., (2018a). Online Virtual Try On using Mannequin Cloth Pictures, Journal of the Korea Industrial Information Systems Research, 23(6), 29-38. https://doi.org/10.9723/JKSIIS.2018.23.6.029
  2. Ahn H., (2018b). Image-based Virtual Try-On System, Journal of Korean Computer Game Society, 31(3), 37-45.
  3. Belongie, S., Malik, J., and Puzicha, J. (2002). Shape Matching and Object Recognition using Shape Contexts, IEEE Transactions on PAMI, 25(4), 509-522. https://doi.org/10.1109/34.993558
  4. Cao, Z., Simon, T., Wei, S. E., and Sheikh, Y. (2017). Realtime Multi-person 2d Pose Estimation using Part Affinity Fields, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291-7299.
  5. Han, X., Wu, Z., Wu, Z., Yu, R., and Davis, L. S. (2018). Viton: An Image-based Virtual Try-on Network, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7543-7552.
  6. Liang, X., Gong, K., Shen, X., and Lin, L. (2018). Look into Person: Joint Body Parsing & Pose Estimation Network and a New Benchmark. IEEE Transactions on PAMI, 41(4), 871-885. https://doi.org/10.1109/TPAMI.2018.2820063
  7. Raj, A., Sangkloy, P., Chang, H., Lu, J., Ceylan, D., and Hays, J. (2018). Swapnet: Garment Transfer in Single View Images, Proceedings of the European Conference on Computer Vision, pp. 666-682.
  8. Wang, B., Zheng, H., Liang, X., Chen, Y., Lin, L., and Yang, M. (2018). Toward Characteristic-preserving Image-based Virtual Try-on Network, Proceedings of the European Conference on Computer Vision, pp. 589-604.
  9. Weng, C. Y., Curless, B., and Kemelmacher-Shlizerman, I. (2019). Photo Wake-up: 3d Character Animation from a Single Photo, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5908-5917.
  10. Zanfir, M., Popa, A. I., Zanfir, A., and Sminchisescu, C. (2018). Human Appearance Transfer, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5391-5399.