Journal of Korea Society of Industrial Information Systems (한국산업정보학회논문지)

Korea Society of Industrial Information Systems (한국산업정보학회)
권호 표지
DOI QR코드

DOI QR Code

An Improved VTON (Virtual-Try-On) Algorithm using a Pair of Cloth and Human Image

이미지를 사용한 가상의상착용을 위한 개선된 알고리즘

  • Received : 2020.01.23
  • Accepted : 2020.02.22
  • Published : 2020.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, a series of studies on virtual try-on (VTON) using images have been published. A comparison study analyzed representative methods, SCMM-based non-deep learning method, deep learning based VITON and CP-VITON, using costumes and user images according to the posture and body type of the person, the degree of occlusion of the clothes, and the characteristics of the clothes. In this paper, we tackle the problems observed in the best performing CP-VTON. The issues tackled are the problem of segmentation of the subject, pixel generation of un-intended area, missing warped cloth mask and the cost function used in the learning, and limited the algorithm to improve it. The results show some improvement in SSIM, and significantly in subjective evaluation.

최근 이미지를 사용한 가상착용기술 (Virtual try-on: VTON)에 대한 일련의 연구들이 발표되었다. 이에 의상과 사용자 이미지를 사용한 대표적 방식 (SCMM 기반의 비-딥러닝 방식, 딥러닝 기반 VITON 과 CP-VITON)에 대해 인물의 자세 및 체형, 의상의 가려짐 정도, 의상의 특성 등에 따라 분석한 연구가 보고되었다. 본 논문에서는 이중 가장 좋은 성능을 보이는 CP-VTON의 문제점을 살펴보고 이에 따른 해결책을 제시한다. 구체적으로 대상인물의 분할 표현 문제, 교체 대상이 아닌 영역이 유지되지 못하는 문제, 합성 마스크 생성네트워크의 학습에 사용되는 비용함수 문제, 합성 네트워크의 마스크 문제를 지적하고 이를 개선하는 알고리즘을 제안하였다. 그 결과 SSIM 등에서 5%내외의 주관적으로는 상당한 개선을 보였다.

Keywords

References

  1. Ahn H. (2018a). Online Virtual Try On using Mannequin Cloth Pictures, Journal of the Korea Industrial Information Systems Research, 23(6), 29-38. https://doi.org/10.9723/JKSIIS.2018.23.6.029
  2. Ahn H. (2018b). Image-based Virtual Try-On System, Journal of Korean Computer Game Society, 31(3), 37-45.
  3. Barratt, S., and Sharma, R. (2018). A Note on the Inception Score, arXiv preprint arXiv:1801.01973.
  4. Cao, Z., Simon, T., Wei, S. E., and Sheikh, Y. (2017). Realtime Multi-person 2d Pose Estimation using Part Affinity Fields, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291-7299.
  5. Han, X., Wu, Z., Wu, Z., Yu, R., and Davis, L. S. (2018). Viton: An Image-based Virtual Try-on Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7543-7552.
  6. Jaderberg, M., Simonyan, K., and Zisserman, A. (2015). Spatial Transformer Networks. Proceedings of Advances in Neural Information Processing Systems, pp. 2017-2025.
  7. Liang, X., Gong, K., Shen, X., and Lin, L. (2018). Look into Person: Joint Body Parsing & Pose Estimation Network and a New Benchmark. IEEE Transactions on PAMI , 41(4), 871-885. https://doi.org/10.1109/TPAMI.2018.2820063
  8. Raj, A., Sangkloy, P., Chang, H., Lu, J., Ceylan, D., and Hays, J. (2018). Swapnet: Garment Transfer in Single View Images. Proceedings of the European Conference on Computer Vision, pp. 666-682.
  9. Tuan, T., Rahman, M., and Ahn, H. (2019). Performance Evaluation of VTON Algorithms using a Pair of Cloth and Human Image, Journal of the Korea Industrial Information Systems Research, 24(6), 24-30.
  10. Wang, B., Zheng, H., Liang, X., Chen, Y., Lin, L., and Yang, M. (2018). Toward Characteristic-preserving Image-based Virtual Try-on Network. Proceedings of the European Conference on Computer Vision, pp. 589-604.
  11. Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. (2018). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586-595.

Cited by

  1. Development of A Uniform And Casual Clothing Recognition System For Patient Care In Nursing Hospitals vol.25, pp.12, 2020, https://doi.org/10.9708/jksci.2020.25.12.045