Journal of IKEEE (전기전자학회논문지)

Institute of Korean Electrical and Electronics Engineers (한국전기전자학회)
권호 표지
DOI QR코드

DOI QR Code

A Scheme for Preventing Data Augmentation Leaks in GAN-based Models Using Auxiliary Classifier

보조 분류기를 이용한 GAN 모델에서의 데이터 증강 누출 방지 기법

  • Shim, Jong-Hwa (Dept. of Electrical Engineering, Korea University) ;
  • Lee, Ji-Eun (Dept. of Electrical Engineering, Korea University) ;
  • Hwang, Een-Jun (Dept. of Electrical Engineering, Korea University)
  • Received : 2022.04.19
  • Accepted : 2022.05.31
  • Published : 2022.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Data augmentation is general approach to solve overfitting of machine learning models by applying various data transformations and distortions to dataset. However, when data augmentation is applied in GAN-based model, which is deep learning image generation model, data transformation and distortion are reflected in the generated image, then the generated image quality decrease. To prevent this problem called augmentation leak, we propose a scheme that can prevent augmentation leak regardless of the type and number of augmentations. Specifically, we analyze the conditions of augmentation leak occurrence by type and implement auxiliary augmentation task classifier that can prevent augmentation leak. Through experiments, we show that the proposed technique prevents augmentation leak in the GAN model, and as a result improves the quality of the generated image. We also demonstrate the superiority of the proposed scheme through ablation study and comparison with other representative augmentation leak prevention technique.

데이터 증강이란 다양한 데이터 변환 및 왜곡을 통해 데이터셋의 크기와 품질을 개선하는 기법으로, 기계학습 모델의 과적합 문제를 해결하기 위한 대표적인 접근법이다. 그러나 심층학습 이미지 생성 모델인 GAN 기반 모델에서 데이터 증강을 적용하면 생성된 이미지에 데이터 변환과 왜곡이 반영되는 증강 누출 문제가 발생하여 생성 이미지의 품질이 하락한다. 이러한 문제를 해결하기 위해 본 논문에서는 데이터 증강의 종류와 수에 관계없이 증강 누출을 방지하는 기법을 제안한다. 증강 누출의 발생 조건을 분석하였으며, 보조적인 데이터 증강 작업 분류기를 GAN 모델에 적용하여 증강 누출을 방지하였다. 정성적 정량적 평가를 통해 제안된 기법을 적용하면 증강 누출이 발생하지 않음을 보이고 추가적으로 생성 이미지의 품질을 향상시키며 기존 기법과 비교하여 발전된 성능을 보임을 입증하였다.

Keywords

Acknowledgement

This research was financially supported by the Ministry of Trade, Industry and Energy(MOTIE) and Korea Institute for Advancement of Technology(KIAT) through the International Cooperative R&D program. (Project No. P0017192)

References

  1. L. Perez. and J. Wang, "The effectiveness of data augmentation in image classification using deep learning," arXiv preprint, 2017. DOI: 10.48550/arXiv.1712.04621
  2. Shorten., Conner, and Taghi M. Khoshgoftaar, "A survey on image data augmentation for deep learning," Journal of big data, Vol.6, No.1, pp. 1-48, 2019. DOI: 10.1186/s40537-019-0197-0
  3. F. Quiroga et al., "Revisiting data augmentation for rotational invariance in convolutional neural networks," International Conference on Modelling and Simulation in Management Sciences 2018, pp.127-141, 2018. DOI: 10.1007/978-3-030-15413-4_10
  4. I. Goodfellow et al., "Generative adversarial nets," Advances in neural information processing systems 2014, 2014.
  5. N. t. Tran et al., "On data augmentation for GAN training," IEEE Transactions on Image Processing, Vol.30, pp.1882-1897, 2021. DOI: 10.1109/TIP.2021.3049346
  6. T. Karras et al., "Training generative adversarial networks with limited data. Advances in Neural Information Processing Systems," Advances in Neural Information Processing Systems 2020, pp. 12104-12114, 2020. DOI: 10.5555/3495724.3496739
  7. Z. Zhao et al, "Image augmentations for GAN training," arXiv preprint, 2020.
  8. H. Zhang et al., "Consistency regularization for generative adversarial networks," International Conference of Leaning Representations, 2020.
  9. Z. Zhao et al., "Improved consistency regularization for GANs," AAAI Conference on Artificial Intelligence, 2021.
  10. A. Radford., L. Metz and S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint, 2015.
  11. A. Krizhevsky, I. Sutskever and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems 2012, 2012.
  12. M. Arjovsky, S. Chintala and L. Bottou, "Wasserstein generative adversarial networks," International conference on machine learning, pp.214-223, 2017.
  13. Gulrajani et al., "Improved training of wasserstein GANs," Advances in neural information processing systems 2017, 2017.
  14. X. Mao et al., "Least squares generative adversarial networks," Proceedings of the IEEE international conference on computer vision, pp.2794-2802, 2017.
  15. T. Karras et al., "Progressive growing of GANs for improved quality, stability, and variation," arXiv preprint, 2017.
  16. T. Karras et al., "A style-based generator architecture for generative adversarial networks." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401-4410, 2019.
  17. K. He et al., "Deep residual learning for image recognition," Proceedings of the IEEE conference on computer vision and pattern recognition, pp.770-778, 2016. DOI: 10.1109/CVPR.2016.90
  18. Z. Cai et al., "Generative adversarial networks: A survey toward private and secure applications," ACM Computing Surveys, Vol.54, No.6, pp.1-38, 2021. DOI: 10.48550/arXiv.2106.03785
  19. Z. Lin et al., "PacGAN: The power of two samples in generative adversarial networks," Advances in neural information processing systems, 2018. DOI: 10.1109/JSAIT.2020.2983071
  20. L. Metz et al., "Unrolled generative adversarial networks," arXiv preprint, 2016.
  21. Y. Yazici et al., "Empirical analysis of overfitting and mode drop in GAN training," IEEE International Conference on Image Processing, pp.1651-1655, 2020. DOI: 10.48550/arXiv.2006.14265
  22. H. Thanh-Tung et al., "Improving generalization and stability of generative adversarial networks," arXiv preprint, 2019. DOI: 10.48550/arXiv.1902.03984
  23. S. Loffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," International conference on machine learning, pp.448-456, 2015. DOI: 10.48550/arXiv.1502.03167
  24. B. Zhou et al., "Learning deep features for discriminative localization," Proceedings of the IEEE conference on computer vision and pattern recognition, pp.2921-2929, 2016. DOI: 10.48550/arXiv.1512.04150
  25. U. Ruby, V. Yendapalli, "Binary cross entropy with deep learning technique for image classification," Int. J. Adv. Trends Comput. Sci. Eng, Vol.9, No.10, 2020. DOI: 10.30534/ijatcse/2020/175942020
  26. S. Yang et al., "From facial parts responses to face detection: A deep learning approach," Proceedings of the IEEE international conference on computer vision, pp.3676-3684, 2015. DOI: 10.1109/ICCV.2015.419
  27. T. Miyato et al,. "Spectral normalization for generative adversarial networks," arXiv preprint, 2018. DOI: 10.48550/arXiv.1802.05957
  28. A. Radford, L. Metz and S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint, 2015.
  29. P. Isola et al., "Image-to-image translation with conditional adversarial networks," Proceedings of the IEEE conference on computer vision and pattern recognition, pp.1125-1134, 2017. DOI: 10.48550/arXiv.1611.07004
  30. D. P. Kingma, and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint, 2014.
  31. M. Heusel et al., "Gans trained by a two time-scale update rule converge to a local nash equilibrium," Advances in neural information processing systems, 2017.
  32. C. Szegedy et al., "Rethinking the inception architecture for computer vision," Proceedings of the IEEE conference on computer vision and pattern recognition, pp.2818-2826, 2016. DOI: 10.48550/arXiv.1512.00567
  33. T. Salimans et al., "Improved techniques for training GANs," Advances in neural information processing systems, 2016. DOI: 10.48550/arXiv.1606.03498
  34. A. Kolesnikov, X. Zhai and L. Beyer "Revisiting self-supervised visual representation learning," Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.1920-1929, 2019. DOI: 10.48550/arXiv.1901.09005