KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Generative Adversarial Networks for single image with high quality image

  • Zhao, Liquan (Key Laboratory of Modern Power System Simulation and Control and Renewable Energy Technology, Ministry of Education, Northeast Electric Power University) ;
  • Zhang, Yupeng (Key Laboratory of Modern Power System Simulation and Control and Renewable Energy Technology, Ministry of Education, Northeast Electric Power University)
  • Received : 2021.05.20
  • Accepted : 2021.10.19
  • Published : 2021.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

The SinGAN is one of generative adversarial networks that can be trained on a single nature image. It has poor ability to learn more global features from nature image, and losses much local detail information when it generates arbitrary size image sample. To solve the problem, a non-linear function is firstly proposed to control downsampling ratio that is ratio between the size of current image and the size of next downsampled image, to increase the ratio with increase of the number of downsampling. This makes the low-resolution images obtained by downsampling have higher proportion in all downsampled images. The low-resolution images usually contain much global information. Therefore, it can help the model to learn more global feature information from downsampled images. Secondly, the attention mechanism is introduced to the generative network to increase the weight of effective image information. This can make the network learn more local details. Besides, in order to make the output image more natural, the TVLoss function is introduced to the loss function of SinGAN, to reduce the difference between adjacent pixels and smear phenomenon for the output image. A large number of experimental results show that our proposed model has better performance than other methods in generating random samples with fixed size and arbitrary size, image harmonization and editing.

Keywords

References

  1. C. G. Turhan and H. S. Bilge, "Recent Trends in Deep Generative Models: a Review," in Proc. of International Conference on Computer Science and Engineering (UBMK), pp. 574-579, 2018.
  2. G. Bombuwala and G. Poravi, "A Review of Generative Image Modeling Techniques," in Proc. of IEEE 5th International Conference for Convergence in Technology (I2CT), pp. 1-5, 2019.
  3. Goodfellow I J, Pouget-Abadie J, Mirza M, et al., "Generative adversarial nets," Advances in Neural Information Processing Systems, vol. 27, pp. 2672-2680, 2014.
  4. Li Y, Wang H, and Dong X, "The Denoising of Desert Seismic Data Based on Cycle-GAN With Unpaired Data Training," IEEE Geoscience and Remote Sensing Letters, vol. 18, pp. 2016-2020, 2020.
  5. Yi X, Walia E, and Babyn P, "Generative adversarial network in medical imaging: A review," Medical image analysis, vol. 58, 2019.
  6. Ledig C, Theis L, Huszar F, et al., "Photo-realistic single image super-resolution using a generative adversarial network," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105-114, 2017.
  7. Shaham T R, Dekel T, and Michaeli T, "Singan: Learning a generative model from a single natural image," in Proc. of IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4569- 4579, 2019.
  8. Mao X, Li Q, Xie H, et al., "Least Squares Generative Adversarial Networks," in Proc. of IEEE International Conference on Computer Vision (ICCV), pp. 2813-2821, 2017.
  9. Arjovsky M, Chintala S, Bottou L, "Wasserstein generative adversarial networks," in Proc. of the 34th International Conference on Machine Learning, PMLR 70, pp. 214-223, 2017.
  10. Gulrajani I, Ahmed F, Arjovsky M, et al., "Improved training of wasserstein GANs," in Proc. of the 31st International Conference on Neural Information Processing Systems (NIPS'17), pp. 5769- 5779, Dec. 2017.
  11. Radford A, Metz L, Chintala S, "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint arXiv:1511.06434, 2015.
  12. Mirza M, Osindero S, "Conditional generative adversarial nets," arXiv preprint arXiv:1411.1784, 2014.
  13. Denton E, Chintala S, Szlam A, et al., "Deep generative image models using a Laplacian pyramid of adversarial networks," in Proc. of the 28th International Conference on Neural Information Processing Systems (NIPS'15), vol.1, pp. 1486-1494, 2015.
  14. Karras T, Laine S, Aila T, "A Style-Based Generator Architecture for Generative Adversarial Networks," in Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4396-4405, 2019.
  15. Xin R, Zhang J, Shao Y, "Complex network classification with convolutional neural network," Tsinghua Science and Technology, vol. 25, no. 4, pp. 447-457, Aug. 2020. https://doi.org/10.26599/tst.2019.9010055
  16. Tilon S, Nex F, Kerle N, et al., "Post-Disaster Building Damage Detection from Earth Observation Imagery Using Unsupervised and Transferable Anomaly Detecting Generative Adversarial Networks," Remote Sensing, vol. 12, p. 4193, 2020. https://doi.org/10.3390/rs12244193
  17. Dong Y, Liu Y, Zhang H, et al., "FD-GAN: Generative Adversarial Networks with FusionDiscriminator for Single Image Dehazing," in Proc. of the AAAI Conference on Artificial Intelligence, pp. 10729-10736, 2020.
  18. Wu B, Liu L, Dai Z, et al, "Detecting Malicious Social Robots with Generative Adversarial Networks," KSII Transactions on Internet and Information Systems, vol. 13, no. 11, pp. 5594-5515, 2019. https://doi.org/10.3837/tiis.2019.11.018
  19. Li C, Wand M, "Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks," Lecture Notes in Computer Science, vol. 9907, pp. 702-716, 2016.
  20. Jetchev N, Bergmann U, Vollgraf R, "Texture synthesis with spatial generative adversarial networks," arXiv preprint arXiv:1611.08207, 2016.
  21. Bergmann U, Jetchev N, Vollgraf R, "Learning texture manifolds with the Periodic Spatial GAN," in Proc. of the 34th International Conference on Machine Learning, PMLR, vol. 70, pp. 469-477, 2017.
  22. Zhou Y, Zhu Z, Bai X, et al., "Non-stationary texture synthesis by adversarial expansion," ACM Transactions on Graphics, vol. 37, no. 4, pp. 1-13, 2018.
  23. Shocher A, Bagon S, Isola P, et al., "InGAN: Capturing and Remapping the "DNA" of a Natural Image," arXiv preprint arXiv:1812.00231, 2018.
  24. Hinz T, Fisher M, Wang O, et al., "Improved Techniques for Training Single-Image GANs," in Proc. of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 1300- 1309, 2021.
  25. Luan F, Paris S, Shechtman E, et al., "Deep painterly harmonization," Computer graphics forum, vol. 37, no. 4, pp. 95-106, 2018. https://doi.org/10.1111/cgf.13478