Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Generation of High-Resolution Chest X-rays using Multi-scale Conditional Generative Adversarial Network with Attention

주목 메커니즘 기반의 멀티 스케일 조건부 적대적 생성 신경망을 활용한 고해상도 흉부 X선 영상 생성 기법

  • 안경진 (연세대학교 심장.혈관 ICT기술연구센터) ;
  • 장영걸 (연세대학교 심장.혈관 ICT기술연구센터) ;
  • 하성민 (연세대학교 심장.혈관 ICT기술연구센터) ;
  • 전병환 (연세대학교 심장.혈관 ICT기술연구센터) ;
  • 홍영택 (연세대학교 심장.혈관 ICT기술연구센터) ;
  • 심학준 (연세대학교 심장.혈관 ICT기술연구센터) ;
  • 장혁재 (연세대학교 의과대학 세브란스병원 심장내과)
  • Received : 2019.05.07
  • Accepted : 2019.11.06
  • Published : 2020.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

In the medical field, numerical imbalance of data due to differences in disease prevalence is a common problem. It reduces the performance of a artificial intelligence network, leading to difficulties in learning a network with good performance. Recently, generative adversarial network (GAN) technology has been introduced as a way to address this problem, and its ability has been demonstrated by successful applications in various fields. However, it is still difficult to achieve good results in solving problems with performance degraded by numerical imbalances because the image resolution of the previous studies is not yet good enough and the structure in the image is modeled locally. In this paper, we propose a multi-scale conditional generative adversarial network based on attention mechanism, which can produce high resolution images to solve the numerical imbalance problem of chest X-ray image data. The network was able to produce images for various diseases by controlling condition variables with only one network. It's efficient and effective in that the network don't need to be learned independently for all disease classes and solves the problem of long distance dependency in image generation with self-attention mechanism.

의료분야에서 질환별 유병률 차이로 인한 데이터 수적 불균형은 흔하게 발생되는 문제로 인공지능 학습 성능을 저하시켜 개발의 어려움을 초래한다. 최근 이러한 데이터 수적 불균형문제를 해결하기 위한 한 방법으로 적대적 생성 신경망(GAN) 기술이 도입되었고 다양한 분야에 성공적으로 적용되어왔다. 그러나 수적 불균형에 의해 저하된 성능 문제를 해결하는데 있어서 기존 연구들의 영상 해상도가 아직 충분하지 않고 영상 내 구조가 전역적으로 일관성 있게 모델링 되지 않아 좋은 결과를 얻기 어렵다. 본 논문에서는, 흉부 X선 영상 데이터의 수적 불균형문제를 해결하기 위하여 고해상도 영상을 생성할 수 있는 주목 메커니즘 기반 멀티 스케일 조건부 적대적 생성 네트워크를 제안한다. 해당 네트워크는 질환제어 조건변수에 의해 하나의 네트워크만으로 다양한 질환 영상을 생성할 수 있어 각 클래스별로 학습을 하는 비효율성을 줄였고, 자기 주목 메커니즘을 통해 영상 내 장거리 종속성 문제를 해결하였다.

Keywords

References

  1. Gyeongwan Kug, Application of Artificial Intelligence Technology and Industry, IITP, pp.22-26, March, 2019.
  2. F Provost, "Machine learning from imbalanced data sets 101," Proceedings of the AAAI'2000 workshop on imbalanced data sets, Vol. 68, No. 2000, AAAI Press, 2000.
  3. Goodfellow, Ian, et al., "Generative adversarial nets," Advances in neural information processing systems, pp. 2672-2680, 2014.
  4. A. Radford, L. Metz, and S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks," in International Conference on Learning Representations (ICLR), 2015.
  5. Mirza, Mehdi, and Simon Osindero, "Conditional generative adversarial nets," arXiv preprint, arXiv:1411.1784, 2014.
  6. Isola, Phillip, et al., "Image-to-image translation with conditional adversarial networks," Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134, 2017.
  7. Zhu, Jun-Yan, et al., "Unpaired image-to-image translation using cycle- consistent adversarial networks," Proceedings of the IEEE International Conference on Computer Vision, pp. 2223-2232, 2017.
  8. Kim, Taeksoo, et al., "Learning to discover cross-domain relations with generative adversarial networks," Proceedings of the 34th International Conference on Machine Learning, Volume 70, JMLR. org, 2017.
  9. Choi, Yunjey, et al., "Stargan: Unified generative adversarial networks for multi-domain image-to-image translation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
  10. Salehinejad, Hojjat, et al., "Generalization of deep neural networks for chest pathology classification in x-rays using generative adversarial networks," 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 990-994, April, 2018.
  11. Salehinejad, Hojjat, et al., "Synthesizing Chest X-Ray Pathology for Training Deep Convolutional Neural Networks," IEEE transactions on medical imaging, 38.5: 1197-1206, 2018. https://doi.org/10.1109/tmi.2018.2881415
  12. Zhang, Han, et al., "Stackgan++: Realistic image synthesis with stacked generative adversarial networks," arXiv preprint, arXiv: 1710.10916, 2017.
  13. Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio, "Neural machine translation by jointly learning to align and translate," arXiv preprint, arXiv:1409.0473, 2014.
  14. Hochreiter, Sepp, and Jurgen Schmidhuber, "Long short-term memory," Neural computation, 9.8: 1735-1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735
  15. Sutskever, Ilya, Oriol Vinyals, and Quoc V. Le, "Sequence to sequence learning with neural networks," Advances in neural information processing systems, 2014.
  16. Zhang, Han, et al., "Self-Attention Generative Adversarial Networks," arXiv preprint, arXiv:1805.08318, 2018.
  17. Mao, Xudong, et al., "Least squares generative adversarial networks," Proceedings of the IEEE International Conference on Computer Vision, 2017.
  18. Heusel, Martin, et al., "Gans trained by a two time-scale update rule converge to a local nash equilibrium," Advances in Neural Information Processing Systems, 2017.
  19. Salimans, Tim, et al., "Improved techniques for training gans," Advances in neural information processing systems, 2016.