Journal of the Korean Society of Radiology (대한영상의학회지)

The Korean Society of Radiology (대한영상의학회)
권호 표지
DOI QR코드

DOI QR Code

Data Augmentation Techniques for Deep Learning-Based Medical Image Analyses

딥러닝 기반 의료영상 분석을 위한 데이터 증강 기법

  • Mingyu Kim (Department of Convergence Medicine, University of Ulsan College of Medicine) ;
  • Hyun-Jin Bae (Promedius Inc.)
  • 김민규 (울산대학교 의과대학 융합의학과) ;
  • 배현진 (프로메디우스 주식회사)
  • Received : 2020.09.02
  • Accepted : 2022.09.24
  • Published : 2020.11.01
Download PDF
⟨ Previous Next ⟩

Abstract

Medical image analyses have been widely used to differentiate normal and abnormal cases, detect lesions, segment organs, etc. Recently, owing to many breakthroughs in artificial intelligence techniques, medical image analyses based on deep learning have been actively studied. However, sufficient medical data are difficult to obtain, and data imbalance between classes hinder the improvement of deep learning performance. To resolve these issues, various studies have been performed, and data augmentation has been found to be a solution. In this review, we introduce data augmentation techniques, including image processing, such as rotation, shift, and intensity variation methods, generative adversarial network-based method, and image property mixing methods. Subsequently, we examine various deep learning studies based on data augmentation techniques. Finally, we discuss the necessity and future directions of data augmentation.

영상처리 기반으로 의료영상을 분석하는 기법은 정상 환자와 비정상 환자를 분류, 병변 검출 및 장기나 병변의 분할 등에 사용되고 있다. 최근 인공지능 기술의 비약적 발전으로 의료영상 분석 연구들이 딥러닝 기술을 활용하여 시도되고 있다. 의료영상은 학습에 필요한 데이터를 충분히 모으기 어렵고 클래스별 데이터 수의 차이 때문에, 딥러닝 모델의 성능을 올리는데 어려움이 있다. 이러한 문제를 해결하기 위해 다양한 연구가 시도되고 있으며, 이 중 하나가 학습 데이터를 증강하는 것이다. 본 종설에서는 회전, 역상, 밝기 변화 등과 같은 영상처리 기반의 데이터 증강, 적대적생성네트워크를 활용한 데이터 증강, 그리고 기존 영상의 속성들을 섞는 등의 최신 데이터 증강 기법을 알아보고, 의료영상 연구에 적용된 사례들과 그 결과를 조사해 보고자 한다. 끝으로 데이터 증강의 필요성을 고찰하고 앞으로의 방향을 짚어본다.

Keywords

References

  1. Kim M, Yun J, Cho Y, Shin K, Jang R, Bae HJ, et al. Deep learning in medical imaging. Neurospine 2019;16:657-668  https://doi.org/10.14245/ns.1938396.198
  2. Kim JH. Imaging informatics: a new horizon for radiology in the era of artificial intelligence, big data, and data science. J Korean Soc Radiol 2019;80:176-201  https://doi.org/10.3348/jksr.2019.80.2.176
  3. Song KD, Kim M, Do S. The latest trends in the use of deep learning in radiology illustrated through the stages of deep learning algorithm development. J Korean Soc Radiol 2019;80:202-212  https://doi.org/10.3348/jksr.2019.80.2.202
  4. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014;15:1929-1958 
  5. Yamashita R, Nishio M, Do RKG, Togashi K. Convolutional neural networks: an overview and application in radiology. Insights Imaging 2018;9:611-629  https://doi.org/10.1007/s13244-018-0639-9
  6. Do S, Song KD, Chung JW. Basics of deep learning: a radiologist's guide to understanding published radiology articles on deep learning. Korean J Radiol 2020;21:33-41  https://doi.org/10.3348/kjr.2019.0312
  7. Jia X, Luo T, Ren S, Guo K, Li F. Small sample-based disease diagnosis model acquisition in medical human-centered computing. J Wireless Com Network 2019;2019:212 
  8. Han C, Rundo L, Araki R, Furukawa Y, Mauri G, Nakayama H, et al. Infinite brain MR images: PGGAN-based data augmentation for tumor detection. In Esposito A, Faundez-Zanuy M, Morabito F, Pasero E, eds. Neural approaches to dynamics of signal exchanges. Singapore: Springer 2020:291-303 
  9. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 2012;25:1097-1105 
  10. Roth HR, Lu L, Liu J, Yao J, Seff A, Cherry K, et al. Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imaging 2015;35:1170-1181  https://doi.org/10.1109/TMI.2015.2482920
  11. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial nets. Adv Neural Inf Process Syst 2014:2672-2680 
  12. Roth HR, Lee CT, Shin HC, Seff A, Kim L, Yao J, et al. Anatomy-specific classification of medical images using deep convolutional nets. New York: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI) 2015:101-104 
  13. Hao R, Namdar K, Liu L, Haider MA, Khalvati F. A comprehensive study of data augmentation strategies for prostate cancer detection in diffusion-weighted MRI using convolutional neural networks. ArXiv Preprint 2020;arXiv:2006.01693 
  14. Zhao Z, Zhang Z, Chen T, Singh S, Zhang H. Image augmentations for GAN training. ArXiv Preprint 2020;arXiv:2006.02595 
  15. Tang C, Li J, Wang L, Li Z, Jiang L, Cai A, et al. Unpaired low-dose CT denoising network based on cycle-consistent generative adversarial network with prior image information. Comput Math Methods Med 2019;2019:8639825 
  16. Yang Q, Yan P, Zhang Y, Yu H, Shi Y, Mou X, et al. Low-dose CT image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. IEEE Trans Med Imaging 2018;37:1348-1357  https://doi.org/10.1109/TMI.2018.2827462
  17. Dong Z, Liu G, Ni G, Jerwick J, Duan L, Zhou C. Optical coherence tomography image denoising using a generative adversarial network with speckle modulation. J Biophotonics 2020;13:e201960135 
  18. Frid-Adar M, Diamant I, Klang E, Amitai M, Goldberger J, Greenspan H. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 2018;321:321-331  https://doi.org/10.1016/j.neucom.2018.09.013
  19. Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. ArXiv Preprint 2015;arXiv:1511.06434 
  20. Zhao D, Zhu D, Lu J, Luo Y, Zhang G. Synthetic medical images using F&BGAN for improved lung nodules classification by multi-scale VGG16. Symmetry 2018;10:519 
  21. Salehinejad H, Valaee S, Dowdell T, Colak E, Barfett J. Generalization of deep neural networks for chest pathology classification in X-rays using generative adversarial networks. Calgary: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2018:990-994 
  22. He T, Zhang Z, Zhang H, Zhang Z, Xie J, Li M. Bag of tricks for image classification with convolutional neural networks. Long Beach: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2019:558-567 
  23. Karras T, Aila T, Laine S, Lehtinen J. Progressive growing of gans for improved quality, stability, and variation. ArXiv Preprint 2017;arXiv:1710.10196 
  24. Redmon J, Farhadi A. Yolov3: an incremental improvement. ArXiv Preprint 2018;arXiv:1804.02767 
  25. Zhao J, Li D, Kassam Z, Howey J, Chong J, Chen B, et al. Tripartite-GAN: synthesizing liver contrast-enhanced MRI to improve tumor detection. Med Image Anal 2020;63:101667 
  26. Bowles C, Chen L, Guerrero R, Bentley P, Gunn R, Hammers A, et al. Gan augmentation: augmenting training data using generative adversarial networks. ArXiv Preprint 2018;arXiv:1810.10863 
  27. Russ T, Goerttler S, Schnurr AK, Bauer DF, Hatamikia S, Schad LR, et al. Synthesis of CT images from digital body phantoms using CycleGAN. Int J Comput Assist Radiol Surg 2019;14:1741-1750  https://doi.org/10.1007/s11548-019-02042-9
  28. Zhu JY, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. Venice: Proceedings of the IEEE Conference on Computer Vision 2017:2223-2232 
  29. Gupta A, Venkatesh S, Chopra S, Ledig C. Generative image translation for data augmentation of bone lesion pathology. ArXiv Preprint 2019;arXiv:1902.02248 
  30. Sandfort V, Yan K, Pickhardt PJ, Summers RM. Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks. Sci Rep 2019;9:16884 
  31. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. In Navab N, Hornegger J, Wells W, Frangi A, eds. International conference on medical image computing and computer-assisted intervention. Cham: Springer 2015:234-241 
  32. Wu E, Wu K, Cox D, Lotter W. Conditional infilling GANs for data augmentation in mammogram classification. In Stoyanov D, Taylor Z, Kainz B, Maicas G, Beichel RR, Martel A, et al. Image analysis for moving organ, breast, and thoracic images. Cham: Springer 2018:98-106 
  33. Onishi Y, Teramoto A, Tsujimoto M, Tsukamoto T, Saito K, Toyama H, et al. Automated pulmonary nodule classification in computed tomography images using a deep convolutional neural network trained by generative adversarial networks. Biomed Res Int 2019;2019:6051939 
  34. Arjovsky M, Chintala S, Bottou L. Wasserstein gan. ArXiv Preprint 2017;arXiv:1701.07875 
  35. Han C, Hayashi H, Rundo L, Araki R, Shimoda W, Muramatsu S, et al. GAN-based synthetic brain MR image generation. Washington D.C.: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) 2018:734-738 
  36. Chuquicusma MJ, Hussein S, Burt J, Bagci U. How to fool radiologists with generative adversarial networks? A visual turing test for lung cancer diagnosis. Washington D.C.: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018) 2018:240-244 
  37. Zhao M, Cong Y, Carin L. On leveraging pretrained GANs for generation with limited data. ArXiv Preprint 2020;arXiv:2002.11810 
  38. DeVries T, Taylor GW. Improved regularization of convolutional neural networks with cutout. ArXiv Preprint 2017;arXiv:1708.04552 
  39. Zhang H, Cisse M, Dauphin YN, Lopez-Paz D. Mixup: beyond empirical risk minimization. ArXiv Preprint 2017;arXiv:1710.09412 
  40. Yun S, Han D, Oh SJ, Chun S, Choe J, Yoo Y. Cutmix: regularization strategy to train strong classifiers with localizable features. Seoul: Proceedings of the IEEE International Conference on Computer Vision 2019:6023-6032 
  41. Hendrycks D, Mu N, Cubuk ED, Zoph B, Gilmer J, Lakshminarayanan B. Augmix: a simple data processing method to improve robustness and uncertainty. ArXiv Preprint 2019;arXiv:1912.02781 
  42. Bae HJ, Kim CW, Kim N, Park B, Kim N, Seo JB, et al. A Perlin noise-based augmentation strategy for deep learning with small data samples of HRCT images. Sci Rep 2018;8:17687 
  43. Perlin K. An image synthesizer. ACM Siggraph Computer Graphics 1985;19:287-296  https://doi.org/10.1145/325165.325247
  44. Perlin K. Improving noise. ACM transactions on graphics. San Antonio: Proceedings of ACM SIGGRAPH 2002;21:681-682  https://doi.org/10.1145/566654.566636
  45. Noguchi S, Nishio M, Yakami M, Nakagomi K, Togashi K. Bone segmentation on whole-body CT using convolutional neural network with novel data augmentation techniques. Comput Biol Med 2020;121;103767 
  46. Takahashi R, Matsubara T, Uehara K. RICAP: random image cropping and patching data augmentation for deep CNNs. Beijing: Asian Conference on Machine Learning 2018:786-798