Journal of Korean Neurosurgical Society

The Korean Neurosurgical Society (대한신경외과학회)
권호 표지
DOI QR코드

DOI QR Code

Spine Computed Tomography to Magnetic Resonance Image Synthesis Using Generative Adversarial Networks : A Preliminary Study

  • Lee, Jung Hwan (Department of Neurosurgery, Pusan National University Hospital) ;
  • Han, In Ho (Department of Neurosurgery, Pusan National University Hospital) ;
  • Kim, Dong Hwan (Department of Neurosurgery, Pusan National University Hospital) ;
  • Yu, Seunghan (Department of Neurosurgery, Pusan National University Hospital) ;
  • Lee, In Sook (Department of Radiology, Pusan National University Hospital) ;
  • Song, You Seon (Department of Radiology, Pusan National University Hospital) ;
  • Joo, Seongsu (Team Elysium Inc.) ;
  • Jin, Cheng-Bin (School of Information and Communication Engineering, Inha University) ;
  • Kim, Hakil (School of Information and Communication Engineering, Inha University)
  • Received : 2019.04.04
  • Accepted : 2019.06.11
  • Published : 2020.05.01
Download PDF
⟨ Previous Next ⟩

Abstract

Objective : To generate synthetic spine magnetic resonance (MR) images from spine computed tomography (CT) using generative adversarial networks (GANs), as well as to determine the similarities between synthesized and real MR images. Methods : GANs were trained to transform spine CT image slices into spine magnetic resonance T2 weighted (MRT2) axial image slices by combining adversarial loss and voxel-wise loss. Experiments were performed using 280 pairs of lumbar spine CT scans and MRT2 images. The MRT2 images were then synthesized from 15 other spine CT scans. To evaluate whether the synthetic MR images were realistic, two radiologists, two spine surgeons, and two residents blindly classified the real and synthetic MRT2 images. Two experienced radiologists then evaluated the similarities between subdivisions of the real and synthetic MRT2 images. Quantitative analysis of the synthetic MRT2 images was performed using the mean absolute error (MAE) and peak signal-to-noise ratio (PSNR). Results : The mean overall similarity of the synthetic MRT2 images evaluated by radiologists was 80.2%. In the blind classification of the real MRT2 images, the failure rate ranged from 0% to 40%. The MAE value of each image ranged from 13.75 to 34.24 pixels (mean, 21.19 pixels), and the PSNR of each image ranged from 61.96 to 68.16 dB (mean, 64.92 dB). Conclusion : This was the first study to apply GANs to synthesize spine MR images from CT images. Despite the small dataset of 280 pairs, the synthetic MR images were relatively well implemented. Synthesis of medical images using GANs is a new paradigm of artificial intelligence application in medical imaging. We expect that synthesis of MR images from spine CT images using GANs will improve the diagnostic usefulness of CT. To better inform the clinical applications of this technique, further studies are needed involving a large dataset, a variety of pathologies, and other MR sequence of the lumbar spine.

Keywords

References

  1. Beechar VB, Zinn PO, Heck KA, Fuller GN, Han I, Patel AJ, et al. : Spinal epidermoid tumors: case report and review of the literature. Neurospine 15 : 117-122, 2018 https://doi.org/10.14245/ns.1836014.007
  2. Bi L, Kim J, Kumar A, Feng D, Fulham M : Synthesis of Positron Emission Tomography (PET) Images via Multi-channel Generative Adversarial Networks (GANs) in Cardoso MJ, Arbel T (eds) : Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment. Cham : Springer, 2017, pp43-51
  3. Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath AA : Generative adversarial networks: an overview. IEEE Signal Processing Magazine 35 : 53-65, 2018 https://doi.org/10.1109/msp.2017.2765202
  4. Feng R, Badgeley M, Mocco J, Oermann EK : Deep learning guided stroke management: a review of clinical applications. J Neurointerv Surg 10 : 358-362, 2018 https://doi.org/10.1136/neurintsurg-2017-013355
  5. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. : Generative adversarial nets. Adv Neural Inf Process Syst 27 : 2672-2680, 2014
  6. He K, Zhang X, Ren S, Sun J : Deep residual learning for image recognition. Proc IEEE Int Conf Comput Vis 770-778, 2016
  7. Hirasawa T, Aoyama K, Tanimoto T, Ishihara S, Shichijo S, Ozawa T, et al. : Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer 21 : 653-660, 2018 https://doi.org/10.1007/s10120-018-0793-2
  8. Isola P, Zhu JY, Zhou T, Efros AA : Image-to-image translation with conditional adversarial networks. Proc IEEE Int Conf Comput Vis 1125-1134, 2017
  9. Jin CB, Kim H, Jung W, Joo S, Park E, Saem AY, et al : Deep CT to MR synthesis using paired and unpaired data. Sensors (Basel) 19 : 2019, 2019 https://doi.org/10.3390/s19092019
  10. Johnson J, Alahi A, Fei-Fei L : Perceptual Losses for Real-Time Style Transfer and Super-Resolution in Leibe B, Matas J, Sebe N, Welling M (eds) : Computer Vision - ECCV 2016. Cham : Springer, 2016, pp694-711
  11. Kingma DP, Ba J : Adam: a method for stochastic optimization. Available at : https://arxiv.org/abs/1412.6980
  12. Nie D, Trullo R, Lian J, Petitjean C, Ruan S, Wang Q, et al. : Medical image synthesis with context-aware generative adversarial networks. Med Image Comput Comput Assist Interv 10435 : 417-425, 2017
  13. Olczak J, Fahlberg N, Maki A, Razavian AS, Jilert A, Stark A, et al. : Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop 88 : 581-586, 2017 https://doi.org/10.1080/17453674.2017.1344459
  14. Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efros AA : Context encoders: feature learning by inpainting. Proc IEEE Int Conf Comput Vis 2536-2544, 2016
  15. Ulyanov D, Vedaldi A, Lempitsky V : Instance normalization: the missing ingredient for fast stylization. Available at : https://arxiv.org/abs/1607.08022
  16. Wolterink JM, Dinkla AM, Savenije MH, Seevinck PR, van den Berg CA, Isgum I : Deep MR to CT synthesis using unpaired data. Available at : https://arxiv.org/abs/1708.01155
  17. Xu B, Wang N, Chen T, Li M : Empirical evaluation of rectified activations in convolutional network. Available at : https://arxiv.org/abs/1505.00853
  18. Zhao R, Zhao JJ, Dai F, Zhao FQ : A new image secret sharing scheme to identify cheaters. Computer Standards & Interfaces 31 : 252-257, 2009 https://doi.org/10.1016/j.csi.2007.10.012

Cited by

  1. A Review on the Use of Artificial Intelligence in Spinal Diseases vol.14, pp.4, 2020, https://doi.org/10.31616/asj.2020.0147
  2. Generative adversarial networks in ophthalmology: what are these and how can they be used? vol.32, pp.5, 2020, https://doi.org/10.1097/icu.0000000000000794
  3. A review on medical imaging synthesis using deep learning and its clinical applications vol.22, pp.1, 2020, https://doi.org/10.1002/acm2.13121
  4. Applications of Artificial Intelligence in Musculoskeletal Imaging: From the Request to the Report vol.72, pp.1, 2021, https://doi.org/10.1177/0846537120947148
  5. Validation of a gyroscope-based wearable device for real-time position monitoring of patients in a hospital vol.29, pp.4, 2021, https://doi.org/10.3233/thc-202575
  6. Synthesizing High-b-Value Diffusion-weighted Imaging of the Prostate Using Generative Adversarial Networks vol.3, pp.5, 2020, https://doi.org/10.1148/ryai.2021200237
  7. Artificial Intelligence and Computer Vision in Low Back Pain: A Systematic Review vol.18, pp.20, 2020, https://doi.org/10.3390/ijerph182010909
  8. AI musculoskeletal clinical applications: how can AI increase my day-to-day efficiency? vol.51, pp.2, 2022, https://doi.org/10.1007/s00256-021-03876-8