Progress in Medical Physics (한국의학물리학회지:의학물리)

Korean Society of Medical Physics (한국의학물리학회)
권호 표지
DOI QR코드

DOI QR Code

Deep-Learning-Based Molecular Imaging Biomarkers: Toward Data-Driven Theranostics

  • Choi, Hongyoon (Department of Nuclear Medicine, Seoul National University Hospital)
  • Received : 2019.04.18
  • Accepted : 2019.05.11
  • Published : 2019.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Deep learning has been applied to various medical data. In particular, current deep learning models exhibit remarkable performance at specific tasks, sometimes offering higher accuracy than that of experts for discriminating specific diseases from medical images. The current status of deep learning applications to molecular imaging can be divided into a few subtypes in terms of their purposes: differential diagnostic classification, enhancement of image acquisition, and image-based quantification. As functional and pathophysiologic information is key to molecular imaging, this review will emphasize the need for accurate biomarker acquisition by deep learning in molecular imaging. Furthermore, this review addresses practical issues that include clinical validation, data distribution, labeling issues, and harmonization to achieve clinically feasible deep learning models. Eventually, deep learning will enhance the role of theranostics, which aims at precision targeting of pathophysiology by maximizing molecular imaging functional information.

Keywords

References

  1. Ravi D, Wong C, Deligianni F, et al. Deep learning for health informatics. IEEE journal of biomedical and health informatics. 2017;21(1):4-21. https://doi.org/10.1109/JBHI.2016.2636665
  2. Russakovsky O, Deng J, Su H, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision. 2015;115(3):211-252. https://doi.org/10.1007/s11263-015-0816-y
  3. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Paper presented at: Advances in neural information processing systems2012.
  4. Weber GM, Mandl KD, Kohane IS. Finding the missing link for big biomedical data. Jama. 2014;311(24):2479-2480. https://doi.org/10.1001/jama.2014.4228
  5. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115-118. https://doi.org/10.1038/nature21056
  6. Rajpurkar P, Irvin J, Ball RL, et al. Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS medicine. 2018;15(11):e1002686. https://doi.org/10.1371/journal.pmed.1002686
  7. Dhungel N, Carneiro G, Bradley AP. Automated mass detection in mammograms using cascaded deep learning and random forests. Paper presented at: Digital Image Computing: Techniques and Applications (DICTA), 2015 International Conference on2015.
  8. Lakhani P, Sundaram B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology. 2017;284(2):574-582. https://doi.org/10.1148/radiol.2017162326
  9. Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Medical image analysis. 2017;42:60-88. https://doi.org/10.1016/j.media.2017.07.005
  10. Choi H. Deep learning in nuclear medicine and molecular imaging: current perspectives and future directions. Nuclear medicine and molecular imaging. 2018:1-10.
  11. Wang H, Zhou Z, Li Y, et al. Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18 F-FDG PET/CT images. EJNMMI research. 2017;7(1):11. https://doi.org/10.1186/s13550-017-0260-9
  12. Kirienko M, Sollini M, Silvestri G, et al. Convolutional Neural Networks Promising in Lung Cancer T-Parameter Assessment on Baseline FDG-PET/CT. Contrast Media & Molecular Imaging. 2018;2018.
  13. Choi H, Jin KH, Initiative AsDN. Predicting cognitive decline with deep learning of brain metabolism and amyloid imaging. Behavioural brain research. 2018;344:103-109. https://doi.org/10.1016/j.bbr.2018.02.017
  14. Ding Y, Sohn JH, Kawczynski MG, et al. A deep learning model to predict a diagnosis of Alzheimer disease by using 18F-FDG PET of the brain. Radiology. 2018;290(2):456-464. https://doi.org/10.1148/radiol.2018180958
  15. Liu M, Cheng D, Yan W. Classification of Alzheimer's Disease by Combination of Convolutional and Recurrent Neural Networks Using FDG-PET images. Frontiers in neuroinformatics. 2018;12:35. https://doi.org/10.3389/fninf.2018.00035
  16. Liu S, Liu S, Cai W, et al. Multimodal neuroimaging feature learning for multiclass diagnosis of Alzheimer's disease. IEEE Transactions on Biomedical Engineering. 2015;62(4):1132-1140. https://doi.org/10.1109/TBME.2014.2372011
  17. Suk H-I, Lee S-W, Shen D, Initiative AsDN. Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. NeuroImage. 2014;101:569-582. https://doi.org/10.1016/j.neuroimage.2014.06.077
  18. Li F, Tran L, Thung K-H, Ji S, Shen D, Li J. A robust deep model for improved classification of AD/MCI patients. IEEE journal of biomedical and health informatics. 2015;19(5):1610-1616. https://doi.org/10.1109/JBHI.2015.2429556
  19. Choi H, Ha S, Im HJ, Paek SH, Lee DS. Refining diagnosis of Parkinson's disease with deep learning-based interpretation of dopamine transporter imaging. NeuroImage: Clinical. 2017.
  20. Martinez-Murcia FJ, Gorriz JM, Ramirez J, Ortiz A. Convolutional Neural Networks for Neuroimaging in Parkinson's Disease: Is Preprocessing Needed? International journal of neural systems. 2018:1850035-1850035. https://doi.org/10.1142/s0129065718500351
  21. Kim DH, Wit H, Thurston M. Artificial intelligence in the diagnosis of Parkinson's disease from ioflupane-123 single-photon emission computed tomography dopamine transporter scans using transfer learning. Nuclear medicine communications. 2018;39(10):887-893. https://doi.org/10.1097/MNM.0000000000000890
  22. Betancur JA, Hu L-H, Commandeur F, et al. Deep Learning Analysis of Upright-Supine High-Efficiency SPECT Myocardial Perfusion Imaging for Prediction of Obstructive Coronary Artery Disease: A Multicenter Study. Journal of Nuclear Medicine. 2018:jnumed.118.213538.
  23. Xu C, Xu L, Gao Z, et al. Direct detection of pixel-level myocardial infarction areas via a deep-learning algorithm. Paper presented at: International Conference on Medical Image Computing and Computer-Assisted Intervention2017.
  24. Betancur J, Commandeur F, Motlagh M, et al. Deep learning for prediction of obstructive disease from fast myocardial perfusion SPECT: a multicenter study. JACC: Cardiovascular Imaging. 2018;11(11):1654-1663. https://doi.org/10.1016/j.jcmg.2018.01.020
  25. Kim K, Wu D, Gong K, et al. Penalized PET reconstruction using deep learning prior and local linear fitting. IEEE transactions on medical imaging. 2018;37(6):1478-1487. https://doi.org/10.1109/tmi.2018.2832613
  26. Gong K, Catana C, Qi J, Li Q. PET Image Reconstruction Using Deep Image Prior. IEEE transactions on medical imaging. 2018.
  27. Gong K, Guan J, Kim K, et al. Iterative PET image reconstruction using convolutional neural network representation. IEEE transactions on medical imaging. 2019;38(3):675-685. https://doi.org/10.1109/tmi.2018.2869871
  28. Zhu B, Liu JZ, Cauley SF, Rosen BR, Rosen MS. Image reconstruction by domain-transform manifold learning. Nature. 2018;555(7697):487. https://doi.org/10.1038/nature25988
  29. Pfaehler E, De Jong JR, Dierckx RA, van Velden FH, Boellaard R. SMART (SiMulAtion and ReconsTruction) PET: an efficient PET simulation-reconstruction tool. EJNMMI physics. 2018;5(1):16. https://doi.org/10.1186/s40658-018-0215-x
  30. Hwang D, Kang SK, Kim KY, et al. Generation of PET attenuation map for whole-body time-of-f light 18F-FDG PET/MRI using a deep neural network trained with simultaneously reconstructed activity and attenuation maps. Journal of Nuclear Medicine. 2019:jnumed. 118.219493.
  31. Han X. MR-based synthetic CT generation using a deep convolutional neural network method. Medical physics. 2017;44(4):1408-1419. https://doi.org/10.1002/mp.12155
  32. Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep learning MR imaging-based attenuation correction for PET/MR imaging. Radiology. 2017;286(2):676-684. https://doi.org/10.1148/radiol.2017170700
  33. Leynes AP, Yang J, Wiesinger F, et al. Direct pseudoCT generation for pelvis PET/MRI attenuation correction using deep convolutional neural networks with multi-parametric MRI: zero echo-time and dixon deep pseudoCT (ZeDD-CT). Journal of Nuclear Medicine. 2017:jnumed.117.198051.
  34. Hwang D, Kim KY, Kang SK, et al. Improving the accuracy of simultaneously reconstructed activity and attenuation maps using deep learning. Journal of Nuclear Medicine. 2018;59(10):1624-1629. https://doi.org/10.2967/jnumed.117.202317
  35. Xiang L, Qiao Y, Nie D, et al. Deep auto-context convolutional neural networks for standard-dose PET image estimation from low-dose PET/MRI. Neurocomputing. 2017;267:406-416. https://doi.org/10.1016/j.neucom.2017.06.048
  36. Chen KT, Gong E, de Carvalho Macruz FB, et al. Ultra-Low-Dose 18F-Florbetaben Amyloid PET Imaging Using Deep Learning with Multi-Contrast MRI Inputs. Radiology. 2018:180940.
  37. Wang Y, Yu B, Wang L, et al. 3D conditional generative adversarial networks for high-quality PET image estimation at low dose. Neuroimage. 2018;174:550-562. https://doi.org/10.1016/j.neuroimage.2018.03.045
  38. Wang T, Lei Y, Tang H, et al. A learning-based automatic segmentation and quantification method on left ventricle in gated myocardial perfusion SPECT imaging: A feasibility study. Journal of Nuclear Cardiology. 2019:1-12.
  39. Belal SL, Sadik M, Kaboteh R, et al. Deep Learning for Segmentation of 49 Selected Bones in CT Scans: First Step in Automated PET/CT-based 3D Quantification of Skeletal Metastases. European Journal of Radiology. 2019.
  40. Chen L, Shen C, Zhou Z, et al. Automatic PET cervical tumor segmentation by combining deep learning and anatomic prior. Physics in medicine and biology. 2019.
  41. Zhong Z, Kim Y, Plichta K, et al. Simultaneous cosegmentation of tumors in PET-CT images using deep fully convolutional networks. Medical physics. 2019;46(2):619-633. https://doi.org/10.1002/mp.13331
  42. Huang B, Chen Z, Wu P-M, et al. Fully Automated Delineation of Gross Tumor Volume for Head and Neck Cancer on PET-CT Using Deep Learning: A Dual-Center Study. Contrast media & molecular imaging. 2018;2018.
  43. Choi H, Lee DS. Generation of structural MR images from amyloid PET: Application to MR-less quantification. Journal of Nuclear Medicine. 2018;59(7):1111-1117. https://doi.org/10.2967/jnumed.117.199414
  44. Kang SK, Seo S, Shin SA, et al. Adaptive template generation for amyloid PET using a deep learning approach. Human brain mapping. 2018;39(9):3769-3778. https://doi.org/10.1002/hbm.24210
  45. Ding Y, Sohn JH, Kawczynski MG, et al. A deep learning model to predict a diagnosis of Alzheimer disease by using 18F-FDG PET of the brain. Radiology. 2018:180958.
  46. Samarin A, Burger C, Wollenweber SD, et al. PET/MR imaging of bone lesions-implications for PET quantification from imperfect attenuation correction. European journal of nuclear medicine and molecular imaging. 2012;39(7):1154-1160. https://doi.org/10.1007/s00259-012-2113-0
  47. Choi H, Cheon GJ, Kim H-J, et al. Segmentation-based MR attenuation correction including bones also affects quantitation in brain studies: an initial result of 18F-FPCIT PET/MR for patients with parkinsonism. Journal of Nuclear Medicine. 2014;55(10):1617-1622. https://doi.org/10.2967/jnumed.114.138636
  48. Park J, Bae S, Seo S, et al. Measurement of Glomerular Filtration Rate using Quantitative SPECT/CT and Deeplearning-based Kidney Segmentation. Scientific reports. 2019;9(1):4223. https://doi.org/10.1038/s41598-019-40710-7
  49. Beam AL, Kohane IS. Translating artificial intelligence into clinical care. Jama. 2016;316(22):2368-2369. https://doi.org/10.1001/jama.2016.17217
  50. He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nature medicine. 2019;25(1):30. https://doi.org/10.1038/s41591-018-0307-0
  51. Saria S, Butte A, Sheikh A. Better medicine through machine learning: What's real, and what's artificial? : Public Library of Science; 2018.
  52. Park SH, Han K. Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology. 2018;286(3):800-809. https://doi.org/10.1148/radiol.2017171920
  53. Redelmeier DA, Shafir E. Medical decision making in situations that offer multiple alternatives. Jama. 1995;273(4):302-305. https://doi.org/10.1001/jama.1995.03520280048038
  54. Gal Y, Ghahramani Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. Paper presented at: international conference on machine learning2016.
  55. Wei Q, Ren Y, Hou R, Shi B, Lo JY, Carin L. Anomaly detection for medical images based on a one-class classification. Paper presented at: Medical Imaging 2018: Computer-Aided Diagnosis2018.
  56. Schlegl T, Seebock P, Waldstein SM, Schmidt-Erfurth U, Langs G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. Paper presented at: International Conference on Information Processing in Medical Imaging2017.
  57. Choi H, Kang H, Lee DS, Initiative AsDN. Predicting aging of brain metabolic topography using variational autoencoder. Frontiers in aging neuroscience. 2018;10.
  58. Le QV, Ranzato MA, Monga R, et al. Building high-level features using large scale unsupervised learning. arXiv preprint arXiv:11126209. 2011.
  59. Bengio Y. Deep learning of representations for unsupervised and transfer learning. Paper presented at: Proceedings of ICML Workshop on Unsupervised and Transfer Learning2012.
  60. Rasmus A, Berglund M, Honkala M, Valpola H, Raiko T. Semi-supervised learning with ladder networks. Paper presented at: Advances in neural information processing systems2015.
  61. Odena A. Semi-supervised learning with generative adversarial networks. arXiv preprint arXiv:160601583. 2016.
  62. Choi H, Na KJ. Integrative analysis of imaging and transcriptomic data of the immune landscape associated with tumor metabolism in lung adenocarcinoma: Clinical and prognostic implications. Theranostics. 2018;8(7):1956. https://doi.org/10.7150/thno.23767
  63. Shin H-C, Lu L, Kim L, Seff A, Yao J, Summers RM. Interleaved text/image deep mining on a very large-scale radiology database. Paper presented at: Proceedings of the IEEE conference on computer vision and pattern recognition2015.
  64. Gomez L, Patel Y, Rusinol M, Karatzas D, Jawahar C. Selfsupervised learning of visual features through embedding images into text topic spaces. Paper presented at: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition2017.
  65. Pan L, Cheng C, Haberkorn U, Dimitrakopoulou-Strauss A. Machine learning-based kinetic modeling: a robust and reproducible solution for quantitative analysis of dynamic PET data. Physics in Medicine & Biology. 2017;62(9):3566. https://doi.org/10.1088/0031-9155/62/9/3566
  66. Landau SM, Breault C, Joshi AD, et al. Amyloid-${\beta}$ imaging with Pittsburgh compound B and florbetapir: comparing radiotracers and quantification methods. Journal of Nuclear Medicine. 2013;54(1):70-77. https://doi.org/10.2967/jnumed.112.109009
  67. Klunk WE, Koeppe RA, Price JC, et al. The Centiloid Project: standardizing quantitative amyloid plaque estimation by PET. Alzheimer's & dementia. 2015;11(1):1-15. e14. https://doi.org/10.1016/j.jalz.2014.07.003