Investigative Magnetic Resonance Imaging

Korean Society of Magnetic Resonance in Medicine (대한자기공명의과학회)
권호 표지
DOI QR코드

DOI QR Code

A Comparative Study of Unsupervised Deep Learning Methods for MRI Reconstruction

  • He, Zhuonan (Department of Electronic Information Engineering, Nanchang University) ;
  • Quan, Cong (Department of Electronic Information Engineering, Nanchang University) ;
  • Wang, Siyuan (Department of Electronic Information Engineering, Nanchang University) ;
  • Zhu, Yuanzheng (Department of Electronic Information Engineering, Nanchang University) ;
  • Zhang, Minghui (Department of Electronic Information Engineering, Nanchang University) ;
  • Zhu, Yanjie (Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences) ;
  • Liu, Qiegen (Department of Electronic Information Engineering, Nanchang University)
  • Received : 2020.07.03
  • Accepted : 2020.11.10
  • Published : 2020.12.31
⟨ Previous Next ⟩

Abstract

Recently, unsupervised deep learning methods have shown great potential in image processing. Compared with a large-amount demand for paired training data of supervised methods with a specific task, unsupervised methods can learn a universal and explicit prior information on data distribution and integrate it into the reconstruction process. Therefore, it can be used in various image reconstruction environments without showing degraded performance. The importance of unsupervised learning in MRI reconstruction appears to be growing. Nevertheless, the establishment of prior formulation in unsupervised deep learning varies a lot depending on mathematical approximation and network architectures. In this work, we summarized basic concepts of unsupervised deep learning comprehensively and compared performances of several state-of-the-art unsupervised learning methods for MRI reconstruction.

Keywords

References

  1. Heidemann RM, Ozsarlak O, Parizel PM, et al. A brief review of parallel magnetic resonance imaging. Eur Radiol 2003;13:2323-2337 https://doi.org/10.1007/s00330-003-1992-7
  2. Donoho DL. Compressed sensing. IEEE Trans Inf Theory 2006;52:1289-1306 https://doi.org/10.1109/TIT.2006.871582
  3. Lustig M, Donoho DL, Santos JM, Pauly JM. Compressed sensing MRI. IEEE Signal Proc Mag 2008;25:72-82 https://doi.org/10.1109/MSP.2007.914728
  4. Liang D, Liu B, Wang J, Ying L. Accelerating SENSE using compressed sensing. Magn Reson Med 2009;62:1574-1584 https://doi.org/10.1002/mrm.22161
  5. Huang J, Zhang S, Metaxas D. Efficient MR image reconstruction for compressed MR imaging. Med Image Anal 2011;15:670-679 https://doi.org/10.1016/j.media.2011.06.001
  6. Lin FH, Kwong KK, Belliveau JW, Wald LL. Parallel imaging reconstruction using automatic regularization. Magn Reson Med 2004;51:559-567 https://doi.org/10.1002/mrm.10718
  7. Dong W, Shi G, Li X, Ma Y, Huang F. Compressive sensing via nonlocal low-rank regularization. IEEE Trans Image Process 2014;23:3618-3632 https://doi.org/10.1109/TIP.2014.2329449
  8. Eksioglu EM. Decoupled algorithm for MRI reconstruction using nonlocal block matching model: BM3D-MRI. J Math Imaging Vis 2016;56:430-440 https://doi.org/10.1007/s10851-016-0647-7
  9. Ravishankar S, Bresler Y. MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans Med Imaging 2011;30:1028-1041 https://doi.org/10.1109/TMI.2010.2090538
  10. Huang Y, Paisley J, Lin Q, Ding X, Fu X, Zhang XP. Bayesian nonparametric dictionary learning for compressed sensing MRI. IEEE Trans Image Process 2014;23:5007-5019 https://doi.org/10.1109/TIP.2014.2360122
  11. Wang S, Su Z, Ying L, et al. Accelerating magnetic resonance imaging via deep learning. Proc IEEE Int Symp Biomed Imaging, 2016:514-517
  12. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. MICCAI 2015: Medical Image Computing and Computer-Assisted Intervention, 2015:234-241
  13. Lee D, Yoo J, Ye JC. Deep residual learning for compressed sensing MRI. Proc IEEE Int Symp Biomed Imaging, 2017:15-18
  14. Schlemper J, Caballero J, Hajnal JV, Price A, Rueckert D. A deep cascade of convolutional neural networks for MR image reconstruction. IMPI 2017: Information Processing in Medical Imaging, 2017:647-658
  15. Aggarwal HK, Mani MP, Jacob M. MoDL: model-based deep learning architecture for inverse problems. IEEE Trans Med Imaging 2019;38:394-405 https://doi.org/10.1109/tmi.2018.2865356
  16. Liu Q, Yang Q, Cheng H, Wang S, Zhang M, Liang D. Highly undersampled magnetic resonance imaging reconstruction using autoencoding priors. Magn Reson Med 2020;83:322-336 https://doi.org/10.1002/mrm.27921
  17. Zhang M, Li M, Zhou J, et al. High-dimensional embedding network derived prior for compressive sensing MRI reconstruction. Med Image Anal 2020;64:101717 https://doi.org/10.1016/j.media.2020.101717
  18. Tezcan KC, Baumgartner CF, Luechinger R, Pruessmann KP, Konukoglu E. MR image reconstruction using deep density Priors. IEEE Trans Med Imaging 2019;38:1633-1642 https://doi.org/10.1109/tmi.2018.2887072
  19. Mardani M, Gong E, Cheng JY, et al. Deep generative adversarial neural networks for compressive sensing MRI. IEEE Trans Med Imaging 2019;38:167-179 https://doi.org/10.1109/TMI.2018.2858752
  20. Mardani M, Monajemi H, Papyan V, Vasanawala S, Donoho D, Pauly J. Recurrent generative adversarial networks for proximal learning and automated compressive image recovery. arXiv preprint arXiv 2017:1711.10046
  21. Luo G, Zhao N, Jiang W, Hui ES, Cao P. MRI reconstruction using deep Bayesian estimation. Magn Reson Med 2020;84:2246-2261 https://doi.org/10.1002/mrm.28274
  22. Hussein S, Kandel P, Bolan CW, Wallace MB, Bagci U. Lung and Pancreatic Tumor Characterization in the deep learning era: novel supervised and unsupervised learning approaches. IEEE Trans Med Imaging 2019;38:1777-1787 https://doi.org/10.1109/tmi.2019.2894349
  23. Antun V, Renna F, Poon C, Adcock B, Hansen AC. On instabilities of deep learning in image reconstruction and the potential costs of AI. Proc Natl Acad Sci U S A & 2020;117:30088-30095 https://doi.org/10.1073/pnas.1907377117
  24. Hastie T, Tibshirani R, Friedman J. The elements of statistical learning: data mining, inference, and prediction. Springer Science & Business Media, 2009
  25. Bengio Y, Courville AC, Vincent P. Unsupervised feature learning and deep learning: a review and new perspectives. CoRR 2012;abs/1206.5538
  26. Erhan D, Courville A, Bengio Y, Vincent P. Why does unsupervised pre-training help deep learning- JMLR Workshop and Conference Proceedings, 2010:201-208
  27. Bengio Y, LeCun Y. Scaling learning algorithms towards AI. Large-scale Kernel Machines 2007;34:1-41
  28. Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 2010;11:3371-3408
  29. Vincent P, Larochelle H, Bengio Y, Manzagol P. Extracting and composing robust features with denoising autoencoders. In Proceedings of International Conference on Machine Learning, 2008:1096-1103
  30. Rifai S, Vincent P, Muller X, Glorot X, Bengio Y. Contractive auto-encoders: explicit invariance during feature extraction. In Proceeding International Conference on Machine Learning (ICML), 2011:833-840
  31. Lee H, Grosse R, Ranganath R, Ng AY. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceeding International Conference on Machine Learning (ICML), 2009:609-616
  32. Bengio Y. Deep learning of representations: looking forward. International Conference on Statistical Language and Speech Processing, 2013:1-37
  33. Rolfe JT. Discrete variational autoencoders. arXiv preprint arXiv:1609.02200, 2016
  34. Dilokthanakul N, Mediano PAM, Garnelo M, et al. Deep unsupervised clustering with gaussian mixture variational autoencoders. arXiv preprint arXiv:1611.02648, 2016
  35. Kipf TN, Welling M. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308, 2016
  36. Sutskever I, Jozefowicz R, Gregor K, Rezende D, Lillicrap T, Vinyals O. Towards principled unsupervised learning. arXiv preprint arXiv:1511.06440, 2015
  37. Yi Z, Zhang H, Tan P, Gong M. Dualgan: Unsupervised dual learning for image-to-image translation. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017:2849-2857
  38. Yuan Y, Liu S, Zhang J, et al. Unsupervised image superresolution using cycle-in-cycle generative adversarial networks. Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018:701-710
  39. Wang C, Macnaught G, Papanastasiou G, ManGillicray T, Newby D. Unsupervised learning for cross-domain medical image synthesis using deformation invariant cycle consistency networks. International Workshop on Simulation and Synthesis in Medical Imaging, 2018:52-60
  40. Lugmayr A, Danelljan M, Timofte R. Unsupervised learning for real-world super-resolution. IEEE/CVF International Conference on Computer Vision Workshop (ICCV Workshops), 2019:3408-3416
  41. Song Y, Ermon S. Generative modeling by estimating gradients of the data distribution. 33rd Conference on Neural Information Processing Systems, 2019:11918-11930
  42. Kingma DP, Dhariwal P. Glow: Generative flow with invertible 1x1 convolution. Advances in Neural Information Processing Systems 31 (NeurIPS 2018), 2018:10215-10224
  43. Oord A, Kalchbrenner N, Kavukcuoglu K. Pixel recurrent neural networks. arXiv preprint arXiv:1601.06759, 2016
  44. Quan TM, Nguyen-Duc T, Jeong WK. Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans Med Imaging 2018;37:1488-1497 https://doi.org/10.1109/TMI.2018.2820120
  45. Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv: 1511.06434, 2015
  46. Lustig M, Donoho DL, Santos JM, Pauly JM. Compressed sensing MRI. IEEE Signal Proc Mag 2008;25:72-82 https://doi.org/10.1109/MSP.2007.914728
  47. Lin FH, Wang FN, Ahlfors SP, Hamalainen MS, Belliveau JW. Parallel MRI reconstruction using variance partitioning regularization. Magn Reson Med 2007;58:735-744 https://doi.org/10.1002/mrm.21356
  48. Akcakaya M, Nam S, Hu P, et al. Compressed sensing with wavelet domain dependencies for coronary MRI: a retrospective study. IEEE Trans Med Imaging 2011;30:1090-1099 https://doi.org/10.1109/TMI.2010.2089519
  49. Liu Q, Leung H. Synthesis-analysis deconvolutional network for compressed sensing. 2017 IEEE International Conference on Image Processing (ICIP), 2017:1940-1944
  50. Liu Q, Wang S, Yang K, Luo J, Zhu Y, Liang D. Highly undersampled magnetic resonance image reconstruction using two-level Bregman method with dictionary updating. IEEE Trans Med Imaging 2013;32:1290-1301 https://doi.org/10.1109/TMI.2013.2256464
  51. Qu X, Hou Y, Lam F, Guo D, Zhong J, Chen Z. Magnetic resonance image reconstruction from undersampled measurements using a patch-based nonlocal operator. Med Image Anal 2014;18:843-856 https://doi.org/10.1016/j.media.2013.09.007
  52. Xiong J, Liu Q, Wang Y, Xu X. A two-stage convolutional sparse prior model for image restoration. J Vis Commun Image R 2017;48:268-280 https://doi.org/10.1016/j.jvcir.2017.07.002
  53. He J, Liu Q, Christodoulou AG, Ma C, Lam F, Liang ZP. Accelerated High-Dimensional MR Imaging With Sparse Sampling Using Low-Rank Tensors. IEEE Trans Med Imaging 2016;35:2119-2129 https://doi.org/10.1109/TMI.2016.2550204
  54. Hammernik K, Knoll F, Sodickson D, Pock T. Learning a variational model for compressed sensing MRI reconstruction. Proceedings of the International Society of Magnetic Resonance in Medicine (ISMRM), 2016:1088
  55. Mardani M, Gong E, Cheng JY, et al. Deep generative adversarial networks for compressed sensing automates MRI. arXiv preprint arXiv:1706.00051, 2017
  56. Ye JC, Han Y, Cha E. Deep convolutional framelets: a general deep learning framework for inverse problems. SIAM J Imaging Sci 2018;11:991-1048 https://doi.org/10.1137/17M1141771
  57. Alain G, Bengio Y. What regularized auto-encoders learn from the data-generating distribution. J Mach Learn Res 2014;15:3743-3773
  58. Nguyen A, Clune J, Bengio Y, Dosovitskiy A, Yosinski J. Plug & play generative networks: conditional iterative generation of images in latent space. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017:4467-4477
  59. Bigdeli SA, Zwicker M. Image restoration using autoencoding priors. arXiv preprint arXiv:1703.09964, 2017
  60. Bigdeli SA, Zwicker M, Favaro P, Jin M. Deep meanshift priors for image restoration. Advances in Neural Information Processing Systems 30 (NIPS 2017), 2017:763-772
  61. Kingma DP, Welling M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013
  62. Rezende DJ, Mohamed S, Wierstra D. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv:1401.4082, 2014
  63. Bishop CM. Pattern recognition and machine learning. New York, NY: Springer, 2006:738
  64. Salimans T, Karpathy A, Chen X, Kingma DP. Pixelcnn++: improving the PixelCNN with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv:1701.05517, 2017
  65. Larochelle H, Murray I. The neural autoregressive distribution estimator. Proceeding International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, 2011:29-37
  66. Grover A, Dhar M, Ermon S. Flow-GAN: combining maximum likelihood and adversarial learning in generative models. arXiv preprint arXiv:1705.08868, 2017
  67. Dinh L, Krueger D, Bengio Y. Nice: non-linear independent components estimation. arXiv preprint arXiv:1410.8516, 2014
  68. Dinh L, Sohl-Dickstein J, Bengio S. Density estimation using real NVP. arXiv preprint arXiv:1605.08803, 2016
  69. Ma X, Kong X, Zhang S, Hovy E. MaCow: masked convolutional generative flow. Advances in Neural Information Processing Systems 32 (NeurIPS 2019), 2019:5893-5902
  70. Hoogeboom E, Berg R, Welling M. Emerging convolutions for generative normalizing flows. arXiv preprint arXiv:1901.11137, 2019
  71. Uecker M, Hohage T, Block KT, Frahm J. Image reconstruction by regularized nonlinear inversion-joint estimation of coil sensitivities and image content. Magn Reson Med 2008;60:674-682 https://doi.org/10.1002/mrm.21691
  72. Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. Advances in neural information processing systems 27 (NIPS 2014), 2014:2672-2680