ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Transfer-learning-based classification of pathological brain magnetic resonance images

  • Serkan Savas (Department of Computer Engineering, Kirikkale University) ;
  • Cagri Damar (Department of Radiology, Gaziantep University)
  • Received : 2022.08.25
  • Accepted : 2023.07.10
  • Published : 2024.04.20
Download PDF
⟨ Previous Next ⟩

Abstract

Different diseases occur in the brain. For instance, hereditary and progressive diseases affect and degenerate the white matter. Although addressing, diagnosing, and treating complex abnormalities in the brain is challenging, different strategies have been presented with significant advances in medical research. With state-of-art developments in artificial intelligence, new techniques are being applied to brain magnetic resonance images. Deep learning has been recently used for the segmentation and classification of brain images. In this study, we classified normal and pathological brain images using pretrained deep models through transfer learning. The EfficientNet-B5 model reached the highest accuracy of 98.39% on real data, 91.96% on augmented data, and 100% on pathological data. To verify the reliability of the model, fivefold cross-validation and a two-tier cross-test were applied. The results suggest that the proposed method performs reasonably on the classification of brain magnetic resonance images.

Keywords

References

  1. Human Brain Facts. Human brain diseases list-causes, symptoms and treatments. 2020. Available from: https://www.humanbrainfacts.org/human-brain-diseases-list.php [last accessed March 2021].
  2. C. G. Jennings, R. Landman, Y. Zhou, J. Sharma, J. Hyman, J. A. Movshon, Z. Qiu, A. C. Roberts, A. W. Roe, X. Wang, H. Zhou, L. Wang, F. Zhang, R. Desimone, G. Feng, Opportunities and challenges in modeling human brain disorders in transgenic primates, Nat. Neurosci. 19 (2016), 1123-1130. DOI 10.1038/nn.4362.
  3. U. R. Acharya, S. L. Fernandes, J. E. WeiKoh, E. J. Ciaccio, M. K. M. Fabell, U. J. Tanik, V. Rajinikanth, and C. H. Yeong, Automated detection of Alzheimer's disease using brain MRI images-a study with various feature extraction techniques, J. Med. Syst. 43 (2019), 302. DOI 10.1007/s10916-019-1428-9.
  4. N. D. Prins and P. Scheltens, White matter hyperintensities, cognitive impairment and dementia: an update, Nat. Rev. Neurol. 11 (2015), 157-165. DOI 10.1038/nrneurol.2015.10.
  5. N. Sarbu, R. Y. Shih, R. V. Jones, I. Horkayne-Szakaly, L. Oleaga, and J. G. Smirniotopoulos, White matter diseases with radiologic-pathologic correlation, Radiographics 36 (2016), 1426-1447. DOI 10.1148/rg.2016160031.
  6. N. S. van der Knaap, R. Schiffmann, F. Mochel, and N. I. Wolf, Diagnosis, prognosis, and treatment of leukodystrophies, Lancet Neurol. 18 (2019), 962-972. DOI 10.1016/S1474-4422(19)30143-7.
  7. J. M. Wardlaw, M. C. Valdes Hernandez, and S. MunozManiega, What are white matter hyperintensities made of? Relevance to vascular cognitive impairment. J. Am. Heart Assoc. 4 (2015), 001140. DOI 10.1161/JAHA.114.001140.
  8. S. Debette and H. S. Markus, The clinical importance of white matter hyperintensities on brain magnetic resonance imaging: systematic review and meta-analysis, BMJ 26 (2010), 341. DOI 10.1136/bmj.c3666.
  9. H. Y. Hu, Y. N. Ou, X. N. Shen, Y. Qu, Y. H. Ma, Z. T. Wang, Q. Dong, L. Tan, and J. T. Yu, White matter hyperintensities and risks of cognitive impairment and dementia: a systematic review and meta-analysis of 36 prospective studies, Neurosci. Biobehav. Rev. 120 (2021), 16-27. DOI 10.1016/j.neubiorev.2020.11.007.
  10. A. Khalaf, K. Edelman, D. Tudorascu, C. Andreescu, C. F. Reynolds, and H. Aizenstein, White matter hyperintensity accumulation during treatment of late-life depression, Neuropsychopharmacology 40 (2015), 3027-3035. DOI 10.1038/npp.2015.158.
  11. M. J. Kempton, J. R. Geddes, U. Ettinger, S. C. Williams, and P. M. Grasby, Meta-analysis, database, and meta-regression of 98 structural imaging studies in bipolar disorder, Arch. Gen. Psychiatry 65 (2008), 1017-1032. DOI 10.1001/archpsyc.65.9.1017.
  12. M. Kim and V. Jewells, Multimodal image analysis for assessing multiple sclerosis and future prospects powered by artificial intelligence, Semin. Ultrasound CT MRI 41 (2020), 309-318. DOI 10.1053/j.sult.2020.02.005.
  13. S. Savas, N. Topaloglu, O. Kazci, and P. N. Kosar, Classification of carotid artery intima media thickness ultrasound images with deep learning, J. Med. Syst. 43 (2019), 273. DOI 10.1007/s10916-019-1406-2.
  14. M. M. Yapici, A. Tekerek, and N. Topaloglu, Literature review of deep learning research areas, Gazi Muhendislik Bilimleri Dergisi 5 (2019), 188-215. DOI 10.30855/gmbd.2019.03.01.
  15. S. Buyrukoglu, Early detection of Alzheimer's disease using data mining: comparison of ensemble feature selection approaches, Konya J. Eng. Sci. 9 (2021), 50-61. DOI 10.36306/konjes.731624.
  16. R. Butuner and M. H. Calp, Diagnosis and detection of COVID19 from lung tomography images using deep learning and machine learning methods, Int. J. Intell. Syst. Appl. Eng. 10 (2022), 190-200.
  17. Y. Yilmaz and S. Buyrukoglu, Hybrid machine learning model coupled with school closure for forecasting COVID-19 cases in the most affected countries, Hittite J. Sci. Eng. 8 (2021), 123-131. DOI 10.17350/HJSE19030000222.
  18. S. Buyrukoglu and A. Akbas, Machine learning based early prediction of type 2 diabetes: a new hybrid feature selection approach using correlation matrix with heatmap and SFS, Balkan J. Electr. Comput. Eng. 10 (2022), 110-117. DOI 10.17694/bajece.973129.
  19. M. Hanefi Calp, Use of deep learning approaches in cancer diagnosis, In Deep learning for cancer diagnosis. Studies in computational intelligence, Vol. 908, Springer, Singapore, 2021, 249-267. DOI 10.1007/978-981-15-6321-8_15.
  20. J. McCarthy, M. L. Minsky, N. Rochester, C. E. Shannon, A proposal for the Dartmouth Summer Research Project on artificial intelligence, August 31, 1955, AI Mag. 27 (2006), 12. DOI 10.1609/aimag.v27i4.1904.
  21. S. Savas, N. Topaloglu, O. Kazci, and P. N. Kosar, Performance comparison of carotid artery intima media thickness classification by deep learning methods, (International Congress on Human-Computer Interaction, Optimization, and Robotic Applications, Urgup, Turkey), 2019, pp. 125-131. DOI 10.36287/setsci.4.5.025.
  22. T. B. Dyrby, E. Rostrup, W. F. Baare, E. C. van Straaten, F. Barkhof, H. Vrenken, S. Ropele, R. Schmidt, T. Erkinjuntti, L.O. Wahlund, L. Pantoni, D. Inzitari, O.B. Paulson, L.K. Hansen, G. Waldemar, LADIS study group, Segmentation of age-related white matter changes in a clinical multi-center study, Neuroimage 41 (2008), 335-345. DOI 10.1016/j.neuroimage.2008.02.024.
  23. R. Guerrero, C. Qin, O. Oktay, C. Bowles, L. Chen, R. Joules, R. Wolz, M.C. Valdes-Hernandez, D.A. Dickie, J. Wardlaw, D. Rueckert, White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks, NeuroImage Clin. 17 (2018), 918-934. DOI 10.1016/j.nicl.2017.12.022.
  24. A. Ari and D. Hanbay, Bolgesel evrisimsel sinir aglari tabanli MR goruntulerinde tumor tespiti, Gazi universitesi Muhendislik Mimarlik Fakultesi Dergisi 34 (2018), 1395-1408. DOI 10.17341/gazimmfd.460535.
  25. P. Roca, A. Attye, L. Colas, A. Tucholka, P. Rubini, S. Cackowski, J. Ding, J.F. Budzik, F. Renard, S. Doyle, E.L. Barbier, I. Bousaid, R. Casey, S. Vukusic, N. Lassau, S. Verclytte, F. Cotton, B. Brochet, R. Casey, F. Cotton, J. de Seze, P. Douek, F. Guillemin, D. Laplaud, C. Lebrun-Frenay, L. Mansuy, T. Moreau, J. Olaiz, J. Pelletier, C. Rigaud-Bully, B. Stankoff, S. Vukusic, R. Marignier, M. Debouverie, G. Edan, J. Ciron, A. Ruet, N. Collongues, C. Lubetzki, P. Vermersch, P. Labauge, G. Defer, M. Cohen, A. Fromont, S. Wiertlewsky, E. Berger, P. Clavelou, B. Audoin, C. Giannesini, O. Gout, E. Thouvenot, O. Heinzlef, A. al-Khedr, B. Bourre, O. Casez, P. Cabre, A. Montcuquet, A. Creange, J.P. Camdessanche, J. Faure, A. Maurousset, I. Patry, K. Hankiewicz, C. Pottier, N. Maubeuge, C. Labeyrie, C. Nifle, R. Ameli, R. Anxionnat, A. Attye, E. Bannier, C. Barillot, D. Ben Salem, M.P. BoncoeurMartel, F. Bonneville, C. Boutet, J.C. Brisset, F. Cervenanski, B. Claise, O. Commowick, J.M. Constans, P. Dardel, H. Desal, V. Dousset, F. Durand-Dubief, J.C. Ferre, E. Gerardin, T. Glattard, S. Grand, T. Grenier, R. Guillevin, C. Guttmann, A. Krainik, S. Kremer, S. Lion, N. Menjot de Champfleur, L. Mondot, O. Outteryck, N. Pyatigorskaya, J.P. Pruvo, S. Rabaste, J.P. Ranjeva, J.A. Roch, J.C. Sadik, D. SappeyMarinier, J. Savatovsky, J.Y. Tanguy, A. Tourbah, T. Tourdias, Artificial intelligence to predict clinical disability in patients with multiple sclerosis using FLAIR MRI, Diagn. Interv. Imaging 101 (2020), 795-802. DOI 10.1016/j.diii.2020.05.009.
  26. P. H. B. Diniz, T. L. Valente, J. O. Diniz, A. C. Silva, M. Gattass, N. Ventura, B. C. Muniz, and E. L. Gasparetto, Detection of white matter lesion regions in MRI using SLIC0 and convolutional neural network, Comput. Methods Programs Biomed. 167 (2018), 49-63. DOI 10.1016/j.cmpb.2018.04.011.
  27. M. Saritha, K. P. Joseph, and A. T. Mathew, Classification of MRI brain images using combined wavelet entropy based spider web plots and probabilistic neural network, Pattern Recogn. Lett. 34 (2013), 2151-2156. DOI 10.1016/j.patrec.2013.08.017.
  28. S. Lu, Z. Lu, J. Yang, M. Yang, and S. Wang, A pathological brain detection system based on kernel based ELM, Multimed. Tools Appl. 77 (2018), 3715-3728. DOI 10.1007/s11042-016-3559-z.
  29. S. Lu, Z. Lu, and Y.-D. Zhang, Pathological brain detection based on AlexNet and transfer learning, J. Comput. Sci. 30 (2019), 41-47. DOI 10.1016/j.jocs.2018.11.008.
  30. A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet classification with deep convolutional neural networks, Adv. Neural Inf. Process. Syst. 25 (2012), 1097-1105.
  31. S. Lu, S.-H. Wang, and Y.-D. Zhang, Detecting pathological brain via ResNet and randomized neural networks, Heliyon 6 (2020), e05625. DOI 10.1016/j.heliyon.2020.e05625.
  32. K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA), 2016, pp. 770-778.
  33. Z. U. H. Usmani, What is Kaggle, why I participate, what is the impact? 2018, Available from: https://www.kaggle.com/getting-started/44916 [last accessed March 2021].
  34. Google Research. What is Colaboratory? 2021, Available from: https://research.google.com/colaboratory/faq.html [last accessed March 2021].
  35. S. Thrun and L. Pratt, Learning to learn: introduction and overview, In Learning to learn, Springer, Boston, MA, USA, 1998, 3-17.
  36. S. J. Pan and Q. Yang, A survey on transfer learning, IEEE Trans. Knowl. Data Eng. 22 (2010), 1345-1359, 2010, 10.1109/TKDE.2009.191.
  37. M. Tan and Q. Le, EfficientNet: rethinking model scaling for convolutional neural networks, (International Conference on Machine Learning, Long Beach, CA, USA), 2019, pp. 6105-6114.
  38. V. Agarwal, Complete architectural details of all EfficientNet models, Available from: https://towardsdatascience.com/complete-architectural-details-of-all-efficientnet-models-5fd5b736142 [last accessed February 2023].
  39. S. Savas, Detecting the stages of Alzheimer's disease with pre-trained deep learning architectures, Arab J. Sci. Eng. 47 (2022), 2201-2218. DOI 10.1007/s13369-021-06131-3.
  40. F. Uysal, F. Hardalac, O. Peker, T. Tolunay, and N. Tokgoz, Classification of shoulder X-ray images with deep learning ensemble models, Appl. Sci. 11 (2021), 2723. DOI 10.3390/app11062723.
  41. A. A. Pandian and R. Balasubramanian, Performance analysis of texture image retrieval for curvelet, contourlet transform and local ternary pattern using MRI brain tumor image, Int. J. Found. Comput. Sci. Technol. 5 (2015), 33-46. DOI 10.5121/ijfcst.2015.5604.
  42. S. Deepak and P. M. Ameer, Brain tumor classification using deep CNN features via transfer learning, Comput. Biol. Med. 111 (2019), 103345. DOI 10.1016/j.compbiomed.2019.103345.
  43. M. Talo, U. B. Baloglu, O. Yildirim, and U. R. Acharya, Application of deep transfer learning for automated brain abnormality classification using MR images, Cogn. Syst. Res. 54 (2019), 176-188. DOI 10.1016/j.cogsys.2018.12.007.
  44. Z. N. Swati, Q. Zhao, M. Kabir, F. Ali, Z. Ali, S. Ahmed, and J. Lu, Brain tumor classification for MR images using transfer learning and fine-tuning, Comput. Med. Imaging Graph. 75 (2019), 34-46. DOI 10.1016/j.compmedimag.2019.05.001.
  45. D. R. Nayak, R. Dash, and B. Majhi, Automated diagnosis of multi-class brain abnormalities using MRI images: a deep convolutional neural network based method, Pattern Recogn. Lett. 138 (2020), 385-391. DOI 10.1016/j.patrec.2020.04.018.
  46. O. Guler and I. Yucedag, Hand gesture recognition from 2D images by using convolutional capsule neural networks, Arab. J. Sci. Eng. 47 (2022), 1211-1225. DOI 10.1007/s13369-021-05867-2.