The Korean Journal of Physiology and Pharmacology

The Korean Society of Pharmacology (대한약리학회)
권호 표지
DOI QR코드

DOI QR Code

Feasibility of fully automated classification of whole slide images based on deep learning

  • Cho, Kyung-Ok (Department of Pharmacology, Seoul St. Mary's Hospital) ;
  • Lee, Sung Hak (Department of Hospital Pathology, Seoul St. Mary's Hospital) ;
  • Jang, Hyun-Jong (Department of Biomedicine & Health Sciences, Seoul St. Mary's Hospital)
  • Received : 2019.09.23
  • Accepted : 2019.11.27
  • Published : 2020.01.01
Download PDF
⟨ Previous Next ⟩

Abstract

Although microscopic analysis of tissue slides has been the basis for disease diagnosis for decades, intra- and inter-observer variabilities remain issues to be resolved. The recent introduction of digital scanners has allowed for using deep learning in the analysis of tissue images because many whole slide images (WSIs) are accessible to researchers. In the present study, we investigated the possibility of a deep learning-based, fully automated, computer-aided diagnosis system with WSIs from a stomach adenocarcinoma dataset. Three different convolutional neural network architectures were tested to determine the better architecture for tissue classifier. Each network was trained to classify small tissue patches into normal or tumor. Based on the patch-level classification, tumor probability heatmaps can be overlaid on tissue images. We observed three different tissue patterns, including clear normal, clear tumor and ambiguous cases. We suggest that longer inspection time can be assigned to ambiguous cases compared to clear normal cases, increasing the accuracy and efficiency of histopathologic diagnosis by pre-evaluating the status of the WSIs. When the classifier was tested with completely different WSI dataset, the performance was not optimal because of the different tissue preparation quality. By including a small amount of data from the new dataset for training, the performance for the new dataset was much enhanced. These results indicated that WSI dataset should include tissues prepared from many different preparation conditions to construct a generalized tissue classifier. Thus, multi-national/multi-center dataset should be built for the application of deep learning in the real world medical practice.

Keywords

References

  1. Fischer AH, Jacobson KA, Rose J, Zeller R. Hematoxylin and eosin staining of tissue and cell sections. CSH Protoc. 2008;2008:pdb.prot4986.
  2. Dalla Palma P, Giorgi Rossi P, Collina G, Buccoliero AM, Ghiringhello B, Gilioli E, Onnis GL, Aldovini D, Galanti G, Casadei G, Aldi M, Gomes VV, Giubilato P, Ronco G; NTCC Pathology Group. The reproducibility of CIN diagnoses among different pathologists: data from histology reviews from a multicenter randomized study. Am J Clin Pathol. 2009;132:125-132. https://doi.org/10.1309/AJCPBRK7D1YIUWFP
  3. Hamilton PW, van Diest PJ, Williams R, Gallagher AG. Do we see what we think we see? The complexities of morphological assessment. J Pathol. 2009;218:285-291. https://doi.org/10.1002/path.2527
  4. Cheng J, Mo X, Wang X, Parwani A, Feng Q, Huang K. Identification of topological features in renal tumor microenvironment associated with patient survival. Bioinformatics. 2018;34:1024-1030. https://doi.org/10.1093/bioinformatics/btx723
  5. Djuric U, Zadeh G, Aldape K, Diamandis P. Precision histology: how deep learning is poised to revitalize histomorphology for personalized cancer care. NPJ Precis Oncol. 2017;1:22. https://doi.org/10.1038/s41698-017-0022-1
  6. Madabhushi A, Lee G. Image analysis and machine learning in digital pathology: challenges and opportunities. Med Image Anal. 2016;33:170-175. https://doi.org/10.1016/j.media.2016.06.037
  7. de Bruijne M. Machine learning approaches in medical image analysis: from detection to diagnosis. Med Image Anal. 2016;33:94-97. https://doi.org/10.1016/j.media.2016.06.032
  8. Chang HY, Jung CK, Woo JI, Lee S, Cho J, Kim SW, Kwak TY. Artificial Intelligence in Pathology. J Pathol Transl Med. 2019;53:1-12. https://doi.org/10.4132/jptm.2018.12.16
  9. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436-444. https://doi.org/10.1038/nature14539
  10. Jang HJ, Cho KO. Applications of deep learning for the analysis of medical data. Arch Pharm Res. 2019;42:492-504. https://doi.org/10.1007/s12272-019-01162-9
  11. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316:2402-2410. https://doi.org/10.1001/jama.2016.17216
  12. Lee JG, Jun S, Cho YW, Lee H, Kim GB, Seo JB, Kim N. Deep learning in medical imaging: general overview. Korean J Radiol. 2017;18:570-584. https://doi.org/10.3348/kjr.2017.18.4.570
  13. Janowczyk A, Madabhushi A. Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases. J Pathol Inform. 2016;7:29. https://doi.org/10.4103/2153-3539.186902
  14. Kwon JM, Kim KH, Jeon KH, Kim HM, Kim MJ, Lim SM, Song PS, Park J, Choi RK, Oh BH. Development and validation of deeplearning algorithm for electrocardiography-based heart failure identification. Korean Circ J. 2019;49:629-639. https://doi.org/10.4070/kcj.2018.0446
  15. Jang HJ, Cho KO. Dual deep neural network-based classifiers to detect experimental seizures. Korean J Physiol Pharmacol. 2019;23:131-139. https://doi.org/10.4196/kjpp.2019.23.2.131
  16. Saltz J, Gupta R, Hou L, Kurc T, Singh P, Nguyen V, Samaras D, Shroyer KR, Zhao T, Batiste R, Van Arnam J; Cancer Genome Atlas Research Network, Shmulevich I, Rao AUK, Lazar AJ, Sharma A, Thorsson V. Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images. Cell Rep. 2018;23:181-193.e7. https://doi.org/10.1016/j.celrep.2018.03.086
  17. Cooper LA, Demicco EG, Saltz JH, Powell RT, Rao A, Lazar AJ. PanCancer insights from the cancer genome atlas: the pathologist's perspective. J Pathol. 2018;244:512-524. https://doi.org/10.1002/path.5028
  18. Senaras C, Niazi MKK, Lozanski G, Gurcan MN. DeepFocus: detection of out-of-focus regions in whole slide digital images using deep learning. PLoS One. 2018;13:e0205387. https://doi.org/10.1371/journal.pone.0205387
  19. Kim JH, Hong JS, Park HJ. Prospects of deep learning for medical imaging. Precis Future Med. 2018;2;37-52. https://doi.org/10.23838/pfm.2018.00030
  20. Cancer Genome Atlas Research Network. Comprehensive molecular profiling of lung adenocarcinoma. Nature. 2014;511:543-550. https://doi.org/10.1038/nature13385
  21. Cancer Genome Atlas Research Network. Comprehensive genomic characterization of squamous cell lung cancers. Nature. 2012;489: 519-525. https://doi.org/10.1038/nature11404
  22. Khan A, Sohail A, Zahoora U, Qureshi AS. A survey of the recent architectures of deep convolutional neural networks. arXiv. 1901.06032 [Preprint]. 2019. Available from: https://arxiv.org/abs/1901.06032.
  23. Glasziou P. The statistical evaluation of medical tests for classification and prediction. Stat Med. 2005;24:2582. https://doi.org/10.1002/sim.2185
  24. Venkatraman ES. A permutation test to compare receiver operating characteristic curves. Biometrics. 2000;56:1134-1138. https://doi.org/10.1111/j.0006-341X.2000.01134.x
  25. Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with deep convolutional neural networks. In: Bartlett P, Pereira FCN, Burges CJC, Bottou L, Weinberger KQ, editors. Advances in neural information processing systems 25: 26th annual conference on neural information processing systems 2012. Red Hook: Curran Associates Inc.; 2012.
  26. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. arXiv. 1409.4842 [Preprint]. 2014. Available from: https://arxiv.org/abs/1409.4842.
  27. Yu KH, Zhang C, Berry GJ, Altman RB, Ré C, Rubin DL, Snyder M. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat Commun. 2016;7:12474. https://doi.org/10.1038/ncomms12474
  28. Ehteshami Bejnordi B, Veta M, Johannes van Diest P, van Ginneken B, Karssemeijer N, Litjens G, van der Laak JAWM; the CAMELYON16 Consortium. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA. 2017;318:2199-2210. https://doi.org/10.1001/jama.2017.14585
  29. Hamilton PW, Bankhead P, Wang Y, Hutchinson R, Kieran D, McArt DG, James J, Salto-Tellez M. Digital pathology and image analysis in tissue biomarker research. Methods. 2014;70:59-73. https://doi.org/10.1016/j.ymeth.2014.06.015
  30. Kather JN, Pearson AT, Halama N, Jager D, Krause J, Loosen SH, Marx A, Boor P, Tacke F, Neumann UP, Grabsch HI, Yoshikawa T, Brenner H, Chang-Claude J, Hoffmeister M, Trautwein C, Luedde T. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat Med. 2019;25:1054-1056. https://doi.org/10.1038/s41591-019-0462-y

Cited by

  1. Generalizability of Deep Learning System for the Pathologic Diagnosis of Various Cancers vol.11, pp.2, 2021, https://doi.org/10.3390/app11020808
  2. A State-of-the-Art Review for Gastric Histopathology Image Analysis Approaches and Future Development vol.2021, 2020, https://doi.org/10.1155/2021/6671417
  3. Deep Learning for Automatic Subclassification of Gastric Carcinoma Using Whole-Slide Histopathology Images vol.13, pp.15, 2020, https://doi.org/10.3390/cancers13153811
  4. Prediction of genetic alterations from gastric cancer histopathology images using a fully automated deep learning approach vol.27, pp.44, 2021, https://doi.org/10.3748/wjg.v27.i44.7687