The Korean journal of internal medicine

The Korean Association of Internal Medicine (대한내과학회)
권호 표지
DOI QR코드

DOI QR Code

Artificial intelligence in colonoscopy: from detection to diagnosis

  • Eun Sun Kim (Department of Gastroenterology, Korea University Anam Hospital) ;
  • Kwang-Sig Lee (AI Center, Korea University Anam Hospital)
  • Received : 2023.08.10
  • Accepted : 2023.11.13
  • Published : 2024.07.01
Download PDF
⟨ Previous Next ⟩

Abstract

This study reviews the recent progress of artificial intelligence for colonoscopy from detection to diagnosis. The source of data was 27 original studies in PubMed. The search terms were "colonoscopy" (title) and "deep learning" (abstract). The eligibility criteria were: (1) the dependent variable of gastrointestinal disease; (2) the interventions of deep learning for classification, detection and/or segmentation for colonoscopy; (3) the outcomes of accuracy, sensitivity, specificity, area under the curve (AUC), precision, F1, intersection of union (IOU), Dice and/or inference frames per second (FPS); (3) the publication year of 2021 or later; (4) the publication language of English. Based on the results of this study, different deep learning methods would be appropriate for different tasks for colonoscopy, e.g., Efficientnet with neural architecture search (AUC 99.8%) in the case of classification, You Only Look Once with the instance tracking head (F1 96.3%) in the case of detection, and Unet with dense-dilation-residual blocks (Dice 97.3%) in the case of segmentation. Their performance measures reported varied within 74.0-95.0% for accuracy, 60.0-93.0% for sensitivity, 60.0-100.0% for specificity, 71.0-99.8% for the AUC, 70.1-93.3% for precision, 81.0-96.3% for F1, 57.2-89.5% for the IOU, 75.1-97.3% for Dice and 66-182 for FPS. In conclusion, artificial intelligence provides an effective, non-invasive decision support system for colonoscopy from detection to diagnosis.

Keywords

Acknowledgement

This work was supported by (1) a Technology Innovation Program grant (Development of AI Base Multimodal Endomicroscope for In Situ Diagnosis Cancer) funded by the Ministry of Trade, Industry, and Energy of South Korea (No. 20001533) and (2) a Korea Health Industry Development Institute grant (Korea Health Technology R&D Project) funded by the Ministry of Health and Welfare of South Korea (No. HI22C1302). The funders had no role in the design of the study; in the collection, analysis, and interpretation of the data; or in the writing and review of the manuscript.

References

  1. Johns Hopkins Medicine. Digestive disorders. Baltimore (MD): Johns Hopkins Medicine, c2023 [cited 2023 Jul 1]. Available from: https://www.hopkinsmedicine.org/health/wellness-and-prevention/digestive-disorders.
  2. Milivojevic V, Milosavljevic T. Burden of gastroduodenal diseases from the global perspective. Curr Treat Options Gastroenterol 2020;18:148-157.
  3. Peery AF, Crockett SD, Murphy CC, et al. Burden and cost of gastrointestinal, liver, and pancreatic diseases in the United States: update 2021. Gastroenterology 2022;162:621-644.
  4. Kim YE, Park H, Jo MW, et al. Trends and patterns of burden of disease and injuries in Korea using disability-adjusted life years. J Korean Med Sci 2019;34(Suppl 1):e75.
  5. Jung HK, Jang B, Kim YH, et al. [Health care costs of digestive diseases in Korea]. Korean J Gastroenterol 2011;58:323-331. Korean.
  6. Cleveland Clinic. Health: gastrointestinal diseases. Cleveland (OH): Cleveland Clinic, c2023 [cited 2023 Jul 1]. Available from: https://my.clevelandclinic.org/health/articles/7040-gastrointestinal-diseases.
  7. Peterse EFP, Meester RGS, de Jonge L, et al. Comparing the cost-effectiveness of innovative colorectal cancer screening tests. J Natl Cancer Inst 2021;113:154-161.
  8. Krzeczewski B, Hassan C, Krzeczewska O, et al. Cost-effectiveness of colonoscopy in an organized screening program. Pol Arch Intern Med 2021;131:128-135.
  9. Areia M, Mori Y, Correale L, et al. Cost-effectiveness of artificial intelligence for screening colonoscopy: a modelling study. Lancet Digit Health 2022;4:e436-e444.
  10. Ren Y, Zhao M, Zhou D, Xing Q, Gong F, Tang W. Cost-effectiveness analysis of colonoscopy and fecal immunochemical testing for colorectal cancer screening in China. Front Public Health 2022;10:952378.
  11. Mori Y, East JE, Hassan C, et al. Benefits and challenges in implementation of artificial intelligence in colonoscopy: World Endoscopy Organization position statement. Dig Endosc 2023;35:422-429.
  12. Lee KS, Son SH, Park SH, Kim ES. Automated detection of colorectal tumors based on artificial intelligence. BMC Med Inform Decis Mak 2021;21:33.
  13. Lee KS, Ahn KH. Application of artificial intelligence in early diagnosis of spontaneous preterm labor and birth. Diagnostics (Basel) 2020;10:733.
  14. Lee KS, Park H. Machine learning on thyroid disease: a review. Front Biosci (Landmark Ed) 2022;27:101.
  15. Lee KS, Ham BJ. Machine learning on early diagnosis of depression. Psychiatry Investig 2022;19:597-605.
  16. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems; 2011 Dec 12-17; Siem Reap, Cambodia: Neural Information Processing Systems, 2011: 1097-1105.
  17. Lee KS, Park KW. Social determinants of the association among cerebrovascular disease, hearing loss and cognitive impairment in a middle-aged or older population: recurrent neural network analysis of the Korean Longitudinal Study of Aging (2014-2016). Geriatr Gerontol Int 2019;19:711-716.
  18. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. arXiv [Preprint]. 2015 [cited 2023 Jul 1]. Available from: https://doi.org/10.48550/arXiv.1505.04597.
  19. Tan M, Le QV. EfficientNet: rethinking model scaling for Convolutional Neural Networks. arXiv [Preprint]. 2019 [cited 2023 Jul 1]. Available from: https://doi.org/10.48550/arXiv.1905.11946.
  20. Tan M, Le QV. EfficientNetV2: smaller models and faster training. arXiv [Preprint]. 2021 [cited 2023 Jul 1]. Availble from: https://doi.org/10.48550/arXiv.2104.00298.
  21. Lai LL, Blakely A, Invernizzi M, et al. Separation of color channels from conventional colonoscopy images improves deep neural network detection of polyps. J Biomed Opt 2021;26:015001.
  22. Golhar M, Bobrow TL, Khoshknab MP, Jit S, Ngamruengphong S, Durr NJ. Improving colonoscopy lesion classification using semi-supervised deep learning. IEEE Access 2021;9:631-640.
  23. Safarov S, Whangbo TK. A-DenseUNet: Adaptive densely connected UNet for polyp segmentation in colonoscopy images with atrous convolution. Sensors (Basel) 2021;21:1441.
  24. Jha D, Ali S, Tomar NK, et al. Real-time polyp detection, localization and segmentation in colonoscopy using deep learning. IEEE Access 2021;9:40496-40510.
  25. Wang Y, Feng Z, Song L, Liu X, Liu S. Multiclassification of endoscopic colonoscopy images based on deep transfer learning. Comput Math Methods Med 2021;2021:2485934.
  26. Su H, Lin B, Huang X, Li J, Jiang K, Duan X. MBFFNet: multi-branch feature fusion network for colonoscopy. Front Bioeng Biotechnol 2021;9:696251.
  27. Sziova B, Nagy S, Fazekas Z. Application of structural entropy and spatial filling factor in colonoscopy image classification. Entropy (Basel) 2021;23:936.
  28. Ma R, Wang R, Zhang Y, et al. RNNSLAM: reconstructing the 3D colon to visualize missing regions during a colonoscopy. Med Image Anal 2021;72:102100.
  29. Li K, Fathan MI, Patel K, et al. Colonoscopy polyp detection and classification: dataset creation and comparative evaluations. PLoS One 2021;16:e0255809.
  30. Yeung M, Sala E, Schonlieb CB, Rundo L. Focus U-Net: a novel dual attention-gated CNN for polyp segmentation during colonoscopy. Comput Biol Med 2021;137:104815.
  31. Syed S, Angel AJ, Syeda HB, et al. The h-ANN Model: comprehensive colonoscopy concept compilation using combined contextual embeddings. Biomed Eng Syst Technol Int Jt Conf BIOSTEC Revis Sel Pap 2022;5:189-200.
  32. Wang YP, Jheng YC, Sung KY, et al. Use of U-Net convolutional neural networks for automated segmentation of fecal material for objective evaluation of bowel preparation quality in colonoscopy. Diagnostics (Basel) 2022;12:613.
  33. Nogueira-Rodriguez A, Reboiro-Jato M, Glez-Pena D, Lopez-Fernandez H. Performance of convolutional neural networks for polyp localization on public colonoscopy image datasets. Diagnostics (Basel) 2022;12:898.
  34. Wang L, Chen L, Wang X, et al. Development of a convolutional neural network-based colonoscopy image assessment model for differentiating Crohn's disease and ulcerative colitis. Front Med (Lausanne) 2022;9:789862.
  35. Chen S, Urban G, Baldi P. Weakly supervised polyp segmentation in colonoscopy images using deep neural networks. J Imaging 2022;8:121.
  36. Sharma P, Balabantaray BK, Bora K, Mallik S, Kasugai K, Zhao Z. An ensemble-based deep convolutional neural network for computer-aided polyps identification from colonoscopy. Front Genet 2022;13:844391.
  37. Byeon SJ, Park J, Cho YA, Cho BJ. Automated histological classification for digital pathology images of colonoscopy specimen via deep learning. Sci Rep 2022;12:12804.
  38. Souaidi M, El Ansari M. Multi-scale hybrid network for polyp detection in wireless capsule endoscopy and colonoscopy images. Diagnostics (Basel) 2022;12:2030.
  39. Yu T, Lin N, Zhang X, et al. An end-to-end tracking method for polyp detectors in colonoscopy videos. Artif Intell Med 2022;131:102363.
  40. Mathew S, Nadeem S, Kaufman A. CLTS-GAN: color-lighting-texture-specular reflection augmentation for colonoscopy. Med Image Comput Comput Assist Interv 2022;2022:519-529.
  41. Ramzan M, Raza M, Sharif MI, Kadry S. Gastrointestinal tract polyp anomaly segmentation on colonoscopy images using graft-U-Net. J Pers Med 2022;12:1459.
  42. Cui R, Yang R, Liu F, Cai C. N-Net: Lesion region segmentations using the generalized hybrid dilated convolutions for polyps in colonoscopy images. Front Bioeng Biotechnol 2022;10:963590.
  43. Yamada M, Shino R, Kondo H, et al. Robust automated prediction of the revised Vienna Classification in colonoscopy using deep learning: development and initial external validation. J Gastroenterol 2022;57:879-889.
  44. Gonzalez-Bueno Puyal J, Brandao P, Ahmad OF, et al. Polyp detection on video colonoscopy using a hybrid 2D/3D CNN. Med Image Anal 2022;82:102625.
  45. Raymann J, Rajalakshmi R. GAR-Net: guided attention residual network for polyp segmentation from colonoscopy video frames. Diagnostics (Basel) 2022;13:123.
  46. Tang CP, Chang HY, Wang WC, Hu WX. A novel computer-aided detection/diagnosis system for detection and classification of polyps in colonoscopy. Diagnostics (Basel) 2023;13:170.
  47. Lewis J, Cha YJ, Kim J. Dual encoder-decoder-based deep polyp segmentation network for colonoscopy images. Sci Rep 2023;13:1183.
  48. Li Y. Deep reinforcement learning: an overview. arXiv [Preprint]. 2017 [cited 2023 Jul 1]. Available from: https://doi.org/10.48550/arXiv.1701.07274.
  49. Silver D, Huang A, Maddison CJ, et al. Mastering the game of Go with deep neural networks and tree search. Nature 2016;529:484-489.