Korean Journal of Radiology

The Korean Society of Radiology (대한영상의학회)
권호 표지
DOI QR코드

DOI QR Code

Use of Artificial Intelligence for Reducing Unnecessary Recalls at Screening Mammography: A Simulation Study

  • Yeon Soo Kim (Department of Radiology, Seoul National University Hospital) ;
  • Myoung-jin Jang (Medical Research Collaborating Center, Seoul National University) ;
  • Su Hyun Lee (Department of Radiology, Seoul National University Hospital) ;
  • Soo-Yeon Kim (Department of Radiology, Seoul National University Hospital) ;
  • Su Min Ha (Department of Radiology, Seoul National University Hospital) ;
  • Bo Ra Kwon (Department of Radiology, Seoul National University Hospital Healthcare System Gangnam Center) ;
  • Woo Kyung Moon (Department of Radiology, Seoul National University Hospital) ;
  • Jung Min Chang (Department of Radiology, Seoul National University Hospital)
  • Received : 2022.04.14
  • Accepted : 2022.09.29
  • Published : 2022.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

Objective: To conduct a simulation study to determine whether artificial intelligence (AI)-aided mammography reading can reduce unnecessary recalls while maintaining cancer detection ability in women recalled after mammography screening. Materials and Methods: A retrospective reader study was performed by screening mammographies of 793 women (mean age ± standard deviation, 50 ± 9 years) recalled to obtain supplemental mammographic views regarding screening mammography-detected abnormalities between January 2016 and December 2019 at two screening centers. Initial screening mammography examinations were interpreted by three dedicated breast radiologists sequentially, case by case, with and without AI aid, in a single session. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and recall rate for breast cancer diagnosis were obtained and compared between the two reading modes. Results: Fifty-four mammograms with cancer (35 invasive cancers and 19 ductal carcinomas in situ) and 739 mammograms with benign or negative findings were included. The reader-averaged AUC improved after AI aid, from 0.79 (95% confidence interval [CI], 0.74-0.85) to 0.89 (95% CI, 0.85-0.94) (p < 0.001). The reader-averaged specificities before and after AI aid were 41.9% (95% CI, 39.3%-44.5%) and 53.9% (95% CI, 50.9%-56.9%), respectively (p < 0.001). The reader-averaged sensitivity was not statistically different between AI-unaided and AI-aided readings: 89.5% (95% CI, 83.1%-95.9%) vs. 92.6% (95% CI, 86.2%-99.0%) (p = 0.053), although the sensitivities of the least experienced radiologists before and after AI aid were 79.6% (43 of 54 [95% CI, 66.5%-89.4%]) and 90.7% (49 of 54 [95% CI, 79.7%-96.9%]), respectively (p = 0.031). With AI aid, the reader-averaged recall rate decreased by from 60.4% (95% CI, 57.8%-62.9%) to 49.5% (95% CI, 46.5%-52.4%) (p < 0.001). Conclusion: AI-aided reading reduced the number of recalls and improved the diagnostic performance in our simulation using women initially recalled for supplemental mammographic views after mammography screening.

Keywords

Acknowledgement

This study was supported by a research grant from DongKook Lifescience Co., Ltd., Republic of Korea, and Seoul National University Hospital (grant no. 06-2020-2840).

References

  1. Duffy SW, Vulkan D, Cuckle H, Parmar D, Sheikh S, Smith RA, et al. Effect of mammographic screening from age 40 years on breast cancer mortality (UK Age trial): final results of a randomised, controlled trial. Lancet Oncol 2020;21:1165-1172
  2. Monticciolo DL, Malak SF, Friedewald SM, Eby PR, Newell MS, Moy L, et al. Breast cancer screening recommendations inclusive of all women at average risk: update from the ACR and Society of Breast Imaging. J Am Coll Radiol 2021;18:1280-1288
  3. Lee CS, Sengupta D, Bhargavan-Chatfield M, Sickles EA, Burnside ES, Zuley ML. Association of patient age with outcomes of current-era, large-scale screening mammography: analysis of data from the National Mammography Database. JAMA Oncol 2017;3:1134-1136
  4. Loberg M, Lousdal ML, Bretthauer M, Kalager M. Benefits and harms of mammography screening. Breast Cancer Res 2015;17:63
  5. Blanchard K, Colbert JA, Kopans DB, Moore R, Halpern EF, Hughes KS, et al. Long-term risk of false-positive screening results and subsequent biopsy as a function of mammography use. Radiology 2006;240:335-342
  6. Lehman CD, Arao RF, Sprague BL, Lee JM, Buist DS, Kerlikowske K, et al. National performance benchmarks for modern screening digital mammography: update from the Breast Cancer Surveillance Consortium. Radiology 2017;283:49-58
  7. Lee CS, Parise C, Burleson J, Seidenwurm D. Assessing the recall rate for screening mammography: comparing the medicare hospital compare dataset with the national mammography database. AJR Am J Roentgenol 2018;211:127-132
  8. Conant EF, Barlow WE, Herschorn SD, Weaver DL, Beaber EF, Tosteson ANA, et al. Association of digital breast tomosynthesis vs digital mammography with cancer detection and recall rates by age and breast density. JAMA Oncol 2019;5:635-642
  9. Marinovich ML, Hunter KE, Macaskill P, Houssami N. Breast cancer screening using tomosynthesis or mammography: a meta-analysis of cancer detection and recall. J Natl Cancer Inst 2018;110:942-949
  10. Dang PA, Freer PE, Humphrey KL, Halpern EF, Rafferty EA. Addition of tomosynthesis to conventional digital mammography: effect on image interpretation time of screening examinations. Radiology 2014;270:49-56
  11. Fenton JJ, Taplin SH, Carney PA, Abraham L, Sickles EA, D'Orsi C, et al. Influence of computer-aided detection on performance of screening mammography. N Engl J Med 2007;356:1399-1409
  12. Lehman CD, Wellman RD, Buist DS, Kerlikowske K, Tosteson AN, Miglioretti DL. Diagnostic accuracy of digital screening mammography with and without computer-aided detection. JAMA Intern Med 2015;175:1828-1837
  13. Dembrower K, Wahlin E, Liu Y, Salim M, Smith K, Lindholm P, et al. Effect of artificial intelligence-based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study. Lancet Digit Health 2020;2:e468-e474
  14. Kim HE, Kim HH, Han BK, Kim KH, Han K, Nam H, et al. Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. Lancet Digit Health 2020;2:e138-e148
  15. Raya-Povedano JL, Romero-Martin S, Elias-Cabot E, Gubern-Merida A, Rodriguez-Ruiz A, Alvarez-Benito M. AI-based strategies to reduce workload in breast cancer screening with mammography and tomosynthesis: a retrospective evaluation. Radiology 2021;300:57-65
  16. Yi C, Tang Y, Ouyang R, Zhang Y, Cao Z, Yang Z, et al. The added value of an artificial intelligence system in assisting radiologists on indeterminate BI-RADS 0 mammograms. Eur Radiol 2022;32:1528-1537
  17. D'Orsi CJ, Sickles EA, Mendelson EB, Morris EA. ACR BI-RADS atlas: breast imaging reporting and data system. Reston, VA: American College of Radiology, 2013
  18. Kim EK, Kim HE, Han K, Kang BJ, Sohn YM, Woo OH, et al. Applying data-driven imaging biomarker in mammography for breast cancer screening: preliminary study. Sci Rep 2018;8:2762
  19. Obuchowski Jr NA, Rockette Jr HE. Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests an anova approach with dependent observations. Commun Stat B: Simul Comput 1995;24:285-308
  20. Smith BJ, Hillis SL. Multi-reader multi-case analysis of variance software for diagnostic performance comparison of imaging modalities. Proc SPIE Int Soc Opt Eng 2020;11316:113160K
  21. Rodriguez-Ruiz A, Krupinski E, Mordang JJ, Schilling K, Heywang-Kobrunner SH, Sechopoulos I, et al. Detection of breast cancer with mammography: effect of an artificial intelligence support system. Radiology 2019;290:305-314
  22. McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, et al. International evaluation of an AI system for breast cancer screening. Nature 2020;577:89-94
  23. Rodriguez-Ruiz A, Lang K, Gubern-Merida A, Broeders M, Gennaro G, Clauser P, et al. Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists. J Natl Cancer Inst 2019;111:916-922
  24. Klompenhouwer EG, Duijm LE, Voogd AC, den Heeten GJ, Nederend J, Jansen FH, et al. Variations in screening outcome among pairs of screening radiologists at non-blinded double reading of screening mammograms: a population-based study. Eur Radiol 2014;24:1097-1104
  25. Sharpe RE Jr, Venkataraman S, Phillips J, Dialani V, FeinZachary VJ, Prakash S, et al. Increased cancer detection rate and variations in the recall rate resulting from implementation of 3D digital breast tomosynthesis into a population-based screening program. Radiology 2016;278:698-706
  26. Kim SH, Lee EH, Jun JK, Kim YM, Chang YW, Lee JH, et al. Interpretive performance and inter-observer agreement on digital mammography test sets. Korean J Radiol 2019;20:218-224
  27. Engelken F, Bremme R, Bick U, Hammann-Kloss S, Fallenberg EM. Factors affecting the rate of false positive marks in CAD in full-field digital mammography. Eur J Radiol 2012;81:e844-e848
  28. Taylor-Phillips S, Wallis MG, Jenkinson D, Adekanmbi V, Parsons H, Dunn J, et al. Effect of using the same vs different order for second readings of screening mammograms on rates of breast cancer detection: a randomized clinical trial. JAMA 2016;315:1956-1965
  29. Timmers JM, den Heeten GJ, Adang EM, Otten JD, Verbeek AL, Broeders MJ. Dutch digital breast cancer screening: implications for breast cancer care. Eur J Public Health 2012;22:925-929
  30. Gur D, Bandos AI, Cohen CS, Hakim CM, Hardesty LA, Ganott MA, et al. The "laboratory" effect: comparing radiologists' performance and variability during prospective clinical and laboratory mammography interpretations. Radiology 2008;249:47-53