Korean Journal of Radiology

  • 제23권3호
  • /
  • Pages.343-354
  • /
  • 2022
  • /
  • 1229-6929(pISSN)
  • /
  • 2005-8330(eISSN)
대한영상의학회 (The Korean Society of Radiology)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning-Assisted Diagnosis of Pediatric Skull Fractures on Plain Radiographs

  • Jae Won Choi (Department of Radiology, Seoul National University College of Medicine) ;
  • Yeon Jin Cho (Department of Radiology, Seoul National University College of Medicine) ;
  • Ji Young Ha (Department of Radiology, Gyeongsang National University Changwon Hospital) ;
  • Yun Young Lee (Department of Radiology, Chonnam National University Hospital) ;
  • Seok Young Koh (Department of Radiology, Seoul National University Hospital) ;
  • June Young Seo (Department of Radiology, Seoul National University Hospital) ;
  • Young Hun Choi (Department of Radiology, Seoul National University College of Medicine) ;
  • Jung-Eun Cheon (Department of Radiology, Seoul National University College of Medicine) ;
  • Ji Hoon Phi (Division of Pediatric Neurosurgery, Seoul National University Children's Hospital) ;
  • Injoon Kim (Department of Emergency Medicine, Armed Forces Yangju Hospital) ;
  • Jaekwang Yang (Army Aviation Operations Command) ;
  • Woo Sun Kim (Department of Radiology, Seoul National University College of Medicine)
  • 투고 : 2021.06.02
  • 심사 : 2021.11.07
  • 발행 : 2022.03.01
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Objective: To develop and evaluate a deep learning-based artificial intelligence (AI) model for detecting skull fractures on plain radiographs in children. Materials and Methods: This retrospective multi-center study consisted of a development dataset acquired from two hospitals (n = 149 and 264) and an external test set (n = 95) from a third hospital. Datasets included children with head trauma who underwent both skull radiography and cranial computed tomography (CT). The development dataset was split into training, tuning, and internal test sets in a ratio of 7:1:2. The reference standard for skull fracture was cranial CT. Two radiology residents, a pediatric radiologist, and two emergency physicians participated in a two-session observer study on an external test set with and without AI assistance. We obtained the area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity along with their 95% confidence intervals (CIs). Results: The AI model showed an AUROC of 0.922 (95% CI, 0.842-0.969) in the internal test set and 0.870 (95% CI, 0.785-0.930) in the external test set. The model had a sensitivity of 81.1% (95% CI, 64.8%-92.0%) and specificity of 91.3% (95% CI, 79.2%-97.6%) for the internal test set and 78.9% (95% CI, 54.4%-93.9%) and 88.2% (95% CI, 78.7%-94.4%), respectively, for the external test set. With the model's assistance, significant AUROC improvement was observed in radiology residents (pooled results) and emergency physicians (pooled results) with the difference from reading without AI assistance of 0.094 (95% CI, 0.020-0.168; p = 0.012) and 0.069 (95% CI, 0.002-0.136; p = 0.043), respectively, but not in the pediatric radiologist with the difference of 0.008 (95% CI, -0.074-0.090; p = 0.850). Conclusion: A deep learning-based AI model improved the performance of inexperienced radiologists and emergency physicians in diagnosing pediatric skull fractures on plain radiographs.

키워드

참고문헌

  1. Marin JR, Weaver MD, Yealy DM, Mannix RC. Trends in visits for traumatic brain injury to emergency departments in the United States. JAMA 2014;311:1917-1919
  2. Greenes DS, Schutzman SA. Clinical indicators of intracranial injury in head-injured infants. Pediatrics 1999;104:861-867
  3. Schutzman SA, Greenes DS. Pediatric minor head trauma. Ann Emerg Med 2001;37:65-74
  4. Expert Panel on Pediatric Imaging, Ryan ME, Pruthi S, Desai NK, Falcone RA Jr, Glenn OA, et al. ACR appropriateness criteria® head trauma-child. J Am Coll Radiol 2020;17:S125-S137
  5. Burstein B, Upton JEM, Terra HF, Neuman MI. Use of CT for head trauma: 2007-2015. Pediatrics 2018;142:e20180814
  6. Kim HB, Kim DK, Kwak YH, Shin SD, Song KJ, Lee SC, et al. Epidemiology of traumatic head injury in Korean children. J Korean Med Sci 2012;27:437-442
  7. Furtado LMF, da Costa Val Filho JA, Dos Santos AR, E Sa RF, Sandes BL, Hon Y, et al. Pediatric minor head trauma in Brazil and external validation of PECARN rules with a cost-effectiveness analysis. Brain Inj 2020;34:1467-1471
  8. Carriere B, Clement K, Gravel J. Variation in the use of skull radiographs by emergency physicians in young children with minor head trauma. CJEM 2014;16:281-287
  9. Expert Panel on Pediatric Imaging, Wootton-Gorges SL, Soares BP, Alazraki AL, Anupindi SA, Blount JP, et al. ACR appropriateness criteria® suspected physical abuse-child. J Am Coll Radiol 2017;14:S338-S349
  10. Tang PH, Lim CC. Imaging of accidental paediatric head trauma. Pediatr Radiol 2009;39:438-446
  11. Paul AR, Adamo MA. Non-accidental trauma in pediatric patients: a review of epidemiology, pathophysiology, diagnosis and treatment. Transl Pediatr 2014;3:195-207
  12. Rajaram S, Batty R, Rittey CD, Griffiths PD, Connolly DJ. Neuroimaging in non-accidental head injury in children: an important element of assessment. Postgrad Med J 2011;87:355-361
  13. Idriz S, Patel JH, Ameli Renani S, Allan R, Vlahos I. CT of normal developmental and variant anatomy of the pediatric skull: distinguishing trauma from normality. Radiographics 2015;35:1585-1601
  14. George CLS, Harper NS, Guillaume D, Cayci Z, Nascene D. Vascular channel mimicking a skull fracture. J Pediatr 2017;181:326
  15. Chung S, Schamban N, Wypij D, Cleveland R, Schutzman SA. Skull radiograph interpretation of children younger than two years: how good are pediatric emergency physicians? Ann Emerg Med 2004;43:718-722
  16. Do S, Song KD, Chung JW. Basics of deep learning: a radiologist's guide to understanding published radiology articles on deep learning. Korean J Radiol 2020;21:33-41
  17. Chea P, Mandell JC. Current applications and future directions of deep learning in musculoskeletal radiology. Skeletal Radiol 2020;49:183-197
  18. Dutta A, Zisserman A. The VIA annotation software for images, audio and video. Proceedings of the 27th ACM International Conference on Multimedia; 2019 Oct 21-25; New York, NY, USA: Association for Computing Machinery; 2019; p. 2276-2279
  19. Redmon J, Farhadi A. YOLOv3: an incremental improvement. arXiv [Preprint]. 2018 [cited 2020 December 14]. Available at: https://arxiv.org/abs/1804.02767
  20. Youden WJ. Index for rating diagnostic tests. Cancer 1950;3:32-35
  21. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837-845
  22. Obuchowski Jr NA, Rockette Jr HE. Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests an anova approach with dependent observations. Commun Stat Simul Comput 1995;24:285-308
  23. Hillis SL. A comparison of denominator degrees of freedom methods for multiple observer ROC analysis. Stat Med 2007;26:596-619
  24. Kuppermann N, Holmes JF, Dayan PS, Hoyle JD Jr, Atabaki SM, Holubkov R, et al. Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study. Lancet 2009;374:1160-1170
  25. Chakraborty DP. Observer performance methods for diagnostic imaging: foundations, modeling, and applications with r-based examples. Boca Raton: CRC Press, 2017
  26. Yang S, Yin B, Cao W, Feng C, Fan G, He S. Diagnostic accuracy of deep learning in orthopaedic fractures: a systematic review and meta-analysis. Clin Radiol 2020;75:713.e17-713.e28
  27. Choi JW, Cho YJ, Lee S, Lee J, Lee S, Choi YH, et al. Using a dual-input convolutional neural network for automated detection of pediatric supracondylar fracture on conventional radiography. Invest Radiol 2020;55:101-110
  28. Miglioretti DL, Johnson E, Williams A, Greenlee RT, Weinmann S, Solberg LI, et al. The use of computed tomography in pediatrics and the associated radiation exposure and estimated cancer risk. JAMA Pediatr 2013;167:700-707
  29. Goldwasser T, Bressan S, Oakley E, Arpone M, Babl FE. Use of sedation in children receiving computed tomography after head injuries. Eur J Emerg Med 2015;22:413-418
  30. Babl FE, Lyttle MD, Bressan S, Borland M, Phillips N, Kochar A, et al. A prospective observational study to assess the diagnostic accuracy of clinical decision rules for children presenting to emergency departments after head injuries (protocol): the Australasian Paediatric Head Injury Rules Study (APHIRST). BMC Pediatr 2014;14:148
  31. Easter JS, Bakes K, Dhaliwal J, Miller M, Caruso E, Haukoos JS. Comparison of PECARN, CATCH, and CHALICE rules for children with minor head injury: a prospective cohort study. Ann Emerg Med 2014;64:145-152
  32. Kim YI, Cheong JW, Yoon SH. Clinical comparison of the predictive value of the simple skull X-ray and 3 dimensional computed tomography for skull fractures of children. J Korean Neurosurg Soc 2012;52:528-533
  33. Oh CK, Yoon SH. The significance of incomplete skull fracture in the birth injury. Med Hypotheses 2010;74:898-900
  34. Martin A, Paddock M, Johns CS, Smith J, Raghavan A, Connolly DJA, et al. Avoiding skull radiographs in infants with suspected inflicted injury who also undergo head CT: "a nobrainer?" Eur Radiol 2020;30:1480-1487
  35. Park SH, Choi J, Byeon JS. Key principles of clinical validation, device approval, and insurance coverage decisions of artificial intelligence. Korean J Radiol 2021;22:442-453
  36. Lorton F, Poullaouec C, Legallais E, Simon-Pimmel J, Chene MA, Leroy H, et al. Validation of the PECARN clinical decision rule for children with minor head trauma: a French multicenter prospective study. Scand J Trauma Resusc Emerg Med 2016;24:98
  37. Ide K, Uematsu S, Tetsuhara K, Yoshimura S, Kato T, Kobayashi T. External validation of the PECARN head trauma prediction rules in Japan. Acad Emerg Med 2017;24:308-314
  38. Hwang EJ, Nam JG, Lim WH, Park SJ, Jeong YS, Kang JH, et al. Deep learning for chest radiograph diagnosis in the emergency department. Radiology 2019;293:573-580
  39. Hwang EJ, Park S, Jin KN, Kim JI, Choi SY, Lee JH, et al. Development and validation of a deep learning-based automated detection algorithm for major thoracic diseases on chest radiographs. JAMA Netw Open 2019;2:e191095
  40. Schutzman SA, Barnes P, Duhaime AC, Greenes D, Homer C, Jaffe D, et al. Evaluation and management of children younger than two years old with apparently minor head trauma: proposed guidelines. Pediatrics 2001;107:983-993
  41. Kim Y, Lee KJ, Sunwoo L, Choi D, Nam CM, Cho J, et al. Deep learning in diagnosis of maxillary sinusitis using conventional radiography. Invest Radiol 2019;54:7-15
  42. Reyes M, Meier R, Pereira S, Silva CA, Dahlweid FM, von Tengg-Kobligk H, et al. On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiol Artif Intell 2020;2:e190043