Korean Journal of Radiology

The Korean Society of Radiology (대한영상의학회)
권호 표지
DOI QR코드

DOI QR Code

Clinical Validation of a Deep Learning-Based Hybrid (Greulich-Pyle and Modified Tanner-Whitehouse) Method for Bone Age Assessment

  • Kyu-Chong Lee (Department of Radiology, Korea University Anam Hospital) ;
  • Kee-Hyoung Lee (Department of Pediatrics, Korea University Anam Hospital) ;
  • Chang Ho Kang (Department of Radiology, Korea University Anam Hospital) ;
  • Kyung-Sik Ahn (Department of Radiology, Korea University Anam Hospital) ;
  • Lindsey Yoojin Chung (Department of Pediatrics, Myongji Hospital) ;
  • Jae-Joon Lee (Crescom) ;
  • Suk Joo Hong (Department of Radiology, Korea University Guro Hospital) ;
  • Baek Hyun Kim (Department of Radiology, Korea University Ansan Hospital) ;
  • Euddeum Shim (Department of Radiology, Korea University Ansan Hospital)
  • Received : 2020.12.17
  • Accepted : 2021.06.28
  • Published : 2021.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

Objective: To evaluate the accuracy and clinical efficacy of a hybrid Greulich-Pyle (GP) and modified Tanner-Whitehouse (TW) artificial intelligence (AI) model for bone age assessment. Materials and Methods: A deep learning-based model was trained on an open dataset of multiple ethnicities. A total of 102 hand radiographs (51 male and 51 female; mean age ± standard deviation = 10.95 ± 2.37 years) from a single institution were selected for external validation. Three human experts performed bone age assessments based on the GP atlas to develop a reference standard. Two study radiologists performed bone age assessments with and without AI model assistance in two separate sessions, for which the reading time was recorded. The performance of the AI software was assessed by comparing the mean absolute difference between the AI-calculated bone age and the reference standard. The reading time was compared between reading with and without AI using a paired t test. Furthermore, the reliability between the two study radiologists' bone age assessments was assessed using intraclass correlation coefficients (ICCs), and the results were compared between reading with and without AI. Results: The bone ages assessed by the experts and the AI model were not significantly different (11.39 ± 2.74 years and 11.35 ± 2.76 years, respectively, p = 0.31). The mean absolute difference was 0.39 years (95% confidence interval, 0.33-0.45 years) between the automated AI assessment and the reference standard. The mean reading time of the two study radiologists was reduced from 54.29 to 35.37 seconds with AI model assistance (p < 0.001). The ICC of the two study radiologists slightly increased with AI model assistance (from 0.945 to 0.990). Conclusion: The proposed AI model was accurate for assessing bone age. Furthermore, this model appeared to enhance the clinical efficacy by reducing the reading time and improving the inter-observer reliability.

Keywords

Acknowledgement

Thanks to Ji-Eun Lee, Statistician, whose statistical expertise was invaluable during the statistical analysis and data interpretation.

References

  1. Zerin JM, Hernandez RJ. Approach to skeletal maturation. Hand Clin 1991;7:53-62 
  2. Satoh M. Bone age: assessment methods and clinical applications. Clin Pediatr Endocrinol 2015;24:143-152 
  3. Manzoor Mughal A, Hassan N, Ahmed A. Bone age assessment methods: a critical review. Pak J Med Sci 2014;30:211-215 
  4. Kim YJ, Kwon A, Jung MK, Kim KE, Suh J, Chae HW, et al. Incidence and prevalence of central precocious puberty in Korea: an epidemiologic study based on a national database. J Pediatr 2019;208:221-228 
  5. Kim JR, Lee YS, Yu J. Assessment of bone age in prepubertal healthy Korean children: comparison among the Korean standard bone age chart, Greulich-Pyle method, and Tanner-Whitehouse method. Korean J Radiol 2015;16:201-205 
  6. Greulich WW, Pyle SI. Radiographic atlas of skeletal development of the hand and wrist. California: Stanford University Press, 1971 
  7. Tanner JM, Healy H, Goldstein H, Cameron N. Assessment of skeletal maturity and prediction of adult height (TW3 method), 3rd ed. London: W.B Saunders Company, 2001 
  8. Kim SY, Oh YJ, Shin JY, Rhie YJ, Lee KH. Comparison of the Greulich-Pyle and Tanner Whitehouse (TW3) methods in bone age assessment. J Korean Soc Pediatr Endocrinol 2008;13:50-55 
  9. Berst MJ, Dolan L, Bogdanowicz MM, Stevens MA, Chow S, Brandser EA. Effect of knowledge of chronologic age on the variability of pediatric bone age determined using the Greulich and Pyle standards. AJR Am J Roentgenol 2001;176:507-510 
  10. Bull RK, Edwards PD, Kemp PM, Fry S, Hughes IA. Bone age assessment: a large scale comparison of the Greulich and Pyle, and Tanner and Whitehouse (TW2) methods. Arch Dis Child 1999;81:172-173 
  11. Spampinato C, Palazzo S, Giordano D, Aldinucci M, Leonardi R. Deep learning for automated skeletal bone age assessment in X-ray images. Med Image Anal 2017;36:41-51 
  12. Lee H, Tajmir S, Lee J, Zissen M, Yeshiwas BA, Alkasab TK, et al. Fully automated deep learning system for bone age assessment. J Digit Imaging 2017;30:427-441 
  13. Fu J, Zheng H, Mei T. Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017 Jul 21-26; Honolulu, HI, USA: IEEE; 2017; p. 4438-4446 
  14. Larson DB, Chen MC, Lungren MP, Halabi SS, Stence NV, Langlotz CP. Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs. Radiology 2018;287:313-322 
  15. Lee BD, Lee MS. Automated bone age assessment using artificial intelligence: the future of bone age assessment. Korean J Radiol 2021;22:792-800 
  16. Oh YJ. Development and evaluation of semi-automatic bone age estimation method. Seoul: The Graduate School of Ewha Womans University, 2011 
  17. Aja-Fernandez S, De Luis-Garcia R, Martin-Fernandez MA, Alberola-Lopez C. A computational TW3 classifier for skeletal maturity assessment. A computing with words approach. J Biomed Inform 2004;37:99-107 
  18. Bui TD, Lee JJ, Shin J. Incorporated region detection and classification using deep convolutional networks for bone age assessment. Artif Intell Med 2019;97:1-8 
  19. Halabi SS, Prevedello LM, Kalpathy-Cramer J, Mamonov AB, Bilbily A, Cicero M, et al. The RSNA pediatric bone age machine learning challenge. Radiology 2019;290:498-503 
  20. University of Southern California. Digital hand atlas - Image processing and informatics lab. Ipilab.usc.edu Web site. https://ipilab.usc.edu/research/baaweb/. Published July, 2017. Accessed March 6, 2020 
  21. Kim JR, Shim WH, Yoon HM, Hong SH, Lee JS, Cho YA, et al. Computerized bone age estimation using deep learning based program: evaluation of the accuracy and efficiency. AJR Am J Roentgenol 2017;209:1374-1380 
  22. Dallora AL, Anderberg P, Kvist O, Mendes E, Diaz Ruiz S, Sanmartin Berglund J. Bone age assessment with various machine learning techniques: a systematic literature review and meta-analysis. PLoS One 2019;14:e0220242 
  23. Tajmir SH, Lee H, Shailam R, Gale HI, Nguyen JC, Westra SJ, et al. Artificial intelligence-assisted interpretation of bone age radiographs improves accuracy and decreases variability. Skeletal Radiol 2019;48:275-283 
  24. Gyftopoulos S, Lin D, Knoll F, Doshi AM, Rodrigues TC, Recht MP. Artificial intelligence in musculoskeletal imaging: current status and future directions. AJR Am J Roentgenol 2019;213:506-513 
  25. Park SH. Artificial intelligence in medicine: beginner's guide. J Korean Soc Radiol 2018;78:301-308 
  26. Castelvecchi D. Can we open the black box of AI? Nature.com Web site. https://www.nature.com/news/can-we-open-theblack-box-of-ai-1.20731. Published October 5, 2016. Accessed November 20, 2019 
  27. Thodberg HH, Martin DD. Validation of a new version of BoneXpert bone age in children with congenital adrenal hyperplasia (CAH), precocious puberty (PP), growth hormone deficiency (GHD), Turner syndrome (TS), and other short stature diagnoses. Proceedings of the 58th Annual ESPE Meeting; 2019 Sep 19-21; Vienna, Austria: European Society for Paediatric Endocrinology; 2019; p. FC2.6 
  28. Thodberg HH, Kreiborg S, Juul A, Pedersen KD. The BoneXpert method for automated determination of skeletal maturity. IEEE Trans Med Imaging 2008;28:52-66 
  29. van Rijn RR, Thodberg HH. Bone age assessment: automated techniques coming of age? Acta Radiol 2013;54:1024-1029 
  30. Zhang A, Sayre JW, Vachon L, Liu BJ, Huang HK. Racial differences in growth patterns of children assessed on the basis of bone age. Radiology 2009;250:228-235 
  31. Ontell FK, Ivanovic M, Ablin DS, Barlow TW. Bone age in children of diverse ethnicity. AJR Am J Roentgenol 1996;167:1395-1398