The korean journal of orthodontics (대한치과교정학회지)

The Korean Association Of Orthodontists (대한치과교정학회)
권호 표지
DOI QR코드

DOI QR Code

Accuracy of one-step automated orthodontic diagnosis model using a convolutional neural network and lateral cephalogram images with different qualities obtained from nationwide multi-hospitals

  • Yim, Sunjin (Department of Orthodontics, School of Dentistry, Seoul National University) ;
  • Kim, Sungchul (Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine) ;
  • Kim, Inhwan (Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine) ;
  • Park, Jae-Woo (Private Practice) ;
  • Cho, Jin-Hyoung (Department of Orthodontics, Chonnam National University School of Dentistry) ;
  • Hong, Mihee (Department of Orthodontics, School of Dentistry, Kyungpook National University) ;
  • Kang, Kyung-Hwa (Department of Orthodontics, School of Dentistry, Wonkwang University) ;
  • Kim, Minji (Department of Orthodontics, College of Medicine, Ewha Womans University) ;
  • Kim, Su-Jung (Department of Orthodontics, Kyung Hee University School of Dentistry) ;
  • Kim, Yoon-Ji (Department of Orthodontics, Asan Medical Center, University of Ulsan College of Medicine) ;
  • Kim, Young Ho (Department of Orthodontics, Institute of Oral Health Science, Ajou University School of Medicine) ;
  • Lim, Sung-Hoon (Department of Orthodontics, College of Dentistry, Chosun University) ;
  • Sung, Sang Jin (Department of Orthodontics, Asan Medical Center, University of Ulsan College of Medicine) ;
  • Kim, Namkug (Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine) ;
  • Baek, Seung-Hak (Department of Orthodontics, School of Dentistry, Dental Research Institute, Seoul National University)
  • Received : 2021.03.23
  • Accepted : 2021.07.02
  • Published : 2022.01.25
Download PDF
⟨ Previous Next ⟩

Abstract

Objective: The purpose of this study was to investigate the accuracy of one-step automated orthodontic diagnosis of skeletodental discrepancies using a convolutional neural network (CNN) and lateral cephalogram images with different qualities from nationwide multi-hospitals. Methods: Among 2,174 lateral cephalograms, 1,993 cephalograms from two hospitals were used for training and internal test sets and 181 cephalograms from eight other hospitals were used for an external test set. They were divided into three classification groups according to anteroposterior skeletal discrepancies (Class I, II, and III), vertical skeletal discrepancies (normodivergent, hypodivergent, and hyperdivergent patterns), and vertical dental discrepancies (normal overbite, deep bite, and open bite) as a gold standard. Pre-trained DenseNet-169 was used as a CNN classifier model. Diagnostic performance was evaluated by receiver operating characteristic (ROC) analysis, t-stochastic neighbor embedding (t-SNE), and gradient-weighted class activation mapping (Grad-CAM). Results: In the ROC analysis, the mean area under the curve and the mean accuracy of all classifications were high with both internal and external test sets (all, > 0.89 and > 0.80). In the t-SNE analysis, our model succeeded in creating good separation between three classification groups. Grad-CAM figures showed differences in the location and size of the focus areas between three classification groups in each diagnosis. Conclusions: Since the accuracy of our model was validated with both internal and external test sets, it shows the possible usefulness of a one-step automated orthodontic diagnosis tool using a CNN model. However, it still needs technical improvement in terms of classifying vertical dental discrepancies.

Keywords

Acknowledgement

This research was supported by a grant from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI18C1638). This article was based on the study of Dr. Yim's PhD dissertation. We thank Professor Won-hee Lim and Dr. Keunoh Lim for their contribution in performing the inter-examiner reliability test.

References

  1. Arik SO, Ibragimov B, Xing L. Fully automated quantitative cephalometry using convolutional neural networks. J Med Imaging (Bellingham) 2017;4:014501. https://doi.org/10.1117/1.JMI.4.1.014501
  2. Kim H, Shim E, Park J, Kim YJ, Lee U, Kim Y. Web-based fully automated cephalometric analysis by deep learning. Comput Methods Programs Biomed 2020;194:105513. https://doi.org/10.1016/j.cmpb.2020.105513
  3. Nishimoto S, Sotsuka Y, Kawai K, Ishise H, Kakibuchi M. Personal computer-based cephalometric landmark detection with deep learning, using cephalograms on the internet. J Craniofac Surg 2019;30:91-5. https://doi.org/10.1097/SCS.0000000000004901
  4. Erkan M, Gurel HG, Nur M, Demirel B. Reliability of four different computerized cephalometric analysis programs. Eur J Orthod 2012;34:318-21. https://doi.org/10.1093/ejo/cjr008
  5. Wen J, Liu S, Ye X, Xie X, Li J, Li H, et al. Comparative study of cephalometric measurements using 3 imaging modalities. J Am Dent Assoc 2017;148:913-21. https://doi.org/10.1016/j.adaj.2017.07.030
  6. Rudolph DJ, Sinclair PM, Coggins JM. Automatic computerized radiographic identification of cephalometric landmarks. Am J Orthod Dentofacial Orthop 1998;113:173-9. https://doi.org/10.1016/S0889-5406(98)70289-6
  7. Mosleh MA, Baba MS, Malek S, Almaktari RA. Ceph-X: development and evaluation of 2D cephalometric system. BMC Bioinformatics 2016;17(Suppl 19):499. https://doi.org/10.1186/s12859-016-1370-5
  8. Yu HJ, Cho SR, Kim MJ, Kim WH, Kim JW, Choi J. Automated skeletal classification with lateral cephalometry based on artificial intelligence. J Dent Res 2020;99:249-56. https://doi.org/10.1177/0022034520901715
  9. Park JH, Hwang HW, Moon JH, Yu Y, Kim H, Her SB, et al. Automated identification of cephalometric landmarks: part 1-comparisons between the latest deep-learning methods YOLOV3 and SSD. Angle Orthod 2019;89:903-9. https://doi.org/10.2319/022019-127.1
  10. Hwang HW, Park JH, Moon JH, Yu Y, Kim H, Her SB, et al. Automated identification of cephalometric landmarks: part 2-might it be better than human? Angle Orthod 2020;90:69-76. https://doi.org/10.2319/022019-129.1
  11. Kunz F, Stellzig-Eisenhauer A, Zeman F, Boldt J. Artificial intelligence in orthodontics: evaluation of a fully automated cephalometric analysis using a customized convolutional neural network. J Orofac Orthop 2020;81:52-68. https://doi.org/10.1007/s00056-019-00203-8
  12. Korean Association of Orthodontics Malocclusion White Paper Publication Committee. Cephalometric analysis of normal occlusion in Korean adults. Seoul: Korean Association of Orthodontists; 1997.
  13. Bujang MA, Baharum N. Guidelines of the minimum sample size requirements for Cohen's Kappa. Epidemiol Biostat Public Health 2017;14:e12267.
  14. McHugh ML. Interrater reliability: the Kappa statistic. Biochem Med (Zagreb) 2012;22:276-82. https://doi.org/10.11613/BM.2012.031
  15. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge. Int J Comput Vis 2015;115:211-52. https://doi.org/10.1007/s11263-015-0816-y
  16. Huang G, Liu Z, van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. Paper presented at: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Jul 21-26; Honolulu, USA. Piscataway: IEEE, 2017. p. 2261-9.
  17. Goyal P, Dollar P, Girshick R, Noordhuis P, Wesolowski L, Kyrola A, et al. Accurate, large minibatch SGD: training ImageNet in 1 hour [Internet]. arxiv; 2017 Jun 8 [updated 2018 Apr 30; cited 2020 Aug 7]. Available from: https://arxiv.org/abs/1706.02677.
  18. Jia X, Song S, He W, Wang Y, Rong H, Zhou F, et al. Highly scalable deep learning training system with mixed-precision: training ImageNet in four minutes [Internet]. arxiv; 2018 Jul 30 [cited 2020 Aug 7]. Available from: https://arxiv.org/abs/1807.11205.
  19. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift [Internet]. arxiv; 2015 Feb 11 [updated 2015 Mar 2; cited 2020 Sep 8]. Available from: https://arxiv.org/abs/1502.03167.
  20. Wu Y, He K. Group normalization. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. ECCV 2018: Computer vision - ECCV 2018. Cham: Springer; 2018. p. 3-19.
  21. Deng J, Guo J, Xue N, Zafeiriou S. ArcFace: additive angular margin loss for deep face recognition. Paper presented at: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2019 Jun 15-20; Long Beach, USA. Piscataway: IEEE, 2019. p. 4690-9.
  22. Dreiseitl S, Ohno-Machado L, Binder M. Comparing three-class diagnostic tests by three-way ROC analysis. Med Decis Making 2000;20:323-31. https://doi.org/10.1177/0272989X0002000309
  23. Li J, Fine JP. ROC analysis with multiple classes and multiple tests: methodology and its application in microarray studies. Biostatistics 2008;9:566-76. https://doi.org/10.1093/biostatistics/kxm050
  24. van der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learn Res 2008;9:2579-605.
  25. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. Paper presented at: 2017 IEEE International Conference on Computer Vision (ICCV); 2017 Oct 22-29; Venice, Italy. Piscataway: IEEE, 2017. p. 618-26.