Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Improved Anatomical Landmark Detection Using Attention Modules and Geometric Data Augmentation in X-ray Images

어텐션 모듈과 기하학적 데이터 증강을 통한 X-ray 영상 내 해부학적 랜드마크 검출 성능 향상

  • Lee, Hyo-Jeong (Department of Computational Medicine, Graduate Program in System Health Science and Engineering) ;
  • Ma, Se-Rie (Division of Mechanical and Biomedical Engineering, Graduate Program in System Health Science and Engineering Ewha Womans University) ;
  • Choi, Jang-Hwan (Division of Mechanical and Biomedical Engineering, Graduate Program in System Health Science and Engineering Ewha Womans University)
  • 이효정 (이화여자대학교 컴퓨터의학과) ;
  • 마세리 (이화여자대학교 휴먼기계바이오공학부) ;
  • 최장환 (이화여자대학교 휴먼기계바이오공학부)
  • Received : 2022.06.10
  • Accepted : 2022.07.05
  • Published : 2022.07.26
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, deep learning-based automated systems for identifying and detecting landmarks have been proposed. In order to train such a deep learning-based model without overfitting, a large amount of image and labeling data is required. Conventionally, an experienced reader manually identifies and labels landmarks in a patient's image. However, such measurement is not only expensive, but also has poor reproducibility, so the need for an automated labeling method has been raised. In addition, in the X-ray image, since various human tissues on the path through which the photons pass are displayed, it is difficult to identify the landmark compared to a general natural image or a 3D image modality image. In this study, we propose a geometric data augmentation technique that enables the generation of a large amount of labeling data in X-ray images. In addition, the optimal attention mechanism for landmark detection was presented through the implementation and application of various attention techniques to improve the detection performance of 16 major landmarks in the skull. Finally, among the major cranial landmarks, markers that ensure stable detection are derived, and these markers are expected to have high clinical application potential.

X-ray 두개골 영상에서 주요 해부학적 부위들 간의 거리를 계측하는 것은 진단과 치료 등 임상적 의미에서 매우 중요하다. 최근에는 딥러닝 기술의 발전을 바탕으로 랜드마크를 식별 및 검출하는 자동화 시스템들이 제시되고 있다. 이러한 딥러닝 기반 모델을 과적합 없이 학습 시키기 위해서는 대량의 영상과 라벨링 데이터가 필요하다. 기존에는 숙련된 판독의가 환자의 영상에서 랜드마크를 수동으로 식별하여 라벨링하는 방식으로 계측이 이루어져 왔다. 그러나 이러한 계측 방식은 많은 비용이 소요될 뿐만 아니라, 재현성이 떨어지기 때문에 자동화된 라벨링 방법에 대한 필요성이 제기되고 있다. 또한, X-ray 영상에는 광자가 통과하는 경로 상의 여러 인체조직들이 표시되기 때문에 랜드마크 식별이 일반 자연 이미지 또는 삼차원 모달리티 영상에 비해 어렵다. 본 연구에서는 X-ray 영상 내에 대량의 라벨링 데이터 생성을 가능하게 하는 기하학적 데이터 증강 기법을 제안하고 있다. 또한, 두개골 내 주요한 16개 랜드마크들의 검출 성능을 향상시키기 위해 다양한 어텐션 기법들의 구현 및 적용을 통해 랜드마크 검출을 위한 최적의 어텐션 메커니즘을 제시하였다. 마지막으로 주요 두개골 랜드마크들 중 안정적인 검출이 보장되는 마커들을 도출하였으며, 이러한 마커들은 임상적인 활용 가능성이 높을 것으로 기대된다.

Keywords

Acknowledgement

This work was partly supported by the Technology development Program of MSS [S3146559] and by the Korea Medical Device Development Fund grant funded by the Korea government (the Ministry of Science and ICT, the Ministry of Trade, Industry and Energy, the Ministry of Health & Welfare, the Ministry of Food and Drug Safety) (Project Number: KMDF_PR_20200901_0016, 9991006689)

References

  1. A. Kaur and C. Singh, "Automatic cephalometric landmark detection using zernike moments and template matching," Signal, Image and Video Processing, pp. 117-132, 2015.
  2. I. El-Fegh, M. Galhood, M. Sid-Ahmed, and M. Ahmadi, "Automated 2-d cephalometric analysis of x-ray by image registration approach based on least square approximator," in 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2008, pp. 3949-3952.
  3. R. D. J.H. Hwang, M.G. Kim, "Evaluation of automated cephalometric analysis based on the latest deep learning method," The Angle Orthodontist, pp. 329-335, 2021.
  4. Daseong and Han, "A supervised learning framework for physics-based controllers using stochastic model predictive control," Journal of the Korea Computer Graphics Society, pp. 9-17, 2021.
  5. Jewon, T. Ahn, T. Gu, and Kwon., "Motion generation of a single rigid body character using deep reinforcement learning," Journal of the Korea Computer Graphics Society, pp. 13-23, 2021.
  6. Z. Feng, J. Kittler, M. Awais, P. Huber, and X. Wu, "Wing loss for robust facial landmark localisation with convolutional neural networks," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2235-2245, 2018.
  7. W. Li, Y. Lu, K. Zheng, H. Liao, C. Lin, J. Luo, C.-T. Cheng, J. Xiao, L. Lu, C.-F. Kuo, et al., "Structured landmark detection via topology-adapting deep graph learning," in European Conference on Computer Vision, 2020, pp. 266-283.
  8. Q. Liu, J. Deng, J. Yang, G. Liu, and D. Tao, "Adaptive cascade regression model for robust face alignment," IEEE Transactions on Image Processing, pp. 797-807, 2016.
  9. S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh, "Convolutional pose machines," in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2016, pp. 4724-4732.
  10. J. Wang, K. Sun, T. Cheng, B. Jiang, C. Deng, Y. Zhao, D. Liu, Y. Mu, M. Tan, X. Wang, et al., "Deep high-resolution representation learning for visual recognition," IEEE transactions on pattern analysis and machine intelligence, pp. 3349-3364, 2020.
  11. H. Lee, M. Park, and J. Kim, "Cephalometric landmark detection in dental x-ray images using convolutional neural networks," in Medical imaging 2017: Computer-aided diagnosis, 2017, p. 101341W.
  12. B. Bier, M. Unberath, J.-N. Zaech, J. Fotouhi, M. Armand, G. Osgood, N. Navab, and A. Maier, "X-ray-transform invariant anatomical landmark detection for pelvic trauma surgery," in International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018, pp. 55-63.
  13. Y. Song, X. Qiao, Y. Iwamoto, and Y.-w. Chen, "Automatic cephalometric landmark detection on x-ray images using a deep-learning method," Applied Sciences, p. 2547, 2020.
  14. W. P. Segars, G. Sturgeon, S. Mendonca, J. Grimes, and B. M. Tsui, "4d xcat phantom for multimodality imaging research," Medical physics, pp. 4902-4915, 2010. https://doi.org/10.1118/1.3480985
  15. D. S. W.R. Proffit, H.W. Fields, "Contemporary orthodontics," Elsevier Health Sciences, p. 8, 2006.
  16. A. Maier, H. G. Hofmann, M. Berger, P. Fischer, C. Schwemmer, H. Wu, K. Muller, J. Hornegger, J.-H. Choi, C. Riess, et al., "Conrad-a software framework for cone-beam imaging in radiology," Medical physics, p. 111914, 2013.
  17. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
  18. J. Hu, L. Shen, and G. Sun, "Squeeze-and-excitation networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132-7141.
  19. S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, "Cbam: Convolutional block attention module," in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 3-19.
  20. P. Ramachandran, N. Parmar, A. Vaswani, I. Bello, A. Levskaya, and J. Shlens, "Stand-alone self-attention in vision models," Advances in Neural Information Processing Systems, vol. 32, 2019.
  21. C.-W. Wang, C.-T. Huang, M.-C. Hsieh, et al., "Evaluation and comparison of anatomical landmark detection methods for cephalometric x-ray images: a grand challenge," IEEE transactions on medical imaging, pp. 1890-1900, 2015.
  22. Jie, W. Yao, T. Zeng, S. He, Y. Zhou, J. Zhang, W. Guo, and Tang, "Automatic localization of cephalometric landmarks based on convolutional neural network," American Journal of Orthodontics and Dentofacial Orthopedics, pp. e250-e259,, 2022.
  23. M. Lee, M. Chung, and Y.-G. Shin, "Cephalometric landmark detection via global and local encoders and patch-wise attentions," Neurocomputing, pp. 182-189, 2022.