Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Chest CT Image Patch-Based CNN Classification and Visualization for Predicting Recurrence of Non-Small Cell Lung Cancer Patients

비소세포폐암 환자의 재발 예측을 위한 흉부 CT 영상 패치 기반 CNN 분류 및 시각화

  • Ma, Serie (Seoul Women's University, Department of Software Convergence) ;
  • Ahn, Gahee (Seoul Women's University, Department of Software Convergence) ;
  • Hong, Helen (Seoul Women's University, Department of Software Convergence)
  • 마세리 (서울여자대학교 소프트웨어융합학과) ;
  • 안가희 (서울여자대학교 소프트웨어융합학과) ;
  • 홍헬렌 (서울여자대학교 소프트웨어융합학과)
  • Received : 2021.11.24
  • Accepted : 2022.01.27
  • Published : 2022.03.01
Download PDF
⟨ Previous Next ⟩

Abstract

Non-small cell lung cancer (NSCLC) accounts for a high proportion of 85% among all lung cancer and has a significantly higher mortality rate (22.7%) compared to other cancers. Therefore, it is very important to predict the prognosis after surgery in patients with non-small cell lung cancer. In this study, the types of preoperative chest CT image patches for non-small cell lung cancer patients with tumor as a region of interest are diversified into five types according to tumor-related information, and performance of single classifier model, ensemble classifier model with soft-voting method, and ensemble classifier model using 3 input channels for combination of three different patches using pre-trained ResNet and EfficientNet CNN networks are analyzed through misclassification cases and Grad-CAM visualization. As a result of the experiment, the ResNet152 single model and the EfficientNet-b7 single model trained on the peritumoral patch showed accuracy of 87.93% and 81.03%, respectively. In addition, ResNet152 ensemble model using the image, peritumoral, and shape-focused intratumoral patches which were placed in each input channels showed stable performance with an accuracy of 87.93%. Also, EfficientNet-b7 ensemble classifier model with soft-voting method using the image and peritumoral patches showed accuracy of 84.48%.

비소세포폐암(NSCLC)은 전체 폐암 중 85%의 높은 비중을 차지하며 사망률(22.7%)이 다른 암에 비해 현저히 높은 암으로 비소세포폐암 환자의 수술 후 예후에 대한 예측은 매우 중요하다. 본 연구에서는 종양을 관심영역으로 갖는 비소세포폐암 환자의 수술 전 흉부 CT 영상 패치의 종류를 종양 관련 정보에 따라 총 다섯 가지로 다양화하고, 이를 입력데이터로 갖는 사전 학습 된 ResNet 과 EfficientNet CNN 네트워크를 사용하여 단일 모델과 간접 투표 방식을 이용한 앙상블 모델, 그리고 3 개의 입력 채널을 활용한 앙상블 모델에서의 실험 결과 및 성능을 오분류의 사례와 Grad-CAM 시각화를 통해 비교 분석한다. 실험 결과, 종양 주변부 패치를 학습한 ResNet152 단일 모델과 EfficientNet-b7 단일 모델은 각각 87.93%와 81.03%의 정확도를 보였다. 또한 ResNet152 에서 총 3 개의 입력 채널에 각각 영상 패치, 종양 주변부 패치, 형상 집중 종양 내부 패치를 넣어 앙상블 모델을 구성한 경우에는 정확도 87.93%를, EfficientNet-b7 에서 간접 투표 방식으로 영상 패치와 종양 주변부 패치 학습 모델을 앙상블 한 경우에는 정확도 84.48%를 도출하며 안정적인 성능을 보였다.

Keywords

Acknowledgement

본 연구는 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원과(No. 2020R1A2C1102140) 과학기술정보통신부 및 정보통신기획평가원의 SW 중심대학지원사업의 연구결과로 수행되었음(2016-0-00022).

References

  1. Korea Central Cancer Registry, National Cancer Center, "Annual report of cancer statistics in Korea in 2018." Ministry of Health and welfare, 2020.
  2. C. Haarburger, P. Weitz, O. Rippel and D. Merhof, "Image-Based Survival Prediction for Lung Cancer Patients Using CNNS." 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1197-1201, 2019.
  3. Ye-Sel Lee, A-Hyun Cho and Helen Hong, "Ensemble Learning Based on Tumor Internal and External Imaging Patch to Predict the Recurrence of Non-small Cell Lung Cancer Patients in Chest CT Image." Journal of Korea Multimedia Society, 24(3), 373-381, 2021. https://doi.org/10.9717/KMMS.2020.24.3.373
  4. Thanh-Hung Vo, Guee-Sang Lee, Hyung-Jeong Yang and In-Jae Oh. "Survival Prediction of Lung Cancer Using Small-Size Clinical Data with a Multiple Task Variational Autoencoder." Electronics 10, no. 12: 1396, 2021. https://doi.org/10.3390/electronics10121396
  5. Tai H Dou, Thibaud P Coroller, Joost J M van Griethuysen, Raymond H Mak and Hugo J W L Aerts, "Peritumoral radiomics features predict distant metastasis in locally advanced NSCLC." PloS one vol. 13,11 e0206108, 2018. https://doi.org/10.1371/journal.pone.0206108
  6. Hansang Lee, Haeil Lee, Helen Hong, Heejin Bae, Joon Seok Lim and Junmo Kim, "Classification of focal liver lesions in CT images using convolutional neural networks with lesion information augmented patches and synthetic data augmentation." Medical physics, 10.1002/mp.15118, 2021.
  7. Tetsuro Baba, Hidetaka Uramoto, Masaru Takenaka, Souichi Oka, Yoshiki Shigematsu, Hidehiko Shimokawa, Takeshi Hanagiri and Fumihiro Tanaka, "The tumour shape of lung adenocarcinoma is related to the postoperative prognosis." Interactive cardiovascular and thoracic surgery vol. 15, 1: 73-6, 2021. https://doi.org/10.1093/icvts/ivs055
  8. K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition." IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
  9. M. Tan and Q. V. Le, "EfficientNet: Rethinking model scaling for convolutional neural networks." 36th Int. Conf. Mach. Learn. ICML, vol. 2019-June, pp. 10691-10700, 2019.
  10. Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh and Dhruv Batra, "Grad-cam: Visual explanations from deep networks via gradient-based localization." Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2016.
  11. Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva and Antonio Torralba "Learning deep features for discriminative localization." Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2921-2929, 2016.
  12. Erico Tjoa and Cuntai Guan "A survey on explainable artificial intelligence (XAI): towards medical XAI." pp. 1-21, 2020
  13. Hualong Yu, Shihe Liu, Chuanyu Zhang, Shaoke Li, Jianan Ren, Jingli Zhang and Wenjian Xu, "Computed tomography and pathology evaluation of lung ground-glass opacity." Experimental and Therapeutic Medicine vol. 16, 5305-5309, 2018.