• The Korean Journal of Applied Statistics
• /
• v.35 no.1
• /
• pp.35-47
• /
• 2022

The receiver operating characteristic (ROC) curve was developed to quantify the classification ability of marker values (covariates) on the response variable and has been extended to survival data with diverse missing data structure. When survival data is understood as binary data (status of being alive or dead) at each time point, the ROC curve expressed at every time point results in time-dependent ROC curve and time-dependent area under curve (AUC). In particular, a follow-up study brings the change of cohort and incomplete data structures such as censoring and competing risk. In this paper, we review time-dependent ROC estimators under several contexts and perform simulation to check the performance of each estimators. We analyzed a dementia dataset to compare the prognostic power of markers.

• Korean Journal of Radiology
• /
• v.22 no.7
• /
• pp.1213-1224
• /
• 2021

Objective: To develop a machine learning (ML) pipeline based on radiomics to predict Coronavirus Disease 2019 (COVID-19) severity and the future deterioration to critical illness using CT and clinical variables. Materials and Methods: Clinical data were collected from 981 patients from a multi-institutional international cohort with real-time polymerase chain reaction-confirmed COVID-19. Radiomics features were extracted from chest CT of the patients. The data of the cohort were randomly divided into training, validation, and test sets using a 7:1:2 ratio. A ML pipeline consisting of a model to predict severity and time-to-event model to predict progression to critical illness were trained on radiomics features and clinical variables. The receiver operating characteristic area under the curve (ROC-AUC), concordance index (C-index), and time-dependent ROC-AUC were calculated to determine model performance, which was compared with consensus CT severity scores obtained by visual interpretation by radiologists. Results: Among 981 patients with confirmed COVID-19, 274 patients developed critical illness. Radiomics features and clinical variables resulted in the best performance for the prediction of disease severity with a highest test ROC-AUC of 0.76 compared with 0.70 (0.76 vs. 0.70, p = 0.023) for visual CT severity score and clinical variables. The progression prediction model achieved a test C-index of 0.868 when it was based on the combination of CT radiomics and clinical variables compared with 0.767 when based on CT radiomics features alone (p < 0.001), 0.847 when based on clinical variables alone (p = 0.110), and 0.860 when based on the combination of visual CT severity scores and clinical variables (p = 0.549). Furthermore, the model based on the combination of CT radiomics and clinical variables achieved time-dependent ROC-AUCs of 0.897, 0.933, and 0.927 for the prediction of progression risks at 3, 5 and 7 days, respectively. Conclusion: CT radiomics features combined with clinical variables were predictive of COVID-19 severity and progression to critical illness with fairly high accuracy.

• Korean Journal of Radiology
• /
• v.25 no.2
• /
• pp.189-198
• /
• 2024

Objective: To investigate the prognostic utility of radiomics features extracted from ¹⁸F-fluorodeoxyglucose (FDG) PET/CT combined with clinical factors and metabolic parameters in predicting progression-free survival (PFS) and overall survival (OS) in individuals diagnosed with extranodal nasal-type NK/T cell lymphoma (ENKTCL). Materials and Methods: A total of 126 adults with ENKTCL who underwent ¹⁸F-FDG PET/CT examination before treatment were retrospectively included and randomly divided into training (n = 88) and validation cohorts (n = 38) at a ratio of 7:3. Least absolute shrinkage and selection operation Cox regression analysis was used to select the best radiomics features and calculate each patient's radiomics scores (RadPFS and RadOS). Kaplan-Meier curve and Log-rank test were used to compare survival between patient groups risk-stratified by the radiomics scores. Various models to predict PFS and OS were constructed, including clinical, metabolic, clinical + metabolic, and clinical + metabolic + radiomics models. The discriminative ability of each model was evaluated using Harrell's C index. The performance of each model in predicting PFS and OS for 1-, 3-, and 5-years was evaluated using the time-dependent receiver operating characteristic (ROC) curve. Results: Kaplan-Meier curve analysis demonstrated that the radiomics scores effectively identified high- and low-risk patients (all P < 0.05). Multivariable Cox analysis showed that the Ann Arbor stage, maximum standardized uptake value (SUVmax), and RadPFS were independent risk factors associated with PFS. Further, β2-microglobulin, Eastern Cooperative Oncology Group performance status score, SUVmax, and RadOS were independent risk factors for OS. The clinical + metabolic + radiomics model exhibited the greatest discriminative ability for both PFS (Harrell's C-index: 0.805 in the validation cohort) and OS (Harrell's C-index: 0.833 in the validation cohort). The time-dependent ROC analysis indicated that the clinical + metabolic + radiomics model had the best predictive performance. Conclusion: The PET/CT-based clinical + metabolic + radiomics model can enhance prognostication among patients with ENKTCL and may be a non-invasive and efficient risk stratification tool for clinical practice.

• Journal of Gastric Cancer
• /
• v.21 no.3
• /
• pp.268-278
• /
• 2021

Purpose: While several prognostic models for the stratification of death risk have been developed for patients with advanced gastric cancer receiving first-line chemotherapy, they have seldom been tested in the Chinese population. This study investigated the performance of these models and identified the optimal tools for Chinese patients. Materials and Methods: Patients diagnosed with metastatic or recurrent gastric adenocarcinoma who received first-line chemotherapy were eligible for inclusion in the validation cohort. Their clinical data and survival outcomes were retrieved and documented. Time-dependent receiver operating characteristic (ROC) and calibration curves were used to evaluate the predictive ability of the models. Kaplan-Meier curves were plotted for patients in different risk groups divided by 7 published stratification tools. Log-rank tests with pairwise comparisons were used to compare survival differences. Results: The analysis included a total of 346 patients with metastatic or recurrent disease. The median overall survival time was 11.9 months. The patients were different into different risk groups according to the prognostic stratification models, which showed variability in distinguishing mortality risk in these patients. The model proposed by Kim et al. showed relative higher predicting abilities compared to the other models, with the highest χ² (25.8) value in log-rank tests across subgroups, and areas under the curve values at 6, 12, and 24 months of 0.65 (95% confidence interval [CI]: 0.59-0.72), 0.60 (0.54-0.65), and 0.63 (0.56-0.69), respectively. Conclusions: Among existing prognostic tools, the models constructed by Kim et al., which incorporated performance status score, neutrophil-to-lymphocyte ratio, alkaline phosphatase, albumin, and tumor differentiation, were more effective in stratifying Chinese patients with gastric cancer receiving first-line chemotherapy.