Korean Journal of Clinical Pharmacy (한국임상약학회지)

Korean College Of Clinical Pharmacy (한국임상약학회)
권호 표지
DOI QR코드

DOI QR Code

Evaluating AI Models and Predictors for COVID-19 Infection Dependent on Data from Patients with Cancer or Not: A Systematic Review

  • Takdon Kim (Clinical Trials Center, Chungnam National University Hospital) ;
  • Heeyoung Lee (College of Pharmacy, Inje University)
  • Received : 2024.05.03
  • Accepted : 2024.06.14
  • Published : 2024.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Background: As preexisting comorbidities are risk factors for Coronavirus Disease 19 (COVID-19), improved tools are needed for screening or diagnosing COVID-19 in clinical practice. Difficulties of including vulnerable patient data may create data imbalance and hinder the provision of well-performing prediction tools, such as artificial intelligence (AI) models. Thus, we systematically reviewed studies on AI prognosis prediction in patients infected with COVID-19 and existing comorbidities, including cancer, to investigate model performance and predictors dependent on patient data. PubMed and Cochrane Library databases were searched. This study included research meeting the criteria of using AI to predict outcomes in COVID-19 patients, whether they had cancer or not. Preprints, abstracts, reviews, and animal studies were excluded from the analysis. Majority of non-cancer studies (54.55 percent) showed an area under the curve (AUC) of >0.90 for AI models, whereas 30.77 percent of cancer studies showed the same result. For predicting mortality (3.85 percent), severity (8.33 percent), and hospitalization (14.29 percent), only cancer studies showed AUC values between 0.50 and 0.69. The distribution of comorbidity data varied more in non-cancer studies than in cancer studies but age was indicated as the primary predictor in all studies. Non-cancer studies with more balanced datasets of comorbidities showed higher AUC values than cancer studies. Based on the current findings, dataset balancing is essential for improving AI performance in predicting COVID-19 in patients with comorbidities, especially considering age.

Keywords

References

  1. Chauhan S. Comprehensive review of coronavirus disease 2019 (covid-19). Biomed J 2020;43(4):334-40. https://doi.org/10.1016/j.bj.2020.05.023
  2. Silk BJ, Scobie HM, Duck WM, et al. Covid-19 surveillance after expiration of the public health emergency declaration - united states, may 11, 2023. MMWR Morb Mortal Wkly Rep 2023;72(19):523-8. https://doi.org/10.15585/mmwr.mm7219e1
  3. Prevention CfDC. Interim guidance on developing a covid-19 case investigation & contact tracing plan: Overview. (2023). 2023. Available from: https://www.cdc.gov/coronavirus/2019-ncov/index.html Accessed 03 August, 2023.
  4. Al-Quteimat OM, Amer AM. The impact of the covid-19 pandemic on cancer patients. Am J Clin Oncol 2020;43(6):452-5. https://doi.org/10.1097/COC.0000000000000712
  5. Dai M, Liu D, Liu M, et al. Patients with cancer appear more vulnerable to sars-cov-2: a multicenter study during the covid-19 outbreak. Cancer Discov 2020;10(6):783-91. https://doi.org/10.1158/2159-8290.CD-20-0422
  6. Salunke AA, Nandy K, Pathak SK, et al. Impact of covid 19 in cancer patients on severity of disease and fatal outcomes: a systematic review and meta-analysis. Diabetes Metab Syndr 2020;14(5):1431-7. https://doi.org/10.1016/j.dsx.2020.07.037
  7. Yousefi L, Saachi L, Bellazzi R, Chiovato L, Tucker A. Predicting comorbidities using resampling and dynamic bayesian networks with latent variables. IEEE Computer Society 2017;30:205-6.
  8. Silaghi-Dumitrescu R, Patrascu I, Lehene M, Bercea I. Comorbidities of covid-19 patients. Medicina (Kaunas) 2023;59(8):1393.
  9. Fok CC, Henry D, Allen J. Maybe small is too small a term: introduction to advancing small sample prevention science. Prev Sci 2015;16(7):943-9.
  10. Jiang F, Jiang Y, Zhi H, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol 2017;2(4):230-43. https://doi.org/10.1136/svn-2017-000101
  11. Kim DK. Prediction models for covid-19 mortality using artificial intelligence. J Pers Med 2022;12(9):1522.
  12. Abdulaal A, Patel A, Charani E, Denny S, Mughal N, Moore L. Prognostic modeling of covid-19 using artificial intelligence in the united kingdom: model development and validation. J Med Internet Res 2020;22(8):e20259.
  13. Ngan Tran HC, Janet Jiang, Jay Bhuyan, Junhua Ding. Effect of class imbalance on the performance of machine learning-based network intrusion detection. Int J Performability Eng 2021;17(9):741-55. https://doi.org/10.23940/ijpe.21.09.p1.741755
  14. Cartmell KB, Bonilha HS, Simpson KN, Ford ME, Bryant DC, Alberg AJ. Patient barriers to cancer clinical trial participation and navigator activities to assist. Adv Cancer Res 2020;146:139-66. https://doi.org/10.1016/bs.acr.2020.01.008
  15. Page MJ, McKenzie JE, Bossuyt PM, et al. The prisma 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71.
  16. Aghakhani A, Shoshtarian Malak J, Karimi Z, Vosoughi F, Zeraati H, Yekaninejad MS. Predicting the covid-19 mortality among iranian patients using tree-based models: a cross-sectional study. Health Sci Rep 2023;6(5):e1279.
  17. Ahamad MM, Aktar S, Uddin MJ, et al. Adverse effects of covid-19 vaccination: machine learning and statistical approach to identify and classify incidences of morbidity and postvaccination reactogenicity. Healthcare (Basel) 2022;11(1):31.
  18. Upadhyay AK, Shukla S. Correlation study to identify the factors affecting covid-19 case fatality rates in india. Diabetes Metab Syndr 2021;15(3):993-9.
  19. Banoei MM, Rafiepoor H, Zendehdel K, et al. Unraveling complex relationships between covid-19 risk factors using machine learning based models for predicting mortality of hospitalized patients and identification of high-risk group: a large retrospective study. Front Med (Lausanne) 2023;10:1170331.
  20. Carbonell G, Del Valle DM, Gonzalez-Kozlova E, et al. Quantitative chest computed tomography combined with plasma cytokines predict outcomes in covid-19 patients. Heliyon 2022;8(8):e10166.
  21. An C, Lim H, Kim DW, Chang JH, Choi YJ, Kim SW. Machine learning prediction for mortality of patients diagnosed with covid-19: a nationwide korean cohort study. Sci Rep 2020;10(1):18716.
  22. Gao Y, Chen L, Chi J, et al. Development and validation of an online model to predict critical covid-19 with immune-inflammatory parameters. J Intensive Care 2021;9(1):19.
  23. Experton B, Tetteh HA, Lurie N, et al. A predictive model for severe covid-19 in the medicare population: a tool for prioritizing primary and booster covid-19 vaccination. Biology (Basel) 2021;10(11):1185.
  24. Khadem H, Nemat H, Eissa MR, Elliott J, Benaissa M. Covid-19 mortality risk assessments for individuals with and without diabetes mellitus: machine learning models integrated with interpretation framework. Comput Biol Med 2022;144:105361.
  25. Heyl J, Hardy F, Tucker K, et al. Frailty, comorbidity, and associations with in-hospital mortality in older covid-19 patients: exploratory study of administrative data. Interact J Med Res 2022;11(2):e41520.
  26. Hilal W, Chislett MG, Snider B, McBean EA, Yawney J, Gadsden SA. Use of ai to assess covid-19 variant impacts on hospitalization, icu, and death. Front Artif Intell 2022;5:927203.
  27. Ikemura K, Bellin E, Yagi Y, et al. Using automated machine learning to predict the mortality of patients with covid-19: prediction model development study. J Med Internet Res 2021;23(2):e23458.
  28. Jamshidi E, Asgary A, Tavakoli N, et al. Using machine learning to predict mortality for covid-19 patients on day 0 in the icu. Front Digit Health 2021;3:681608.
  29. Razjouyan J, Helmer DA, Lynch KE, et al. Smoking status and factors associated with covid-19 in-hospital mortality among us veterans. Nicotine Tob Res 2022;24(5):785-93. https://doi.org/10.1093/ntr/ntab223
  30. Edqvist J, Lundberg C, Andreasson K, et al. Severe covid-19 infection in type 1 and type 2 diabetes during the first three waves in sweden. Diabetes Care 2023;46(3):570-8. https://doi.org/10.2337/dc22-1760
  31. Karasneh RA, Khassawneh BY, Al-Azzam S, et al. Risk factors associated with mortality in covid-19 hospitalized patients: data from the middle east. Int J Clin Pract 2022;2022:9617319.
  32. Lee BH, Lee KS, Kim HI, et al. Blood transfusion, all-cause mortality and hospitalization period in covid-19 patients: machine learning analysis of national health insurance claims data. Diagnostics (Basel) 2022;12(12):2970.
  33. Modelli de Andrade LG, de Sandes-Freitas TV, Requiao-Moura LR, et al. Development and validation of a simple web-based tool for early prediction of covid-19-associated death in kidney transplant recipients. Am J Transplant 2022;22(2):610-25. https://doi.org/10.1111/ajt.16807
  34. Kivrak M, Guldogan E, Colak C. Prediction of death status on the course of treatment in sars-cov-2 patients with deep learning and machine learning methods. Comput Methods Programs Biomed 2021;201:105951.
  35. Rahman MM, Islam MM, Manik MMH, Islam MR, Al-Rakhami MS. Machine learning approaches for tackling novel coronavirus (covid-19) pandemic. SN Comput Sci 2021;2(5):384.
  36. Lore NI, De Lorenzo R, Rancoita PMV, et al. Cxcl10 levels at hospital admission predict covid-19 outcome: hierarchical assessment of 53 putative inflammatory biomarkers in an observational study. Mol Med 2021;27(1):129.
  37. Rasmy L, Nigo M, Kannadath BS, et al. Recurrent neural network models (covrnn) for predicting outcomes of patients with covid-19 on admission to hospital: model development and validation using electronic health record data. Lancet Digit Health 2022;4(6):e415-25. https://doi.org/10.1016/S2589-7500(22)00049-8
  38. Wollenstein-Betech S, Cassandras CG, Paschalidis IC. Personalized predictive models for symptomatic covid-19 patients using basic preconditions: hospitalizations, mortality, and the need for an icu or ventilator. Int J Med Inform 2020;142:104258.
  39. Schmidt M, Guidet B, Demoule A, et al. Predicting 90-day survival of patients with covid-19: survival of severely ill covid (sosic) scores. Ann Intensive Care 2021;11(1):170.
  40. Alle S, Kanakan A, Siddiqui S, et al. Covid-19 risk stratification and mortality prediction in hospitalized indian patients: harnessing clinical data for public health benefits. PLoS One 2022;17(3):e0264785.
  41. Nojiri S, Irie Y, Kanamori R, Naito T, Nishizaki Y. Mortality prediction of covid-19 in hospitalized patients using the 2020 diagnosis procedure combination administrative database of japan. Intern Med 2023;62(2):201-13.
  42. Snider JM, You JK, Wang X, et al. Group iia secreted phospholipase a2 is associated with the pathobiology leading to covid-19 mortality. J Clin Invest 2021;131(19):e149236.
  43. Subudhi S, Verma A, Patel AB, et al. Comparing machine learning algorithms for predicting icu admission and mortality in covid-19. NPJ Digit Med 2021;4(1):87.
  44. Kar S, Chawla R, Haranath SP, et al. Multivariable mortality risk prediction using machine learning for covid-19 patients at admission (aicovid). Sci Rep 2021;11(1):12801.
  45. Wu JT, de la Hoz MA A, Kuo PC, et al. Developing and validating multi-modal models for mortality prediction in covid-19 patients: a multi-center retrospective study. J Digit Imaging 2022;35(6):1514-29. https://doi.org/10.1007/s10278-022-00674-z
  46. Guan X, Zhang B, Fu M, et al. Clinical and inflammatory features based machine learning model for fatal risk prediction of hospitalized covid-19 patients: results from a retrospective cohort study. Ann Med 2021;53(1):257-66.
  47. Jung C, Excoffier JB, Raphael-Rousseau M, Salaun-Penquer N, Ortala M, Chouaid C. Evolution of hospitalized patient characteristics through the first three covid-19 waves in paris area using machine learning analysis. PLoS One 2022;17(2):e0263266.
  48. Zhao C, Bai Y, Wang C, et al. Risk factors related to the severity of covid-19 in wuhan. Int J Med Sci 2021;18(1):120-7. https://doi.org/10.7150/ijms.47193
  49. Jiao Z, Choi JW, Halsey K, et al. Prognostication of patients with covid-19 using artificial intelligence based on chest x-rays and clinical data: a retrospective study. Lancet Digit Health 2021;3(5):e286-94. https://doi.org/10.1016/S2589-7500(21)00039-X
  50. Kang J, Chen T, Luo H, Luo Y, Du G, Jiming-Yang M. Machine learning predictive model for severe covid-19. Infect Genet Evol 2021;90:104737.
  51. Wong KC, Xiang Y, Yin L, So HC. Uncovering clinical risk factors and predicting severe covid-19 cases using uk biobank data: machine learning approach. JMIR Public Health Surveill 2021;7(9):e29544.
  52. Rojas-Garcia M, Vazquez B, Torres-Poveda K, Madrid-Marina V. Lethality risk markers by sex and age-group for covid-19 in mexico: a cross-sectional study based on machine learning approach. BMC Infect Dis 2023;23(1):18.
  53. Burns SM, Woodworth TS, Icten Z, Honda T, Manjourides J. A machine learning approach to identify predictors of severe covid-19 outcome in patients with rheumatoid arthritis. Pain Physician 2022;25(8):593-602.
  54. Wang R, Jiao Z, Yang L, et al. Artificial intelligence for prediction of covid-19 progression using ct imaging and clinical data. Eur Radiol 2022;32(1):205-12.
  55. Chen Y, Ouyang L, Bao FS, et al. A multimodality machine learning approach to differentiate severe and nonsevere covid-19: model development and validation. J Med Internet Res 2021;23(4):e23948.
  56. De Freitas VM, Chiloff DM, Bosso GG, et al. A machine learning model for predicting hospitalization in patients with respiratory symptoms during the covid-19 pandemic. J Clin Med 2022;11(15):4574.
  57. Jehi L, Ji X, Milinovich A, et al. Development and validation of a model for individualized prediction of hospitalization risk in 4,536 patients with covid-19. PLoS One 2020;15(8):e0237419.
  58. Hao B, Sotudian S, Wang T, et al. Early prediction of level-of-care requirements in patients with covid-19. Elife 2020;9:e60519.
  59. Aminu M, Yadav D, Hong L, et al. Habitat imaging biomarkers for diagnosis and prognosis in cancer patients infected with covid-19. Cancers (Basel) 2022;15(1):275.
  60. Chen Z, Russo NW, Miller MM, Murphy RX, Burmeister DB. An observational study to develop a scoring system and model to detect risk of hospital admission due to covid-19. J Am Coll Emerg Physicians Open 2021;2(2):e12406.
  61. Churpek MM, Gupta S, Spicer AB, et al. Machine learning prediction of death in critically ill patients with coronavirus disease 2019. Crit Care Explor 2021;3(8):e0515.
  62. Elghamrawy SM, Hassanien AE, Vasilakos AV. Genetic-based adaptive momentum estimation for predicting mortality risk factors for covid-19 patients using deep learning. Int J Imaging Syst Technol 2022;32(2):614-28. https://doi.org/10.1002/ima.22644
  63. Khadem H, Nemat H, Elliott J, Benaissa M. Interpretable machine learning for inpatient covid-19 mortality risk assessments: diabetes mellitus exclusive interplay. Sensors (Basel) 2022;22(22):8757.
  64. Kablan R, Miller HA, Suliman S, Frieboes HB. Evaluation of stacked ensemble model performance to predict clinical outcomes: a covid-19 study. Int J Med Inform 2023;175:105090.
  65. Ovcharenko E, Kutikhin A, Gruzdeva O, et al. Cardiovascular and renal comorbidities included into neural networks predict the outcome in covid-19 patients admitted to an intensive care unit: three-center, cross-validation, age- and sex-matched study. J Cardiovasc Dev Dis 2023;10(2):39.
  66. Passarelli-Araujo H, Passarelli-Araujo H, Urbano MR, Pescim RR. Machine learning and comorbidity network analysis for hospitalized patients with covid-19 in a city in southern brazil. Smart Health (Amst) 2022;26:100323.
  67. Pournazari P, Spangler AL, Ameer F, et al. Cardiac involvement in hospitalized patients with covid-19 and its incremental value in outcomes prediction. Sci Rep 2021;11(1):19450.
  68. Pyrros A, Rodriguez Fernandez J, Borstelmann SM, et al. Validation of a deep learning, value-based care model to predict mortality and comorbidities from chest radiographs in covid-19. PLOS Digit Health 2022;1(8):e0000057.
  69. Yazdani A, Bigdeli SK, Zahmatkeshan M. Investigating the performance of machine learning algorithms in predicting the survival of covid-19 patients: a cross section study of iran. Health Sci Rep 2023;6(4):e1212.
  70. Wang JM, Liu W, Chen X, McRae MP, McDevitt JT, Fenyo D. Predictive modeling of morbidity and mortality in patients hospitalized with covid-19 and its clinical implications: algorithm development and interpretation. J Med Internet Res 2021;23(7):e29514.
  71. Woo SH, Rios-Diaz AJ, Kubey AA, et al. Development and validation of a web-based severe covid-19 risk prediction model. Am J Med Sci 2021;362(4):355-62.
  72. Ageno W, Cogliati C, Perego M, et al. Clinical risk scores for the early prediction of severe outcomes in patients hospitalized for covid-19. Intern Emerg Med 2021;16(4):989-96. https://doi.org/10.1007/s11739-020-02617-4
  73. Carr E, Bendayan R, Bean D, et al. Evaluation and improvement of the national early warning score (news2) for covid-19: a multi-hospital study. BMC Med 2021;19(1):23.
  74. Min K, Cheng Z, Liu J, et al. Early-stage predictors of deterioration among 3145 nonsevere sars-cov-2-infected people community-isolated in wuhan, china: a combination of machine learning algorithms and competing risk survival analyses. J Evid Based Med 2023;16(2):166-77. https://doi.org/10.1111/jebm.12529
  75. Sun L, Song F, Shi N, et al. Combination of four clinical indicators predicts the severe/critical symptom of patients infected covid-19. J Clin Virol 2020;128:104431.
  76. Liptak P, Banovcin P, Rosolanka R, et al. A machine learning approach for identification of gastrointestinal predictors for the risk of covid-19 related hospitalization. PeerJ 2022;10:e13124.
  77. Nakamichi K, Shen JZ, Lee CS, et al. Hospitalization and mortality associated with sars-cov-2 viral clades in covid-19. Sci Rep 2021;11(1):4802.
  78. Tariq A, Celi LA, Newsome JM, et al. Patient-specific covid-19 resource utilization prediction using fusion ai model. NPJ Digit Med 2021;4(1):94.
  79. Shakibfar S, Nyberg F, Li H, et al. Artificial intelligence-driven prediction of covid-19-related hospitalization and death: a systematic review. Front Public Health 2023;11:1183725.
  80. Giang Hoang N, Abdesselam B, Son Lam P. Learning pattern classification tasks with imbalanced data sets. In: Peng-Yeng Y, editor. Pattern recognition. Rijeka: IntechOpen, 2009:Ch. 10:193-208.
  81. Tasci E, Zhuge Y, Camphausen K, Krauze AV. Bias and class imbalance in oncologic data-towards inclusive and transferrable ai in large scale oncology data sets. Cancers (Basel) 2022;14(12):2897.
  82. Navlakha S, Morjaria S, Perez-Johnston R, Zhang A, Taur Y. Projecting covid-19 disease severity in cancer patients using purposefully-designed machine learning. BMC Infect Dis 2021;21(1):391.
  83. Ricci Lara MA, Echeveste R, Ferrante E. Addressing fairness in artificial intelligence for medical imaging. Nature Communications 2022;13(1):4581.
  84. Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol 2010;5(9):1315-6. https://doi.org/10.1097/JTO.0b013e3181ec173d
  85. Huang S, Cai N, Pacheco PP, Narrandes S, Wang Y, Xu W. Applications of support vector machine (svm) learning in cancer genomics. Cancer Genomics Proteomics 2018;15(1):41-51.
  86. Dessie ZG, Zewotir T. Mortality-related risk factors of covid-19: a systematic review and meta-analysis of 42 studies and 423,117 patients. BMC Infect Dis 2021;21(1):855.
  87. Wu Z, McGoogan JM. Characteristics of and important lessons from the coronavirus disease 2019 (covid-19) outbreak in china: summary of a report of 72 314 cases from the chinese center for disease control and prevention. JAMA 2020;323(13):1239-42. https://doi.org/10.1001/jama.2020.2648
  88. Cazeau N, Palazzo M, Savani M, Shroff RT. Covid-19 vaccines and immunosuppressed patients with cancer: critical considerations. Clin J Oncol Nurs 2022;26(4):367-73.
  89. Bartleson JM, Radenkovic D, Covarrubias AJ, Furman D, Winer DA, Verdin E. Sars-cov-2, covid-19 and the aging immune system. Nature Aging 2021;1(9):769-82. https://doi.org/10.1038/s43587-021-00114-7
  90. Kuderer NM, Choueiri TK, Shah DP, et al. Clinical impact of covid-19 on patients with cancer (ccc19): a cohort study. Lancet 2020;395(10241):1907-18. https://doi.org/10.1016/S0140-6736(20)31187-9
  91. Garassino MC, Whisenant JG, Huang LC, et al. Covid-19 in patients with thoracic malignancies (teravolt): first results of an international, registry-based, cohort study. Lancet Oncol 2020;21(7):914-22. https://doi.org/10.1016/S1470-2045(20)30314-4
  92. Tehrani D, Wang X, Rafique AM, et al. Impact of cancer and cardiovascular disease on in-hospital outcomes of covid-19 patients: results from the american heart association covid-19 cardiovascular disease registry. Cardiooncology 2021;7(1):28.
  93. Asokan I, Rabadia SV, Yang EH. The covid-19 pandemic and its impact on the cardio-oncology population. Curr Oncol Rep 2020;22(6):60.
  94. Momtazmanesh S, Shobeiri P, Hanaei S, Mahmoud-Elsayed H, Dalvi B, Malakan Rad E. Cardiovascular disease in covid-19: a systematic review and meta-analysis of 10,898 patients and proposal of a triage risk stratification tool. The Egyptian Heart Journal 2020;72(1):41.
  95. Kazemi E, Soldoozi Nejat R, Ashkan F, Sheibani H. The laboratory findings and different covid-19 severities: a systematic review and meta-analysis. Annals of Clinical Microbiology and Antimicrobials 2021;20(1):17.