Journal of The Korean Society of Emergency Medicine (대한응급의학회지)

The Korean Society of Emergency Medicine (대한응급의학회)
권호 표지

Predicting the mortality of pneumonia patients visiting the emergency department through machine learning

기계학습모델을 통한 응급실 폐렴환자의 사망예측 모델과 기존 예측 모델의 비교

  • Bae, Yeol (Department of Emergency Medicine, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea) ;
  • Moon, Hyung Ki (Department of Emergency Medicine, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea) ;
  • Kim, Soo Hyun (Department of Emergency Medicine, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea)
  • 배열 (가톨릭대학교 의과대학 응급의학교실) ;
  • 문형기 (가톨릭대학교 의과대학 응급의학교실) ;
  • 김수현 (가톨릭대학교 의과대학 응급의학교실)
  • Received : 2018.06.25
  • Accepted : 2018.08.20
  • Published : 2018.10.31

⟨ Previous Next ⟩

Abstract

Objective: Machine learning is not yet widely used in the medical field. Therefore, this study was conducted to compare the performance of preexisting severity prediction models and machine learning based models (random forest [RF], gradient boosting [GB]) for mortality prediction in pneumonia patients. Methods: We retrospectively collected data from patients who visited the emergency department of a tertiary training hospital in Seoul, Korea from January to March of 2015. The Pneumonia Severity Index (PSI) and Sequential Organ Failure Assessment (SOFA) scores were calculated for both groups and the area under the curve (AUC) for mortality prediction was computed. For the RF and GB models, data were divided into a test set and a validation set by the random split method. The training set was learned in RF and GB models and the AUC was obtained from the validation set. The mean AUC was compared with the other two AUCs. Results: Of the 536 investigated patients, 395 were enrolled and 41 of them died. The AUC values of PSI and SOFA scores were 0.799 (0.737-0.862) and 0.865 (0.811-0.918), respectively. The mean AUC values obtained by the RF and GB models were 0.928 (0.899-0.957) and 0.919 (0.886-0.952), respectively. There were significant differences between preexisting severity prediction models and machine learning based models (P<0.001). Conclusion: Classification through machine learning may help predict the mortality of pneumonia patients visiting the emergency department.

Keywords

References

  1. Musher DM, Thorner AR. Community-acquired pneumonia. N Engl J Med 2014;371:1619-28. https://doi.org/10.1056/NEJMra1312885
  2. Restrepo MI, Mortensen EM, Rello J, Brody J, Anzueto A. Late admission to the ICU in patients with communityacquired pneumonia is associated with higher mortality. Chest 2010;137:552-7. https://doi.org/10.1378/chest.09-1547
  3. Niederman MS, McCombs JS, Unger AN, Kumar A, Popovian R. The cost of treating community-acquired pneumonia. Clin Ther 1998;20:820-37. https://doi.org/10.1016/S0149-2918(98)80144-6
  4. Fine MJ, Auble TE, Yealy DM, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med 1997;336:243-50. https://doi.org/10.1056/NEJM199701233360402
  5. Lim WS, van der Eerden MM, Laing R, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax 2003;58:377-82. https://doi.org/10.1136/thorax.58.5.377
  6. Charles PG, Wolfe R, Whitby M, et al. SMART-COP: a tool for predicting the need for intensive respiratory or vasopressor support in community-acquired pneumonia. Clin Infect Dis 2008;47:375-84. https://doi.org/10.1086/589754
  7. Obermeyer Z, Emanuel EJ. Predicting the future: big data, machine learning, and clinical medicine. N Engl J Med 2016;375:1216-9. https://doi.org/10.1056/NEJMp1606181
  8. Yoon JG, Heo J, Kim M, et al. Machine learning-based diagnosis for disseminated intravascular coagulation (DIC): development, external validation, and comparison to scoring systems. PLoS One 2018;13:e0195861. https://doi.org/10.1371/journal.pone.0195861
  9. Dybowski R, Weller P, Chang R, Gant V. Prediction of outcome in critically ill patients using artificial neural network synthesised by genetic algorithm. Lancet 1996;347:1146-50. https://doi.org/10.1016/S0140-6736(96)90609-1
  10. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 2016;316:2402-10. https://doi.org/10.1001/jama.2016.17216
  11. Way GP, Sanchez-Vega F, La K, et al. Machine learning detects pan-cancer Ras pathway activation in The Cancer Genome Atlas. Cell Rep 2018;23:172-80.e3. https://doi.org/10.1016/j.celrep.2018.03.046
  12. Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform 2002;35:352-9. https://doi.org/10.1016/S1532-0464(03)00034-0
  13. Genuer R, Poggi JM, Tuleau-Malot C. Variable selection using random forests. Pattern Recognit Lett 2010;31:2225-36. https://doi.org/10.1016/j.patrec.2010.03.014
  14. Mandell LA, Wunderink RG, Anzueto A, et al. Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of communityacquired pneumonia in adults. Clin Infect Dis 2007;44 Suppl 2:S27-72. https://doi.org/10.1086/511159
  15. American Thoracic Society. Infectious Diseases Society of Amerrica. Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcareassociated pneumonia. Am J Respir Crit Care Med 2005;171:388-416. https://doi.org/10.1164/rccm.200405-644ST
  16. Breiman L. Random forests. Mach Learn 2001;45:5-32. https://doi.org/10.1023/A:1010933404324
  17. Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Stat 2001;29:1189-232.
  18. Capelastegui A, Espana PP, Quintana JM, et al. Validation of a predictive rule for the management of communityacquired pneumonia. Eur Respir J 2006;27:151-7. https://doi.org/10.1183/09031936.06.00062505
  19. Aujesky D, Auble TE, Yealy DM, et al. Prospective comparison of three validated prediction rules for prognosis in community-acquired pneumonia. Am J Med 2005;118:384-92. https://doi.org/10.1016/j.amjmed.2005.01.006
  20. Zhang BT. Next-generation machine learning technologies. Commun Korean Inst Inf Sci Eng 2007;25:96-107.
  21. Launay CP, Riviere H, Kabeshova A, Beauchet O. Predicting prolonged length of hospital stay in older emergency department users: use of a novel analysis method, the Artificial Neural Network. Eur J Intern Med 2015;26:478-82. https://doi.org/10.1016/j.ejim.2015.06.002
  22. Weng SF, Reps J, Kai J, Garibaldi JM, Qureshi N. Can machine-learning improve cardiovascular risk prediction using routine clinical data? PLoS One 2017;12:e0174944. https://doi.org/10.1371/journal.pone.0174944
  23. Olden JD, Jackson DA. Illuminating the "black box": a randomization approach for understanding variable contributions in artificial neural networks. Ecol Modell 2002;154:135-50. https://doi.org/10.1016/S0304-3800(02)00064-9
  24. Welling SH, Refsgaard HH, Brockhoff PB, Clemmensen LH. Forest floor visualizations of random forests. arXiv 2016:1605.09196.