International Journal of Computer Science & Network Security

International Journal of Computer Science & Network Security (국제컴퓨터통신보호논문지학회)
권호 표지
DOI QR코드

DOI QR Code

A Prediction Triage System for Emergency Department During Hajj Period using Machine Learning Models

  • Huda N. Alhazmi (Umm Al-Qura University, College of Computers, Department of Computer Science and Artificial Intelligence)
  • Received : 2024.07.05
  • Published : 2024.07.30
Download PDF
⟨ Previous Next ⟩

Abstract

Triage is a practice of accurately prioritizing patients in emergency department (ED) based on their medical condition to provide them with proper treatment service. The variation in triage assessment among medical staff can cause mis-triage which affect the patients negatively. Developing ED triage system based on machine learning (ML) techniques can lead to accurate and efficient triage outcomes. This study aspires to develop a triage system using machine learning techniques to predict ED triage levels using patients' information. We conducted a retrospective study using Security Forces Hospital ED data, from 2021 through 2023 during Hajj period in Saudia Arabi. Using demographics, vital signs, and chief complaints as predictors, two machine learning models were investigated, naming gradient boosted decision tree (XGB) and deep neural network (DNN). The models were trained to predict ED triage levels and their predictive performance was evaluated using area under the receiver operating characteristic curve (AUC) and confusion matrix. A total of 11,584 ED visits were collected and used in this study. XGB and DNN models exhibit high abilities in the predicting performance with AUC-ROC scores 0.85 and 0.82, respectively. Compared to the traditional approach, our proposed system demonstrated better performance and can be implemented in real-world clinical settings. Utilizing ML applications can power the triage decision-making, clinical care, and resource utilization.

Keywords

References

  1. PG. Jones, D. Mountain, and R. Forero, "Review article: Emergency department crowding measures associations with quality of care: A systematic review," Emerg Med Australas, vol. 33, no.4, pp. 592-600, 2021, doi: 10.1111/1742-6723.13743. 
  2. K. Stobbe, D. Dewar, C. Thornton, et al., "Canadian Emergency Department Triage and Acuity Scale (CTAS): Rural Implementation Statement," CJEM,52, pp. 104-7, 2003.  https://doi.org/10.1017/S1481803500008241
  3. Australasian College for Emergency Medicine. Guidelines on the Implementation of the Australasian Triage Scale in Emergency Department. Doc No G24. Australasian College for Emergency Medicine; 2013:1-8. 
  4. JM. Zachariasse, N. Seiger, et al., "Validity of the Manchester Triage System in emergency care: A prospective observational study," PLoS One, vol. 12, no. 2, 2017, doi: 10.1371/journal.pone.0170811. 
  5. D. Sax, E. Warton, et al., "Evaluation of the Emergency Severity Index in US Emergency Departments for the Rate of Mistriage," JAMA Netw Open, vol. 6, no. 3, 2023, doi: 10.1001/jamanetworkopen.2024.23536. 
  6. Y. Raita, T. Goto, M.K. Faridi, et al. "Emergency department triage prediction of clinical outcomes using machine learning models," Crit Care, vol. 23, 2019, https://doi.org/10.1186/s13054-019-2351-7. 
  7. C. Morley, M. Unwin, GM. Peterson, et al., "Emergency department crowding: a systematic review of causes, consequences and solutions," PloS One, vol. 13, no. 8, 2018, doi:10.1371/journal.pone.0203316. 
  8. B. Asplin, D. Magid, K. Rhodes, et al., "A conceptual model of emergency department crowding," Ann Emerg Med., vol. 42, no. 2, 2003, doi: 10.1067/mem.2003.302. 
  9. A. Salmon-Rousseau, E. Piednoir, V. Cattoir, et al., "Hajj-associated infections," Med Mal Infect, vol. 46, pp. 346-354, 2016.  https://doi.org/10.1016/j.medmal.2016.04.002
  10. B. Mistry, S. Stewart De Ramirez, G. Kelen, et al., "Accuracy and reliability of emergency department triage using the emergency severity index: An international multicenter assessment," Ann. Emerg. Med vol. 71, no. 5, pp. 581-587, 2018, doi: 10.1016/j.annemergmed.2017.09.036. 
  11. G.umhee Baek, D. Baik, and N.Yi, "The effects of triage applying artificial intelligence on triage in the emergency department: A systematic review of prospective studies", 2023, PREPRINT available at Research Square., doi.org/10.21203/rs.3.rs-3288343/v1 
  12. M. Fernandes, SM. Vieira, F. Leite, et al., "Clinical decision support systems for triage in the emergency department using intelligent systems: a review," Artif Intell Med, vol 102, 2020, doi: 10.1016/j.artmed.2019.101762. 
  13. S, Jalal, W. Parker, D. Ferguson, et al., "Exploring the Role of Artificial Intelligence in an Emergency and Trauma Radiology Department," Can Assoc Radiol J., vol. 72, no. 1, 2021, doi: 10.1177/0846537120918338. 
  14. T. Panch, P. Szolovits, and R. Atun, "Artificial intelligence, machine learning, and health systems," J Glob Health, vol. 8, no. 2, 2018, doi: 10.7189/jogh.08.020303. 
  15. S. Lee, N. M. Mohr, W. N. Street, et al., "Machine learning in relation to emergency medicine clinical and operational scenarios: an overview," West J. Emerg. Med, vol. 20, no. 2, pp. 219-227, 2019, doi: 10.5811/westjem.2019.1.41244. 
  16. B. Mueller, T. Kinoshita, A. Peebles, et al. "Artificial intelligence and machine learning in emergency medicine: a narrative review," Acute Med Surg, vol. 9, no. 1, 2022, doi: 10.1002/ams2.740. 
  17. Y. Raita, T. Goto, M. K. Faridi, et al. "Emergency department triage prediction of clinical outcomes using machine learning models," Crit Care, 2019 vol. 23, no. 1, 2019, doi: 10.1186/s13054-019-2351-7. 
  18. Z. Gao, X. Qi, X. Zhang, et al., "Developing and Validating an Emergency Triage Model Using Machine Learning Algorithms with Medical Big Data," Risk Manag Healthc Policy, vol. 15, pp.1545-1551, 2022, doi: 10.2147/RMHP.S355176. 
  19. H. Yun, J. Choi, and J. H. Park, "Prediction of Critical Care Outcome for Adult Patients Presenting to Emergency Department Using Initial Triage Information: An XGBoost Algorithm Analysis," JMIR Med Inform, vol. 9, no. 9, 2021, doi: 10.2196/30770. 
  20. W. S. Hong, A. D. Haimovich, and R. A. Taylor, "Predicting hospital admission at emergency department triage using machine learning," PLoS One, vol. 13, no. 7, 2018, doi: 10.1371/journal.pone.0201016. 
  21. M. Fernandes, R. Mendes, S. M. Vieira, et al., "Predicting Intensive Care Unit admission among patients presenting to the emergency department using machine learning and natural language processing," PLoS One, vol. 15, no. 3, 2020, doi: 10.1371/journal.pone.0229331. 
  22. CW. Sung, J. Ho, CY. Fan, et al., "Prediction of high-risk emergency department revisits from a machine-learning algorithm: a proof-of-concept study," BMJ Health Care Inform, vol. 31, no. 1, 2024, doi: 10.1136/bmjhci-2023-100859. 
  23. H. Elhaj, N. Achour, M. H. Tania, et al., "A comparative study of supervised machine learning approaches to predict patient triage outcomes in hospital emergency departments," Array, vol. 17, 2023, doi.org/10.1016/j.array.2023.100281. 
  24. YT. Lin, YX. Deng, CL. Tsai, et al., "Interpretable Deep Learning System for Identifying Critical Patients Through the Prediction of Triage Level, Hospitalization, and Length of Stay: Prospective Study," JMIR Med Inform, vol. 12, 2024, doi: 10.2196/48862. 
  25. LH. Yao, KC. Leung, CL. Tsai, et al., "A Novel Deep Learning-Based System for Triage in the Emergency Department Using Electronic Medical Records: Retrospective Cohort Study," J Med Internet Res, vol. 12, 2021, doi: 10.2196/27008. 
  26. Y. Liu, J. Gao, J. Liu, et al., "Development and validation of a practical machine-learning triage algorithm for the detection of patients in need of critical care in the emergency department," Sci Rep, vol. 11, no. 1, 2021, doi: 10.1038/s41598-021-03104-2. 
  27. A. Ekwall, M. Gerdtz, and E. Manias, "The influence of patient acuity on satisfaction with emergency care: perspectives of family, friends and carers" J Clin Nurs, vol. 17, no, 6, 2008, doi: 10.1111/j.1365-2702.2007.02052. 
  28. G. FitzGerald, GA. Jelinek, D. Scott D, et al., "Emergency department triage revisited," Emerg Med J, vol. 27, no. 2, pp. 86-92, 2010, doi.org/10.1136/emj.2009.077081. 
  29. Cioffi J. Triage decision making: educational strategies. Accident and emergency nursing, 1999, 7:106-11.  https://doi.org/10.1016/S0965-2302(99)80031-9
  30. A. Chorzempa and A. LaMotte, "The role of the triage nurse practitioner in general medical practice: an analysis of the role," Clinical excellence for nurse practitioners, vol. 3, no. 3, pp. 189-90, 1999. 
  31. S. George, S. Read, L. Westlake, et al.," Nurse triage in theory and in practice," Arch Emerg Med, vol. 10, no. 3, pp. 220-8, 1993, doi: 10.1136/emj.10.3.220. 
  32. A. M. Al Yasin, M. Alyaseen, and S. Alyaseen, "The Effectiveness of Emergency Triage Systems: A Systematic Review," vol. 6, pp. 272-282, 2023, DOI: 10.36348/sjnhc.2023.v06i08.004. 
  33. M. Alquraini, E. Awad, and R. Hijazi, "Reliability of Canadian Emergency Department Triage and Acuity Scale (CTAS) in Saudi Arabia," Int J Emerg Med, vol. 8, no. 1, 2015, doi.org/10.1186/s12245-015-0080-5. 
  34. R. Alhaqbani, R. Bahmaid, M. Almutairi, et al.," Patient's conception and attitude regarding triage system and waiting time at emergency department at Riyadh, Saudi Arabia," Saudi Journal of Emergency Medicine, vol. 3, pp. 130-137, 2022, DOI: 10.24911/SJEMED/72-1649358740. 
  35. I. Kaysi, B. Alshalalfeh, M. Sayour, et al.," Rapid transit service in the unique context of Holy Makkah: assessing the first year of operation during the 2010 pilgrimage season," in Proceedings of the 18th International Conference on Urban Transport and the Environment, 2012, pp. 253-267. 
  36. D. F. Gaieski, A. K. Agarwal, M. E. Mikkelsen, et al., "The impact of ED crowding on early interventions and mortality in patients with severe sepsis," Am J Emerg Med, vol. 35, no. 7, pp. 953-60, 2017, DOI: 10.1016/j.ajem.2017.01.061 
  37. E. J. Weber, "Triage: making the simple complex?" Emerg Med J, vol. 36, no. 6, pp. 64-65, 2019, doi: 10.1136/emermed-2018-207659 
  38. F. Rahimian, G. Salimi-Khorshidi, A. H. Payberah, et al., "Predicting the risk of emergency admission with machine learning: Development and validation using linked electronic health records," PLoS Med, vol. 15, no. 11, 2018, doi: 10.1371/journal.pmed.1002695 
  39. S. Farahmand, O. Shabestari, M. Pakrah, et al., "Artificial Intelligence-Based Triage for Patients with Acute Abdominal Pain in Emergency Department; a Diagnostic Accuracy Study," Adv J Emerg Med, vol. 21, no. 1, 2017, doi: 10.22114/AJEM.v1i1.11. 
  40. DY. Kang, KJ. Cho, O.Kwon, et al., "Artificial intelligence algorithm to predict the need for critical care in prehospital emergency medical services," Scand J Trauma Resusc Emerg Med, vol. 28, no. 1, 2020, doi: 10.1186/s13049-020-0713-4F. 
  41. Pedregosa, G. Varoquaux, A. Gramfort, et al., "Scikit-learn: Machine Learning in Python," Journal of Machine Learning Research, vol. 12, pp. 2825-2830, 2011.
  42. T. Chen, C. Guestrin, "XGBoost: A Scalable Tree Boosting System," in: Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., Association for Computing Machinery, New York, NY, USA, 2016, 785-794. 
  43. M.A. Deif, A.A.A. Solyman, M.H. Alsharif, et al., "Automated Triage System for Intensive Care Admissions during the COVID-19 Pandemic Using Hybrid XGBoost-AHP Approach," Sensors (Basel), vol 21, no. 19, 2021, doi: 10.3390/s21196379. 
  44. SK. Mun and M. Chang, "Development of prediction models for the incidence of pediatric acute otitis media using Poisson regression analysis and XGBoost," Environ Sci Pollut Res Int, vol. 29, no. 13, pp. 18629-18640, 2022, doi: 10.1007/s11356-021-17135-9. 
  45. A. Ogunleye and Q. -G. Wang, "XGBoost Model for Chronic Kidney Disease Diagnosis," in IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 17, no. 6, pp. 2131-2140, 1 Nov.-Dec. 2020, doi: 10.1109/TCBB.2019.2911071. 
  46. D. Spangler, T. Hermansson, D. Smekal D, et al., "A validation of machine learning-based risk scores in the prehospital setting," PLoS One, vol. 14, no. 12, 2019. doi: 10.1371/journal.pone.0226518. 
  47. Z. Huang, C. Hu, C. Chi C, et al., "An Artificial Intelligence Model for Predicting 1-Year Survival of Bone Metastases in Non-Small-Cell Lung Cancer Patients Based on XGBoost Algorithm," Biomed Res Int, 2020, doi: 10.1155/2020/3462363. 
  48. J. Baek, and Y. Choi, "Deep Neural Network for Predicting Ore Production by Truck-Haulage Systems in Open-Pit Mines," Applied Sciences, vol. 10, 2020, doi.org/10.3390/app10051657. 
  49. M. Kuhn, and K. Johnson, Applied predictive modeling. New York: Springer-Verlag, 2013. 
  50. Xgboost, Extreme gradient boosting. https://CRAN.R-project.org/package=xgboost. Accessed: 15 Jun 2024. 
  51. R Interface to Keras. https://keras.rstudio.com/. Accessed: 15 Jun 2024. 
  52. D.P. Kingma and J. Ba, "Adam: A method for stochastic optimization. ArXiv14126980. http://arxiv.org/abs/1412.6980. Accessed: 15 Jun 2024. 
  53. W.S. Hong, A.D. Haimovich, and R.A. Taylor, "Predicting hospital admission at emergency department triage using machine learning," PLoS One, vol. 13, no. 7, pp. 1-13, 2018. doi: 10.1371/journal.pone.0201016. 
  54. E. Alpaydin, Introduction to Machine Learning, 4th ed, Adaptive Computation and Machine Learning Series, MIT Press, 2020. 
  55. G. Chenais, E. Lagarde, and C. Gil-Jardine, "Artificial Intelligence in Emergency Medicine: Viewpoint of Current Applications and Foreseeable Opportunities and Challenges," J Med Internet Res, vol. 25, 2023, doi:10.2196/40031.