Use of Artificial Intelligence to Manage Patient Flow in Emergency Department during the COVID-19 Pandemic: A Prospective, Single-Center Study
Abstract
:1. Introduction
2. Materials and Methods
2.1. Study Design
2.2. Participants
2.3. Intervention
2.4. Statistical Analysis
3. Results
3.1. Characteristics of the Study Population
3.2. The Model’s Performance
3.3. Individual Predictions
4. Discussion
4.1. The 3P-U threshold
4.2. Related Work
4.3. Implications
4.4. Strengths and Limitations
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A. Characteristics of the Patients
Characteristics | Overall | Missing Value | Imputation Strategy |
Demographic characteristics | |||
Number of patients, n (%) | 10,5457 (100%) | ||
Age, mean ± SD | 51 ± 22 | 0 (00%) | None |
Sex, n (%) | 0 (00%) | None | |
Male | 50,639 (48%) | ||
Female | 54,818 (52%) | ||
Clinical triage characteristics | |||
Heart rate (/min), mean ± SD | 86 ± 18 | 20,253 (19%) | Fixed: 80 |
Systolic blood pressure (mmHg), mean ± SD | 138 ± 24 | 25,558 (24%) | Fixed: 120 |
Diastolic blood pressure (mmHg), mean ± SD | 79 ± 23 | 25,558 (24%) | Fixed: 80 |
Blood oxygen saturation (%), mean ± SD | 99 ± 2 | 38 (<1%) | Fixed: 100 |
Body temperature (°C), mean ± SD | 36.4 ± 0.7 | 19,769 (18%) | Fixed: 37.4 |
Capillary blood glucose level (mmol/L), mean ± SD | 7.33 ± 4.19 | 83,606 (79%) | Fixed: 5.0 |
Capillary blood ketone level (mmol/L), mean ± SD | 0.98 ± 1.97 | 104,207 (98%) | Fixed: 0.0 |
Oxygen flow (L/min), mean ± SD | 0.6 ± 3.7 | 73,410 (69%) | Fixed: 100 |
Capillary blood hemoglobin level (dg/dL), mean ± SD | 11.72 ± 2.96 | 102,533 (97%) | Fixed: 12.0 |
Expired breath alcohol level (g/L), mean ± SD | 1.81 ± 0.78 | 102,625 (97%) | Fixed: 0.0 |
Bladder volume (mL), mean ± SD | 334 ± 305 | 105,078 (99%) | Fixed: 0.0 |
Pain intensity, mean ± SD | 3 ± 3 | 17,095 (16%) | Fixed: 0 |
FRENCH triage scale rating, n (%) | 1356 (1%) | Most frequent | |
1 | 235 (<1%) | ||
2 | 3975 (4%) | ||
3 | 56,679 (54%) | ||
4 | 28,363 (27%) | ||
5 | 14,849 (14%) | ||
Urine tests | |||
Blood in urine, n (%) | 104,696 (99%) | Fixed: 0 | |
0 | 223 (29%) | ||
Traces | 167 (22%) | ||
+ | 93 (12%) | ||
++ | 93 (12%) | ||
+++ | 184 (24%) | ||
++++ | 1 (<1%) | ||
Urine nitrite, n (%) | 104,669 (99%) | Fixed: 0 | |
0 | 661 (87%) | ||
+ | 89 (87%) | ||
++ | 5 (<1%) | ||
+++ | 1 (<1%) | ||
++++ | 2 (<1%) | ||
Urine leukocyte count, n (%) | 104,698 (99%) | Fixed: 0 | |
0 | 475 (63%) | ||
Traces | 93 (12%) | ||
+ | 94 (12%) | ||
++ | 37 (5%) | ||
+++ | 60 (8%) | ||
Urine glucose level, n (%) | 104,698 (99%) | Fixed: 0 | |
0 | 690 (90%) | ||
Traces | 33 (4%) | ||
+ | 7 (<1%) | ||
++ | 25 (3%) | ||
+++ | 3 (<1%) | ||
++++ | 1 (<1%) | ||
Urine ketone level, n (%) | 104,698 (99%) | Fixed: 0 | |
0 | 564 (74%) | ||
Traces | 66 (9%) | ||
+ | 58 (8%) | ||
++ | 40 (5%) | ||
+++ | 13 (2%) | ||
++++ | 18 (2%) | ||
Outcome | 0 (0%) | ||
Admission to a medical ward | 21,466 (20%) | ||
Admission to a surgical ward | 7604 (7%) | ||
Admission to the ICU | 4640 (5%) | ||
Discharge | 71,747 (68%) | ||
Non-clinical triage characteristics | |||
Accompanying person, n (%) | 47,236 (44%) | Most frequent | |
Spouse | 26,047 (45%) | ||
Unrelated person | 13,928 (24%) | ||
Parent | 9437 (16%) | ||
Other family member | 4984 (9%) | ||
Grandparent | 2355 (4%) | ||
Police | 1147 (2%) | ||
Waiting status, n (%) | 25,109 (23%) | Most frequent | |
Stretcher | 56,146 (70%) | ||
Wheelchair | 12,418 (15%) | ||
Standing | 11,517 (14%) | ||
Other | 267 (<1%) | ||
Circumstances, n (%) | 70,653 (67%) | Most frequent | |
Other | 16,887 (48%) | ||
Referred by the EMSs | 8068 (23%) | ||
Fall | 2046 (6%) | ||
Accident in the workspace | 1783 (5%) | ||
Accident at home | 1466 (4%) | ||
Fainting | 447 (<1%) | ||
Sports accident | 375 (<1%) | ||
Road traffic accident | 336 (<1%) | ||
Family context, n (%) | 34,097 (32%) | Most frequent | |
Informed | 45,183 (64%) | ||
Present | 20,657 (29%) | ||
Family due to be informed | 5124 (7%) | ||
Informing the family refused by the patient | 396 (<1%) | ||
Week of the year | 0 (0%) | ||
Day of the week | 0 (0%) | ||
Time of day | 0 (0%) | ||
The urinary tests are qualitative scaled from 0 to ++++ and refer to the strength of the detection. |
Appendix B. Receiver Operating Characteristic Curve for the 3P-U Model and the Study Data
Appendix C. 3P-U’s Automated Dashboard
Appendix D. F1-Score
References
- Yesudhas, D.; Srivastava, A.; Gromiha, M.M. COVID-19 outbreak: History, mechanism, transmission, structural studies and therapeutics. Infect 2021, 49, 199–213. [Google Scholar] [CrossRef] [PubMed]
- SAMU Centres 15 Référentiel et Guide d’évaluation. Available online: https://www.sfmu.org/upload/referentielsSFMU/sfmu-sudf_referentiel_samu_2015.pdf (accessed on 1 July 2022).
- Duchateau, F.X.; Garnier-Connois, D.; Ricard-Hibon, A.; Josseaume, J.; Casalino, E. Response time evaluation for emergency medical service as a part of its performance. Emerg. Med. J. 2013, 30, 763–765. [Google Scholar] [CrossRef]
- Terrasi, B.; Arnaud, E.; Guilbart, M.; Besserve, P.; Mahjoub, Y. French ICUs fight back: An example of regional ICU organisation to tackle the SARS-CoV-2 outbreak. Anaesth. Crit. Care Pain Med. 2020, 39, 355–357. [Google Scholar] [CrossRef]
- Hong, W.S.; Haimovich, A.D.; Taylor, R.A. Predicting hospital admission at emergency department triage using machine learning. PLoS ONE 2018, 13, e0201016. [Google Scholar] [CrossRef] [Green Version]
- Zhang, X.; Kim, J.; Patzer, R.E.; Pitts, S.R.; Patzer, A.; Schrager, J.D. Prediction of Emergency Department Hospital Admission Based on Natural Language Processing and Neural Networks. Methods Inf. Med. 2017, 56, 377–389. [Google Scholar] [CrossRef]
- Kraaijvanger, N.; Rijpsma, D.; Roovers, L.; van Leeuwen, H.; Kaasjager, K.; van den Brand, L.; Horstink, L.; Edwards, M. Development and validation of an admission prediction tool for emergency departments in the Netherlands. Emerg. Med. J. 2018, 35, 464–470. [Google Scholar] [CrossRef]
- Richardson, D.B. Increase in patient mortality at 10 days associated with emergency department overcrowding. Med. J. Aust. 2006, 184, 4. [Google Scholar] [CrossRef]
- Sprivulis, P.C.; Silva, J.A.D.; Jacobs, I.G.; Frazer, A.R.L.; Jelinek, G.A. The association between hospital overcrowding and mortality among patients admitted via Western Australian emergency departments. Med. J. Aust. 2006, 184, 5. [Google Scholar] [CrossRef]
- Guttmann, A.; Schull, M.J.; Vermeulen, M.J.; Stukel, T.A. Association between waiting times and short term mortality and hospital admission after departure from emergency department: Population based cohort study from Ontario, Canada. BMJ 2011, 342, d2983. [Google Scholar] [CrossRef] [Green Version]
- Jones, S.; Moulton, C.; Swift, S.; Molyneux, P.; Black, S.; Mason, N.; Oakley, R.; Mann, C. Association between delays to patient admission from the emergency department and all-cause 30-day mortality. Emerg. Med. J. 2022, 39, 168–173. [Google Scholar] [CrossRef]
- Weiss, S.J.; Ernst, A.A.; Derlet, R.; King, R.; Bair, A.; Nick, T.G. Relationship between the National ED Overcrowding Scale and the number of patients who leave without being seen in an academic ED. Am. J. Emerg. Med. 2005, 23, 288–294. [Google Scholar] [CrossRef] [PubMed]
- Higginson, I.; Boyle, A. What should we do about crowding in emergency departments? Br. J. Hosp. Med. 2018, 79, 500–503. [Google Scholar] [CrossRef] [PubMed]
- Yarmohammadian, M.; Rezaei, F.; Haghshenas, A.; Tavakoli, N. Overcrowding in emergency departments: A review of strategies to decrease future challenges. J. Res. Med. Sci. 2017, 22, 23. [Google Scholar] [PubMed]
- Hoot, N.R.; Aronsky, D. Systematic Review of Emergency Department Crowding: Causes, Effects, and Solutions. Ann. Emerg. Med. 2008, 52, 126–136. [Google Scholar] [CrossRef]
- Ghazali, D.A.; Choquet, C.; Bouzid, D.; Peyrony, O.; Fontaine, J.P.; Sonja, C.; Javaud, N.; Plaisance, P.; Revue, E.; Chauvin, A.; et al. The Response of Emergency Departments (EDs) to the COVID-19 Pandemic: The Experience of 5 EDs in a Paris-Based Academic Hospital Trust. Qual. Manag. Health Care. 2022. Available online: https://journals.lww.com/10.1097/QMH.0000000000000351 (accessed on 1 July 2022). [CrossRef]
- Arnaud, E.; Elbattah, M.; Gignon, G. Deep Learning to Predict Hospitalization at Triage: Integration of Structured Data and Unstructured Text. In Proceedings of the 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 10–13 December 2020; pp. 4836–4841. Available online: https://ieeexplore.ieee.org/document/9378073/ (accessed on 1 July 2022).
- Collins, G.S.; Reitsma, J.B.; Altman, D.G.; Moons, K.G.M. Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD). BMC. Medicine. 2015, 62, 9. [Google Scholar]
- Taboulet, P.; Moreira, V.; Haas, L.; Porcher, R.; Braganca, A.; Fontaine, J.P.; Poncet, M.C. Triage with the French Emergency Nurses Classi-fication in Hospital scale: Reliability and validity. Eur. J. Emerg. Med. 2009, 16, 61–67. [Google Scholar] [CrossRef] [Green Version]
- Données de Laboratoires pour le Dépistage (A COMPTER DU 18/05/2022)—SI-DEP. Available online: https://www.data.gouv.fr/fr/datasets/donnees-de-laboratoires-pour-le-depistage-a-compter-du-18-05-2022-si-dep/ (accessed on 1 July 2022).
- Fernandes, M.; Mendes, R.; Vieira, S.M.; Leite, F.; Palos, C.; Johnson, A.; Finkelstein, S.; Horng, S.; Celi, L.A. Predicting Intensive Care Unit admission among patients presenting to the emergency department using machine learning and natural language processing. PLoS ONE 2020, 15, e0229331. [Google Scholar] [CrossRef]
- Sánchez-Salmerón, R.; Gómez-Urquiza, J.L.; Albendín-García, L.; Correa-Rodríguez, M.; Martos-Cabrera, M.B.; Velando-Soriano, A.; Suleiman-Martos, N. Machine learning methods applied to triage in emergency services: A systematic review. Int. Emerg. Nursing. 2022, 60, 101109. [Google Scholar] [CrossRef]
- Murray, N.M.; Unberath, M.; Hager, G.D.; Hui, F.K. Artificial intelligence to diagnose ischemic stroke and identify large vessel occlusions: A systematic review. J. NeuroInterv. Surg. 2020, 12, 156–164. [Google Scholar] [CrossRef]
- Kim, K.H.; Park, J.H.; Ro, Y.S.; Hong, K.J.; Song, K.J.; Shin, S.D. Emergency department routine data and the diagnosis of acute ischemic heart disease in patients with atypical chest pain. PLoS ONE 2020, 15, e0241920. [Google Scholar] [CrossRef] [PubMed]
- Graham, B.; Bond, R.; Quinn, M.; Mulvenna, M. Using Data Mining to Predict Hospital Admissions From the Emergency De-partment. IEEE Access. 2018, 6, 10458–10469. [Google Scholar] [CrossRef]
- Tan, T.H.; Hsu, C.C.; Chen, C.J.; Hsu, S.L.; Liu, T.L.; Lin, H.J.; Wang, J.J.; Liu, C.F.; Huang, C.C. Predicting outcomes in older ED patients with influenza in real time using a big data-driven and machine learning approach to the hospital information system. BMC Geriatr. 2021, 21, 280. [Google Scholar] [CrossRef] [PubMed]
- Leow, S.H.; Dean, W.; MacDonald-Nethercott, M.; MacDonald-Nethercott, E.; Boyle, A.A. The Attend Study: A Retrospective Observational Study of Emergency Department Attendances During the Early Stages of the COVID-19 Pandemic. Cureus 2020, 12, e9328. Available online: https://www.cureus.com/articles/34364-the-attend-study-a-retrospective-observational-study-of-emergency-department-attendances-during-the-early-stages-of-the-covid-19-pandemic (accessed on 1 July 2022). [CrossRef]
Demographic Characteristics | Overall |
---|---|
Number of patients, n (%) | 105,457 (100%) |
Age, mean ± SD | 51 ± 22 |
Sex, n (%) | |
Male | 50,639 (48%) |
Female | 54,818 (52%) |
Clinical triage characteristics | |
Heart rate (/min), mean ± SD | 86 ± 18 |
Systolic blood pressure (mmHg), mean ± SD | 138 ± 24 |
Diastolic blood pressure (mmHg), mean ± SD | 79 ± 23 |
Blood oxygen saturation (%), mean ± SD | 99 ± 2 |
Body temperature (°C), mean ± SD | 36.4 ± 0.7 |
Capillary blood glucose level (mmol/L), mean ± SD | 7.33 ± 4.19 |
Capillary blood ketone level (mmol/L), mean ± SD | 0.98 ± 1.97 |
Oxygen flow (L/min), mean ± SD | 0.6 ± 3.7 |
Capillary blood hemoglobin level (dg/dL), mean ± SD | 11.72 ± 2.96 |
Expired breath alcohol level (g/L), mean ± SD | 1.81 ± 0.78 |
Bladder volume (mL), mean ± SD | 334 ± 305 |
Pain intensity, mean ± SD | 3 ± 3 |
Patient rating on the FRENCH triage scale, n (%) | |
1 | 235 (< 1%) |
2 | 3975 (4%) |
3 | 56,679 (54%) |
4 | 28,363 (27%) |
5 | 14,849 (14%) |
Outcome | |
Admission to a medical ward | 21,470 (21%) |
Admission to a surgical ward | 7604 (7%) |
Admission to the ICU | 4641 (4%) |
Discharge | 71,616 (68%) |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Arnaud, E.; Elbattah, M.; Ammirati, C.; Dequen, G.; Ghazali, D.A. Use of Artificial Intelligence to Manage Patient Flow in Emergency Department during the COVID-19 Pandemic: A Prospective, Single-Center Study. Int. J. Environ. Res. Public Health 2022, 19, 9667. https://doi.org/10.3390/ijerph19159667
Arnaud E, Elbattah M, Ammirati C, Dequen G, Ghazali DA. Use of Artificial Intelligence to Manage Patient Flow in Emergency Department during the COVID-19 Pandemic: A Prospective, Single-Center Study. International Journal of Environmental Research and Public Health. 2022; 19(15):9667. https://doi.org/10.3390/ijerph19159667
Chicago/Turabian StyleArnaud, Emilien, Mahmoud Elbattah, Christine Ammirati, Gilles Dequen, and Daniel Aiham Ghazali. 2022. "Use of Artificial Intelligence to Manage Patient Flow in Emergency Department during the COVID-19 Pandemic: A Prospective, Single-Center Study" International Journal of Environmental Research and Public Health 19, no. 15: 9667. https://doi.org/10.3390/ijerph19159667
APA StyleArnaud, E., Elbattah, M., Ammirati, C., Dequen, G., & Ghazali, D. A. (2022). Use of Artificial Intelligence to Manage Patient Flow in Emergency Department during the COVID-19 Pandemic: A Prospective, Single-Center Study. International Journal of Environmental Research and Public Health, 19(15), 9667. https://doi.org/10.3390/ijerph19159667