Can a Generative Artificial Intelligence Model Be Used to Create Mass Casualty Incident Simulation Scenarios? A Feasibility Study
Abstract
1. Introduction
2. Materials and Methods
2.1. Data and Analytical Methods
2.1.1. Database Creation
2.1.2. Custom LLM Development
2.1.3. Integration
LLM Architecture, Hyperparameters, and Prompting
- Compute Resources
- Scenario Generation
- LLM Output
- Summarization and Outcome Assessment
- Quantitative Review
- Subjective Review
- Revision Process
- Statistical Analysis
- Deployment of Research Tool
3. Results
- Scenario Scoring
- Interrater reliability
- Revision process
- Post-Revision Scoring
4. Discussion
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
| AI | Artificial Intelligence |
| API | Application Programming Interface |
| ED | Emergency Department |
| EMS | Emergency Medical Services |
| FAST | Focused Assessment with Sonography for Trauma |
| GCS | Glasgow Coma Scale |
| GenAI | Generative Artificial Intelligence |
| GPT | Generative Pre-trained Transformer |
| HR | Heart Rate |
| IO | Intraosseous |
| LLM | Large Language Model |
| MAP | Mean Arterial Pressure |
| MCI | Mass Casualty Incident |
| NEISS | National Electronic Injury Surveillance System |
| NFPA | National Fire Protection Agency |
| NHTSA | National Highway Traffic Safety Administration |
| NTDB | National Trauma Data Bank |
| NTSB | National Traffic and National Transportation Safety Board |
| RLHF | Reinforcement Learning from Human Feedback |
| RR | Respiratory Rate |
| SBP | Systolic Blood Pressure |
| SSET | Surgical Simulation Evaluation Tool |
| US | United States |
| USCG | United States Coast Guard |
References
- King, D.R.; Patel, M.B.; Feinstein, A.J.; Earle, S.A.; Topp, R.F.; Proctor, K.G. Simulation training for a mass casualty incident: Two-year experience at the Army Trauma Training Center. J. Trauma 2006, 61, 943–948. [Google Scholar] [CrossRef] [PubMed]
- Quick, J.A. Simulation Training in Trauma. Mo. Med. 2018, 115, 447–450. [Google Scholar] [PubMed] [PubMed Central]
- Khajehaminian, M.R.; Ardalan, A.; Keshtkar, A.; Hosseini Boroujeni, S.M.; Nejati, A.; Ebadati, E.O.M.E.; Rahimi Foroushani, A. A systematic literature review of criteria and models for casualty distribution in trauma related mass casualty incidents. Injury 2018, 49, 1959–1968. [Google Scholar] [CrossRef] [PubMed]
- Igra, N.M.; Schmulevich, D.; Geng, Z.; Guzman, J.; Biddinger, P.D.; Gates, J.D.; Spinella, P.C.; Yazer, M.H.; Cannon, J.W.; THOR-AABB Workgroup. Optimizing Mass Casualty Triage: Using Discrete Event Simulation to Minimize Time to Resuscitation. J. Am. Coll. Surg. 2024, 238, 41–53. [Google Scholar] [CrossRef] [PubMed]
- Castoldi, L.; Greco, M.; Carlucci, M.; Lennquist Montan, K.; Faccincani, R. Mass Casualty Incident (MCI) training in a metropolitan university hospital: Short-term experience with MAss Casualty SIMulation system MACSIM®. Eur. J. Trauma Emerg. Surg. 2022, 48, 283–291. [Google Scholar] [CrossRef]
- Tallach, R.; Schyma, B.; Robinson, M.; O’NEill, B.; Edmonds, N.; Bird, R.; Sibley, M.; Leitch, A.; Cross, S.; Green, L.; et al. Refining Mass Casualty Plans With Simulation-Based Iterative Learning. Br. J. Anaesth. 2022, 128, e180–e189. [Google Scholar] [CrossRef]
- Gabbe, B.J.; Veitch, W.; Mather, A.; Curtis, K.; Holland, A.J.; Gomez, D.; Civil, I.; Nathens, A.; Fitzgerald, M.; Martin, K.; et al. Review of the Requirements for Effective Mass Casualty Preparedness for Trauma Systems. A Disaster Waiting to Happen? Br. J. Anaesth. 2022, 128, e158–e167. [Google Scholar] [CrossRef]
- Moss, R.; Gaarder, C. Exercising for Mass Casualty Preparedness. Br. J. Anaesth. 2022, 128, e67–e70. [Google Scholar] [CrossRef]
- Debacker, M.; Van Utterbeeck, F.; Ullrich, C.; Dhondt, E.; Hubloue, I. SIMEDIS: A Discrete-Event Simulation Model for Testing Responses to Mass Casualty Incidents. J. Med. Syst. 2016, 40, 273. [Google Scholar] [CrossRef]
- Hick, J.L.; Einav, S.; Hanfling, D.; Kissoon, N.; Dichter, J.R.; Devereaux, A.V.; Christian, M.D. Surge Capacity Principles: Care of the Critically Ill and Injured During Pandemics and Disasters: CHEST Consensus Statement. Chest 2014, 146 (Suppl. 4), e1S–e16S. [Google Scholar] [CrossRef]
- Heldring, S.; Jirwe, M.; Wihlborg, J.; Lindström, V. Acceptability and Applicability of Using Virtual Reality for Training Mass Casualty Incidents- A Mixed Method Study. BMC Med. Educ. 2025, 25, 728. [Google Scholar] [CrossRef] [PubMed]
- Bauchwitz, B.; Nguyen, J.; Woods, K.; Albagli, K.; Sawitz, M.; Hatch, M.; Broach, J. The Use of Smartphone-Based Highly Realistic MCI Training as an Adjunct to Traditional Training Methods. Mil. Med. 2024, 189 (Suppl. 3), 775–783. [Google Scholar] [CrossRef]
- Blimark, M.; Robinson, Y.; Jacobson, C.; Lönroth, H.; Boffard, K.D.; Montán, K.L.; Laesser, I.; Örtenwall, P. Determining Surgical Surge Capacity with a Hybrid Simulation Exercise. Front. Public Health 2023, 11, 1157653. [Google Scholar] [CrossRef]
- Mills, B.; Dykstra, P.; Hansen, S.; Miles, A.; Rankin, T.; Hopper, L.; Brook, L.; Bartlett, D. Virtual Reality Triage Training Can Provide Comparable Simulation Efficacy for Paramedicine Students Compared to Live Simulation-Based Scenarios. Prehospital Emerg. Care 2020, 24, 525–536. [Google Scholar] [CrossRef]
- Newton, J.; Smith, A.D.A.C. Developing Emotional Preparedness and Mental Resilience Through High-Fidelity Simulation: A ‘Bridge Too Far’ for Institutions Teaching Major Trauma Management and Mass-Casualty Medicine? BMC Med. Educ. 2024, 24, 544. [Google Scholar] [CrossRef]
- Christiano, P.F.; Leike, J.; Brown, T.; Martic, M.; Legg, S.; Amodei, D. Deep reinforcement learning from human preferences. arXiv 2017. arXiv:1706.03741. [Google Scholar] [CrossRef]
- Hernandez, J.; Frallicciardi, A.; Nadir, N.A.; Gothard, M.D.; Ahmed, R.A. Development of a Simulation Scenario Evaluation Tool (SSET): Modified Delphi study. BMJ Simul. Technol. Enhanc. Learn. 2020, 6, 344–350. [Google Scholar] [CrossRef]
- Koo, T.K.; Li, M.Y. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J. Chiropr. Med. 2016, 15, 155–163. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
- Bentley, S.; Iavicoli, L.; Boehm, L.; Agriantonis, G.; Dilos, B.; LaMonica, J.; Smith, C.; Wong, L.; Lopez, T.; Galer, A.; et al. A Simulated Mass Casualty Incident Triage Exercise: SimWars. Mededportal 2019, 15, 10823. [Google Scholar] [CrossRef] [PubMed]
- Ko, P.Y.; Escobar, S.L.; Wallus, H.J.; Knych, M.K.J.; Rossettie, A.S.; Dunham, C.A.; Scott, J.M. Mass Casualty Triage and Tagging Scenario in the Pre-hospital Setting Simulated Event. Mededportal 2012, 8, 9264. [Google Scholar] [CrossRef]
- Savage, T.; Nayak, A.; Gallo, R.; Rangan, E.; Chen, J.H. Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine. npj Digit. Med. 2024, 7, 20. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
- von Wedel, D.; Schmitt, R.A.; Thiele, M.; Leuner, R.; Shay, D.; Redaelli, S.; Schaefer, M.S. Affiliation Bias in Peer Review of Abstracts by a Large Language Model. JAMA 2024, 331, 252–253. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
- Goodman, K.E.; Yi, P.H.; Morgan, D.J. AI-Generated Clinical Summaries Require More Than Accuracy. JAMA 2024, 331, 637–638. [Google Scholar] [CrossRef] [PubMed]
- Schaffter, T.; Buist, D.S.M.; Lee, C.I.; Nikulin, Y.; Ribli, D.; Guan, Y.; Lotter, W.; Jie, Z.; Du, H.; Wang, S.; et al. Evaluation of Combined Artificial Intelligence and Radiologist Assessment to Interpret Screening Mammograms. JAMA Netw. Open 2020, 3, e200265. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
- Schulz, F.; Hultin, M.; Gyllencreutz, L. Self-Assessment of Learning Outcomes in Prehospital Disaster Response Skills: Instrument Development and Validation for Mass Casualty Incident Training. BMJ Open 2025, 15, e098284. [Google Scholar] [CrossRef]
- Regev, S.; Mitchnik, I.Y. Mastering Multicausality Trauma Care With the Trauma Non-Technical Skills Scale. J. Trauma Acute Care Surg. 2024, 97 (Suppl. 1), S60–S66. [Google Scholar] [CrossRef] [PubMed]
- Ferber, D.; El Nahhas, O.S.M.; Wölflein, G.; Wiest, I.C.; Clusmann, J.; Leßmann, M.E.; Foersch, S.; Lammert, J.; Tschochohei, M.; Jäger, D.; et al. Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology. Nat. Cancer 2025, 6, 1337–1349. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
- Debray, T.P.; Collins, G.S.; Riley, R.D.; Snell, K.I.; Van Calster, B.; Reitsma, J.B.; Moons, K.G. Transparent reporting of multivariable prediction models developed or validated using clustered data: TRIPOD-Cluster checklist. BMJ 2023, 380, e071018. [Google Scholar] [CrossRef]
- Debray, T.P.; Collins, G.S.; Riley, R.D.; Snell, K.I.; Van Calster, B.; Reitsma, J.B.; Moons, K.G. Transparent reporting of multivariable prediction models developed or validated using clustered data (TRIPOD-Cluster): Explanation andelaboration. BMJ 2023, 380, e071058. [Google Scholar] [CrossRef]
- Moons, K.G.; Altman, D.G.; Reitsma, J.B.; Ioannidis, J.P.; Macaskill, P.; Steyerberg, E.W.; Vickers, A.J.; Ransohoff, D.F.; Collins, G.S. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and elaboration. Ann. Intern. Med. 2015, 162, W1–W73. [Google Scholar] [CrossRef] [PubMed]



| Prior to Review | Post Review (Changes in Italics) | SSET Domain | How Review Improved the Score |
|---|---|---|---|
| Initial Assessment: This scenario is designed to simulate a traumatic event involving multiple gunshot wound victims. It focuses on rapid trauma assessment, management, and surgical consultation for gunshot wounds. | Initial Assessment: This scenario is designed to simulate a traumatic event involving multiple gunshot wound victims. It focuses on rapid trauma assessment, management, and surgical consultation for gunshot wounds. | Learning Objectives (Element 1) | No change |
| Background: Gunshot wounds can lead to significant morbidity and mortality depending on the location and trajectory. Immediate assessment, intervention, and surgical consultation are crucial. | Background: Gunshot wounds can lead to significant morbidity and mortality depending on the location and trajectory. Immediate assessment, intervention, and surgical consultation are crucial. | Learning Objectives (Element 1) | No change |
| Initial Presentation: Six patients are brought in by EMS after an active shooter incident. | Initial Presentation: Six patients are brought in by EMS after an active shooter incident. | Clinical Context (Element 2) | No change |
| Scenario Unfolding: The team must stabilize, diagnose, and manage each patient’s injuries. Some patients may need immediate surgical intervention, while others may require imaging and observation. | Scenario Unfolding: The team must stabilize, diagnose, and manage each patient’s injuries. Some patients may need immediate surgical intervention, while others may require imaging and observation. | Clinical Actions (Element 3) | No change |
| Actions: Immediate trauma assessment, imaging, and surgical consultation. | Actions: Immediate trauma assessment, imaging, and surgical consultation. | Clinical Actions (Element 3) | No change |
| Minute-by-Minute Review: 0–1 min: Arrival of patients. Briefing by EMS on the nature of injuries for a quick overview. 1–3 min: Rapid primary survey for all six patients. Immediate interventions such as cervical spine immobilization and supplemental oxygen provision. Two trauma bays should be prepped to receive the most critical patients. 3–5 min: Airway assessment and management for patients with altered mental status or respiratory distress. Initiation of two large-bore IVs for fluid and blood products as needed. 5–7 min: Rapid assessment of chest wounds. Needle decompression for suspected tension pneumothorax. Emergency chest tube insertion for patients with significant hemothorax or persistent air leak. 7–10 min: Secondary survey with a specific focus on gunshot wound trajectories. FAST (Focused Assessment with Sonography for Trauma) exams for patients with abdominal gunshot wounds. Direct pressure on thigh wound and assessment of distal pulses. 10–12 min: X-ray ordered for chest gunshot wound victims to ascertain bullet trajectory and organ involvement. CT scan ordered for abdominal gunshot wound victims after FAST exams. 12–15 min: Communication with surgical teams (general surgery for abdominal wounds, vascular surgery for thigh wound, and cardiothoracic surgery for chest wounds). 15–17 min: Blood products (PRBCs, FFP, Platelets) being administered as required. Continued monitoring and reassessment of all patients. 17–20 min: Results from imaging return, guiding surgical and interventional decisions. Decision made on immediate surgical interventions vs. observation vs. non-operative management. 20–25 min: Continued stabilization, preparation for surgery for those who need it. Wound dressing and pain management for patients not going immediately to the OR. 25–30 min: Handoff to surgical teams or ICU/ward teams for further management. Debriefing of the initial trauma team. | Minute-by-Minute Review: 0–1 min: Arrival of patients. Briefing by EMS on the nature of injuries for a quick overview. 1–3 min: Rapid primary survey for all six patients. Immediate interventions such as cervical spine immobilization and supplemental oxygen provision. Two trauma bays should be prepped to receive the most critical patients. 3–5 min: Airway assessment and management for patients with altered mental status or respiratory distress. Initiation of two large-bore IVs for fluid and blood products as needed. Mass Transfusion protocol (MTP) activation as needed for those with hemodynamic instability. 5–7 min: Rapid assessment of chest wounds. Needle decompression for suspected tension pneumothorax. Emergency chest tube insertion for patients with significant hemothorax or persistent air leak. 7–10 min: Secondary survey with a specific focus on gunshot wound trajectories. FAST (Focused Assessment with Sonography for Trauma) exams for patients with and without thoracic abdominal gunshot wounds. Tourniquet takedown as needed with assessment of thigh wounds. Direct pressure on thigh wound and assessment of distal pulses. 10–12 min: X-ray ordered for chest gunshot wound victims to ascertain bullet trajectory and organ involvement. CT scan ordered for abdominal gunshot wound victims after negative FAST exams. 12–15 min: Communication with surgical teams (general surgery for abdominal wounds, vascular surgery for thigh wound, and immediate operating room (OR) for positive FAST exam in the thoracic cavity with hemodynamic instability. Transport to CT for imaging if stable. 15–17 min: Blood products (Packed Red Blood Cells (PRBCs), Fresh Frozen Plasma (FFP), Platelets) being administered as required. Continued monitoring and reassessment of all patients. 17–20 min: Results from imaging return, guiding surgical and interventional decisions. Decision made on immediate surgical interventions vs. observation vs. non-operative management. 20–25 min: Continued stabilization, preparation for surgery for those who need it. Wound dressing and pain management for patients not going immediately to the OR. 25–30 min: Handoff to surgical teams or ICU/ward teams for further management. Debriefing of the initial trauma team. | Patient States (Element 4) | Improved Clinical Appropriate Case Management (Element 4, Item 13), Improved Effectiveness of Meeting Learning Objectives (Element 4, Item 15). |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Navarro, S.M.; Atkinson, A.G.; Donagay, E.; Jabaay, M.; Lund, S.; Park, M.S.; Loomis, E.A.; Zietlow, J.M.; Diem Vu, T.N.; Rivera, M.; et al. Can a Generative Artificial Intelligence Model Be Used to Create Mass Casualty Incident Simulation Scenarios? A Feasibility Study. Healthcare 2025, 13, 3184. https://doi.org/10.3390/healthcare13243184
Navarro SM, Atkinson AG, Donagay E, Jabaay M, Lund S, Park MS, Loomis EA, Zietlow JM, Diem Vu TN, Rivera M, et al. Can a Generative Artificial Intelligence Model Be Used to Create Mass Casualty Incident Simulation Scenarios? A Feasibility Study. Healthcare. 2025; 13(24):3184. https://doi.org/10.3390/healthcare13243184
Chicago/Turabian StyleNavarro, Sergio M., Angie G. Atkinson, Ege Donagay, Maxwell Jabaay, Sarah Lund, Myung S. Park, Erica A. Loomis, John M. Zietlow, T. N. Diem Vu, Mariela Rivera, and et al. 2025. "Can a Generative Artificial Intelligence Model Be Used to Create Mass Casualty Incident Simulation Scenarios? A Feasibility Study" Healthcare 13, no. 24: 3184. https://doi.org/10.3390/healthcare13243184
APA StyleNavarro, S. M., Atkinson, A. G., Donagay, E., Jabaay, M., Lund, S., Park, M. S., Loomis, E. A., Zietlow, J. M., Diem Vu, T. N., Rivera, M., & Stephens, D. (2025). Can a Generative Artificial Intelligence Model Be Used to Create Mass Casualty Incident Simulation Scenarios? A Feasibility Study. Healthcare, 13(24), 3184. https://doi.org/10.3390/healthcare13243184

