Participatory Modeling with Discrete-Event Simulation: A Hybrid Approach to Inform Policy Development to Reduce Emergency Department Wait Times
Abstract
:1. Introduction
- building trust in the modeling approach and seeking buy-in from the project leads for further development during the project initialization;
- leveraging existing conceptual mapping tools used in the health system for conceptual modeling;
- using co-production methods to build trust in the model and its outputs;
- identify and prioritize intervention scenarios using a multi-criteria framework.
2. Brief Review of Participatory Modeling Approaches
2.1. Motivation
2.2. Applied Fields and Purpose
2.3. Specific Participatory Modeling Approaches and Tools
2.3.1. Soft Operation Research Methodology
2.3.2. Group Model Building
2.3.3. PartiSim Framework for DES
2.3.4. Other Participatory Modeling Approaches Used in Case Studies
2.3.5. Participatory Modeling Tools
2.4. Hybrid Modeling and Simulation
3. Case Study Context
3.1. The Problem of Emergency Department Crowding and Wait Times
3.2. Whole-System ED Patient Flow Modeling
4. Materials and Methods
4.1. Overview
4.2. Stage 1: Project Initialization
Team Composition
4.3. Stage 2: Conceptual Modeling
4.4. Stage 3: Model Implementation
4.4.1. Turning Conceptual Modeling into DES Models
4.4.2. Capacity Building and Health System Modeling Workshop
4.5. Stage 4: Model Use and Policy Co-Development
4.5.1. The First Meeting
4.5.2. The Second Meeting
4.5.3. The Third and Fourth Meetings
4.6. A Multi-Criteria Framework for Identifying and Prioritizing Policy Options with Stakeholders
4.6.1. The Intake Process for Modeling Scenario
- The proposed intervention scenario is not within the scope of the project;
- Data are not available or the proposed intervention requires primary data collection that cannot be completed within the current timeline and budget cycle;
- There is a lack of evidence regarding the efficacy of the proposed intervention in reducing ED wait times or improving ED patient flow;
- If the intervention scenario is unsuitable for modeling, as determined by the modeling team, it will not be pursued;
- If the intervention is not feasible for implementation in the current local context.
4.6.2. The Scoring Matrix
5. Application of the Participatory Modeling Approach
5.1. Stage 1: Project Initialization
5.2. Stage 2: Conceptual Modeling
5.2.1. Problem Conceptualization
5.2.2. Core Patient Flow Processes Emerging from the Value Stream Mapping
5.2.3. Model Scope and Level of Details
5.3. Stage 3: Model Implementation
5.4. Stage 4: Model Use and Policy Co-Development
6. Discussion
7. Limitation and Future Work
8. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Concannon, T.W.; Fuster, M.; Saunders, T.; Patel, K.; Wong, J.B.; Leslie, L.K.; Lau, J. A Systematic Review of Stakeholder Engagement in Comparative Effectiveness and Patient-Centered Outcomes Research. J. Gen. Intern. Med. 2014, 29, 1692–1701. [Google Scholar] [CrossRef] [PubMed]
- Kotiadis, K.; Tako, A.A.; Vasilakis, C. A Participative and Facilitative Conceptual Modelling Framework for Discrete Event Simulation Studies in Healthcare. J. Oper. Res. Soc. 2014, 65, 197–213. [Google Scholar] [CrossRef] [Green Version]
- Freebairn, L.; Rychetnik, L.; Atkinson, J.-A.; Kelly, P.; McDonnell, G.; Roberts, N.; Whittall, C.; Redman, S. Knowledge Mobilisation for Policy Development: Implementing Systems Approaches through Participatory Dynamic Simulation Modelling. Health Res. Policy Syst. 2017, 15, 83. [Google Scholar] [CrossRef] [Green Version]
- Lamé, G.; Simmons, R.K. From Behavioural Simulation to Computer Models: How Simulation Can Be Used to Improve Healthcare Management and Policy. BMJ Simul. Technol. Enhanc. Learn. 2020, 6, 95. [Google Scholar] [CrossRef] [PubMed]
- Pitt, M.; Monks, T.; Crowe, S.; Vasilakis, C. Systems Modelling and Simulation in Health Service Design, Delivery and Decision Making. BMJ Qual. Saf. 2016, 25, 38–45. [Google Scholar] [CrossRef]
- Lich, K.H.; Tian, Y.; Beadles, C.A.; Williams, L.S.; Bravata, D.M.; Cheng, E.M.; Bosworth, H.B.; Homer, J.B.; Matchar, D.B. Strategic Planning to Reduce the Burden of Stroke among Veterans: Using Simulation Modeling to Inform Decision Making. Stroke 2014, 45, 2078–2084. [Google Scholar] [CrossRef] [Green Version]
- Atkinson, J.-A.; O’Donnell, E.; Wiggers, J.; McDonnell, G.; Mitchell, J.; Freebairn, L.; Indig, D.; Rychetnik, L. Dynamic Simulation Modelling of Policy Responses to Reduce Alcohol-Related Harms: Rationale and Procedure for a Participatory Approach. Public Health Res. Pract. 2017, 27, 2711707. [Google Scholar] [CrossRef] [Green Version]
- Hirsch, G.; Homer, J.; Evans, E.; Zielinski, A. A System Dynamics Model for Planning Cardiovascular Disease Interventions. Am. J. Public Health 2010, 100, 616–622. [Google Scholar] [CrossRef]
- Freebairn, L.; Atkinson, J.-A.; Osgood, N.D.; Kelly, P.M.; McDonnell, G.; Rychetnik, L. Turning Conceptual Systems Maps into Dynamic Simulation Models: An Australian Case Study for Diabetes in Pregnancy. PLoS ONE 2019, 14, e0218875. [Google Scholar] [CrossRef]
- Brailsford, S.C.; Harper, P.R.; Patel, B.; Pitt, M. An Analysis of the Academic Literature on Simulation and Modelling in Health Care. J. Simul. 2009, 3, 130–140. [Google Scholar] [CrossRef]
- Atkinson, J.A.; Wells, R.; Page, A.; Dominello, A.; Haines, M.; Wilson, A. Applications of System Dynamics Modelling to Support Health Policy. Public Health Res. Pract. 2015, 25, e2531531. [Google Scholar] [CrossRef] [Green Version]
- Mohiuddin, S.; Busby, J.; Savović, J.; Richards, A.; Northstone, K.; Hollingworth, W.; Donovan, J.L.; Vasilakis, C. Patient Flow within UK Emergency Departments: A Systematic Review of the Use of Computer Simulation Modelling Methods. BMJ Open 2017, 7, e015007. [Google Scholar] [CrossRef] [Green Version]
- Taylor, S.J.E.; Eldabi, T.; Riley, G.; Paul, R.J.; Pidd, M. Simulation Modelling Is 50! Do We Need a Reality Check? J. Oper. Res. Soc. 2009, 60, S69–S82. [Google Scholar] [CrossRef] [Green Version]
- Rutter, H.; Savona, N.; Glonti, K.; Bibby, J.; Cummins, S.; Finegood, D.T.; Greaves, F.; Harper, L.; Hawe, P.; Moore, L.; et al. The Need for a Complex Systems Model of Evidence for Public Health. Lancet 2017, 390, 2602–2604. [Google Scholar] [CrossRef] [Green Version]
- Jahangirian, M.; Borsci, S.; Shah, S.G.S.; Taylor, S.J.E. Causal Factors of Low Stakeholder Engagement: A Survey of Expert Opinions in the Context of Healthcare Simulation Projects. Simulation 2015, 91, 511–526. [Google Scholar] [CrossRef] [Green Version]
- Vázquez-Serrano, J.I.; Peimbert-García, R.E.; Cárdenas-Barrón, L.E. Discrete-Event Simulation Modeling in Healthcare: A Comprehensive Review. Int. J. Environ. Res. Public Health 2021, 18, 12262. [Google Scholar] [CrossRef] [PubMed]
- Sobolev, B.G.; Sanchez, V.; Vasilakis, C. Systematic Review of the Use of Computer Simulation Modeling of Patient Flow in Surgical Care. J. Med. Syst. 2011, 35, 1–16. [Google Scholar] [CrossRef]
- Brownson, R.C.; Royer, C.; Ewing, R.; McBride, T.D. Researchers and Policymakers: Travelers in Parallel Universes. Am. J. Prev. Med. 2006, 30, 164–172. [Google Scholar] [CrossRef] [PubMed]
- Harper, A.; Mustafee, N.; Yearworth, M. The Issue of Trust and Implementation of Results in Healthcare Modeling and Simulation Studies. In Proceedings of the 2022 Winter Simulation Conference (WSC), Singapore, 11–14 December 2022; pp. 1104–1115. [Google Scholar]
- Harper, A.; Mustafee, N.; Yearworth, M. Facets of Trust in Simulation Studies. Eur. J. Oper. Res. 2021, 289, 197–213. [Google Scholar] [CrossRef]
- Tako, A.A.; Kotiadis, K. PartiSim: A Multi-Methodology Framework to Support Facilitated Simulation Modelling in Healthcare. Eur. J. Oper. Res. 2015, 244, 555–564. [Google Scholar] [CrossRef] [Green Version]
- Voinov, A.; Jenni, K.; Gray, S.; Kolagani, N.; Glynn, P.D.; Bommel, P.; Prell, C.; Zellner, M.; Paolisso, M.; Jordan, R.; et al. Tools and Methods in Participatory Modeling: Selecting the Right Tool for the Job. Environ. Model. Softw. 2018, 109, 232–255. [Google Scholar] [CrossRef] [Green Version]
- O’Donnell, E.; Atkinson, J.-A.; Freebairn, L.; Rychetnik, L. Participatory Simulation Modelling to Inform Public Health Policy and Practice: Rethinking the Evidence Hierarchies. J. Public Health Policy 2017, 38, 203–215. [Google Scholar] [CrossRef] [PubMed]
- Tian, Y.; Osgood, N.D.; Stempien, J.; Onaemo, V.; Danyliw, A.; Fast, G.; Osman, B.A.; Reynolds, J.; Basran, J. The Impact of Alternate Level of Care on Access Block and Operational Strategies to Reduce Emergency Wait Times: A Multi-Center Simulation Study. Can. J. Emerg. Med. 2023, 25, 608–616. [Google Scholar] [CrossRef] [PubMed]
- Jun, J.B.; Jacobson, S.H.; Swisher, J.R. Application of Discrete-Event Simulation in Health Care Clinics: A Survey. J. Oper. Res. Soc. 1999, 50, 109–123. [Google Scholar] [CrossRef]
- Fone, D.; Hollinghurst, S.; Temple, M.; Round, A.; Lester, N.; Weightman, A.; Roberts, K.; Coyle, E.; Bevan, G.; Palmer, S. Systematic Review of the Use and Value of Computer Simulation Modelling in Population Health and Health Care Delivery. J. Public Health Med. 2003, 25, 325–335. [Google Scholar] [CrossRef]
- Morris, Z.S.; Boyle, A.; Beniuk, K.; Robinson, S. Emergency Department Crowding: Towards an Agenda for Evidence-Based Intervention. Emerg. Med. J. 2012, 29, 460–466. [Google Scholar] [CrossRef]
- Günal, M.M.; Pidd, M. Discrete Event Simulation for Performance Modelling in Health Care: A Review of the Literature. J. Simul. 2010, 4, 42–51. [Google Scholar] [CrossRef]
- Laker, L.F.; Torabi, E.; France, D.J.; Froehle, C.M.; Goldlust, E.J.; Hoot, N.R.; Kasaie, P.; Lyons, M.S.; Barg-Walkow, L.H.; Ward, M.J.; et al. Understanding Emergency Care Delivery Through Computer Simulation Modeling. Acad. Emerg. Med. 2018, 25, 116–127. [Google Scholar] [CrossRef] [Green Version]
- Salmon, A.; Rachuba, S.; Briscoe, S.; Pitt, M. A Structured Literature Review of Simulation Modelling Applied to Emergency Departments: Current Patterns and Emerging Trends. Oper. Res. Health Care 2018, 19, 1–13. [Google Scholar] [CrossRef]
- Hare, M. Forms of Participatory Modelling and Its Potential for Widespread Adoption in the Water Sector. Environ. Policy Gov. 2011, 21, 386–402. [Google Scholar] [CrossRef]
- Boaz, A.; Borst, R.; Kok, M.; O’Shea, A. How Far Does an Emphasis on Stakeholder Engagement and Co-Production in Research Present a Threat to Academic Identity and Autonomy? A Prospective Study across Five European Countries. Res. Eval. 2021, 30, 361–369. [Google Scholar] [CrossRef]
- Tako, A.A.; Kotiadis, K. Participative Simulation (Partisim): A Facilitated Simulation Approach for Stakeholder Engagement. In Proceedings of the 2018 Winter Simulation Conference (WSC), Göteborg, Sweden, 9–12 December 2018; pp. 192–206. [Google Scholar]
- Voinov, A.; Kolagani, N.; McCall, M.K.; Glynn, P.D.; Kragt, M.E.; Ostermann, F.O.; Pierce, S.A.; Ramu, P. Modelling with Stakeholders—Next Generation. Environ. Model. Softw. 2016, 77, 196–220. [Google Scholar] [CrossRef]
- Ulibarri, N. Collaborative Model Development Increases Trust in and Use of Scientific Information in Environmental Decision-Making. Environ. Sci. Policy 2018, 82, 136–142. [Google Scholar] [CrossRef] [Green Version]
- Stave, K. Participatory System Dynamics Modeling for Sustainable Environmental Management: Observations from Four Cases. Sustain. Sci. Pract. Policy 2010, 2, 2762–2784. [Google Scholar] [CrossRef] [Green Version]
- Beall, A.M.; Ford, A. Reports from the Field: Assessing the Art and Science of Participatory Environmental Modeling. Int. J. Inf. Syst. Soc. Chang. 2010, 1, 72–89. [Google Scholar] [CrossRef] [Green Version]
- Richardson, G.P.; Andersen, D.F. Teamwork in Group Model Building. Syst. Dyn. Rev. 1995, 11, 113–137. [Google Scholar] [CrossRef] [Green Version]
- Hovmand, P.S. Community Based System Dynamics; Springer: New York, NY, USA, 2014. [Google Scholar]
- Adams, S.; Rhodes, T.; Lancaster, K. New Directions for Participatory Modelling in Health: Redistributing Expertise in Relation to Localised Matters of Concern. Glob. Public Health 2021, 17, 1827–1841. [Google Scholar] [CrossRef]
- Seidl, R. A Functional-Dynamic Reflection on Participatory Processes in Modeling Projects. Ambio 2015, 44, 750–765. [Google Scholar] [CrossRef] [Green Version]
- Freebairn, L.; Atkinson, J.-A.; Kelly, P.M.; McDonnell, G.; Rychetnik, L. Decision Makers’ Experience of Participatory Dynamic Simulation Modelling: Methods for Public Health Policy. BMC Med. Inform. Decis. Mak. 2018, 18, 131. [Google Scholar] [CrossRef]
- Freebairn, L.; Atkinson, J.-A.; Qin, Y.; Nolan, C.J.; Kent, A.L.; Kelly, P.M.; Penza, L.; Prodan, A.; Safarishahrbijari, A.; Qian, W.; et al. “Turning the Tide” on Hyperglycemia in Pregnancy: Insights from Multiscale Dynamic Simulation Modeling. BMJ Open Diabetes Res. Care 2020, 8, e000975. [Google Scholar] [CrossRef]
- Frerichs, L.; Lich, K.H.; Dave, G.; Corbie-Smith, G. Integrating Systems Science and Community-Based Participatory Research to Achieve Health Equity. Am. J. Public Health 2016, 106, 215–222. [Google Scholar] [CrossRef] [PubMed]
- Gerritsen, S.; Harré, S.; Rees, D.; Renker-Darby, A.; Bartos, A.E.; Waterlander, W.E.; Swinburn, B. Community Group Model Building as a Method for Engaging Participants and Mobilising Action in Public Health. Int. J. Environ. Res. Public Health 2020, 17, 3457. [Google Scholar] [CrossRef]
- Augustsson, H.; Churruca, K.; Braithwaite, J. Re-Energising the Way We Manage Change in Healthcare: The Case for Soft Systems Methodology and Its Application to Evidence-Based Practice. BMC Health Serv. Res. 2019, 19, 666. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Hovmand, P.S. Group Model Building and Community-Based System Dynamics Process. In Community Based System Dynamics; Springer: New York, NY, USA, 2014. [Google Scholar]
- Reed, M.S. Stakeholder Participation for Environmental Management: A Literature Review. Biol. Conserv. 2008, 141, 2417–2431. [Google Scholar] [CrossRef]
- Viswanathan, M.; Ammerman, A.; Eng, E.; Garlehner, G.; Lohr, K.N.; Griffith, D.; Rhodes, S.; Samuel-Hodge, C.; Maty, S.; Lux, L.; et al. Community-Based Participatory Research: Assessing the Evidence. Evid. Rep. Summ. 2004, 18, 1–8. [Google Scholar]
- Checkland, P.; Poulter, J. Soft Systems Methodology. In Systems Approaches to Making Change: A Practical Guide; Reynolds, M., Holwell, S., Eds.; Springer: London, UK, 2020; pp. 201–253. ISBN 9781447174721. [Google Scholar]
- Water, H.; van de Schinkel, M.; Rozier, R. Fields of Application of SSM: A Categorization of Publications. J. Oper. Res. Soc. 2007, 58, 271–287. [Google Scholar] [CrossRef]
- Jones, W.; Kotiadis, K.; O’Hanley, J.R. Maximising Stakeholder Learning by Looping Again through the Simulation Life-Cycle: A Case Study in Public Transport. J. Oper. Res. Soc. 2021, 73, 2640–2659. [Google Scholar] [CrossRef]
- Kotiadis, K.; Mingers, J. Combining Problem Structuring Methods with Simulation: The Philosophical and Practical Challenges. In Discrete-Event Simulation and System Dynamics for Management Decision Making; John Wiley & Sons Ltd.: Chichester, UK, 2014; pp. 52–75. ISBN 9781118762745. [Google Scholar]
- Andersen, D.F.; Vennix, J.A.M.; Richardson, G.P.; Rouwette, E.A.J.A. Group Model Building: Problem Structuring, Policy Simulation and Decision Support. J. Oper. Res. Soc. 2007, 58, 691–694. [Google Scholar] [CrossRef] [Green Version]
- Vennix, J.A.M. Group Model-Building: Tackling Messy Problems. Syst. Dyn. Rev. 1999, 15, 379–401. [Google Scholar] [CrossRef]
- Esensoy, A.V.; Carter, M.W. Health System Modelling for Policy Development and Evaluation: Using Qualitative Methods to Capture the Whole-System Perspective. Oper. Res. Health Care 2015, 4, 15–26. [Google Scholar] [CrossRef]
- Tako, A.A.; Kotiadis, K. Facilitated Conceptual Modelling: Practical Issues and Reflections. In Proceedings of the 2012 Winter Simulation Conference (WSC), Berlin, Germany, 9–12 December 2012; pp. 1–12. [Google Scholar]
- Tako, A.A.; Vasilakis, C.; Kotiadis, K. A Participative Modelling Framework for Developing Conceptual Models in Healthcare Simulation Studies. In Proceedings of the 2010 Winter Simulation Conference, Baltimore, MD, USA, 5–8 December 2010; pp. 500–512. [Google Scholar]
- Kotiadis, K.; Tako, A.A. Facilitated Post-Model Coding in Discrete Event Simulation (DES): A Case Study in Healthcare. Eur. J. Oper. Res. 2018, 266, 1120–1133. [Google Scholar] [CrossRef]
- Tako, A.A.; Kotiadis, K.; Vasilakis, C.; Miras, A.; le Roux, C.W. Improving Patient Waiting Times: A Simulation Study of an Obesity Care Service. BMJ Qual. Saf. 2014, 23, 373–381. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Brailsford, S.C.; Eldabi, T.; Kunc, M.; Mustafee, N.; Osorio, A.F. Hybrid Simulation Modelling in Operational Research: A State-of-the-Art Review. Eur. J. Oper. Res. 2019, 278, 721–737. [Google Scholar] [CrossRef]
- Powell, J.H.; Mustafee, N. Widening Requirements Capture with Soft Methods: An Investigation of Hybrid M&S Studies in Health Care. J. Oper. Res. Soc. Abingdon 2017, 68, 1211–1222. [Google Scholar] [CrossRef]
- Tolk, A.; Harper, A.; Mustafee, N. Hybrid Models as Transdisciplinary Research Enablers. Eur. J. Oper. Res. 2021, 291, 1075–1090. [Google Scholar] [CrossRef]
- Gao, A.; Osgood, N.D.; An, W.; Dyck, R.F. A Tripartite Hybrid Model Architecture for Investigating Health and Cost Impacts and Intervention Tradeoffs for Diabetic End-Stage Renal Disease. In Proceedings of the Winter Simulation Conference 2014, Savannah, GA, USA, 7–10 December 2014; pp. 1676–1687. [Google Scholar]
- Qin, Y.; Freebairn, L.; Atkinson, J.-A.; Qian, W.; Safarishahrbijari, A.; Osgood, N.D. Multi-Scale Simulation Modeling for Prevention and Public Health Management of Diabetes in Pregnancy and Sequelae. In Proceedings of the Social, Cultural, and Behavioral Modeling; Springer International Publishing: Berlin/Heidelberg, Germany, 2019; pp. 256–265. [Google Scholar]
- Zhu, H.; Liu, S.; Li, X.; Zhang, W.; Osgood, N.; Jia, P. Using a Hybrid Simulation Model to Assess the Impacts of Combined COVID-19 Containment Measures in a High-Speed Train Station. J. Simul. 2023, 1–25. [Google Scholar] [CrossRef]
- Gao, A.; Osgood, N.D.; Jiang, Y.; Dyck, R.F. Projecting Prevalence, Costs and Evaluating Simulated Interventions for Diabetic End Stage Renal Disease in a Canadian Population of Aboriginal and Non-Aboriginal People: An Agent Based Approach. BMC Nephrol. 2017, 18, 283. [Google Scholar] [CrossRef] [Green Version]
- Flynn, T.; Tian, Y.; Masnick, K.; McDonnell, G.; Huynh, E.; Mair, A.; Osgood, N. Discrete Choice, Agent Based and System Dynamics Simulation of Health Profession Career Paths. In Proceedings of the Winter Simulation Conference 2014, Savannah, GA, USA, 7–10 December 2014; pp. 1700–1711. [Google Scholar]
- Vickers, D.M.; Osgood, N.D. A Unified Framework of Immunological and Epidemiological Dynamics for the Spread of Viral Infections in a Simple Network-Based Population. Theor. Biol. Med. Model. 2007, 4, 49. [Google Scholar] [CrossRef] [Green Version]
- Mustafee, N.; Harper, A.; Fakhimi, M. From Conceptualization of Hybrid Modelling & Simulation to Empirical Studies in Hybrid Modelling. In Proceedings of the 2022 Winter Simulation Conference (WSC), Singapore, 11–14 December 2022; pp. 1199–1210. [Google Scholar]
- Mustafee, N.; Harper, A.; Onggo, B.S. Hybrid Modelling and Simulation (M&S): Driving Innovation in the Theory and Practice of M&S. In Proceedings of the 2020 Winter Simulation Conference (WSC), Orlando, FL, USA, 14–18 December 2020. [Google Scholar]
- Mustafee, N.; Powell, J.H. From Hybrid Simulation to Hybrid Systems Modelling. In Proceedings of the 2018 Winter Simulation Conference (WSC), Göteborg, Sweden, 9–12 December 2018. [Google Scholar]
- McDonald, G.W.; Bradford, L.; Neapetung, M.; Osgood, N.D.; Strickert, G.; Waldner, C.L.; Belcher, K.; McLeod, L.; Bharadwaj, L. Case Study of Collaborative Modeling in an Indigenous Community. Water 2022, 14, 2601. [Google Scholar] [CrossRef]
- Jiang, L.; Li, X.; Wang, M.C.; Osgood, N.; Whaley, S.E.; Crespi, C.M. Estimating the Population Impact of Hypothetical Breastfeeding Interventions in a Low-Income Population in Los Angeles County: An Agent-Based Model. PLoS ONE 2020, 15, e0231134. [Google Scholar] [CrossRef] [Green Version]
- Tian, Y.; Hassmiller Lich, K.; Osgood, N.D.; Eom, K.; Matchar, D.B. Linked Sensitivity Analysis, Calibration, and Uncertainty Analysis Using a System Dynamics Model for Stroke Comparative Effectiveness Research. Med. Decis. Mak. 2016, 36, 1043–1057. [Google Scholar] [CrossRef] [PubMed]
- Kreuger, L.K.; Qian, W.; Osgood, N.; Choi, K. Agile Design Meets Hybrid Models: Using Modularity to Enhance Hybrid Model Design and Use. In Proceedings of the 2016 Winter Simulation Conference (WSC), Washington, DC, USA, 11–14 December 2016; pp. 1428–1438. [Google Scholar]
- Morley, C.; Unwin, M.; Peterson, G.M.; Stankovich, J.; Kinsman, L. Emergency Department Crowding: A Systematic Review of Causes, Consequences and Solutions. PLoS ONE 2018, 13, e0203316. [Google Scholar] [CrossRef] [PubMed]
- Pines, J.M.; Griffey, R.T. What We Have Learned from a Decade of ED Crowding Research. Acad. Emerg. Med. 2015, 22, 985–987. [Google Scholar] [CrossRef] [Green Version]
- Guttmann, A.; Schull, M.J.; Vermeulen, M.J.; Stukel, T.A. Association between Waiting Times and Short Term Mortality and Hospital Admission after Departure from Emergency Department: Population Based Cohort Study from Ontario, Canada. BMJ 2011, 342, d2983. [Google Scholar] [CrossRef] [Green Version]
- Singer, A.J.; Thode, H.C., Jr.; Viccellio, P.; Pines, J.M. The Association between Length of Emergency Department Boarding and Mortality. Acad. Emerg. Med. 2011, 18, 1324–1329. [Google Scholar] [CrossRef]
- Tekwani, K.L.; Kerem, Y.; Mistry, C.D.; Sayger, B.M.; Kulstad, E.B. Emergency Department Crowding Is Associated with Reduced Satisfaction Scores in Patients Discharged from the Emergency Department. West. J. Emerg. Med. 2013, 14, 11–15. [Google Scholar] [CrossRef]
- Moskop, J.C.; Sklar, D.P.; Geiderman, J.M.; Schears, R.M.; Bookman, K.J. Emergency Department Crowding, Part 1—Concept, Causes, and Moral Consequences. Ann. Emerg. Med. 2009, 53, 605–611. [Google Scholar] [CrossRef]
- Vogel, L. Canadians Still Waiting for Timely Access to Care. Can. Med. Assoc. J. 2017, 189, E375–E376. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Pines, J.M.; Hilton, J.A.; Weber, E.J.; Alkemade, A.J.; Al Shabanah, H.; Anderson, P.D.; Bernhard, M.; Bertini, A.; Gries, A.; Ferrandiz, S.; et al. International Perspectives on Emergency Department Crowding. Acad. Emerg. Med. 2011, 18, 1358–1370. [Google Scholar] [CrossRef]
- Bond, K.; Ospina, M.B.; Blitz, S.; Afilalo, M.; Campbell, S.G.; Bullard, M.; Innes, G.; Holroyd, B.; Curry, G.; Schull, M.; et al. Frequency, Determinants and Impact of Overcrowding in Emergency Departments in Canada: A National Survey. Healthc. Q. 2007, 10, 32–40. [Google Scholar] [CrossRef]
- Saskatchewan ED Waits and Patient Flow Initiative Emergency Department Waits and Patient Flow Initiative Fall 2015 Newsletter. Available online: https://www.hqc.sk.ca/Portals/0/documents/ReducingERWaitsImprovingFlow/Emerg-Dept-Waits-Update-Fall-2015.pdf (accessed on 16 July 2017).
- Saghafian, S.; Austin, G.; Traub, S.J. Operations Research/Management Contributions to Emergency Department Patient Flow Optimization: Review and Research Prospects. IIE Trans. Healthc. Syst. Eng. 2015, 5, 101–123. [Google Scholar] [CrossRef]
- Paul, S.A.; Reddy, M.C.; DeFlitch, C.J. A Systematic Review of Simulation Studies Investigating Emergency Department Overcrowding. Simulation 2010, 86, 559–571. [Google Scholar] [CrossRef]
- Zhang, X. Application of Discrete Event Simulation in Health Care: A Systematic Review. BMC Health Serv. Res. 2018, 18, 687. [Google Scholar] [CrossRef]
- Rashwan, W.; Abo-Hamad, W.; Arisha, A. A System Dynamics View of the Acute Bed Blockage Problem in the Irish Healthcare System. Eur. J. Oper. Res. 2015, 247, 276–293. [Google Scholar] [CrossRef] [Green Version]
- Vanderby, S.; Carter, M.W. An Evaluation of the Applicability of System Dynamics to Patient Flow Modelling. J. Oper. Res. Soc. 2010, 61, 1572–1581. [Google Scholar] [CrossRef]
- Shi, P.; Chou, M.C.; Dai, J.G.; Ding, D.; Sim, J. Models and Insights for Hospital Inpatient Operations: Time-Dependent ED Boarding Time. Manag. Sci. 2016, 62, 1–28. [Google Scholar] [CrossRef] [Green Version]
- Esensoy, A.V. Whole-System Patient Flow Modelling for Strategic Planning in Healthcare. Ph.D. Thesis, University of Toronto, Toronto, ON, Canada, 2016. [Google Scholar]
- Rouwette, E.A.J.A.; Vennix, J.A.M.; van Mullekom, T. Group Model Building Effectiveness: A Review of Assessment Studies. Syst. Dyn. Rev. 2002, 18, 5–45. [Google Scholar] [CrossRef]
- Robinson, S. Conceptual Modelling for Simulation Part II: A Framework for Conceptual Modelling. J. Oper. Res. Soc. Abingdon 2008, 59, 291–304. [Google Scholar] [CrossRef]
- Institute for Healthcare Improvement. Quality Improvement Essentials Toolkit; Institute for Healthcare Improvement: Boston, MA, USA, 2017. [Google Scholar]
- Feld, W.M. Lean Manufacturing: Tools, Techniques, and How to Use Them; CRC Press: Boca Raton, FL, USA, 2000; ISBN 9781420025538. [Google Scholar]
- Hines, P.; Rich, N. The Seven Value Stream Mapping Tools. Int. J. Oper. Prod. Manag. 1997, 17, 46–64. [Google Scholar] [CrossRef] [Green Version]
- Kinsman, L.; Rotter, T.; Stevenson, K.; Bath, B.; Goodridge, D.; Harrison, L.; Dobson, R.; Sari, N.; Jeffery, C.; Bourassa, C.; et al. “The Largest Lean Transformation in the World”: The Implementation and Evaluation of Lean in Saskatchewan Healthcare. Healthc. Q. 2014, 17, 29–32. [Google Scholar] [CrossRef]
- Saskatchewan Health Quality Council Health Care Leaders to Participate in Innovative Computer Modelling Boot Camp. Available online: https://www.saskhealthquality.ca/about-us/news/health-care-leaders-to-participate-in-innovative-computer-modelling-boot-camp/ (accessed on 30 April 2023).
- Straus, S.E.; Tetroe, J.; Graham, I.D. Knowledge Translation in Health Care: Moving from Evidence to Practice, 2nd ed.; Wiley: Hoboken, NJ, USA, 2013; ISBN 9781118413548. [Google Scholar]
- Komal, B.; Janjua, U.I.; Anwar, F.; Madni, T.M.; Cheema, M.F.; Malik, M.N.; Shahid, A.R. The Impact of Scope Creep on Project Success: An Empirical Investigation. IEEE Access 2020, 8, 125755–125775. [Google Scholar] [CrossRef]
- Asplin, B.R.; Magid, D.J.; Rhodes, K.V.; Solberg, L.I.; Lurie, N.; Camargo, C.A., Jr. A Conceptual Model of Emergency Department Crowding. Ann. Emerg. Med. 2003, 42, 173–180. [Google Scholar] [CrossRef] [PubMed]
- Hoot, N.R.; Aronsky, D. Systematic Review of Emergency Department Crowding: Causes, Effects, and Solutions. Ann. Emerg. Med. 2008, 52, 126–136. [Google Scholar] [CrossRef] [PubMed]
- Doupe, M.B.; Chateau, D.; Chochinov, A.; Weber, E.; Enns, J.E.; Derksen, S.; Sarkar, J.; Schull, M.; Lobato de Faria, R.; Katz, A.; et al. Comparing the Effect of Throughput and Output Factors on Emergency Department Crowding: A Retrospective Observational Cohort Study. Ann. Emerg. Med. 2018, 72, 410–419. [Google Scholar] [CrossRef] [PubMed]
- Asaro, P.V.; Lewis, L.M.; Boxerman, S.B. The Impact of Input and Output Factors on Emergency Department Throughput. Acad. Emerg. Med. 2007, 14, 235–242. [Google Scholar] [CrossRef] [PubMed]
- Moineddin, R.; Meaney, C.; Agha, M.; Zagorski, B.; Glazier, R.H. Modeling Factors Influencing the Demand for Emergency Department Services in Ontario: A Comparison of Methods. BMC Emerg. Med. 2011, 11, 13. [Google Scholar] [CrossRef] [Green Version]
- Forster, A.J.; Stiell, I.; Wells, G.; Lee, A.J.; van Walraven, C. The Effect of Hospital Occupancy on Emergency Department Length of Stay and Patient Disposition. Acad. Emerg. Med. 2003, 10, 127–133. [Google Scholar] [CrossRef]
- Yarmohammadian, M.H.; Rezaei, F.; Haghshenas, A.; Tavakoli, N. Overcrowding in Emergency Departments: A Review of Strategies to Decrease Future Challenges. J. Res. Med. Sci. 2017, 22, 23. [Google Scholar] [CrossRef]
- Moskop, J.C.; Sklar, D.P.; Geiderman, J.M.; Schears, R.M.; Bookman, K.J. Emergency Department Crowding, Part 2—Barriers to Reform and Strategies to Overcome Them. Ann. Emerg. Med. 2009, 53, 612–617. [Google Scholar] [CrossRef]
- Mason, S.; Mountain, G.; Turner, J.; Arain, M.; Revue, E.; Weber, E.J. Innovations to Reduce Demand and Crowding in Emergency Care; a Review Study. Scand. J. Trauma Resusc. Emerg. Med. 2014, 22, 55. [Google Scholar] [CrossRef] [Green Version]
- Saskatchewan Health Quality Council Connected Care: A Summary of Learnings from the Emergency Department Waits and Patient Flow Initiative. Available online: https://www.saskhealthquality.ca/wp-content/uploads/2021/06/Connected-Care-Summary-of-Learnings-from-ED-Waits-and-Patient-Flow-Initiative.pdf (accessed on 26 July 2021).
- Khan, S.; Sanmartin, C. LHAD research team Hospitalizations for Ambulatory Care Sensitive Conditions (ACSC): The Factors That Matter; Statistics: Ottawa, ON, Canada, 2011. [Google Scholar]
- Canadian Institute for Health Information. Continuity of Care with Family Medicine Physicians: Why It Matters; CIHI: Ottawa, ON, Canada, 2015. [Google Scholar]
- Jones, W.; Kotiadis, K.; O’Hanley, J. Engaging Stakeholders to Extend the Lifecycle of Hybrid Simulation Models. In Proceedings of the 2019 Winter Simulation Conference (WSC), National Harbor, MD, USA, 8–11 December 2019; pp. 1304–1315. [Google Scholar]
- Powell, J.; Mustafee, N. Soft OR Approaches in Problem Formulation Stage of a Hybrid M&S Study. In Proceedings of the Winter Simulation Conference 2014, Savannah, GA, USA, 7–10 December 2014; pp. 1664–1675. [Google Scholar]
- Bal, A.; Ceylan, C.; Taçoğlu, C. Using Value Stream Mapping and Discrete Event Simulation to Improve Efficiency of Emergency Departments. Int. J. Healthc. Manag. 2017, 10, 196–206. [Google Scholar] [CrossRef]
- Robinson, S.; Radnor, Z.J.; Burgess, N.; Worthington, C. SimLean: Utilising Simulation in the Implementation of Lean in Healthcare. Eur. J. Oper. Res. 2012, 219, 188–197. [Google Scholar] [CrossRef] [Green Version]
- Robinson, S.; Worthington, C.; Burgess, N.; Radnor, Z.J. Facilitated Modelling with Discrete-Event Simulation: Reality or Myth? Eur. J. Oper. Res. 2014, 234, 231–240. [Google Scholar] [CrossRef] [Green Version]
- Diallo, S.Y.; Gore, R.; Lynch, C.J.; Padilla, J.J. Formal Methods, Statistical Debugging and Exploratory Analysis in Support of System Development: Towards a Verification and Validation Calculator Tool. Int. J. Model. Simul. Sci. Comput. 2016, 7, 1641001. [Google Scholar] [CrossRef] [Green Version]
Component | Criteria | Description | Scoring (in the Range of 0 to 5) |
---|---|---|---|
Data | Regional Data Availability | Degree to which regional data is collected or available for understanding the current state or the targeted cohort for the intervention scenario | 0 = no data; 1 = limited data with poor quality; 2 = partial data with poor quality; 3 = partial data with good quality; 4 = almost complete data; 5 = we have everything we need |
Administrative Data Availability | Degree to which data are readily available in administrative databases for modeling or understanding the current state or target cohort. | ||
Evidence | Literature Support for Modeling | Degree to which evidence (e.g., meta-analysis and systematic reviews) provides relevant outcome metrics for modeling | 0 = very low; 1 = low; 2 = low to moderate; 3 = moderate; 4 = moderate to high; 5 = high |
Modeling | Modeling Effort | The amount of effort and time it will take for the modeler to incorporate and simulate the scenario using the DES model. | 0 = extreme effort; 1 = significant effort; 2 = moderate effort; 3 = moderate to low effort; 4 = low effort; 5 = almost no effort at all |
Stakeholders Inputs | Organizational Readiness | Degree to which the organization is ready and committed to implementing the idea | 0 = not ready at all; 1 = not ready; 2 = neural or uncertain; 3 = moderately ready; 4 = mostly ready; 5 = ready to go immediately |
Regional Priority | Degree to which the health region agrees that this will address their ED wait times or patient flow issues | 0 = not at all; 1 = unlikely; 2 = somewhat unlikely; 3 = neutral or uncertain; 4 = somewhat likely; 5 = very likely | |
Provincial Priority | Degree to which the provincial stakeholders and ED waits initiative agrees that this will address ED wait times or patient flow issues | ||
Length of Time to Impact Drivers | The length of time it would take to see an effect on ED waits, volume, ED LOS or acute LOS. | 0 = >7 years; 1 = 5–7 years 2 = 3–5 years; 3 = 1–3 years 4 = within 1 year; 5 = immediate | |
Length of Time to Get Service Ready | The length of time required to alter or design service. |
Mean Time Waiting for Physician Initial Assessment, Hours | Mean Time Waiting for Inpatient Bed, Hours | |||||
---|---|---|---|---|---|---|
Site | Simulated | Actual | Δ | Simulated | Actual | Δ |
Site 1 | 1.99 | 2.04 | −0.05 | 10.17 | 10.18 | −0.01 |
Site 2 | 1.18 | 1.27 | −0.09 | 7.81 | 7.88 | −0.07 |
Site 3 | 1.26 | 1.26 | 0.00 | 11.13 | 11.01 | 0.12 |
Site 4 | 2.29 | 2.35 | −0.06 | 4.65 | 4.61 | 0.04 |
Site 5 | 1.36 | 1.43 | −0.07 | 1.38 | 1.29 | 0.09 |
Site 6 | 0.95 | 0.94 | 0.01 | 0.68 | 0.69 | −0.01 |
Region | Scenario Name | Organizational Readiness | Regional Data Availability | Administrative Data Availability | Literature Support for Modeling | Regional Priority | Provincial Priority | Modeling Effort | Length of Time to Impact Drivers | Length of Time to Get Service Ready | Total Score |
---|---|---|---|---|---|---|---|---|---|---|---|
A | Attachment to primary care provider | 3 | 3 | 5 | 5 | 5 | 4 | 4 | 3 | 3 | 35 |
Reducing admissions for ACSCs | 3 | 3 | 4 | 3 | 5 | 4 | 4 | 3 | 3 | 32 | |
Reducing ED visits for family practice sensitive conditions | 3 | 3 | 4 | 1 | 5 | 4 | 4 | 3 | 3 | 30 | |
Alternate level of care reduction strategy | 3 | 2 | 1 | 3 | 5 | 5 | 2 | 3 | 3 | 27 | |
High-quality care transitions (discharge planning and coordination) | 3 | 3 | 4 | 4 | 4 | 4 | 1 | 4 | 3 | 30 | |
Hospital at home (early supported discharge) for surgical and neuro patients | 4 | 4 | 4 | 5 | 4 | 4 | 5 | 3 | 3 | 36 | |
B | Expansion of community nurse practitioner services for COPD patients | 5 | 5 | 3 | 2 | 5 | 4 | 3 | 5 | 4 | 36 |
Hospitalist model for medical and surgical units | 5 | 3 | 3 | 3 | 4 | 4 | 3 | 3 | 2 | 30 | |
Additional ED physician coverage | 5 | 5 | 2 | 1 | 3 | 2 | 4 | 4 | 4 | 30 | |
C | Reducing ED visits for family practice sensitive conditions | 4 | 4 | 4 | 1 | 4 | 3 | 4 | 3 | 3 | 30 |
ALC reduction strategy | - | 1 | 3 | 3 | - | 5 | 2 | 3 | 3 | 20 | |
COPD clinical pathway | 4 | 2 | 2 | 3 | 4 | 3 | 2 | 3 | 3 | 26 | |
High-quality care transitions (discharge planning and coordination) | - | 3 | 4 | 4 | 3 | 4 | 1 | 3 | 4 | 26 | |
Hospital at home (early supported discharge) for surgical and neuro patients | 4 | 4 | 4 | 4 | 4 | 3 | 5 | 3 | 5 | 36 |
Scenario | Simulated Mean Time Waiting for Physician Initial Assessment, Hours | Simulated Mean Time Waiting for Inpatient Bed, Hours | ||||
---|---|---|---|---|---|---|
Pre | Post | Δ | Pre | Post | Δ | |
Reduce ACSC-related hospitalizations by 10% (input) | ||||||
Site 1 | 1.99 | 1.95 | −0.04 | 10.17 | 9.38 | −0.79 |
Site 2 | 1.18 | 1.16 | −0.02 | 7.81 | 7.43 | −0.38 |
Site 3 | 1.26 | 1.22 | −0.04 | 11.13 | 10.62 | −0.51 |
Site 4 | 2.29 | 2.28 | −0.01 | 4.65 | 4.47 | −0.18 |
Site 5 | 1.36 | 1.36 | 0 | 1.38 | 1.22 | −0.16 |
Reduce LTC-related ALC hospital days by 30% (output) | ||||||
Site 1 | 1.99 | 1.94 | −0.05 | 10.17 | 7.83 | −2.34 |
Site 2 | 1.18 | 1.12 | −0.06 | 7.81 | 7.08 | −0.73 |
Site 3 | 1.26 | 1.26 | 0 | 11.13 | 10.99 | −0.14 |
Site 4 | 2.29 | 2.25 | −0.04 | 4.65 | 3.95 | −0.7 |
Site 5 | 1.36 | 1.36 | 0 | 1.38 | 1.23 | −0.15 |
Reduce Non-LTC-related ALC hospital days by 30% (output) | ||||||
Site 1 | 1.99 | 1.93 | −0.06 | 10.17 | 7.66 | −2.51 |
Site 2 | 1.18 | 1.07 | −0.11 | 7.81 | 6.21 | −1.6 |
Site 3 | 1.26 | 1.26 | 0 | 11.13 | 10.91 | −0.22 |
Site 4 | 2.29 | 2.25 | −0.04 | 4.65 | 4.10 | −0.55 |
Site 5 | 1.36 | 1.36 | 0 | 1.38 | 1.28 | −0.1 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Tian, Y.; Basran, J.; Stempien, J.; Danyliw, A.; Fast, G.; Falastein, P.; Osgood, N.D. Participatory Modeling with Discrete-Event Simulation: A Hybrid Approach to Inform Policy Development to Reduce Emergency Department Wait Times. Systems 2023, 11, 362. https://doi.org/10.3390/systems11070362
Tian Y, Basran J, Stempien J, Danyliw A, Fast G, Falastein P, Osgood ND. Participatory Modeling with Discrete-Event Simulation: A Hybrid Approach to Inform Policy Development to Reduce Emergency Department Wait Times. Systems. 2023; 11(7):362. https://doi.org/10.3390/systems11070362
Chicago/Turabian StyleTian, Yuan, Jenny Basran, James Stempien, Adrienne Danyliw, Graham Fast, Patrick Falastein, and Nathaniel D. Osgood. 2023. "Participatory Modeling with Discrete-Event Simulation: A Hybrid Approach to Inform Policy Development to Reduce Emergency Department Wait Times" Systems 11, no. 7: 362. https://doi.org/10.3390/systems11070362