Next Article in Journal
Efficiency Improvement Strategies for Public Health Systems: Developing and Evaluating a Taxonomy in the Australian Healthcare System
Previous Article in Journal
Factors Associated with Depression, Anxiety, and Somatic Symptoms among International Salespeople in the Medical Device Industry: A Cross-Sectional Study in China
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Healthcare
Volume 11
Issue 15
10.3390/healthcare11152175
healthcare-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Study Protocol

Looking Back, Looking Forward: A Study Protocol for a Mixed-Methods Multiple-Case Study to Examine Improvement Sustainability of Large-Scale Initiatives in Tertiary Hospitals

by
Sarah E. J. Moon
1,2,*,
Anne Hogden
1,3,
Kathy Eljiz
3 and
Nazlee Siddiqui
1
1
Australian Institute of Health Service Management, College of Business and Economics, University of Tasmania, Sydney 2040, Australia
2
Statewide Quality and Patient Safety Service, Department of Health Tasmania, Launceston 7250, Australia
3
School of Population Health, Faculty of Medicine and Health, University of New South Wales, Sydney 2052, Australia
*
Author to whom correspondence should be addressed.
Healthcare 2023, 11(15), 2175; https://doi.org/10.3390/healthcare11152175
Submission received: 30 June 2023 / Revised: 26 July 2023 / Accepted: 28 July 2023 / Published: 31 July 2023
(This article belongs to the Section Healthcare Quality and Patient Safety)
Browse Figures
Review Reports Versions Notes

Abstract

:
Background Hospitals invest extensive resources in large-scale initiatives to improve patient safety and quality at an organizational level. However, initial success, if any, does not guarantee longer-term improvement. Empirical and theoretical knowledge that informs hospitals on how to attain sustained improvement from large-scale change is lacking. Aim The proposed study aims to examine improvement sustainability of two large-scale initiatives in an Australian tertiary hospital and translate the lessons into strategies for achieving sustained improvement from large-scale change in hospital settings. Design and Methods The study employs a single-site, multiple-case study design to evaluate the initiatives separately and comparatively using mixed methods. Semi-structured staff interviews will be conducted in stratified cohorts across the organizational hierarchy to capture different perspectives from various staff roles involved in the initiatives. The output and impact of the initiatives will be examined through organizational documents and relevant routinely collected organizational indicators. The obtained data will be analyzed thematically and statistically before being integrated for a synergic interpretation. Implications Capturing a comprehensive organizational view of large-scale change, the findings will have the potential to guide the practice and contribute to the theoretical understandings for achieving meaningful and longer-term organizational improvement in patient safety and quality.

1. Introduction

Making and maintaining improvements is vital for hospitals to deliver safer and higher-quality care. Hospitals face escalating public scrutiny and regulatory requirements for safer and higher-quality care [1]. In response, hospitals dedicate resources to implementing change for improvement, such as quality improvement initiatives [2,3,4]. Large-scale quality initiatives, such as those implemented at an organizational level in hospitals, typically entail mobilizing larger groups of stakeholders whose priorities may diverge or compete. A successful introduction of change necessitates the concerted efforts of those involved at various levels of the organization as well as more time and resource investments. Additionally, achieving the stability of these achievements over the long term cannot be guaranteed by any initial success. Consequently, it poses a major challenge to attain sustained improvement through time and resource intensive endeavors with multiple stakeholder engagements. Based on previously established definitions [5,6], in this study protocol, improvement sustainability is conceptualized as the continuation of change effectiveness after the active implementation phase.
Change implementation at the hospital level presents unique challenges due to the large scale and complexity of the organization. Most hospitals, as complex systems [7,8], comprise many layers of structures and processes, within which multiple departments are interdependent. Within this system, multiple groups of healthcare providers are interrelated across hierarchies [9,10], and they may have different ways to fulfill organizational and professional priorities.
Despite the recognized challenges, empirical evidence and theoretical underpinnings to support sustained improvement from hospital-based large-scale change are lacking. It is acknowledged in the literature that the longer-term success of large-scale improvement initiatives in hospital settings is seldom demonstrated [11,12,13]. Previous studies, which did document the outcome of large-scale change, reported that hospital-wide improvement initiatives infrequently sustained the improvements made [14,15]. An estimated success rate of change over the longer term has been reported as less than one in three [16,17].
From a theoretical perspective, there is a limitation in the existing change theories/models to sufficiently support improvement sustainability in hospital settings [9,18,19,20]. While organizational factors for achieving initial change success have been identified in healthcare settings [21,22,23,24,25], the sufficiency of those factors for sustaining improvement is not well understood. Studies have highlighted theoretical limitations in existing conceptual frameworks toward improvement that is comprehensively supported and sustained [26,27,28].
Lewin’s Unfreezing-Moving-Freezing model takes change from a development perspective, in which implemented change is consolidated and institutionalized through the Freezing phase [29,30]. As such, Lewin’s model demonstrates the potential to inform the sustainability of change effectiveness. However, the factors and processes of each phase, particularly Freezing, need to be more explicit to facilitate meaningful application of the model in complex settings [31,32,33], such as hospitals.
The dearth of empirical evidence and limited theoretical underpinnings to support sustained change effectiveness leads to a problem. Without sufficient evidence to support the longer-term success of larger-scale quality initiatives, initial implementation success may be short lived. In hospitals, such shortfalls could have a negative impact on the provision of safer and higher-quality care and may require repeated endeavors at additional cost when improvements are unsustainable.
Learning from previous change and translating the lessons into actionable strategies is central to achieving sustainable change outcomes and development [30,32,34]. Organizational learning involves a change in the organization’s knowledge that occurs through experience and is manifested in behavior embedded in individuals, routines, and memory systems [35]. From an organizational learning perspective, applications of the learning lead to changes in staff actions (i.e., error correction), processes (e.g., policy), and context (e.g., basic assumptions) within the organization [35,36]. The organizational learning approach provides a useful theoretical lens for sustaining change outcomes. For hospitals, this means that these learnings could help avoid similar mistakes and improve upon identified strengths in implementing planned change, such as organizational initiatives. However, insufficient opportunity to learn from experience is a major challenge to change success [32]. The low success rate of longer-term change effectiveness could be addressed by context-specific learning from change experience and using the lessons to inform current and future change strategies. Sustainable change in healthcare requires deliberate and proactive planning [37,38]. For hospitals, forming their own organizational knowledge that is grounded in learning and outcomes from large-scale change could contribute to the effective embedding of organizational initiatives with longer-term benefits.
A recent review [39] of empirical evidence has identified that sustained improvement from hospital-wide quality initiatives requires concerted efforts across hospital hierarchies (i.e., change actors) before, during, and after the active implementation phase. The review provides a comprehensive framework of interrelationships among the identified factors for improvement sustainability. The evidence-based framework includes the process aspects that resemble Lewin’s Unfreeze-Moving-Freezing model [29,30] and interplays between each aspect, which could be used to further expand the model for its use in hospital settings. However, the findings of this review indicate the need to examine improvement sustainment in real-world settings due to the context dependency of change implementation in the complex hospital environment [40,41]. Robust empirical research on large-scale change in various hospital settings will add to practice-based evidence [42]. Moreover, further research can bring a balanced focus on the end users, such as frontline staff, who determine the success of embedded change [43].

Study Aims and Research Questions

We propose to examine improvement sustainability of two large-scale initiatives in an Australian tertiary hospital and translate the lessons into strategies for achieving sustained improvement from large-scale change in hospital settings. Specifically, Aim 1 is to examine retrospectively two quality improvement initiatives implemented across a hospital (i.e., hospital-wide), including the relevant organizational indicator outcome and the experience of the change actors involved in the initiatives. Aim 2 is to obtain lessons from the initiative implementations that can be translated prospectively into change strategies to design and support improvement sustainability.
We pose an overarching research question, “How is hospital-wide change embedded for sustained improvement in patient safety and quality?”, to address evidence gaps in the empirical and theoretical knowledge base. The following sub-questions will be addressed:
(1)
What are the enablers and barriers to sustaining the improvement?
(2)
How do relevant organizational records demonstrate the initiative outcome?
(3)
What factors in hospital-wide change implementation lead to sustained improvement?
(4)
What hospital-wide change processes enable sustained improvement?

2. Design and Methods

2.1. Study Design

We propose to conduct a single-site, multiple-case study with a mixed-methods approach to fulfill the study’s aims. The reporting elements of the mixed-method approach provided in this document have been informed by the reporting guideline for Mixed-Methods Research Manuscript Preparation and Review in health services management studies [44].

2.1.1. Multiple-Case Study

Case studies are used to obtain richer and context-specific insights on a particular phenomenon [45,46], such as improvement sustainability. A single case may not reflect sufficiently the organizational variables within a complex hospital environment [8,9,10], by which to examine change implementation and improvement sustainability. Therefore, this study proposes to examine two real-world hospital-wide initiatives at the same hospital (Initiative #1, and Initiative #2). Each initiative will be treated as a unit of case and examined separately and sequentially (Initiative #1 then Initiative #2) to capture and fully immerse in the distinct experience and indicator outcomes of each case [46].
Two initiatives will be chosen based on the similarities in (1) change scale (i.e., hospital-wide), (2) change scope (i.e., multiple stakeholder groups), and (3) the aim of the initiative (i.e., to improve patient safety and quality). The study will be conducted in one hospital setting to ensure a consistent change context and eliminate variation introduced by different organizational settings that may influence the experience and indicator outcome of the initiatives. In addition, the quality-improvement oriented objective of the two initiatives will allow comparisons of the various change elements utilized [47]. This approach has been chosen because it will enhance rigor while ascertaining a variety of commonalities and differences among the cases in a single-site hospital context.
This study will select a completed initiative (Initiative #1) and another that is under implementation (Initiative #2) across the study site. This is to obtain actionable learnings from the first initiative and inform the strategies and actions of the second initiative’s implementation. Together, this approach will assist hospitals in a similar context to sustain any gains resulting from large-scale change initiatives.

2.1.2. Mixed Methods

This study will adopt the one-phase (convergent) mixed-methods approach described by Creswell and Creswell [48], which involves the concurrent collection of separate sets of quantitative and qualitative data and integrating them for interpretation for each case. This approach will optimize the comprehensiveness and rigor of data collection and analysis to address the research questions. Integrated quantitative and qualitative data will provide stronger insights than a single type of data [48]. In a complex hospital environment [9], organizational indicators (i.e., quantitative data) alone may not fully reflect the outcome and process of the change implementation without considering the experience of the participants (i.e., qualitative data), and vice versa.
Together, this study will comprehensively capture the experience and relevant indicator outcomes from the two initiatives to inform change strategies for optimizing improvement sustainability. This approach, namely ‘Looking back, looking forward’ [49], is illustrated in Figure 1.

2.2. Study Setting

This study will be conducted at a public tertiary referral hospital in a regional area of Australia. The area is part of a state with a population of approximately 560,000 [50]. As a tertiary hospital, the study site is the referral center for patients who require intensive care and has extensive laboratory and clinical service facilities [51]. The hospital provides a range of general and specialty treatments ranging from acute, sub-acute, mental health, and inpatient aged care to approximately 250,000 people. It is a major clinical research and teaching facility in the area in affiliation with a university. The hospital is the largest referral center in the area, with approximately 63,000 admissions and 65,000 emergency department visits as of 2020. There are approximately 500 beds, employing around 3000 staff, and this facility is part of the broader public health service of the state.

2.3. Initiatives Selected for This Study

The two selected initiatives are Right Time Every Time (Initiative #1) and Speaking Up For SafetyTM (Initiative #2). These initiatives have been selected for (1) the change scale being hospital-wide, (2) the complexity involving multi-disciplinary stakeholder groups and various staff roles, and (3) the aims of both initiatives being improvement in patient safety and quality of care.
The scale of change and complexity of the selected initiatives are sufficiently representative of the phenomenon being examined (i.e., improvement sustainability of large-scale change), and this will allow rich data to address the research questions [46].

2.3.1. Right Time Every Time (Initiative #1)

The aim of this initiative was to reduce incidences of medication omission, delay, and duplication across the study site. The initiative was implemented between June 2014 and June 2015. It included multiple interventions, such as developing and updating relevant protocols within the study site, a hospital-wide promotional campaign, and changes in the pharmacy systems for prescribing and dispensing prescribed medication. The compliance rate of omitted drugs within the National Inpatient Medication Chart (NIMC) Audit was used to measure the effectiveness of this initiative. The Audit is standardized and coordinated nationally by the Australian Commission on Safety and Quality in Health Care (ACSQHC), an Australian government agency that sets regulatory standards for safety, quality, and clinical care [52]. The NIMC Audit evaluated the safety and quality of paper-based medication charting in Australian hospitals and was replaced by the National Standard Medication Chart Audit in 2018 [52].

2.3.2. Speaking Up for SafetyTM (Initiative #2)

The second initiative was launched in 2020 to implement the Speaking Up For SafetyTM program developed by the Cognitive Institute. This program aims to improve communication when escalating patient safety concerns to build a safety culture [53]. The program is being implemented across the broader region of the health service in the state. Currently, the study site is undergoing an ‘awareness phase’, during which all staff members are invited to attend a seminar presentation. The presentations are delivered by the service’s staff, who have been trained and licensed (called ‘Champions’) to provide education on this initiative. Presently, there is no agreed measurement by which to evaluate the effectiveness of the program. However, a routinely conducted organizational staff survey has been identified for utilization, as it includes questions relevant to the program’s intent and objective.

3. Study Procedures

The sequential approach to the multiple cases will involve examining Initiative #1, followed by Initiative #2. This approach will allow immersion in the unique experience and indicator outcomes of each initiative [45,46]. Each case will be examined through the data collected using the one-phase mixed-methods approach which will involve staff experience, routinely collected organizational indicator data and other organizational records, such as relevant organizational protocols [48]. A summary of the mixed-methods data, to be collected for this study is provided in Table 1.

3.1. Quantitative Data Collection—Routinely Collected Organizational Data

For Initiative #1, the annual medication omission rate within the NMIC Audit results for the period of 2012–2021 will be collected. The omission rate at a baseline (prior to the initiative implementation), during the initiative implementation, and in the post-implementation phase will be assessed. The audit tool was developed and managed by the ACSQHC for national use. This measure is consistent with that used during the initiative’s implementation in 2014–2015. The audit results are collected and published internally at the hospital level; therefore, no identifiable or personal information will be included in this dataset. The annual trend of the medication omission rate for the study site will be assessed to identify any rate changes during and after the implementation of the initiative. The audit results of the post-implementation years will inform the longer-term outcome of the initiative in relation to Aim 1.
For Initiative #2, the responses to selected questions within a routinely conducted Staff Engagement Survey will be used. Examples of the selected questions are “Employees in my work unit make every effort to deliver safe, error-free care” and “I can report patient safety mistakes without fear of punishment”. The selected questions indicate sufficient alignment with the aim of Initiative #2. Responders are asked to rate their agreement with these statements on a Likert scale of 1 (Strongly disagree) to 5 (Strongly agree). Survey participation is voluntary and conducted anonymously by a provider, Insync®, with an internally developed questionnaire. The survey is used among healthcare organizations in Australia and internationally, particularly for the external benchmarking feature provided by the survey company. At the study site, the survey commenced in 2018 for nursing staff and expanded in 2020 to include medical, allied health, and clinical support staff. The survey is conducted biennially. Changes in the ratings for selected questions with the introduction of Initiative #2 will be assessed as an interim outcome measure. Anonymized responses will be used, with no identifiable or personal information to be collected.

3.2. Qualitative Data Collection

We will conduct semi-structured, one-on-one staff interviews and collect organizational documents, such as relevant policies and internal reports, as the qualitative data collection method [48] to capture both individual experience and organizational evidence about the initiatives’ implementation.

3.2.1. Semi-Structured Individual Staff Interviews

The first author will conduct semi-structured interviews with individual hospital staff members. Eligibility criteria include clinical or non-clinical staff from the study site who took part in Initiative #1 and those currently taking part in Initiative #2. Eligible staff will be invited to reflect on their experience with the relevant initiative. Semi-structured interviews can facilitate deep understanding by providing the opportunity to explore participants’ experiences and reflections on the initiatives [54]. Interviews will last approximately 45–60 min and be conducted in a quiet setting via video conferencing or in person at the study site or negotiated venue. Interviews will be recorded and transcribed verbatim with participant consent. A demographic questionnaire will be administered to capture the characteristics of the participants, including their professional background, years of employment, and previous education relevant to change implementation. Written informed consent will be obtained from all participants prior to the interview.

Participant Stratification

Implementing large-scale change in the complex hospital environment involves multiple groups of individuals professionally and hierarchically [4,9,10]. This complexity requires capturing diverse perspectives from different functions within the role hierarchy, such as senior managers, first-level managers, frontline staff, and the implementation team involved in the initiative’s implementation. Therefore, we will stratify participants into homogenous role groups to allow exploration of unique perspectives on initiative implementation and improvement sustainability across role variations (Table 2). Role stratification helps to capture the distinctive roles of the healthcare workforce at different levels of hierarchy to achieve and sustain improvement [4,39,55].
The total number of interview participants will be between 32 and 36 (Table 2). Data sufficiency can be reached using a relatively small number of participants who have higher “information power” [56], the degree to which participants can provide necessary insight to answer the research question based on their relevant experience [56]. This approximation considers the number of participants needed in each role/function for adequate capture of potentially distinctive experiences with the initiative’s implementation and to explore improvement sustainability.

Interview Questions

Interviews will explore staff experience of the implementation of the initiatives across three main domains—people, process, and organizational environment—and 11 subdomains, empirically evidenced as the factors to sustaining improvement [39]. Examples of subdomains that will inform the interview discussion are leadership, change management and perceived hospital culture supported for embedding the initiatives (Figure 2). Some interview questions will be customized for the stratified participant groups relevant to the role variation. For example, for frontline staff, there will be additional questions about perceived leadership at the unit and hospital levels. For the senior manager group, questions will explore more of their experience with the organizational embedding of the initiatives.

3.2.2. Organizational Records

The electronic strategic document system and the Intranet within the study site will be utilized to search and retrieve relevant electronic organizational records relevant to the two initiatives. Strategic documents, such as policies and protocols, will be searched to identify any outputs or evidence of the organizational embedding of the initiatives. Operational documents such as reports produced from the initiative and staff communications such as newsletters and memos will be obtained to capture the activities and endorsed messages from the hospital management regarding the initiatives.

3.3. Data Analysis

3.3.1. Multiple-Case Analysis

Each initiative, as a case, will be analyzed separately (i.e., within a case) and comparatively (i.e., across cases) to identify similarities and differences [45,47,57]. The observed themes and outcomes evidenced by the merged and interpreted data through mixed methods [48] will address the study aims and provide a fuller picture of change implementation and improvement sustainability within the study site (Figure 1).

3.3.2. Quantitative Data Analysis

Descriptive statistical analysis will be conducted on the numerical data collected: (1) the omitted medication audit results (Initiative #1), (2) the selected survey responses (Initiative #2), and (3) the demographic questionnaire responses. The statistical analysis will address Aim 1 by quantitatively summarizing measurable hospital indicator outcomes relevant to the two initiatives and the characteristics of interview participants. The analysis for the first two data items will be a time series (trend) analysis of the indicator outcome before, during (Initiative #1 and #2) and after (Initiative #1) the initiative’s implementation. Microsoft Excel (version 2302) or IBM SPSS Statistics V.29.12 will be used for the time series analysis. The time series analysis will assess whether the implementation affected the indicator outcome and whether the impact of Initiative #1 was sustained. For Initiative #2, staff survey responses prior to the initiative introduction will be compared with those after the introduction. Demographic data from the interview participants will be analyzed through Research Electronic Data Capture (REDCap), a secure web-based research data capture tool [58].
Although the statistical analyses will examine potential long-term (Initiative #1) and interim (Initiative #2) impacts of the intervention, caution will be taken that the initiatives will not be the sole factor in any numerical changes in the indicator outcome and that there may be a variety of confounding factors. Accordingly, the statistical analysis is not expected to identify the causal factors for the changes. However, inferential statistical analysis may be conducted to identify correlations within the staff survey responses, such as a relationship between other questionnaire items and the selected items relevant to patient safety.

3.3.3. Qualitative Data Analysis

The qualitative data from interviews and organizational documents will be analyzed to address Aim 1 and Aim 2 by exploring the participant experience and relevant organizational documents from the perspective of improvement sustainability. Data collected through interviews and free-text responses from the previously conducted surveys will be analyzed using the thematic analysis method elaborated by Nowell et al. [59]. This method involves: (1) familiarizing oneself with the collected data; (2) generating initial codes; (3) searching for themes; (4) reviewing themes; (5) defining and naming themes; and (6) producing the report. Such a structured approach will allow examining different perspectives through similarities and differences captured in the data [59]. A coding framework will be developed and aligned with the interview question themes [39]. The stratified participants will be assigned a unique code to be indirectly identified in separate groups (Table 3). For example, a doctor who participated in Initiative #1 without managerial responsibilities will be coded as 1G4DR, and a Nurse Unit Manager who is participating in Initiative #2 will be coded as 2G3NR.
The collected organizational documents, such as protocols and staff communication, will be examined through document analysis, which involves (1) reading the material, (2) extracting data, (3) analyzing data and (4) distilling the findings [60]. This approach supports rigor through systematic and practical guidance for evaluating and analyzing organizational documents.
These qualitative analyses will occur concurrently with the data collection. This approach will allow the interview questions to be refined based on findings that emerge from the interviews [61]. NVivo software (Release 1.7, version 1533) [62] will be used to manage qualitative data.

3.3.4. Integrating Mixed-Methods Data

The mixed-methods datasets for each case will be integrated by merging [63] within the one-phase (convergent) approach [48] (Figure 1). The integrated data will provide a triangulated and synergetic overview of the improvement sustainability of Initiatives #1 and #2 within the study site [48,64]. The findings of the integrated data for each initiative will be compared, and divergence and inconsistencies between the datasets will be identified [44]. The described analysis approach has been adopted to allow sufficient rigor through rich mixed-methods data from two different initiatives [65]. The findings obtained will address Aim 1.
Further, lessons emerging from the qualitative datasets relevant to improvement sustainability will be synthesized. These will be translated into actionable strategies to guide large-scale change with sustained improvement within the study site and potentially for broader application in similar settings. The synthesized actions will be informed by the triangulated approach consistent with the above. This approach has been chosen for the comprehensive, context-driven evidence necessary to sufficiently address Aim 2.
Together, the integration of the data is designed to draw a fuller picture of the improvement sustainability of large-scale change within the study site as well as utilize lessons to inform improvement sustainability. This approach is chosen as a recognized strategy to embed large-scale change through learning from experience in context [32,35].

4. Discussion

By addressing the research questions, this study aims to comprehensively examine the past and current large-scale quality initiatives in a tertiary hospital and obtain actionable knowledge for attaining sustained improvement from large-scale change in comparable settings. This study seeks to contribute relevant knowledge for practice and theoretical underpinnings to support organizational development through sustained improvement from change efforts.
For the study site, information obtained from the research questions will provide evidence to develop site-specific organizational knowledge and learning in improvement sustainability of large-scale change [30,32,34,35]. The advanced knowledge, based on evidence from the local context, has the potential to assist hospitals in optimizing time and resource investment for large-scale change and sustaining these outcomes. Previous studies have shown that the organizational context determines the implementation outcome [23,66]. The information gained through the sub-questions will provide context-based evidence that may demonstrate actionable strategies to consistently deliver safer and higher quality care. Learning from change experience is a recognized opportunity toward successful and meaningful organizational change [7,32] and an essential step to transforming organizational experience into knowledge [35]. The ability to learn in such a way is a vital element in adapting to the dynamic healthcare landscape [67].
The study findings will add to the evidence base to support longer-term improvement from large-scale change in hospital settings. The multiple-case study design allows comprehensive reflections of organizational variables within the complex environment of hospitals [8,9,10] that may be unique to other settings such as care homes [68]. Particularly, the change outcomes of and the participant experience with Initiative #1 will address calls for longer-term change evaluation [11,12,13]. The organizational learning approach of prospectively translating the lesson from previous change to current and future change will contribute to the evidence base to inform change strategies for longer-term effectiveness [32]. Additionally, improvement sustainability from the perspectives of the participants could add to the change management literature that is traditionally driven by managerial viewpoints [32]. The value of understanding change participants’ views and their interpretation of change as a recognized opportunity for successful and meaningful change [32,69]. The participant interviews in this study will provide a lens to evaluate organizational change through the examined initiatives and the ability to sustain the benefits of the change.
From a theoretical perspective, the study findings will present the potential to advance the theoretical underpinnings to support and design organizational improvement sustainability in hospital settings. The knowledge obtained of the factors that facilitate or constrain improvement sustainability will provide organizational variables and/or determinants in large-scale change. These factors could be as considered as potential organizational determinants in conjunction with those included in frameworks that are widely used in healthcare [70,71] to enhance improvement sustainability in change implementation. The identified elements of change processes that lead to sustaining improvement will offer evidence to guide organizational change in hospital settings or be incorporated within current change process models to elaborate on improvement sustainability. Lewin’s Unfreezing-Moving-Freezing model [29,30] provides a flexible platform that is well established in healthcare [33]. Available evidence has identified a high affinity between Lewin’s model and the review findings of the change process that enabled sustained improvement [39].
The proposed study has strengths and limitations. The mixed methods and multiple-case study design of this study cater for the complexity that is inherent to the large hospital environment [7,8,9,10]. Those approaches enable capturing a fuller picture of change implementation and improvement sustainability in the study organization through triangulating organizational records and participant experience. Mixed methods will allow unique and balanced perspectives on interim and longer-term outcomes from the initiatives and their improvement sustainability. The stratified cohorts for interview participants are designed to ascertain the holistic perspectives of the participants involved in the examined initiatives across organizational hierarchies and various disciplines. This structure is reflective of the intricate layers of contemporary healthcare organizations [4,55]. Improvement sustainability seen through the lens of participant experience across different role positions will offer knowledge that may be of actionable value for hospital managers and health service researchers. Therefore, this proposed study could be appropriate for audiences across different healthcare disciplines in complex settings where change is introduced. Limitations of this study include reduced generalizability due to the single-site study design, although the findings will provide a rich organizational knowledge base for the study site to develop strategies for sustaining improvement from change. A relatively small sample size for interview participants is another limitation of this study, which aims to capture an organizational view. Stratifying participant groups to capture various viewpoints across organizational hierarchy and disciplines is a mitigation strategy to prevent misrepresenting an organizational view from a single group.

5. Conclusions

This study aims to obtain a comprehensive organizational view of improvement sustainability through large-scale initiatives and translate the lessons into actionable strategies for achieving sustained improvement in hospital settings. The multiple-case study design with mixed methods will allow rich data to comprehensively address evidence gaps and produce knowledge that can be applied in practice and research settings. Examining past and current initiatives, this study will generate lessons for guiding organizational change in which the gained improvement can be maintained in longer-term. We seek to optimize the outcome of the efforts and investment for organizational change toward improved patient care outcome in complex hospital settings.

6. Dissemination

Anonymized results of this study will be disseminated locally and internationally through peer-reviewed journals, conference presentations, and a doctoral thesis (SEJM). The stakeholders of the study site will be shared with the study results in a variety of forms, such as reports and presentations.

Author Contributions

S.E.J.M. conceptualized and designed this study in consultation with A.H. and K.E.; A.H., K.E. and N.S. provided guidance to data collection and analysis methods; S.E.J.M. drafted the initial manuscript. All authors critically reviewed the manuscript and contributed to the refinement of the final manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

S.E.J.M. is funded by the Australian Government Research Training Program PhD scholarship.

Institutional Review Board Statement

The proposed study has been reviewed and approved by the relevant Human Research Ethics Committee in Australia (H0026463) and by the study site (2023/STE01308). The Committee is anonymized due to the high risk of identifiability of the study site and study participants by disclosing the committee’s name. There is no anticipated harm in the health or safety of the interview participants or other aspects of the proposed study. However, should any participants express discomfort or distress during the interview appropriate support will be offered. No published data will identify individuals. This study will be conducted as part of a doctoral study by the first author. Confidentiality will be maintained in accordance with the University’s data management policy.

Informed Consent Statement

Informed consent will be obtained from all participants involved in the study.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Slawomirski, L.; Auraaen, A.; Klazinga, N. The Economics of Patient Safety: Strengthening a Value-Based Approach to Reducing Patient Harm at National Level; Organisation for Economic Co-operation and Development: Paris, France, 2017. [Google Scholar]
  2. Donnelly, L.F. Avoiding failure: Tools for successful and sustainable quality-improvement projects. Pediatr. Radiol. 2017, 47, 793–797. [Google Scholar] [CrossRef] [PubMed]
  3. Lawson, T.; Weekes, L.; Hill, M. Ensuring success and sustainability of a quality improvement project. BJA Educ. 2018, 18, 147–152. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Scoville, R.; Little, K.; Rakover, J.; Luther, K.; Mate, K. Sustaining Improvement—IHI White Paper; Institute for Healthcare Improvement: Cambridge, MA, USA, 2016. [Google Scholar]
  5. Moore, J.E.; Mascarenhas, A.; Bain, J.; Straus, S.E. Developing a comprehensive definition of sustainability. Implement. Sci. 2017, 12, 110. [Google Scholar] [CrossRef] [PubMed]
  6. Braithwaite, J.; Ludlow, K.; Testa, L.; Herkes, J.; Augustsson, H.; Lamprell, G.; McPherson, E.; Zurynski, Y. Built to last? The sustainability of healthcare system improvements, programmes and interventions: A systematic integrative review. BMJ Open 2020, 10, e036453. [Google Scholar] [CrossRef]
  7. Braithwaite, J.; Churruca, K.; Long, J.C.; Ellis, L.A.; Herkes, J. When complexity science meets implementation science: A theoretical and empirical analysis of systems change. BMC Med. 2018, 16, 63. [Google Scholar] [CrossRef] [Green Version]
  8. Thompson, D.S.; Fazio, X.; Kustra, E.; Patrick, L.; Stanley, D. Scoping review of complexity theory in health services research. BMC Health Serv. Res. 2016, 16, 87. [Google Scholar] [CrossRef] [Green Version]
  9. Cowie, J.; Nicoll, A.; Dimova, E.D.; Campbell, P.; Duncan, E.A. The barriers and facilitators influencing the sustainability of hospital-based interventions: A systematic review. BMC Health Serv. Res. 2020, 20, 588. [Google Scholar] [CrossRef]
  10. De Silva, D. What’s Getting in the Way? Barriers to Improvement in the NHS; The Health Foundation: London, UK, 2015. [Google Scholar]
  11. Baker, D.; Quinn, B.; Ewan, V.; Giuliano, K.K. Sustaining quality improvement: Long-term reduction of nonventilator hospital-acquired pneumonia. J. Nurs. Care Qual. 2019, 34, 223–229. [Google Scholar] [CrossRef]
  12. Baloh, J.; Zhu, X.; Ward, M.M. What influences sustainment and nonsustainment of facilitation activities in implementation? Analysis of organizational factors in hospitals implementing TeamSTEPPS. Med. Care Res. Rev. 2021, 78, 146–156. [Google Scholar] [CrossRef]
  13. Silver, S.A.; McQuillan, R.; Harel, Z.; Weizman, A.V.; Thomas, A.; Nesrallah, G.; Bell, C.M.; Chan, C.T.; Chertow, G.M. How to sustain change and support continuous quality improvement. Clin. J. Am. Soc. Nephrol. 2016, 11, 916–924. [Google Scholar] [CrossRef] [Green Version]
  14. Glasgow, J.M.; Davies, M.L.; Kaboli, P.J. Findings from a national improvement collaborative: Are improvements sustained? BMJ Qual. Saf. 2012, 21, 663. [Google Scholar] [CrossRef] [Green Version]
  15. Robert, G.; Sarre, S.; Maben, J.; Griffiths, P.; Chable, R. Exploring the sustainability of quality improvement interventions in healthcare organisations: A multiple methods study of the 10-year impact of the ‘Productive Ward: Releasing Time to Care’ programme in English acute hospitals. BMJ Qual. Saf. 2020, 29, 31–40. [Google Scholar] [CrossRef] [Green Version]
  16. Pieterse, J.H.; Caniëls, M.C.J.; Homan, T. Professional discourses and resistance to change. J. Organ. Chang. Manag. 2012, 25, 798–818. [Google Scholar] [CrossRef] [Green Version]
  17. NHS Institute of Innovation and Improvement. Improvement Leaders’ Guide: Sustainability and its Relationship with Spread and Adoption. Available online: https://www.england.nhs.uk/improvement-hub/publication/improvement-leaders-guide-sustainability-and-its-relationship-with-spread-and-adoption-general-improvement-skills/ (accessed on 13 April 2023).
  18. Shelton, R.C.; Cooper, B.R.; Stirman, S.W. The sustainability of evidence-based interventions and practices in public health and health care. Annu. Rev. Public Health 2018, 39, 55–76. [Google Scholar] [CrossRef] [Green Version]
  19. Lennox, L.; Maher, L.; Reed, J. Navigating the sustainability landscape: A systematic review of sustainability approaches in healthcare. Implement. Sci. 2018, 13, 27. [Google Scholar] [CrossRef] [Green Version]
  20. Cadilhac, D.A.; Andrew, N.E.; Stroil Salama, E.; Hill, K.; Middleton, S.; Horton, E.; Meade, I.; Kuhle, S.; Nelson, M.R.; Grimley, R. Improving discharge care: The potential of a new organisational intervention to improve discharge after hospitalisation for acute stroke, a controlled before–after pilot study. BMJ Open 2017, 7, e016010. [Google Scholar] [CrossRef]
  21. Li, S.A.; Jeffs, L.; Barwick, M.; Stevens, B. Organizational contextual features that influence the implementation of evidence-based practices across healthcare settings: A systematic integrative review. Syst. Rev. 2018, 7, 72. [Google Scholar] [CrossRef] [Green Version]
  22. Cummings, A.; Lund, S.; Campling, N.; May, C.R.; Richardson, A.; Myall, M. Implementing communication and decision-making interventions directed at goals of care: A theory-led scoping review. BMJ Open 2017, 7, e017056. [Google Scholar] [CrossRef]
  23. Kaplan, H.C.; Brady, P.W.; Dritz, M.C.; Hooper, D.K.; Linam, W.M.; Froehle, C.M.; Margolis, P. The influence of context on quality improvement success in health care: A systematic review of the literature. Milbank Q. 2010, 88, 500–559. [Google Scholar] [CrossRef] [Green Version]
  24. Ross, J.; Stevenson, F.; Lau, R.; Murray, E. Factors that influence the implementation of e-health: A systematic review of systematic reviews (an update). Implement. Sci. 2016, 11, 146. [Google Scholar] [CrossRef] [Green Version]
  25. Hulscher, M.E.; Schouten, L.M.; Grol, R.P.; Buchan, H. Determinants of success of quality improvement collaboratives: What does the literature show? BMJ Qual. Saf. 2013, 22, 19–31. [Google Scholar] [CrossRef] [PubMed]
  26. Ament, S.M.C.; Gillissen, F.; Moser, A.; Maessen, J.M.C.; Dirksen, C.D.; von Meyenfeldt, M.F.; van der Weijden, T. Factors associated with sustainability of 2 quality improvement programs after achieving early implementation success. A qualitative case study. J. Eval. Clin. Pract. 2017, 23, 1135–1143. [Google Scholar] [CrossRef] [PubMed]
  27. Ilott, I.; Gerrish, K.; Booth, A.; Field, B. Testing the Consolidated Framework for Implementation Research on health care innovations from South Yorkshire. J. Eval. Clin. Pract. 2013, 19, 915–924. [Google Scholar] [CrossRef] [PubMed]
  28. Skingley, A.; Marshall, J. Challenges of implementing and embedding a programme to improve care for older people with dementia on hospital wards. Nurs. Older People 2018, 30, 29–33. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  29. Lewin, K. Frontiers in group dynamics: Concept, method and reality in social science; social equilibria and social change. Hum. Relat. 1947, 1, 5–41. [Google Scholar] [CrossRef]
  30. Burnes, B. The origins of Lewin’s Three-Step Model of Change. J. Appl. Behav. Sci. 2020, 56, 32–59. [Google Scholar] [CrossRef]
  31. Burnes, B. Kurt Lewin and complexity theories: Back to the future? J. Chang. Manag. 2004, 4, 309–325. [Google Scholar] [CrossRef]
  32. Stouten, J.; Rousseau, D.M.; De Cremer, D. Successful organizational change: Integrating the management practice and scholarly literatures. Acad. Manag. Ann. 2018, 12, 752–788. [Google Scholar] [CrossRef]
  33. Harrison, R.; Fischer, S.; Walpola, R.; Chauhan, A.; Babalola, T.; Mears, S.; Le-Dao, H. Where do models for change management, improvement and implementation meet? A systematic review of the applications of change management models in healthcare. J. Healthc. Leadersh. 2021, 13, 85–108. [Google Scholar] [CrossRef]
  34. French, W.L.; Bell, C.M. A definition and history of Organisation Development: Some comments. Acad. Manag. Proc. 1971, 1971, 146–153. [Google Scholar] [CrossRef]
  35. Argote, L.; Miron-Spektor, E. Organizational learning: From experience to knowledge. Organ. Sci. 2011, 22, 1123–1137. [Google Scholar] [CrossRef] [Green Version]
  36. Tosey, P.; Visser, M.; Saunders, M.N. The origins and conceptualizations of ‘triple-loop’ learning: A critical review. Manag. Learn. 2012, 43, 291–307. [Google Scholar] [CrossRef] [Green Version]
  37. Dixon-Woods, M.; McNicol, S.; Martin, G. Ten challenges in improving quality in healthcare: Lessons from the Health Foundation’s programme evaluations and relevant literature. BMJ Qual. Saf. 2012, 21, 876. [Google Scholar] [CrossRef] [Green Version]
  38. Mitchell, S.E.; Weigel, G.M.; Laurens, V.; Martin, J.; Jack, B.W. Implementation and adaptation of the Re-Engineered Discharge (RED) in five California hospitals: A qualitative research study. BMC Health Serv. Res. 2017, 17, 291. [Google Scholar] [CrossRef]
  39. Moon, S.E.; Hogden, A.; Eljiz, K. Sustaining improvement of hospital-wide initiative for patient safety and quality: A systematic scoping review. BMJ Open Qual. 2022, 11, e002057. [Google Scholar] [CrossRef]
  40. Coles, E.; Anderson, J.; Maxwell, M.; Harris, F.M.; Gray, N.M.; Milner, G.; MacGillivray, S. The influence of contextual factors on healthcare quality improvement initiatives: A realist review. Syst. Rev. 2020, 9, 94. [Google Scholar] [CrossRef] [Green Version]
  41. Fulop, N.J.; Ramsay, A.I.G. How organisations contribute to improving the quality of healthcare. BMJ 2019, 365, l1773. [Google Scholar] [CrossRef] [Green Version]
  42. Ammerman, A.; Smith, T.W.; Calancie, L. Practice-Based Evidence in Public Health: Improving Reach, Relevance, and Results. Annu. Rev. Public Health 2014, 35, 47–63. [Google Scholar] [CrossRef] [Green Version]
  43. Higgs, M.J.; Kuipers, B.S.; Steijn, B. Change leadership and change embeddedness in public organizations: Connecting macrolevel reform to microlevel implementation. Public Adm. 2022, 100, 1–15. [Google Scholar] [CrossRef]
  44. Lee, S.-Y.D.; Iott, B.; Banaszak-Holl, J.; Shih, S.-F.; Raj, M.; Johnson, K.E.; Kiessling, K.; Moore-Petinak, N.d. Application of mixed methods in health services management research: A systematic review. Med. Care Res. Rev. 2022, 79, 331–344. [Google Scholar] [CrossRef]
  45. Stake, R.E. Multiple Case Study Analysis; Guilford Press: New York, NY, USA, 2005. [Google Scholar]
  46. Yin, R.K. Case Study Research and Applications: Design and Methods, 6th ed.; Sage: Thousand Oaks, CA, USA, 2017. [Google Scholar]
  47. Crowe, S.; Cresswell, K.; Robertson, A.; Huby, G.; Avery, A.; Sheikh, A. The case study approach. BMC Med. Res. Methodol. 2011, 11, 100. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  48. Creswell, J.W.; Creswell, J.D. Research Design: Qualitative, Quantitative and Mixed Methods Approaches, 5th ed.; Sage Publications: Thousand Oaks, CA, USA, 2018. [Google Scholar]
  49. Hill, J.N.; Guihan, M.; Hogan, T.P.; Smith, B.M.; LaVela, S.L.; Weaver, F.M.; Anaya, H.D.; Evans, C.T. Use of the PARIHS Framework for retrospective and prospective Implementation evaluations. Worldviews Evid. Based Nurs. 2017, 14, 99–107. [Google Scholar] [CrossRef] [PubMed]
  50. Australian Bureau of Statistics. Snapshot of Australia. Available online: https://www.abs.gov.au/statistics/people/people-and-communities/snapshot-australia/2021 (accessed on 21 May 2023).
  51. Australian Institute of Health and Welfare. Glossary. Available online: https://www.aihw.gov.au/reports-data/myhospitals/content/glossary (accessed on 16 July 2023).
  52. Australian Commission on Safety and Quality in Health Care. National Standard Medication Chart (NSMC) Audit. Available online: https://www.safetyandquality.gov.au/our-work/medication-safety/national-standard-medication-chart-nsmc-audit (accessed on 21 May 2023).
  53. Cognitive Institute. Speaking Up for Safety™ Programme. Available online: https://www.cognitiveinstitute.org/healthcare-courses/speaking-up-for-safety-programme/ (accessed on 18 March 2023).
  54. De Jonckheere, M.; Vaughn, L.M. Semistructured interviewing in primary care research: A balance of relationship and rigour. Fam. Med. Community Health 2019, 7, e000057. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  55. Nyström, M.E.; Höög, E.; Garvare, R.; Andersson Bäck, M.; Terris, D.D.; Hansson, J. Exploring the potential of a multi-level approach to improve capability for continuous organizational improvement and learning in a Swedish healthcare region. BMC Health Serv. Res. 2018, 18, 376. [Google Scholar] [CrossRef] [Green Version]
  56. Malterud, K.; Siersma, V.D.; Guassora, A.D. Sample size in qualitative interview studies: Guided by information power. Qual. Health Res. 2016, 26, 1753–1760. [Google Scholar] [CrossRef]
  57. Taher, S.; Muramatsu, N.; Odoms-Young, A.; Peacock, N.; Michael, C.F.; Courtney, K.S. An embedded multiple case study: Using CFIR to map clinical food security screening constructs for the development of primary care practice guidelines. BMC Public Health 2022, 22, 97. [Google Scholar] [CrossRef]
  58. Patridge, E.F.; Bardyn, T.P. Research Electronic Data Capture (REDCap). J. Med. Libr. Assoc. 2018, 106, 142–144. [Google Scholar] [CrossRef] [Green Version]
  59. Nowell, L.S.; Norris, J.M.; White, D.E.; Moules, N.J. Thematic analysis: Striving to meet the trustworthiness criteria. Int. J. Qual. Methods 2017, 16, 1609406917733847. [Google Scholar] [CrossRef]
  60. Dalglish, S.L.; Khalid, H.; McMahon, S.A. Document analysis in health policy research: The READ approach. Health Policy Plan. 2020, 35, 1424–1431. [Google Scholar] [CrossRef]
  61. Pomare, C.; Churruca, K.; Long, J.C.; Ellis, L.A.; Gardiner, B.; Braithwaite, J. Exploring the ripple effects of an Australian hospital redevelopment: A protocol for a longitudinal, mixed-methods study. BMJ Open 2019, 9, e027186. [Google Scholar] [CrossRef]
  62. QSR International Pyt Ltd. NVivo (Released in March 2020). 2020. Available online: https://support.qsrinternational.com/s/ (accessed on 27 July 2023).
  63. Fetters, M.D.; Curry, L.A.; Creswell, J.W. Achieving integration in mixed methods designs-principles and practices. Health Serv. Res. 2013, 48, 2134–2156. [Google Scholar] [CrossRef] [Green Version]
  64. Curry, L.A.; Krumholz, H.M.; O’Cathain, A.; Clark, V.L.P.; Cherlin, E.; Bradley, E.H. Mixed methods in biomedical and health services research. Circ. Cardiovasc. Qual. Outcomes 2013, 6, 119–123. [Google Scholar] [CrossRef] [Green Version]
  65. Brown, K.M.; Elliott, S.J.; Leatherdale, S.T.; Robertson-Wilson, J. Searching for rigour in the reporting of mixed methods population health research: A methodological review. Health Educ. Res. 2015, 30, 811–839. [Google Scholar] [CrossRef]
  66. Nilsen, P.; Bernhardsson, S. Context matters in implementation science: A scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv. Res. 2019, 19, 189. [Google Scholar] [CrossRef] [Green Version]
  67. Chambers, D.A.; Glasgow, R.E.; Stange, K.C. The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implement. Sci. 2013, 8, 117. [Google Scholar] [CrossRef] [Green Version]
  68. Marshall, M.; Silva, D.d.; Cruickshank, L.; Shand, J.; Wei, L.; Anderson, J. What we know about designing an effective improvement intervention (but too often fail to put into practice). BMJ Qual. Saf. 2017, 26, 578–582. [Google Scholar] [CrossRef] [Green Version]
  69. Oreg, S.; Vakola, M.; Armenakis, A. Change recipients’ reactions to organisational change: A 60-year review of quantitative studies. J. Appl. Behav. Sci. 2011, 47, 461–524. [Google Scholar] [CrossRef]
  70. Damschroder, L.J.; Reardon, C.M.; Widerquist, M.A.O.; Lowery, J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement. Sci. 2022, 17, 75. [Google Scholar] [CrossRef]
  71. Harvey, G.; Kitson, A. PARIHS revisited: From heuristic to integrated framework for the successful implementation of knowledge into practice. Implement. Sci. 2016, 11, 33. [Google Scholar] [CrossRef] [Green Version]
Healthcare 11 02175 g001 550
Figure 1. Illustration of the research design—looking back and looking forward—a single-site, multiple-case study [45] involving a convergent mixed-methods approach [48].
Figure 1. Illustration of the research design—looking back and looking forward—a single-site, multiple-case study [45] involving a convergent mixed-methods approach [48].
Healthcare 11 02175 g001
Healthcare 11 02175 g002 550
Figure 2. Domains of factors for sustained improvement from hospital-wide initiatives. Adapted from Moon et al. [39].
Figure 2. Domains of factors for sustained improvement from hospital-wide initiatives. Adapted from Moon et al. [39].
Healthcare 11 02175 g002
Table
Table 1. Proposed mixed-methods data collection.
Table 1. Proposed mixed-methods data collection.
Mixed-Methods DataInitiative #1 (2014–2015)Initiative #2 (2020–Present)
Quantitative data: Routinely collected organizational data
  • National Inpatient Medication Chart Audit results on omitted medication (2012–2019)
  • Selected questionnaire items relating to patient safety from Staff Engagement Surveys (2018–latest)
  • A demographic questionnaire accompanying the interview
  • A demographic questionnaire accompanying the interview
Qualitative data: Interviews and organizational records
  • Semi-structured interview with involved staff
  • Semi-structured interview with involved staff
  • Organizational records relevant with the initiative (e.g., Report, policy)
  • Organizational records relevant with the initiative (e.g., Report, policy)
Table
Table 2. Target number of interview participants by stratified group and initiative.
Table 2. Target number of interview participants by stratified group and initiative.
Participant GroupInitiative #1Initiative #2Total
Group 1: Senior managers3–458–9
Group 2: Implementation team members3–458–9
Group 3: Frontline managers3–458–9
Group 4: Frontline staff3–458–9
Total Interview participant no.12–162032–36
Table
Table 3. Coding scheme for the interview participants.
Table 3. Coding scheme for the interview participants.
Initiative CodeParticipant Group CodeProfession Code
Initiative #1: 1Group 1: G1Nurse: NR
Initiative #2: 2Group 2: G2Doctor: DR
Group 3: G3Allied Health: AH
Group 4: G4Non-clinical professionals: NC
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Moon, S.E.J.; Hogden, A.; Eljiz, K.; Siddiqui, N. Looking Back, Looking Forward: A Study Protocol for a Mixed-Methods Multiple-Case Study to Examine Improvement Sustainability of Large-Scale Initiatives in Tertiary Hospitals. Healthcare 2023, 11, 2175. https://doi.org/10.3390/healthcare11152175

AMA Style

Moon SEJ, Hogden A, Eljiz K, Siddiqui N. Looking Back, Looking Forward: A Study Protocol for a Mixed-Methods Multiple-Case Study to Examine Improvement Sustainability of Large-Scale Initiatives in Tertiary Hospitals. Healthcare. 2023; 11(15):2175. https://doi.org/10.3390/healthcare11152175

Chicago/Turabian Style

Moon, Sarah E. J., Anne Hogden, Kathy Eljiz, and Nazlee Siddiqui. 2023. "Looking Back, Looking Forward: A Study Protocol for a Mixed-Methods Multiple-Case Study to Examine Improvement Sustainability of Large-Scale Initiatives in Tertiary Hospitals" Healthcare 11, no. 15: 2175. https://doi.org/10.3390/healthcare11152175

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop