The aim of this study was to report on the implementation outcomes of the BPSO program at two BPSO pre-designate sites within a complex health service context and to illustrate how implementation affected service outcomes using examples from these sites. The first three years of a BPSO program at a site presented a unique opportunity to observe factors that affected the implementation of the program and allowed for identification of those factors critical to its success. This is important when considering translocation of programs to new settings [27
]. This study was the first to apply an implementation evaluation framework to an EBP program, and complements other similar research focused on identifying factors salient to EBP implementation and sustainability [10
4.1. Facilitators and Barriers to Implementation and Impact on Service Delivery
Proctor’s framework allowed for structured evaluation of the BPSO program implementation performance with respect to identifying facilitators and barriers to implementation. Both sites considered broader strategic initiatives being progressed within their site, which was a strategically wise decision when selecting the three BPGs as part of their pre-designate period. For example, the SA Health suicide prevention plan, published by the Office of the Chief Psychiatrist, was closely aligned to activities undertaken within Site 2 (e.g., connecting with people training). This facilitated many of the implementation outcome domains including acceptability, adoption, feasibility, and sustainability. However, this also made it difficult to directly ascribe all activities to initiatives being progressed under the banner of the BPSO program. This was noted for other BPGs, and potentially, exemplifies that alignment to broader initiatives is required, especially when attempting systemic change and/or a difficult BPG (e.g., Woman’s abuse).
Partnerships between units were also critical to success. For example, at Site 1 the partnership of the site BPSO program lead unit (the clinical practice development unit) with the consumer and community engagement unit was critical. The BPSO program’s service delivery was greatly augmented through this partnership with many people commenting on the value of having consumer presence and voice at training. In addition, many of their BPSO activities were directly aligned with existing works of the Consumer and Community Engagement Unit, which reduced duplication of effort.
Extensive documentation of organizational level activities related to the BPSO program and changes helped to support the demonstration of the program’s impact, as well as efficiently building in program accountability and recognition. These included activity reporting, evaluation of Champion’s training, consumer and community engagement strategies, practice level activities, and changes and awards that have occurred against each BPG.
The organizational commitment at the site, and the commitment of the external parties—in this instance ANMF (SA Branch), SA Health, and the RNAO—who each had roles to play to support the site, was critical to implementation success. Senior stakeholders interviewed commented that, given the numerous pressures a large healthcare providers faces in Australia and therefore the potential risk the organization is taking when committing to an organization-wide clinical quality improvement program like the BPSO program, understanding and communicating the benefits such a program would bring to the organization, and reinforcing this with evidence as the program rolled-out was necessary to maintain this commitment. Financing the BPSO program also supported this commitment. The financing arrangement in this instance was a three-way commitment between the three South Australian parties, and so there was internal and external interest in its success. In this sense, the funds acted more than just as money to pay for people and activities. It gave the program a budget line in the organization, it represented a financial risk to multiple parties, and combined gave the program greater substance.
Champions training and the Best Practice Champions Network were vital elements in the uptake and embedding of EBP across organizations. While nurses may have positive attitudes and beliefs toward EBP, their beliefs are associated with the extent to which EBP is implemented [28
]. The BPSO Program received many positive comments from nurses on its value, adaptability, and relevance to their organization and the South Australian healthcare context.
Implementation barriers were not always the antithesis of facilitators. While it was observed that factors critical to implementation success were also barriers when not in place (e.g., executive stability), there were also other barriers identified that presented separate challenges. As noted, there was resistance amongst clinicians to the introduction of the Woman’s Abuse BPG at Site 1. This was in part due to uncertainties regarding screening and responsibilities of clinicians around this sensitive area. Consequently, it took this BPG considerably longer to bring to a mature training and practice improvement program compared with the two other BPGs adopted by Site 1.
4.2. Implications for Clinical Practice
The results of this study provide program and service-level considerations related to BPG and EBP program implementation, as well as sustained practice change. First, this study reinforces previous research that there is no single solution to the application of EBP in healthcare [29
], but that theoretically driven, structured programs, like the BPSO program, facilitate this process [30
]. This study identified that strategic alignment of BPGs to service initiatives, embedding of the program that supports their implementation, and minimizing duplication of effort are key to the long-term sustainability and maintenance of EBP. The study showed that positive workplace environments influence uptake and application of BPG in the provision of safe, quality care, similar to findings from previous research [31
]. Clinically focused BPGs are less likely to succeed in the long term without the organizational culture and processes to support them. For example, governance of BPGs being embedded in established clinical and corporate procedures, network-wide audits in collaboration with quality and safety representatives, consumer engagement activities, maintaining BPSO and BPG communication, and embedding the BPSO program into staff orientation and other courses.
Second, the study identified that the BPSO program was more than just a ‘vehicle’ to implement BPGs; its structure was acting as an ‘anchor’ for establishing and sustaining EBP change in the healthcare setting. In a stable, more positive workplace environment (e.g., Site 1), this assisted with supporting continual quality improvement and provided a vehicle for sites to improve organization-wide policy and evidence-based practice. In an organizational environment undergoing continual, widespread, and/or significant reform (e.g., Site 2), the program acted as a stabilizing function to maintain course direction towards continual quality improvement of safe and quality care. This is not to be underestimated when there may be changes in one or more levels of leadership, budget cuts, or changes in the political landscape that may result in decisions that impact on practice improvement programs.
Third, this study demonstrated that non-discipline specific change to improve the safety and quality of healthcare could be implemented irrespective of which discipline is nominally ‘leading’ it. As the single largest providers of health care in Australia, nurses have the potential to significantly influence and address gaps in EBP. While some have questioned the need for discipline-specific guidelines for EBP, it is the responsibility of members across all disciplines to be informed by best available research and some of this is discipline-specific [32
]. However, challenges do exist when moving EBP across discipline boundaries, which as evidenced in this study, require other facilitating factors to be present in the environment in order to succeed (e.g., executive sponsorship, strategic alignment to policy, and time for discussion and negotiation). This is important when health services are planning large change; careful planning before acting, negotiation with and on-boarding of all stakeholders, especially those key influencers, can save significant time and organizational disruption during the ‘roll-out’ phase.
Fourth, the study highlighted the need for measurement and reporting as part of program success. Implementation of BPGs is a legal directive as well as a professional requirement in Canada [33
] and measurement and reporting are embedded as part of the program through a standardized database [30
]. As part of this study, interviewees responsible for monitoring and reporting the BPSO program noted that for each BPG well-defined and sensical (to the healthcare system) KPIs facilitated implementation and subsequent sustainability. However, when such KPIs were lacking, it presented a challenge for BPG implementation. In Australia, the development of the NSQHS standards has given rise to a number of developments to address the lack of reliable data for monitoring the quality of care and patient safety [34
]. This presents an opportunity to closely align BPG KPIs to NSQHS standards, given that many of which are nurse sensitive. However, nurse sensitive indicators, while increasingly accepted that they assess nursing and midwifery’s impact on patients’ safety and health status [35
], are not standardized and can vary in definition and measurement between organizations. The BPSO program provided a structured framework for promoting nurse or midwife-led EBP, their impact on patient safety and quality of care, and aligning BPG monitoring across organizations to nurse sensitive indicators, which served a dual purpose of improving care and satisfying NSQHS reporting requirements.
4.3. Implications for Implementation Evaluation
This study also served to demonstrate the practical application of an implementation framework to guide the structuring of the data analysis and reporting. Use of Proctor’s framework was important for aligning service outcomes to indices meaningful to healthcare services, especially as it relates to hospital accreditation safety and quality standards. This top-down approach focused the data analysis process on extracting only those bits of information relevant to the pre-defined domains. In an environment that is awash with data (i.e., the health system), this expedited the data extraction and reporting process, which may serve others interested in reviewing programs in their healthcare settings.
The IOM standards are quite broad terms, and are agnostic to the health system, so are useful outcomes to report against. However, it may be more suitable to use system context-specific measure of quality and safety to demonstrate benefit locally (e.g., using the Australian NSQHS standards to align with accreditation).
4.4. Study Limitations
The main limitation to the study was the limited amount of raw, quantitative data available to complement other data sources, as part of assessing the implementation outcomes and service outcomes. For example, it was not possible to extract firm evidence regarding impact on health service efficiency. While this is not a priority of the BPSO program (i.e., to have an explicit strategy to improve system efficiency), the cost of healthcare delivery is an important component to capture, especially if planning to affect large-scale change. Similar rationale could be argued for the other service outcome domains.
Greater emphasis was placed on qualitative data sourced from staff perceptions, perspectives, and experiences with respect to the implementation of the BPSO program, and how it affected the healthcare delivery. This was in part due to the nature of the methodological design and analysis, and the inherent limitations of having a small number of questionnaire respondents meaning less exploratory analysis were possible (e.g., meaningful assessment of work environment outcomes between those aware of vs. not aware of the BPSO program).
Response bias was a risk given the stake or interest many of the participants had, especially those who were interviewed or part of the focus group. The questionnaire assisted here by gathering feedback from people with very little awareness of or interest in the program. However, the questionnaire results should not be viewed as a generalizable representation of each site, and likely has other biases.