SUMAMOS EXCELENCIA® Project: Results of the Implementation of Best Practice in a Spanish National Health System (NHS)

The use of certain strategies for the implementation of a specific recommendation yields better results in clinical practice. The aim of this study was to assess the effectiveness of an evidence-based model using clinical audits (GRIP model), for the implementation of recommendations in pain and urinary incontinence management as well as fall prevention, in the Spanish National Health System during the period 2015–2018. A quasi-experimental study has been conducted. The subjects were patients treated in hospitals, primary care units and nursing home centers. There were measures related to pain, fall prevention and urinary incontinence. Measurements were taken at baseline and at months 3, 6, 9, and 12. The sample consisted of 22,114 patients. The frequency of pain assessment increased from 59.9% in the first cycle to a mean of 71.6% in the last cycle, assessments of risk of falling increased from 56.8% to 87.8% in the last cycle; and finally, the frequency of assessments of urinary incontinence increased from a 43.4% in the first cycles to a mean of 62.2% in the last cycles. The implementation of specific evidence-based recommendations on pain, fall prevention, and urinary incontinence using a model based on clinical audits improved the frequency of assessments and their documentation.


Introduction
For more than four decades, studies have reported the existence of a large gap between research and clinical practice. This means that research results are not contributing to the improvement of care, and there is a high level of variability which complicates decision making and implementing the "best care" available. This happens despite the greater awareness of and accessibility to evidence [1,2].
This lack of consensus increases the risk of making mistakes, the inappropriate use of resources, and the continuation of practices that are ineffective or even potentially harmful [1,3]. However, the use of research is a complex process, in which the characteristics of the healthcare professionals, the organizations (context) and the innovation itself are related [3][4][5]. The GRIP model was developed by the Joanna Briggs Institute for Health Care. It is a model designed to establish interprofessional process to examine the barriers (obstacles) that hinder the use of evidence in prom tice; and to help develop actions to be implemented to overcome the Smaller-scale studies have used audit and feedback based on the GRIP results have been favorable [22][23][24].
In this context, proposals for change that require the involvement detection of difficulties in achieving change, the selection of actions to ov and the definition and monitoring of indicators to measure success, ar ments for changing practice and implementing evidence [25].
In 2012, the Nursing & Healthcare Research Unit and the Spanish Co tre of the Joanna Briggs Institute created a national network of centers ta menting Clinical Practice Guidelines (CPG) in healthcare. This project ha plementation of evidence-based recommendations from CPG improves p A project was started, as a way of continuing the research program being pursued by both the above-mentioned centers, in which nursing staff and other health professionals could implement specific evidence-based recommendations, instead of implementing full CPGs; this project is Sumamos Excelencia® [29].
Pain, falls and urinary incontinence (UI) are health problems that are highly prevalent in the general population, especially in individuals over 65 years of age, and there is strong evidence to support their requirement for nursing care. However, these health problems are scarcely assessed, managed, and monitored in current clinical practice [30,31] According to International Association for the Study of Pain, pain is defined as "An unpleasant sensory and emotional experience associated with, or resembling that associated with, actual or potential tissue damage". Pain has a profound impact on the quality of life and may have physical, psychological and social consequences. It has globally been estimated that one in five adults suffer from pain. Moreover, this prevalence increases in the institutionalized population subjected to painful procedures [32]. According to the US Joint Commission's standards of care, all patients should be evaluated, treated and monitored of their pain level. Even so, in current clinical practice assessment of pain is infrequent, as is its management and monitoring [31,33].
Urinary incontinence (UI) is the involuntary loss of a person's bladder control at an inappropriate time and place, and it is known to cause in the person hygienic, social and psychological difficulties [34]. Urinary incontinence is one of the biggest problems affecting older people and is of great significance in public health [35]. According to the majority of studies, the estimated prevalence of urinary incontinence ranges from 23% to 44%. According to Moehrer [36], approximately 20% of women aged 25-64 years experience some UI-related symptom.
Depending on its form of presentation, UI can be urge incontinence, when there is an urgent sensation of bladder emptying that is difficult to postpone; stress incontinence, when the loss coincides with an increase in pressure in the intra-abdominal cavity; and mixed incontinence, in which there is a combination of the two [37].
On the other hand, falls are one of the principal cause of preventable adverse events in health institutions, and are the second leading cause of death due to accidental or unintentional injuries worldwide [38]. It is established that each year one third of the population over the age of 65 years, experience a fall, a proportion that rises among institutionalized patients [39], and most fatal falls are suffered by people over 65. Approximately, 28-35% of elderlies aged 65, and older, fall every year, and that number increases to 32-42% for people over 70. Organizations should evaluate their patients' risk of falling, in the context of the attended population, care should be offered and they should take measures to reduce the risk of falls and fall-induced injuries [33].
The purpose of this study is to assess a model for the implementation of specific evidence-based recommendations, as measured using clinical audits, that will enable a better assessment of patients and a reduction of the practice variation in decision making. We also aim to determine the baseline degree of compliance with recommendations in the participating units and nursing home centers; to analyze the initial and final levels of compliance with the best practice criteria; to analyze the implementation barriers detected in the participating units and nursing home centers, and the actions to be taken to overcome them.

Hypothesis
The implementation of evidence-based recommendations using the GRIP model improves the assessment and management of pain, urinary incontinence, and fall prevention in patients [22][23][24].

Design
This is a quasi-experimental multi-centered study. The intervention consisted in the application of recommendations in the chosen issue by each unit, with outcome measurements taken at baseline and, subsequently, 3, 6, 9, and 12 months after the baseline measurement was taken.

Setting and Participants
The study was conducted in healthcare units and nursing home centers of the Spanish National Health System (SNHS) between 2015 and 2018. Patients seen in SNHS units (primary care, hospitals) and nursing home centers. In Spain, the nursing home centers are institutions, where care is provided in form of personal assistance in daily living, as well as nursing and medical care, for older people who live there for an undefined period of time.

Inclusion Criteria for the Units
Units that committed themselves to implementing recommendations regarding one of the following issues: pain, falls, or urinary incontinence. For the purposes of this study, "units" were considered to be those services (primary care), centers (hospitals), or institutions (nursing homes) providing healthcare services to patients who share similar characteristics.

Inclusion Criteria for Patients
Patients seen in the participating units. Pain: individuals admitted to hospitals, primary care, or nursing home institutions who could potentially suffer from some type of pain. Falls: individuals admitted to hospitals, primary care, or nursing home institutions, age of 65 or over, presenting one or more risk factors for falls according to the assessment criteria established in the risk measurement instruments used. Urinary incontinence: individuals residing in the community or institutionalized individuals who are susceptible to urinary incontinence.

Recruitment of Study Subjects and Sample Size
Units: intentional non-probabilistic sampling. A participation process was launched and published through the channels provided by the Nursing and Healthcare Research Unit (Investen-isciii) and the Spanish Center for Evidence-Based Nursing and Healthcare: which is a JBI Centre of Excellence. We selected the units that met the inclusion criteria and agreed with the project requirements. It was estimated that 100 units of each group of recommendations should be included, which would account a total of 300 units.
All patients who met the inclusion criteria when data were collected from their unit were included. All data were collected during the last five working days of each evaluation quarter, to homogenize the collection periods. Data collected from different types of units were as follows: Primary care units: patients seen in consultations during the last five working days of each assessment trimester. Hospital units: patients discharged in the last five working days of each assessment trimester. Nursing home centers: patients admitted in the last five working days of each assessment trimester.

Intervention
The intervention consisted of the implementation of specific evidence-based recommendations regarding the assessment and management of pain, the detection and management of urinary incontinence, and the fall prevention through assessment and management of the risk of falling (Supplementary Material File S1). This was done using a multi-component strategy that included: training, facilitators, auditing and feedback.
Phase 0: Recruitment. In this phase, the units were recruited through a public call for participation. Phase 1: Preparation. This phase was guided by external and internal facilitators. The external facilitators were the project researchers who provided training and methodological and logistical support to the participating units. The internal facilitators (researchers responsible) were nursing professionals from the participating units who were identified as project leaders within each unit. They received, along with the other professionals from the participating unit, an intensive online training program addressing the following: knowledge of evidence-based practice, evidence-based recommendations on the topic to be implemented, process and outcome indicators, and management of clinical audits. The training program lasted two months and ended with a knowledge assessment test. Internal facilitators were trained in the use of the online data collection platform to ensure consistent data collection. Phase 2: Implementation. The implementation phase was based on reports from the participating units about the baseline assessment of compliance to evidence-based recommendations specific to the health problem to be addressed. An analysis of the situation: identification of barriers to achieving compliance, identification of actions to address these barriers, and resources associated with each action of improvement. Incorporation of recommendations into clinical practice in as many as permitted from all the barriers detected. An online data collection platform was created for the project, which generated reports with the data on compliance, as well as recommendations, an action plan with the barriers identified, the actions to be implemented, the responsible parties, the necessary resources, and a date. Phase 3: Follow-up. After a baseline assessment, the process was monitored with measurements at month 3, 6, 9, and 12 while continuing to identify barriers and establish actions for improvement.

Measurements
The data related to the indicators regarding the assessment, detection and management of pain, fall prevention and urinary incontinence was collected. All indicators were selected based on CPGs and relevant scientific evidence [40,41]. In Spain there were no updated guidelines in the context of nursing care. The content of these guides has been translated, adapted and used previously in projects for the implementation of evidence in the Spanish healthcare context [27,42]. The indicators and variables used can be consulted in the study protocol [29] (Supplementary Material File S1). The primary outcomes were the assessment in pain, fall risk and urinary incontinence, the second outcomes were the implementation of care plan and education. All the units that completed the monitoring for at least nine months were considered for analysis. An initial audit of all the indicators was conducted to determine the degree of compliance to the evidence-based recommendations included in the project. Subsequently, a follow-up was conducted every three months (at month 3, 6, 9, and 12) to reassess compliance with all the indicators measured in the initial audit (April-September). Patient and indicator data were collected by the responsible researchers (internal facilitators) for each unit. They were collected from the medical and nursing records and entered into the platform created for the project.

Data Analysis
Initially, we calculated the percentages (with a 95% confidence interval) of compliance for each item (assessment, care plan, patient education, comprehensive assessment and restraining measures) and for each health problem assessed (pain, fall prevention, and urinary incontinence). Subsequently, due the dichotomic nature of the outcomes, we analyzed the changes between the cycles using a generalized logistic mixed methods model including units nested within hospitals as a random factor and patients' sex and age as covariates. Finally, in order to assess the impact of the intervention, we calculated the percentages of compliance in each unit and the estimated average change observed from baseline cycles (cycles 0 and 1) to post-implementation cycles (cycles 4 and 5) using a mixed methods model to account for dependency between units within the same hospital.
Analyses were performed using the Lme4 packages (Bates et al., 2015) [43] and LmerTest (Alexandra Kuznetsova et al. 2020) [44] in the R programming language (Pinheiro, Bates, DebRoy, Sarkar, and R Core Team 2015) [45]. The statistical significance threshold for the results was set at p < 0.05, with a 95% confidence interval for all cases.

Description of the Sample
A total of 288 units responded to the recruitment process, of which a total of 135 finally participated. Figure 2 is a flow chart of the recruitment process.

Description of the Sample
A total of 288 units responded to the recruitment process, of which a total finally participated. Figure 2 is a flow chart of the recruitment process. A total of 22,114 patients were analyzed, whose distribution and characteris reflected in Table 1. A total of 22,114 patients were analyzed, whose distribution and characteristics are reflected in Table 1. In the case of pain assessment, the final sample consisted of 10,192 patients. We found that the frequency of pain assessment increased from 59.9% in the first cycle to a mean of 71.6% in the last cycle (Z = 15.88; p < 0.001) (Tables 2 and 3). We found no differences in pain assessment in relation to patients' sex (Z = 1.00; p = 0.318) or age (Z = 0.63; p = 0.531) ( Table 2).   In the case of risk of falling, the final sample was 7782 patients. We found an increase in the frequency of assessments of risk of falling across study cycles, from 56.8% in the first cycle to a mean of 87.8% in the last cycle (Z = 20.49; p < 0.001) (Tables 2 and 3).
We also found a positive effect of the age factor, since older patients were more frequently assessed than younger patients (Z = 6.58; p < 0.001). We did not find sex differences in fall risk assessment (Z = 0.40; p = 0.692) ( Table 2).
Finally, in the case of urinary incontinence, the final sample was 2492. We found an increase on the frequency of urinary incontinence assessments across study cycles from a 43.4% in the first cycles to a mean of 62.2% in the last cycles (Z = 10.87; p < 0.001) (Tables 2 and 3). We also found a positive effect of the age on the assessments: older patients were more frequently assessed than younger patients (Z = 7.16; p < 0.001). We did not find sex differences in urinary incontinence assessments (Z = −0.10; p = 0.917) ( Table 2).

Secondary Outcome: Efficacy of the Implementation Model in the Provision of Care Plans and Patient Education
Regarding pain secondary outcomes, we also found increases in care plan implementation (Z = 16.53; p < 0.001), frequency of patient education (Z = 25.30; p < 0.001), and comprehensive pain assessment (Z = 15.99; p < 0.001) across study cycles (Table 3). In the case of risk of falling, as well as we found an increase in the implementation of the care plan (Z = 11.52; p < 0.001) and restraining measures (Z = 4.407; p < 0.001) across study cycles. We found a positive effect of age regarding restraining measures (Z = 16.41; p < 0.001) and a marginal non-significant trend on the implementation of the care plans (Z = 1.94; p = 0.052). Finally, in the case of urinary incontinence, we found as well an increase in the implementation of care plans (Z = 4.72; p < 0.001) and in the frequency of patient education (Z = 6.44; p < 0.001) across study cycles.

Impact of the Intervention
In order to assess the impact of the intervention, we aggregated data from baseline cycles (0 and 1) and post-intervention cycles (4 and 5) and calculated the mean change and Cohen's effect size (d). We found medium to large changes in all evaluated outcomes except in the implementation of restraining measures, with the largest changes occurring in the field of pain. The intervention caused increases of up to 30-40% (e.g., in the development of patient education), and over 15-20% in patient evaluation in all fields.
In the case of pain, we found a mean increase of 18.

Barriers Detected during Measurement Cycles and Planned Actions
Forty-six units (34%) reported barriers, with a total of 218 barriers and 218 actions. The most reported barrier was the lack of recording tools for the clinical records (30.3%, n = 66), followed by the lack of action procedures (25.2%, n = 55) and the incompletion of records (19.7%, n = 43). Regarding the reported actions, training of professionals (52.3%, n = 114) was the most implemented, followed by the creation of computer tools for the registers (39%, n = 85). The frequencies of the barriers and actions implemented by the units are shown in Table 4.

Discussion
SUMAMOS EXCELENCIA ® is a Spanish national project that aims to create a network for the translation of evidence into practice through specific recommendations from CPGs.
In this project, a model for implementing evidence-based recommendations (Getting Research into Practice-GRIP) has been evaluated. To this end, we used a multi-component strategy that integrates educational elements (training of professionals), the figure of facilitators (both internal and external), the performance of audits and the provision of feedback. This model has previously been used in other studies with a smaller number of units [23,24,46,47]. In this first cohort, a total of 135 units participated and a total of 22,114 patients were analyzed. This has meant the involvement in the project of more than 500 nursing professionals from all over the SNHS.

Baseline Situation of the Degree of Compliance with the Recommendations in the Participating Units and Final Levels of Compliance to the Best Practice Criteria
At baseline, the participating units complied with recommendations in the 59.93%, 56.8%, and 43.4% of the patients for pain assessment, risk of falling, and detection of urinary incontinence, respectively. With regard to pain assessment, the study by Stevens et al. reported assessment percentages of around 67-69% [48]. Perhaps this difference with respect to our study may be explained by the pediatric setting of their research, where there is greater awareness of pain assessment. The study by Daniels reported baseline frequencies of 56%, and another similar study, conducted in the context of cancer pain with the same implementation model, reported a baseline assessment rate below 50% [46]. This indicates that, despite the Joint Commission's consideration of pain assessment as the fifth vital sign, there is still an under-assessment of pain in clinical practice [33].
In regard to the risk of falling, a study conducted with the same implementation model, reported baseline frequencies in the assessment of the risk of falling between 90-93%, depending on the type of unit [24]. Similarly, a study by Johnson et al. indicated a baseline assessment of 95% [49]. In SUMAMOS EXCELENCIA ® we started from lower baseline levels (56.8%). This may be due to the fact that the project includes different types of units (medical, surgical, medical-surgical) and different types of centers (hospitals, primary care centers, and nursing home centers), while the rest of the studies focused on more homogeneous centers, which may decrease variability. Lastly, regarding to the assessment of urinary incontinence, a study using the same implementation model reported baseline data of 100% compliance in assessing urinary incontinence [23]. Perhaps because it was carried out in a medium-to long-term hospital for patients over 65, where the assessment criteria for urinary incontinence are more present, given that it is a highly prevalent problem in individuals over 65.
Concerning the final levels of compliance to the evidence-based recommendations and taking into account the studies mentioned previously, in SUMAMOS EXCELENCIA ® the assessment of pain has increased by 11.7%. Similar studies report increases ranging from 13% in Daniels et al., 25% in Stevens et al., 33% in Ang & Chow and to 43% in Dulko et al. [22,46,48,50]. This variability in final compliance may be conditioned by the baseline data used, which were lower in the study reporting the greatest increase [50]. The results of the study by Daniels are the closest to our results, with similar baseline data. The increase in compliance was between 13 and 43%. This 30% range coincides, although with a more optimistic interval, with that reported by Lau et al., who pointed out that only a third of the evidence containing recommendations was routinely followed, with compliance to recommendations ranging from 20 to 80% [51].
Regarding the assessment of the risk of falling, which was increased by 30.7% in our study, the rest of the studies showed a smaller increase, between 3% and 10% in the case of Szymaniak [24], and even a decrease of 1% in the case of Johnson et al. [49].
Finally, concerning the detection of urinary incontinence, the increase in compliance in our study was 18.8%. The only study available which was similar to our intervention reported no increase in compliance, since it started from a baseline value of 100% [23].

Assessment Indicators, Barriers and Actions in the Implementation Process
This project is based on a model that integrates simple and specific indicators. The homogenization of indicators implies a decrease in inter-professional variability; moreover, benchmarking will allow professionals to measure results and evaluate the quality of the care provided [52]. These indicators were used in other implementation studies based on the GRIP model [22][23][24].
On the other hand, previous studies recommend "mapping the territory" before starting an implementation project and developing specific actions to identify barriers, thus reducing their impact. In the project SUMAMOS EXCELENCIA ® , barriers were identified, among which the lack of specific registration tools in clinical records and the lack of training in professionals stand out. As a result, the majority of the actions undertaken were focused on these aspects. This is in line with other studies that highlight professional training as one of the barriers detected and which design educational interventions to implement the evidence available [13,[22][23][24]. Identifying barriers and taking action enables improvements to be made in the current knowledge of the causal mechanisms behind the success or failure of an intervention [1,4]. Indeed, a 2010 Cochrane systematic review recommends describing explicitly how to identify these barriers and propose actions to overcome them [53].

Facilitators in Implementation Projects
The figure of the facilitator is a contextual element included in the PARiHS model [54]. This figure has been used in other studies, reporting that facilitation based on the facilitator's role and the relationship established with the individuals, may affect the changes [22][23][24]48].
This project has included both internal facilitators (well-trained staff of the participating units), who prepare, guide, and support the members of their team during the implementation process, and external facilitators professionals from the research team with experience in projects implementing evidence into clinical practice [26,27]. External facilitators were responsible for training the internal facilitators and monitoring their performance throughout the project.

Clinical Audits and Feedback as Implementation Strategies
The GRIP model used in this project is based on a process of improvement in response to a previous baseline clinical audit. Interventions using educational activities and audits are the most widely used, although the effectiveness of both remains unclear [55]. Audits and feedback have been shown to change behaviors in physicians, and it is recommended that their effectiveness in the field of nursing should be studied [7,56]. Within the field of nursing, there are studies that have used auditing and feedback based on the GRIP model on a smaller scale, with favorable results [22][23][24]. For the audit and feedback process to be effective, it is important that the professionals involved understand the purpose of the audit [19,55]. The project SUMAMOS EXCELENCIA ® provides a pre-training stage where internal facilitators and unit professionals receive online training in clinical auditing and evidence-based practice. This enables professionals to understand the purpose of the audit and how to manage it. Each unit adapted the project to its local circumstances, allowing professionals to perceive the opportunity for change provided by the project and thus adjust it to the needs of each unit.

Limitations
The conclusions of this study may be limited by the biases inherent in a non-controlled and non-randomized design. The data should therefore be interpreted with this in mind. The necessary sample size was estimated to be 100 units for each health problem; in the case of pain, 63% of the sample size was reached, with 55% for falls and 17% for urinary incontinence. It is true that the calculation was an estimate without reference data, which is a limitation of our study. The low response rate in the area of urinary incontinence may be related to two elements: on the one hand, the low priority given to the treatment of urinary incontinence, considering it as part of ageing and difficult to manage; on the other hand, the fact that the participating units were asked to choose only one of the three areas (pain, urinary incontinence or falls) which may have influenced the choice of the other areas in detriment of incontinence [57,58].
There were contextual elements that may have influenced the improvement described, one of them being the monitoring process inherent in the audit procedure, which makes the participants aware of the process and may lead them to be affected by the Hawthorne effect. However, with a follow-up of 12 months, this effect is estimated to be minimal.

Conclusions
This study shows that the implementation of specific evidence-based recommendations on assessment pain, fall prevention, and urinary incontinence, using a model based on clinical audits, improves the frequency of assessments and their documentation.
It also demonstrates that an easy-to-apply model, based on clinical audits, contributes to improved pain assessment and management, fall prevention and detection of urinary incontinence.
In the project SUMAMOS EXCELENCIA ® , barriers have been identified, among which the lack of specific registration tools in clinical records and the lack of training in professionals stand out. As a result, the majority of the actions undertaken were focused on these aspects.
For this project we used a multi-component strategy that integrates educational elements (training of professionals), the figure of facilitators (both internal and external), the performance of audits and the provision of feedback. These components are key elements of evidence implementation strategies.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy/ethical restrictions.