Next Article in Journal
Changes in the Impacts of Topographic Factors, Soil Texture, and Cropping Systems on Topsoil Chemical Properties in the Mountainous Areas of the Subtropical Monsoon Region from 2007 to 2017: A Case Study in Hefeng, China
Next Article in Special Issue
Mapping Research Conducted on Long-Term Care Facilities for Older People in Brazil: A Scoping Review
Previous Article in Journal
The Influence of Child-Related Factors on Caregiver Perceptions of Their Child’s Sustained Participation in a Community Football Program: A Study of Children with and without Neurodevelopmental Disorders
Previous Article in Special Issue
Personal and Work-Related Factors Associated with Good Care for Institutionalized Older Adults
 
 
Due to planned maintenance work on our platforms, there might be short service disruptions on Saturday, December 3rd, between 15:00 and 16:00 (CET).
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Pilot Study to Develop and Test Palliative Care Quality Indicators for Nursing Homes

1
VUB-UGhent End-of-life Care Research Group, Vrije Universiteit Brussel (VUB), 1090 Brussel, Belgium
2
Department of Family Medicine and Chronic Care, Vrije Universiteit Brussel (VUB), 1090 Brussel, Belgium
3
Department of Radiotherapy, University Hospital Brussels, 1090 Brussel, Belgium
4
Department of Public Health and Primary Care, Ghent University, 9000 Ghent, Belgium
*
Authors to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(2), 829; https://doi.org/10.3390/ijerph18020829
Received: 16 December 2020 / Revised: 15 January 2021 / Accepted: 15 January 2021 / Published: 19 January 2021
(This article belongs to the Special Issue Enhancing the Quality of Care in Long-Term Care Settings)

Abstract

:
An increasingly frail population in nursing homes accentuates the need for high quality care at the end of life and better access to palliative care in this context. Implementation of palliative care and its outcomes can be monitored by using quality indicators. Therefore, we developed a quality indicator set for palliative care in nursing homes and a tailored measurement procedure while using a mixed-methods design. We developed the instrument in three phases: (1) literature search, (2) interviews with experts, and (3) indicator and measurement selection by expert consensus (RAND/UCLA). Second, we pilot tested and evaluated the instrument in nine nursing homes in Flanders, Belgium. After identifying 26 indicators in the literature and expert interviews, 19 of them were selected through expert consensus. Setting-specific themes were advance care planning, autonomy, and communication with family. The quantitative and qualitative analyses showed that the indicators were measurable, had good preliminary face validity and discriminative power, and were considered to be useful in terms of quality monitoring according to the caregivers. The quality indicators can be used in a large implementation study and process evaluation in order to achieve continuous monitoring of the access to palliative care for all of the residents in nursing homes.

1. Background

In the past decade, in many Western countries, an increasing number of elderly persons were admitted to nursing homes. Projection studies concerning numbers of deaths and place of death suggest that, by the year 2040, the majority of deaths will occur in nursing homes [1,2]. Moreover, prediction studies also indicate that the need for high quality care at the end of life will most likely double in the nursing home setting, because of an increasing prevalence of frailty and multimorbidity in the resident population [2,3], being linked to a strongly reduced average length of stay of residents in recent years [4]. The quality of care at the end of life in long-term care facilities is currently high on the agenda: WHO and other health care organizations have advocated for good palliative care for older people already for years [5,6]. Additionally, in research, emphasis in the past decade has been placed on the quality of advance care planning, autonomy of residents, and the implementation of palliative care in nursing homes [7,8]. Until now, palliative care has been insufficiently developed in nursing homes and international studies show the late initiation of palliative care and even mostly for residents with a cancer diagnosis. An urgent need arises for better access for all elderly persons to high quality palliative care provision in the nursing home context.
When implementing palliative care in a nursing home context, it is important to evaluate its success and measure its outcomes for residents. One way to do so is the use of quality indicators within a continuous cycle of implementing, monitoring, and improvement [9]. By continuously monitoring quality of care and its outcomes and conducting implementation trajectories that are based on the results of these measurements, care teams are able to optimize the quality of care based on information, patient experiences, and best practice examples [10,11,12,13]. Quality indicators can be used within this monitoring cycle to provide data on subjective and objective aspects of quality of care over time. They are defined as measurable aspects of care, calculated as a percentage with a predefined numerator and denominator [14,15]. These indicators give caregivers information on their performance in terms of care processes and outcomes and which elements of care may need improvement [16]. Several national health care monitoring programs have been started in Western Countries, including Belgium, based on quality indicators. However, they mainly focus on the hospital or home setting or circumstances surrounding death such as symptoms and place of death [17,18,19,20,21,22,23,24,25]. Although initiatives have been taken for improving palliative care in nursing homes [24,26,27], researchers were not yet able to validate and implement solid quality indicators for palliative care in this specific setting.
A previous program to develop and implement palliative care quality indicators, the Belgian Q-PAC study, used a rigorous development method combining literature review, expert consultation, and pilot testing, resulting in a core set of 31 quality indicators covering a broad range of aspects of palliative care. The set was meant to be used for all palliative care services and settings, including nursing homes [24,28,29]. However, because the nursing home setting appeared to be too different from the specialized palliative care services in terms of organization and structure of care (e.g., no dedicated palliative care teams), and in characteristics of the population cared for (e.g., specific population with frailty, dementia, cognitive decline), the quality indicator set was implemented into all specialized palliative care services, but not in the nursing homes from 2014 onwards [30]. Therefore, a need persisted to investigate which indicators can be used for monitoring the quality of palliative care in the nursing home context.
Because of the increasing need for development of palliative care and its monitoring in nursing homes, we started a project to develop a set of quality indicators for the quality of palliative care in nursing homes. Previous research already highlighted the importance of person centered care through autonomy and involvement of family, but also communication and advance care planning in nursing homes [31,32,33,34]; hence, we decided to develop quality indicators specifically targeting advance care planning, palliative care, and end of life care. The main aim for nursing home teams is to obtain insights in their care processes and outcomes, and further develop missing elements in the care for their residents. In this study, we develop and test a quality indicator set and measurement procedure for palliative care in nursing home context.

2. Methods

2.1. Design

We used a two-step approach with a mixed-method design based on a standardized indicator testing protocol for generic quality indicators in order to develop the quality indicator set and measurement procedure [29,35]. First, we developed the set of quality indicators using the Rand/UCLA appropriateness method in three phases: (1) literature review to develop a preliminary set of quality indicators, (2) interviews with experts to test face validity of the preliminary set of quality indicators, and (3) indicator selection by expert evaluation [36]. Second, we evaluated the face validity, feasibility, discriminative power, and usefulness of the quality indicators in a quantitative cross-sectional application of the quality indicators in combination with qualitative interviews. Figure 1 shows an overview of the development process and pilot testing.

2.2. Step 1: Indicator and Questionnaire Development

2.2.1. Phase 1 and 2: Literature Study and Expert Interviews

In phase 1, we identified a comprehensive set of candidate quality indicators for palliative care in nursing homes. Therefore, we adapted the existing Belgium quality indicators for specialized palliative care listed in the Q-PAC (Quality indicators for Palliative Care) study [24] to the nursing home context. To do so, we searched literature in PubMed while using a snowball method starting with the reviews of Pasman et al. and De Roo et al. [37,38]. We searched for existing quality indicators, domains of quality of care for elderly and questionnaires or instruments for quality of palliative care, advance care planning, and end-of-life care in the nursing home context. Candidate quality indicators could be processed or outcome indicators and emphasis were placed particularly on subjective quality indicators in order to reflect the user perspective on quality of care. In phase 2, we performed interviews with relevant stakeholders (i.e., healthcare professionals, community-based organizations, and policy makers) that are involved in Flemish nursing home care, to test face the validity of the candidate quality indicators and gather additional indicators and domains for this specific context not found in literature. This way, an iterative process of literature search and interviews lead to the selection of a preliminary set of quality indicators representing all of the identified domains. Furthermore, to be able to calculate the quality indicators, we operationalized each of them into questions for residents, bereaved family and nursing home staff, accompanied with measurement instructions. This was done based on input that was provided by the experts and questionnaires identified in the literature.

2.2.2. Phase 3: Expert Consensus

The preliminary set of candidate quality indicators was sent to 15 experts (see results infra). They were asked to score the quality indicators with “1” as “not appropriate” to “9” as “very appropriate” in order to measure the quality of palliative care in nursing homes. Experts were provided with the candidate indicator’s description, rationale, numerator and denominator, question (per response type: residents, bereaved family, or nursing home staff) and literature source. They were also able to suggest missing domains or themes.
The median scores on appropriateness were calculated per candidate indicator. The quality indicators were then categorized based on the RAND/UCLA consensus method: accepted, to be decided, or rejected. Indicators were immediately accepted if they had a median score of 7 or more and if no more than two experts scored the indicator with a 1, 2, or 3 (strict positive consensus). Indicators with a median score of 3 or lower and for which no more than two experts scored the indicator 7, 8, or 9 were immediately rejected (strict negative consensus) [36]. All other to-be-decided indicators were discussed during the one-day plenary discussion until consensus was found regarding rejection or acceptance among the experts. During the discussion, additional selection criteria were to be considered as defined by the researchers: (1) a maximum of eight quality indicators per questionnaire (i.e., response type) was suggested to ensure feasibility of the quality monitoring in nursing homes, without overburdening nursing home staff, and (2) experts were encouraged to consider a good balance between process and outcome indicators, as well as objective and subjective quality indicators. As such, experts were asked to select the eight most important indicators for each response type.

2.2.3. Questionnaires to Measure the Quality Indicators

After defining the quality indicators together with the experts, we developed four questionnaires to be able to calculate the performance score per quality indicator. These questionnaires were based on questions of validated scales as much as possible, or if no good question gathering the right information for a specific indicator existed in the literature, it was developed by the researchers, together with the experts. An overview of all indicators, accompanying questions and evidence can be found in the Supplementary Table S1.
Two questionnaires were developed to measure indicators of quality of care for residents who currently lived in the facility: one for the resident [1] and one for the most involved professional caregiver [2]. To be able to question every resident, we decided, in consultation with the experts, to create an adapted version of the questionnaire for residents who needed help to fill out a questionnaire because of physical or mental health issues. The questions in this version are the same as the questions in the standard resident questionnaire, but are reformulated from second to third person. They can be read to the resident or filled out by the resident’s informal caregiver (or professional caregivers if no informal caregiver was noted in the patient record), preferably together with the resident.
A questionnaire was developed for the closest family caregiver (as noted in the health record) [3] and a separate one for the most involved professional caregiver [4] in order to measure indicators of quality of care for residents who passed away in the facility within the last six months. We performed a cognitive testing for all questionnaires in the corresponding responder group (i.e., residents, family, and professional caregivers). We tested the comprehensibility and response burden: recommendations resulted in minor linguistic changes for both residents and family caregivers.

2.3. Step 2: Pilot Testing

2.3.1. Design

We used a mixed-method design, including a quantitative application of the quality indicators and qualitative interviews with the nursing home staff using the instrument, in order to evaluate the face validity, feasibility, discriminative power, and usefulness of the instrument.

2.3.2. Setting and Participants

Nursing homes were recruited on a voluntary basis through a call for participation via involved community-based organizations. From the 24 candidates, we selected a purposive sample of nine nursing homes, while considering the number of beds (between 64 and 290 beds), the organizational structure (i.e., six profit and three non-profit) and the geographical location (every Flemish province was represented).
Nursing homes were able to measure the quality indicators via questionnaires through a cross-sectional inclusion design. This method allows for nursing homes to gather information on residents who were currently living in the nursing home as well as those who had passed away. Following inclusion criteria were used:
Residents who were currently living in the nursing home and:
  • lived for a minimum of one month in the facility;
Residents who had passed away and:
  • lived for a minimum of one month in the facility; and,
  • passed away four weeks to six months earlier in the nursing home.

2.3.3. Measurement Procedure

All nursing homes followed the same measurement procedure based on a previously developed and tested method in order to measure the quality indicators via questionnaires [16]. Before the start of the pilot test, a coordinator per nursing home was appointed in consultation with the researchers. The researchers visited the coordinator (in the nursing homes) in order to explain the study, expectations, measurement procedure, how to work with the online questionnaires, going through the detailed instruction manual. The coordinator responsibilities include the supervision of the measurement procedure, communication within the nursing home (e.g., informing the staff about the instrument and procedure, announcing start date), drafting the list for including residents (in concordance with the researcher), and distribution of the questionnaire among residents, family caregivers and staff (Figure 2). The coordinator was also asked to keep a diary and note thoughts regarding the workload, setbacks, and/or other findings (Figure 2).
Nursing homes were asked to include minimum of 2/3 of all residents at random and all deceased residents who met the inclusion criteria. We developed an inclusion matrix, depending on the number of residents per nursing home. We used online questionnaires via Limesurvey because nursing home staff were responsible for the distribution of questionnaires and to ensure responders privacy. No IP addresses were saved to guarantee anonymity. Residents could fill out the questionnaire via portable computers or tablets available in the nursing homes; family members received a link to the questionnaire via email; inhouse caregivers accessed the online questionnaire via computers in the nursing homes or on their private computers.

2.3.4. Feedback and Evaluation

Per nursing home, a report was created, summarizing the individual and overall performance scores in a structured and standardized way. Nursing home coordinators were responsible for communicating the results to the nursing home staff (step 3 and 4 in Figure 2). After the report was sent to the nursing homes, the researchers visited the nursing homes for an evaluation interview with the coordinator, while using an interview guide with open-ended questions. The coordinators kept a diary during the measurement and delivered it in advance to the researcher. During the interview, the workload of the coordinator and the nursing home staff was evaluated and barriers and facilitators in the use of the quality indicators were identified. Coordinators could also share their thoughts on future use of the instrument and wider implementation. The researcher kept a diary of evaluation points, remarks, and questions for further qualitative evaluation during the whole period of the pilot test.

2.3.5. Analyses

Data collection was closed after one month. Performance scores (non-adjusted mean) per quality indicator were calculated while using the defined numerators and denominators (range 0–100). In order to evaluate feasibility and discriminative power for individual quality indicator, we used descriptive and psychometric analyses in Microsoft Excel and SAS. Furthermore, the interviews were conducted with all coordinators in order to evaluate the face validity and usability of the indicators and the feasibility of the procedure. Together with the diaries of the coordinators and the field notes of the researcher, these interviews were analyzed while using a thematic framework approach, which was based on the barriers and facilitators for implementation framework of Grol and Wensing [10,39,40]. Table 1 presents an overview of all evaluation aspects, accompanying methods, and criterions.

2.4. Ethical and Language Issues

This study is approved by the Ethical Review Board of Brussels University Hospital of the Vrije Universiteit Brussel (protocol: QPACWZC01 BUN: 143201838240). All of the respondents (i.e., residents, family, and nursing home staff) received an online questionnaire, including cover letter and informed consent. Only questionnaires with signed informed consent were used to calculate performance scores. No IP addresses, names, or other personal identifiers were saved in the online questionnaire system.
All of the indicators and questionnaires were developed and evaluated in Dutch. All of the interviews and trainings were performed in Dutch. The English translation was done specifically for this article. Dutch versions of the indicators or questionnaires are available on request.

3. Results

3.1. Step 1: Indicator Development

3.1.1. Phase 1 and 2: Literature Study and Expert Interviews

Based on the existing QPAC quality indicator set for specialized palliative care, the additional literature search (phase 1) and interviews with relevant stakeholders (n = 10) (phase 2), we identified 26 candidate quality indicators in eight domains of quality of palliative care for elderly persons in nursing homes. Table 2 shows the difference between the Q-PAC domains (specializes palliative care) and the domains for the nursing homes based on literature search and stakeholder interviews in phase 2.

3.1.2. Phase 3: Expert Consensus

Based on their individual evaluation of the 26 candidate quality indicators, seven quality indicators were immediately accepted and included. None were immediately rejected, so the remaining 19 quality indicators were debated in a one-day plenary discussion until consensus was found. Nine of 19 quality indicators were eventually accepted and three were newly developed during the meeting and added to the draft set. After the discussion, a set of in total 19 quality indicators were drafted and per email consented by all experts (Table 3 and Table 4). In Supplementary Table S1, the full list of quality indicators, as was tested in the pilot phase, is presented with accompanying numerator, denominator, question, and source.

3.2. Step 2: Pilot Test

3.2.1. Responder Characteristics

Nine nursing homes tested the quality indicator set and measurement procedure. In total, 294 residents, 393 professional caregivers (345 for residents who currently lived in the facility and 48 for deceased residents), and 34 family caregivers completed the whole questionnaire and hence were included for the pilot study. We asked nursing homes to list the total number of inclusions, but four of them did not perform this assignment correctly; hence, we lack information on the response rates in this study. In total 214 of the residents were female and the majority (53%) of residents was between 85- and 94-years old. Table 5 presents an overview of characteristics.

3.2.2. Psychometric Analyses: Feasibility and Discriminative Power

None of the indicators had too many missing (>10%) answers. The quality indicators showed good discriminative power, as there were no indicators that had 95% or more answers in an extreme category (Table 3). Only two indicators had a variation range (min-max) smaller than 20 percentage points between different nursing homes, i.e., ‘Recognizing the approaching death’ and ‘Satisfied by care delivered’ and, hence, showed problems with sensitivity to change.

3.2.3. Qualitative Analyses; Feasibility, Usefulness and Face Validity

We interviewed all nine coordinators of the included nursing homes. With regard to face validity, all of the coordinators confirmed that the appeared to reflect their practice and seemed valid. As indicated by one on the coordinators: “The results indicate clear work points and results are recognizable”. They also agreed the results were easy to interpret and useful in terms of improving their service, but they indicated that they struggled in establishing concrete improvement goals that are based on the quality indicator scores. The coordinators evaluated the length of all four questionnaires as feasible, but four coordinators declared that they would prefer paper questionnaires for residents and family, as this may improve response rates. As indicated by one of the coordinators: “We would prefer paper questionnaires … we [staff in de nursing home] don’t have professional email addresses and I didn’t want to send the questionnaires to their private email. Also, our residents don’t know how to use a computer or tablet and therefore some residents who normally could fill in a questionnaire alone, now couldn’t”.
Moreover, all of the coordinators indicated that they would use the instrument again and evaluated the instruction manual as useful and sufficient and assumed the instrument could be executed without the researchers. One of the coordinators said: “Training was okay, the manual is clear and I think we could have managed without [the manual]”. Additionally, all of the coordinators indicated that the workload was feasible and worthwhile, although they declared that preparing the list of respondents was time-consuming, as they had to acquire their own approach.
Based on field notes, the interviews with the coordinators, and diaries of the same coordinator, we made an overview of facilitators and barriers regarding the use of the instrument (Table 6) in general terms and per step of the measurement procedure, as described in Figure 2.

4. Discussion

In this study, we developed and evaluated a quality indicator set and a tailored measurement procedure consisting of 19 indicators to monitor the quality of palliative care in Belgian nursing homes. The composition of this indicator set is based on previously developed quality indicators for specialized palliative care, but, after adaptation to the nursing home context by experts and stakeholders, the themes differ somewhat: more emphasis is placed on autonomy and dignity of the nursing home residents. From this first pilot study, the quality indicators seem to be valid and the measurement procedure feasible for caregivers in nursing homes who are interested in improving the quality of end-of-life care within their center. From the psychometric analyses, we found that most of the quality indicators were feasible and they showed good discriminative power. The instrument appeared to reflect practice and hence confirmed face validity, according to coordinators during the qualitative interviews. The measurement procedure was evaluated by the interviewed coordinators as feasible and they indicated the measurement of the quality indicators could be performed based on the manual without extra help of the researchers. Overall, this study shows that the quality indicators are ready for further use in a large implementation study in Flemish nursing homes in order to further evaluate their feasibility, usefulness, discriminative power, and potential for quality improvement.
An evaluation of the quality of care with quality indicators best includes process as well as outcomes indicators of care in one monitoring cycle [16,41]. The quality indicator set for palliative care that we developed for nursing homes uses both types of indicators. We also included objective as well as subjective quality indicators. The psychometric analyses in this pilot study showed good results for all of the indicators on discriminative power expect for ’recognizing the approaching death’ and ‘satisfied by care delivered’, which are both indicators subjectively measured by caregivers. Both of the indicators might have been influenced by response bias, due to social desirability or a tendency to overestimate their skills [42] and were discarded from the quality indicator set. From this finding we might conclude that, when using self-assessment instruments for quality monitoring, caregivers should report as much as possible on objective information of care, i.e., information that can be found in the patient file. Such biases can best be monitored by regularly evaluating the quality indicator set for psychometric criteria, in order to keep the quality data sensitive to changes in quality of care over time and between health care services.
An important strength of our study is the rigorous, systematic development method while using stakeholders and the mixed-method design, including the RAND/UCLA method for indicator development, quantitative analysis of data, and qualitative interviews with the coordinators in the nursing homes to evaluate the instrument. Hence, we were able to evaluate the instrument and its measurement procedure in terms of face validity, feasibility, discriminative power, and usefulness. Additionally, because the thoroughly follow-up with the involved coordinators before, during and after the pilot, barriers and facilitators influencing the course of the measurement were identified. The small database is one of the limitations of this study. Psychometric analyses were limited and a study on further implementation is necessary to evaluate and validate the instrument including the quality indicators. The absence of response rates is another limitation of our study. Although coordinators drafted a list of included residents, we were unable to match them with the questionnaires because of GDPR policies. Additionally, although we aimed to include as many residents as possible while using three versions of the questionnaire in the resident’s evaluation, we have no insights regarding whether residents with cognitive problems, such as dementia, were sufficiently involved in the quality monitoring.
Worldwide initiatives have been taken to monitor and improve the quality of palliative care in different settings [43,44,45,46,47]. Several studies have pointed out that the quality of dying and end of life care is not optimal across Western countries [48,49,50]. Some of these studies also used quality indicators in order to evaluate quality of palliative care in this setting, albeit being mostly focused on cancer patients, hospital and home setting, and administrative data in order to gather information on care processes and patient outcomes. Therefore, these measures are labeled as objective indicators and, although they provide a good basis for quality monitoring, they are not enough to point out strengths and weaknesses in specific long-term care organizations. Additionally, user perspective needs to be considered through subjective quality measures [51]. With our instrument, we focused on nursing homes and tried to combine both objective and subjective measures into different stakeholder perspectives in order to reach a comprehensive picture on quality of palliative care. Only this way, important themes for elderly persons, such as dignity and autonomy, can be properly addressed in order to improve the quality of care in the light of also improving the quality of life for residents in nursing homes. According to our qualitative analysis in this pilot study, coordinators indeed found the results of their measurement recognizable for their nursing home, supporting the face validity of these indicators (i.e., they are measuring what they aim to measure) from a caregiver point of view. This is an important finding, because, in order to reach effective change in health care, the value of timely and recognizable feedback is a crucial incentive for caregivers in order to continuously engage in these monitoring and improvement processes [11,40,45,52,53,54].
In light of care improvement in the field of palliative care in a nursing home context, a large-scale research project, ‘PACE steps to success’, has recently been implementing a combination of tailored improvement initiatives focusing on communication, advanced care planning, and knowledge and skills on end of life care while using a train-the-trainer implementation model. Although the intervention did not show significant effect on their primary outcome (comfort in the last week of life for residents), the process evaluation showed that the implementation rate was highly variable between countries and teams, and several challenges arose, such as attitude and motivation of staff, and skills and expertise of the trainer appointed to the individual nursing homes [26,48]. Our previous implementation research in palliative care already showed that caregivers are willing to invest in quality improvement trajectories and learn from other teams, but they need support from their management and financial reimbursement or staff to engage in these activities [40]. In this pilot study, we found the same barriers and facilitators pointing out the importance of setting the right preconditions for implementation in the nursing home context, throughout research and policy. This might be done by primordially evaluating nursing home readiness in order to increase the use and correct application of the quality indicators [55].

5. Conclusions

In this study, we developed and evaluated a quality indicator set and a tailored measurement procedure consisting of 19 indicators to monitor quality of palliative care in Belgian nursing homes. We combined both objective and subjective measures into four questionnaires for different perspectives in order to reach a comprehensive picture on quality of palliative care, end-of-life care and advance care planning in nursing homes. Care teams in nursing homes are able to monitor themselves based on these indicator scores. We found, while using both quantitative as qualitative analyses, the developed instrument had good face validity, feasibility, discriminative power, and it is useful in terms of quality monitoring according to caregivers, though establishing concrete improvement goals based on quality indicator scores remains difficult for them. The quality indicators are ready for further use in a large implementation study and process evaluation in Flemish nursing homes in order to further evaluate their feasibility, usefulness, discriminative power, and potential for quality improvement.

Supplementary Materials

The following are available online at https://www.mdpi.com/1660-4601/18/2/829/s1, Table S1: list of quality indicators with numerators, denominators and questions.

Author Contributions

All authors have substantially contributed to the research. Conceptualization, J.C., L.V.d.B., L.D. and K.L. Methodology, J.C., L.V.d.B., L.D., M.D.R., R.D.S. and K.L.; Formal Analysis, R.D.S., C.D. and K.L.; Investigation, R.D.S., C.D. and K.L.; Writing—Original Draft Preparation, C.D., R.D.S. and K.L.; Writing—Review & Editing, J.C., L.V.d.B., L.D., and M.D.R.; Supervision, J.C., L.V.d.B., L.D., M.D.R. and K.L.; Project Administration, R.D.S. and C.D.; Funding Acquisition, J.C., L.V.d.B., L.D., M.D.R. and K.L. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by the Flemish Agency of Healthcare (AZG/WEL/TPKO/2019/20190405).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethical Review Board of Brussels University Hospital of the Vrije Universiteit Brussel (protocol: QPACWZC01 BUN: 143201838240)

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

We would like to thank all nursing homes and their residents who voluntarily participated in this pilot study for their time and effort. We would also like to thank all stakeholders and experts as well as the staff members of the Flemish Agency of Healthcare for their comments and expertise in this study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Etkind, S.N.; Bone, A.E.; Gomes, B.; Lovell, N.; Evans, C.J.; Higginson, I.J.; Murtagh, F.E.M. How many people will need palliative care in 2040? Past trends, future projections and implications for services. BMC Med. 2017, 15, 102. [Google Scholar] [CrossRef] [PubMed][Green Version]
  2. Bone, A.E.; Gomes, B.; Etkind, S.N.; Verne, J.; Murtagh, F.E.M.; Evans, C.J.; Higginson, I.J. What is the impact of population ageing on the future provision of end-of-life care? Population-Based projections of place of death. Palliat. Med. 2018, 32, 329–336. [Google Scholar] [CrossRef] [PubMed][Green Version]
  3. Kojima, G. Prevalence of Frailty in Nursing Homes: A Systematic Review and Meta-Analysis. J. Am. Med. Dir. Assoc. 2015, 16, 940–945. [Google Scholar] [CrossRef] [PubMed]
  4. Hoben, M.; Chamberlain, S.A.; Gruneir, A.; Knopp-Sihota, J.A.; Sutherland, J.M.; Poss, J.W.; Doupe, M.B.; Bergsrom, V.; Norton, P.G.; Schalm, C.; et al. Nursing Home Length of Stay in 3 Canadian Health Regions: Temporal Trends, Jurisdictional Differences, and Associated Factors. J. Am. Med. Dir. Assoc. 2019, 20, 1121–1128. [Google Scholar] [CrossRef] [PubMed]
  5. Centeno, C.; Lynch, T.; Donea, O.; Rocafort, J.; Clark, D. EAPC Atlas of Palliative Care in Europe 2013—Full Edition; EAPC Press: Charleroi, Belgium, 2013. [Google Scholar]
  6. Organisation, W.H. Definition of Palliative Care. Available online: http://www.who.int/cancer/palliative/definition/en/ (accessed on 9 November 2020).
  7. Ten Koppel, M.; Onwuteaka-Philipsen, B.D.; Van den Block, L.; Deliens, L.; Gambassi, G.; Heymans, M.W.; Kylanen, M.; Pasman, H.R.W.; Payne, S.; Smets, T.; et al. Palliative care provision in long-term care facilities differs across Europe: Results of a cross-sectional study in six European countries (PACE). Palliat. Med. 2019, 33, 1176–1188. [Google Scholar] [CrossRef] [PubMed]
  8. Honinx, E.; van Dop, N.; Smets, T.; Deliens, L.; Van Den Noortgate, N.; Froggatt, K.; Gambassi, G.; Kylanen, M.; Onwuteaka-Philipsen, B.; Van den Block, L.; et al. Dying in long-term care facilities in Europe: The PACE epidemiological study of deceased residents in six countries. BMC Public Health 2019, 19, 1199. [Google Scholar] [CrossRef][Green Version]
  9. Wang, L.-R.; Wang, Y.; Lou, Y.; Li, Y.; Zhang, X.-G. The role of quality control circles in sustained improvement of medical quality. Springerplus 2013, 2, 141. [Google Scholar] [CrossRef][Green Version]
  10. Grol, R.; Wensing, M. What drives change? Barriers to and incentives for achieving evidence-based practice. Med. J. Aust. 2004, 180, S57–S60. [Google Scholar] [CrossRef]
  11. Peters, M.A.J.; Harmsen, M.; Laurant, M.G.H.; Wensing, M. Room for Improvement? Barriers to and Facilitators for Improvement of Patient Care; Centre for Quality of Care Research (WOK): Nijmegen, The Netherlands, 2002.
  12. Grol, R.P.; Bosch, M.C.; Hulscher, M.E.; Eccles, M.P.; Wensing, M. Planning and studying improvement in patient care: The use of theoretical perspectives. Milbank Q. 2007, 85, 93–138. [Google Scholar] [CrossRef][Green Version]
  13. Grol, R.; Wensing, M. Implementatie: Effectieve Verbetering van de Patiëntenzorg; Reed Business: Amsterdam, The Netherlands, 2011. [Google Scholar]
  14. Ostgathe, C.; Voltz, R. Quality indicators in end-of-life care. Curr. Opin. Support. Palliat. Care 2010, 4, 170–173. [Google Scholar] [CrossRef]
  15. Teno, J.M.; Coppola, K.M. For every numerator, you need a denominator: A simple statement but key to measuring the quality of care of the “dying”. J. Pain Symptom Manag. 1999, 17, 109–113. [Google Scholar] [CrossRef]
  16. Cohen, J.; Leemans, K. How can you prove that you are delivering good care? Monitoring the quality of palliative care using quality indicators. Eur. J. Palliat. Care 2014, 21, 228–231. [Google Scholar]
  17. Eagar, K.; Watters, P.; Currow, D.C.; Aoun, S.M.; Yates, P. The Australian Palliative Care Outcomes Collaboration (PCOC)—Measuring the quality and outcomes of palliative care on a routine basis. Aust. Health Rev. 2009, 34, 186–192. [Google Scholar] [CrossRef] [PubMed]
  18. Canadian Government D of H. Canadian Strategy on Palliative and End-Of-Life Care. Available online: http://www.hc-sc.gc.ca/hcs-sss/palliat/support-soutien/strateg-eng.php (accessed on 2 September 2014).
  19. New Zealand Government M of H. The New Zealand Palliative Care Strategy. Available online: http://www.health.govt.nz/publication/new-zealand-palliative-care-strategy (accessed on 2 September 2014).
  20. Department of Health. End of life Care Strategy: Quality Markers and Measures for End of Life Care; Crown: London, UK, 2009. Available online: http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/documents/digitalasset/dh_101684.pdf (accessed on 9 November 2020).
  21. Lundstrom, S.; Axelsson, B.; Heedman, P.A.; Fransson, G.; Furst, C.J. Developing a national quality register in end-of-life care: The Swedish experience. Palliat. Med. 2012, 26, 313–321. [Google Scholar] [CrossRef] [PubMed]
  22. Ministry of Health, Welfare and Sports. Plan van Aanpak Palliatieve Zorg 2008–2010; Ministry of Health, Welfare and Sports: The Hague, The Netherlands, 2008. Available online: http://www.palliatief.nl/Portals/31/Planvanaanpak2008-2010/planvanaanpakpalliatievezorg2008-2010.pdf (accessed on 9 November 2020).
  23. Martinsson, L.; Furst, C.J.; Lundstrom, S.; Nathanaelsson, L.; Axelsson, B. Registration in a quality register: A method to improve end-of-life care--a cross-sectional study. BMJ Open 2012, 2, e001328. [Google Scholar] [CrossRef] [PubMed][Green Version]
  24. Leemans, K.; Deliens, L.; Van den Block, L.; Vander, S.R.; Francke, A.L.; Cohen, J. Systematic Quality Monitoring for Specialized Palliative Care Services: Development of a Minimal Set of Quality Indicators for Palliative Care Study (QPAC). Am. J. Hosp. Palliat Care 2017, 34. [Google Scholar] [CrossRef]
  25. Plessers, M.; Ghekiere, A.; De Wachter, D.; Deneckere, S.; Tambuyzer, E.; Ramaekers, D. Het ontwikkelen van evidence-based indicatoren van kwaliteit van zorg in Vlaanderen: Een methodologieitle. Preprints 2019, 2020120424. [Google Scholar] [CrossRef]
  26. Oosterveld-Vlug, M.; Onwuteaka-Philipsen, B.; Ten Koppel, M.; van Hout, H.; Smets, T.; Pivodic, L.; Tanghe, M.; Hockley, J.; Kijowska, V.; Engels, Y.; et al. Evaluating the implementation of the PACE Steps to Success Programme in long-term care facilities in seven countries according to the RE-AIM framework. Implement. Sci. 2019, 14, 107. [Google Scholar] [CrossRef][Green Version]
  27. Van Riet Paap, J.; Vissers, K.; Iliffe, S.; Radbruch, L.; Hjermstad, M.J.; Chattat, R.; Vernooij-Dassen, M.; Engels, Y.; IMPACT Research Team. Strategies to implement evidence into practice to improve palliative care: Recommendations of a nominal group approach with expert opinion leaders. BMC Palliat. Care 2015, 14, 47. [Google Scholar] [CrossRef][Green Version]
  28. Leemans, K.; Deliens, L.; Francke, A.L.; Vander Stichele, R.; Van Den Block, L.; Cohen, J. Quality indicators for palliative care services: Mixed-method study testing for face validity, feasibility, discriminative power and usefulness. Palliat. Med. 2015, 29. [Google Scholar] [CrossRef][Green Version]
  29. Leemans, K.; Cohen, J.; Francke, A.L.; Stichele, R.V.; Claessen, S.J.J.; Block, L.V.D.; Deliens, L. Towards a standardized method of developing quality indicators for palliative care: Protocol of the Quality indicators for Palliative Care (Q-PAC) study. BMC Palliat. Care 2013, 12, 6. [Google Scholar] [CrossRef] [PubMed][Green Version]
  30. Dodd, S.R.; Payne, S.A.; Preston, N.J.; Walshe, C.E. Understanding the Outcomes of Supplementary Support Services in Palliative Care for Older People. A Scoping Review and Mapping Exercise. J. Pain Symptom Manag. 2020, 60, 449–459. [Google Scholar] [CrossRef] [PubMed][Green Version]
  31. Lood, Q.; Kirkevold, M.; Sjögren, K.; Bergland, Å.; Sandman, P.-O.; Edvardsson, D. Associations between person-centred climate and perceived quality of care in nursing homes: A cross-sectional study of relatives’ experiences. J. Adv. Nurs. 2019, 75, 2526–2534. [Google Scholar] [CrossRef] [PubMed]
  32. Bailey, C.; Kinghorn, P.; Orlando, R.; Armour, K.; Perry, R.; Jones, L.; Coast, J. “The ICECAP-SCM tells you more about what I’m going through”: A think-aloud study measuring quality of life among patients receiving supportive and palliative care. Palliat. Med. 2016, 30, 642–652. [Google Scholar] [CrossRef][Green Version]
  33. Brownie, S.; Nancarrow, S. Effects of person-centered care on residents and staff in aged-care facilities: A systematic review. Clin. Interv. Aging 2013, 8, 1–10. [Google Scholar] [CrossRef][Green Version]
  34. Smets, T.; Onwuteaka-Philipsen, B.B.D.; Miranda, R.; Pivodic, L.; Tanghe, M.; van Hout, H.; Engels, Y.; Piers, R.; Baranska, I.; Deliens, L.; et al. Integrating palliative care in long-term care facilities across Europe (PACE): Protocol of a cluster randomized controlled trial of the “PACE Steps to Success” intervention in seven countries. BMC Palliat. Care 2018, 17, 47. [Google Scholar] [CrossRef][Green Version]
  35. Campbell, S.M.; Kontopantelis, E.; Hannon, K.; Burke, M.; Barber, A.; Lester, H.E. Framework and indicator testing protocol for developing and piloting quality indicators for the UK quality and outcomes framework. BMC Fam. Pract. 2011, 12, 85. [Google Scholar] [CrossRef]
  36. Fitch, K.; Bernstein, S.J.; Aguilar, M.S.; Burnand, B.; LaCalle, J.R.; Lazaro, P.; van het Loo, M.; McDonnell, J.; Vader, J.; Kahan, J.P. The RAND/UCLA Appropriateness Method User’s Manual; RAND Corporation: Santa Monica, CA, USA, 2001; Available online: http://www.rand.org/pubs/monograph_reports/MR1269 (accessed on 9 November 2020).
  37. Pasman, H.R.; Brandt, H.E.; Deliens, L.; Francke, A.L. Quality indicators for palliative care: A systematic review. J. Pain Symptom Manag. 2009, 38, 145–156. [Google Scholar] [CrossRef]
  38. De Roo, M.L.; Leemans, K.; Claessen, S.J.J.; Cohen, J.W.; Pasman, H.R.; Deliens, L.; Francke, A.L.; EURO IMPACT. Quality indicators for palliative care: Update of a systematic review. J. Pain Symptom Manag. 2013, 46, 556–572. [Google Scholar] [CrossRef]
  39. Pope, C.; Ziebland, S.; Mays, N. Qualitative research in health care. Analysing qualitative data. Br. Med. J. 2000, 320, 114–146. [Google Scholar] [CrossRef]
  40. Leemans, K.; Van den Block, L.; Vander, S.R.; Francke, A.L.; Deliens, L.; Cohen, J. How to implement quality indicators successfully in palliative care services: Perceptions of team members about facilitators of and barriers to implementation. Support. Care Cancer 2015, 23, 3503–3511. [Google Scholar] [CrossRef] [PubMed][Green Version]
  41. Donabedian, A. Quality assurance. Structure, process and outcome. Nurs. Stand. 1992, 7, 4–5. [Google Scholar] [PubMed]
  42. Mazor, K.M.; Clauser, B.E.; Field, T.; Yood, R.A.; Gurwitz, J.H. A demonstration of the impact of response bias on the results of patient satisfaction surveys. Health Serv. Res. 2002, 37, 1403–1417. [Google Scholar] [CrossRef] [PubMed][Green Version]
  43. Shekelle, P.G.; Maclean, C.H.; Morton, S.C.; Wenger, N.S. Assessing care of vulnerable elders: Methods for developing quality indicators. Ann. Intern. Med. 2001, 135, 647–652. [Google Scholar] [CrossRef] [PubMed][Green Version]
  44. Lorenz, K.A.; Rosenfeld, K.; Wenger, N. Quality indicators for palliative and end-of-life care in vulnerable elders. J. Am. Geriatr. Soc. 2007, 55, S318–S326. [Google Scholar] [CrossRef] [PubMed]
  45. Hall, S.; Goddard, C.; Stewart, F.; Higginson, I.J. Implementing a quality improvement programme in palliative care in care homes: A qualitative study. BMC Geriatr. 2011, 11, 31. [Google Scholar] [CrossRef][Green Version]
  46. Amin, A.; Besdine, R.C.; Blazer, D.G.; Cohen, H.; Fulmer, T.; Ganz, P.A.; Grunwald, M.; Hall, W.J.; Katz, P.R.; Kitzman, D.W.; et al. Assessing care of vulnerable elders-3 quality indicators. J. Am. Geriatr. Soc. 2007, 55, S464–S487. [Google Scholar]
  47. Detering, K.M.; Hancock, A.D.; Reade, M.C.; Silvester, W. The impact of advance care planning on end of life care in elderly patients: Randomised controlled trial. Br. Med. J. 2010, 340, 1345. [Google Scholar] [CrossRef][Green Version]
  48. Van den Block, L.; Honinx, E.; Pivodic, L.; Miranda, R.; Onwuteaka-Philipsen, B.D.; van Hout, H.; Pasman, R.W.; Piers, R.; Froggatt, K.; Payne, S.; et al. Evaluation of a Palliative Care Program for Nursing Homes in 7 Countries: The PACE Cluster-Randomized Clinical Trial. JAMA Intern. Med. 2020, 180, 233–242. [Google Scholar] [CrossRef]
  49. Van Soest-Poortvliet, M.C.; van der Steen, J.T.; Zimmerman, S.; Cohen, L.W.; Reed, D.; Achterberg, W.P.; Ribbe, M.W.; de Vet, H.C.W. Selecting the best instruments to measure quality of end-of-life care and quality of dying in long term care. J. Am. Med. Dir. Assoc. 2013, 14, 179–186. [Google Scholar] [CrossRef]
  50. Hall, S.; Kolliakou, A.; Petkova, H.; Froggatt, K.; Higginson, I.J. Interventions for improving palliative care for older people living in nursing care homes. Cochrane Database Syst. Rev. 2011, CD007132. [Google Scholar] [CrossRef]
  51. Sawatzky, R.; Laforest, E.; Schick-Makaroff, K.; Stajduhar, K.; Reimer-Kirkham, S.; Krawczyk, M.; Ohlen, J.; McLeod, B.; Hilliard, N.; Cohen, R.; et al. Design and introduction of a quality of life assessment and practice support system: Perspectives from palliative care settings. J. Patient Rep. Outcomes 2017, 2, 36. [Google Scholar] [CrossRef] [PubMed]
  52. De Vos, M.L.; van der Veer, S.N.; Graafmans, W.C.; de Keizer, N.F.; Jager, K.J.; Westert, G.P.; van der Vort, P.H.J. Process evaluation of a tailored multifaceted feedback program to improve the quality of intensive care by using quality indicators. Br. Med. J. Qual. Saf. 2013, 22, 233–241. [Google Scholar] [CrossRef] [PubMed]
  53. Van Riet Paap, J.; Vernooij-Dassen, M.; Brouwer, F.; Meiland, F.; Iliffe, S.; Davies, N.; Leppert, W.; Jaspers, B.; Mariani, E.; Vissers, K.; et al. Improving the organization of palliative care: Identification of barriers and facilitators in five European countries. Implement. Sci. 2014, 9, 130. [Google Scholar] [CrossRef] [PubMed][Green Version]
  54. Antunes, B.; Harding, R.; Higginson, I.J. Implementing patient-reported outcome measures in palliative care clinical practice: A systematic review of facilitators and barriers. Palliat. Med. 2014, 28, 158–175. [Google Scholar] [CrossRef][Green Version]
  55. Bunn, F.; Goodman, C.; Corazzini, K.; Sharpe, R.; Handley, M.; Lynch, J.; Meyer, J.; Dening, T.; Gordon, A.L. Setting Priorities to Inform Assessment of Care Homes’ Readiness to Participate in Healthcare Innovation: A Systematic Mapping Review and Consensus Process. Int. J. Environ. Res. Public Health 2020, 17, 987. [Google Scholar] [CrossRef][Green Version]
Figure 1. Overview of study phases and accompanying results.
Figure 1. Overview of study phases and accompanying results.
Ijerph 18 00829 g001
Figure 2. Measurement procedure.
Figure 2. Measurement procedure.
Ijerph 18 00829 g002
Table 1. Overview of the evaluation and accompanying methods and criterions.
Table 1. Overview of the evaluation and accompanying methods and criterions.
AspectDefinitionEvaluation MethodCriterion to Judge Aspect as
Adequate
Individual quality indicators (QI’s)
Face validityThe extent to which QI’s are subjectively viewed as covering the concept it purports to measureQualitative: interview: feedback on every single quality indicator was asked in terms of face validitySubjective confirmation of validity of quality indicator scores
FeasibilityThe extent to which the QI’s are measurableQuantitative: psychometric analysesNot more than 10% missing values per question
Discriminative powerThe extent to which a QI discriminates between good and bad qualityQuantitative: psychometric analysesNot more that 95% of answers in an extreme category
Meaningful range between QI scores (min–max ≥20%)
UsefulnessThe extent to which the QI scores can be used to improve careQualitative: interview question “Were you able to define improvement point based on the quality indicator scores and feedback report?”Subjective confirmation of usefulness
Overall quality indicator measurement
FeasibilityThe extent to which the measurement procedure is feasible for caregivers in nursing homesQualitative: interview question “Do you have the feeling you are able to measure the quality indicators without any support in the future?”Subjective information on work-load for caregivers
Qualitative: interview question “How did you feel about the length of the questionnaire?”Subjective information on survey completion time for caregivers
Table 2. Eight thematic domains for quality indicators.
Table 2. Eight thematic domains for quality indicators.
Original QPAC Set [24]QPAC for Nursing Homes
1Physical aspects of carePhysical aspects of care1
2Psychological, social and spiritual aspects of carePsychological, social and spiritual aspects of care2
3Care planning, information and communication with patientsAutonomy and dignity3
Care planning and communication with residents4
4Care planning, information and communication with familyCommunication with family5
5Care planning, information and communication between caregiversCommunication between caregivers6
6Circumstances surrounding deathCare and circumstances surrounding death7
7Coordination and continuity of care
8Support for familyCare for family8
Table 3. Participants in expert consultation rounds.
Table 3. Participants in expert consultation rounds.
Total
Professional caregivers from care homes7
Head nurse/Referent nurse3
Paramedic1
Care personnel1
Physician 1
Quality coordinator1
Representatives from residents and next-of-kin3
Flemish Expertise Centre for Dementia1
Alzheimer League, family council1
Flemish elderly council1
Palliative care research and policy5
KU Leuven—LUCAS research group2
Flemish Federation Palliative Care 1
Local Palliative home care network Westhoek-Oostende1
Flemish agency for care and health1
Table 4. Quality indicators for palliative care in nursing homes.
Table 4. Quality indicators for palliative care in nursing homes.
Domain: Physical Aspects of Care
NShort titleDescription of the indicatorRespondentMean score (%)Range
(min–max)
PC-1Being in painPercentage of residents with a pain score of 3 or more in the last three daysResidents30.737.1 (19.1–56.3)
Domain: Psychological, Social and Spiritual Aspects of Care
NShort titleDescription of the indicatorRespondentMean score (%)Range
(min–max)
PC-2Feeling worried or anxious, or a burdenPercentage of residents who indicate they were most of the times or always feeling worried or anxious, or a burden to othersResidents9.223 (4.3–27.3)
PC-3Being around people who care about youPercentage of residents who indicate that they were most of the times or always able to be around people who cared about themResidents57.142.9 (29.8–72.7)
Domain: Autonomy and Dignity
NShort titleDescription of the indicatorRespondentMean score (%)Range
(min–max)
PC-4Personal wishes and beliefs respectedPercentage of residents who indicate that their caregivers most of the times or always respecting their personal wishes and beliefsResidents63.355.7 (35.2–90.9)
PC-5Decisions about life and carePercentage of residents who indicate that they most of the times or always can make their own decisions about their life and careResidents44.235.4 (31.3–66.7)
PC-6Treated with respect Percentage of residents who indicate that they most of the times or always were treated with respect Residents68.647.2 (43.8–90.9)
Domain: Care Planning and Communication with Residents
NShort titleDescription of the indicatorRespondentMean score (%)Range
(min–max)
ACP-1Information comprehensible and not contradictoryPercentage of residents who indicate that they most of the times or always receive comprehensible information and almost never of never contradictory informationResidents79.521.4 (72.3–93.8)
ACP-2Conversation with familyPercentage of residents for whom the next-of-kin indicates that more than once a conversation took place with the caregivers, the next-of-kin and, when possible, the residentNext-of-kin47.6100 (0–100)
ACP-3Knowledge about care goals and life wishesPercentage of residents for whom their professional caregiver indicates that they have knowledge about the residents’ care goals and life wishes. Professional caregiver63.840 (47.1–87.1)
ACP-4Encouraging ACPPercentage of residents for whom their professional caregiver indicates that they often or very often encourage residents and their next-of-kins to involve in advance care planning. Professional caregiver37.772.5 (10.8–83.3)
Domain: Communication with Family
NShort titleDescription of the indicatorRespondentMean score (%)Range
(min–max)
ACP-5Next-of-kin involved in decisionsPercentage of next-of-kin who indicate that they often or very often felt involved in the decisions taken about the resident. Next-of-kin64.775 (25–100)
EOL-1Information about approaching deathPercentage of next-of-kin who indicate that they received the right amount of information on the approaching death of the resident. Next-of-kin73.535.7 (64.3–100)
Domain: Communication between Caregivers
NShort titleDescription of the indicatorRespondentMean score (%)Range
(min–max)
PC-8Information in resident filePercentage of residents for whom the professional caregiver finds sufficient information in the resident file when needed. Professional caregiver69.337.6 (52.7–90.3)
Domain: Care and Circumstances Surrounding Death
NShort titleDescription of the indicatorRespondentMean score (%)Range
(min–max)
EOL-3Comfortable in last week of lifePercentage of next-of-kin who indicate that many or a lot of measures were taken to make the resident comfortable in the last week of life. Next-of-kin67.6100 (0–100)
EOL-4Recognizing the approaching deathPercentage of residents for whom the professional caregiver indicates they could recognize the approaching death well or very well by physical changes. Professional caregiver91.716.7 (83.3–100)
EOL-5Satisfied by care deliveredPercentage of residents for whom the professional caregiver indicates they are satisfied with the care delivered to the resident. Professional caregiver95.816.7 (83.3–100)
EOL-6Support by specialized palliative carePercentage of residents for whom the professional caregiver indicates a palliative care referent or specialized team was involved in the care for the resident. Professional caregiver68.8100 (0–100)
Domain: Care for Family
PC-7Attention for wishes and feelings of next-of-kin Percentage of next-of-kin who indicate that the professional caregivers had attention for their wishes and feelings. Next-of-kin67.630 (50–80)
EOL-2Supported immediate after deathPercentage of next-of-kin who indicate that they felt sufficiently supported by the professional caregivers immediate after the death of the resident. Next-of-kin85.366.7 (33.3–100)
PC = Palliative care; ACP = advance care planning; EOL = end of life.
Table 5. Characteristics per response type in the pilot test.
Table 5. Characteristics per response type in the pilot test.
Response TypeTotalFemale (%)Age of ResidentDementia B (%)Length of Stay
C
<75 (%)75–84 (%)85–94 (%)>94 (%)<12 (%)12–24 (%)>24
(%)
Residents294214 (73)26
(9)
74
(25)
157 (53)37
(13)
NANANANA
Resident him/herself11483 (73)11 (10)24 (21)67 (59)12 (11)NANANANA
Together with family caregiver6343 (68)5
(8)
14 (22)38 (60)6 (10)NANANANA
Family caregivers in the name of the resident116 A87 (75)10
(9)
35 (30)52 (45)19 (16)56 (48)37 (33)22
(20)
52
(47)
Professional caregivers393 305 (73)27 (7)97 (25)218 (55)51 (13)204 (49)125 (32)60 (15)208
(53)
Residents who lived in the facility345 257 (74)25 (7)88 (26)193 (56)39 (11)162 (47)109 (32)54 (16)182
(53)
Deceased residents4831 (65)2 (4)9
(19)
25 (52)12 (25)29 (60)16 (33)6
(13)
26
(54)
Family caregivers3422 (65)1
(3)
7
(21)
17 (50)9 (26)16 (47)14 (41)5
(15)
15
(44)
A: 5 missings for length of stay. B: Questioned only when the family caregivers completed the questionnaire in the name of the resident. C: Length of stay in months.
Table 6. Facilitators and barriers based on the interviews and dairies regarding the use of the instrument.
Table 6. Facilitators and barriers based on the interviews and dairies regarding the use of the instrument.
Barrier (b) or Facilitator (f)Quote from Caregivers or Field NotesDiary by Coordinator Interview with Coordinator
The use of the instrument in general terms
Lack of time and staff to perform quality measurement (b)To sell the instrument: make it a sort of an obligation, otherwise it will not happen, I think. So much extra is added [next to the regular work], and also many projects that are already there anyway” (coordinator nursing home)XX
Readiness of the team to perform quality monitoring together (f)[experience with implementation of the quality assessment] it was ok. It also depends on the enthusiasm and commitment of the persons who are doing it.” (coordinator nursing home) X
Step 1: Appointing coordinator
Presence of a good coordinator to guide the quality measurement (f)“Appointment of the coordinator: one is not enough. Depends on the size of the nursing home.” (coordinator nursing home)
“Announced [the quality assessment] during team meeting. They [coordinators] had made a step-by-step plan and mailed it to the staff, how they could easily find it and fill it in … everything went smoothly” (coordinator nursing home)
XX
Step 2: Data collection with the quality indicators
Bad timing regarding the start of measurement (i.e., sick staff, loss of coordinator) (b)Some of the coordinators became absent during the procedure and the person who took over didn’t have all the needed paperwork. (field notes researchers).
Some nursing homes forgot to record the total of included participants, didn’t sent out the recruited number of questionnaires or didn’t sent questionnaires to family caregivers. The reason they indicated was the moment of the measurement was not convenient (field notes researchers).
X
Lack of computer literacy in all participants (b)They [family and residents] had no e-mail and some [family] had to come to the nursing home to fill it [the questionnaire] in.
In some nursing homes professional caregivers didn’t had a work email and in one of these homes, the coordinator had to aid each included professional caregivers with opening the link [which made available on the desktop] to the questionnaire (field notes researchers).
X
Lack of technology in the nursing homes (b)“It was a lot of time investment, there was only one iPad available in the nursing home, so we had to arrange a lot.
WIFI connection was also not reliable, which limited usability.” (coordinator nursing home)
XX
Feasible workload (f)All coordinators found the overall workload feasible (field notes researchers)
“A lot of work in preparation by the coordinator so the coordinator should certainly have time to prepare. Once it runs [there is] little follow-up work.” (coordinator nursing home)
XX
Step 3: Analysis of results by researchers
Low(er) response rate because of measurement procedure (b)
AND
Inclusion of deceased residents due to low mortality (b)
“With a longer measurement period, they [respondents] could fill in more” (coordinator nursing home)XX
Fast (within two weeks) analysis of questionnaires because of the use of digital data (f)Because we used online questionnaires the researchers didn’t need to input any data but could directly analyse resulting in fast feed-back to the nursing homes X
Step 4: Interpretation of results by coordinator and nursing home team
Easy to interpret results (f)The results indicate clear work points. Results are recognizable” (coordinator nursing home) X
Struggle to go from interpretation to establishing improvement goals (b)Most coordinators indicate they recognize the results, but they cannot (yet) make clear improvement goals. (field notes researchers) X
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Dupont, C.; De Schreye, R.; Cohen, J.; De Ridder, M.; Van den Block, L.; Deliens, L.; Leemans, K. Pilot Study to Develop and Test Palliative Care Quality Indicators for Nursing Homes. Int. J. Environ. Res. Public Health 2021, 18, 829. https://doi.org/10.3390/ijerph18020829

AMA Style

Dupont C, De Schreye R, Cohen J, De Ridder M, Van den Block L, Deliens L, Leemans K. Pilot Study to Develop and Test Palliative Care Quality Indicators for Nursing Homes. International Journal of Environmental Research and Public Health. 2021; 18(2):829. https://doi.org/10.3390/ijerph18020829

Chicago/Turabian Style

Dupont, Charlèss, Robrecht De Schreye, Joachim Cohen, Mark De Ridder, Lieve Van den Block, Luc Deliens, and Kathleen Leemans. 2021. "Pilot Study to Develop and Test Palliative Care Quality Indicators for Nursing Homes" International Journal of Environmental Research and Public Health 18, no. 2: 829. https://doi.org/10.3390/ijerph18020829

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop