Next Article in Journal
Effect of Short-Term Low-Nitrogen Addition on Carbon, Nitrogen and Phosphorus of Vegetation-Soil in Alpine Meadow
Previous Article in Journal
Developing a Dynamic Model for Assessing Green Infrastructure Investments in Urban Areas
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 18
Issue 20
10.3390/ijerph182010995
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Design and Validation of the INCUE Questionnaire: Assessment of Primary Healthcare Nurses’ Basic Training Needs in Palliative Care

by
Isidro García-Salvador
1,2,3,
Encarna Chisbert-Alapont
2,3,4,*,
Amparo Antonaya Campos
2,3,5,
Jorge Casaña Mohedo
2,3,6,
Clara Hurtado Navarro
2,3,7,
Silvia Fernández Peris
2,8,
José Bonías López
2,3,9 and
Maria Luisa De la Rica Escuín
2,10
1
Nurse Oncology Service, Valencia Health Department, Doctor Peset, 46017 Valencia, Spain
2
Foundation for the Promotion of Health and Biomedical Research in the Valencian Region (FISABIO), 46020 Valencia, Spain
3
Research Group INCUE, Valencia Health Department, Doctor Peset, 46017 Valencia, Spain
4
Nurse Oncology Service, Valencia Health Department La Fe, 46026 Valencia, Spain
5
Primary Care Nursing Director, Valencia Health Department, Doctor Peset, 46017 Valencia, Spain
6
Nursing Department, Faculty of Medicine and Health Sciences, Universidad Católica San Vicente Mártir, 46001 Valencia, Spain
7
Nurse Training Service, Valencia Health Department, Doctor Peset, 46017 Valencia, Spain
8
Carena Association of Psycho-Oncology, Valencia Health Department, Doctor Peset, 46017 Valencia, Spain
9
Nurse Primary Care Center of San Marcelino, Valencia Health Department, Doctor Peset, 46017 Valencia, Spain
10
Nurse Research Group on Care in End-of-Life Processes, Institute for Health Research Aragón, 50009 Zaragoza, Spain
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(20), 10995; https://doi.org/10.3390/ijerph182010995
Submission received: 12 September 2021 / Revised: 12 October 2021 / Accepted: 14 October 2021 / Published: 19 October 2021
Browse Figure
Review Reports Versions Notes

Abstract

:
Many instruments have been created to measure knowledge and attitudes in palliative care. However, not only is it important to acquire knowledge, but also that this knowledge should reach patients and their relatives through application in clinical practice. This study aimed to develop and psychometrically test the INCUE questionnaire (Investigación Cuidados Enfermeros/Investigation into Nurses’ Care Understanding of End-of-Life) to assess the basic training needs of primary or home healthcare nurses in palliative care. A questionnaire was developed based on the classical theory of tests and factor analysis models. Initially, 18 experts developed 67 items in two blocks and determined content validity by two rounds of expert panels. Exploratory factor analysis and reliability testing were conducted with a non-probabilistic sample of 370 nurses. Some items were observed to have very low homogeneity indices or presented convergence problems and were eliminated. Questionnaire reliability was 0.700 in the theoretical block (KR20 Index) and 0.941 in the practical block (Cronbach’s alpha). The model converges and shows an adequate fit, specifically CFI = 0.977, TLI = 0.977 and RMSEA = 0.05. The correlation between the two factors in the model is ρ = 0.63. The questionnaire objectively evaluates primary or home healthcare nurses’ knowledge of palliative care and its practical application, thereby facilitating more efficient training plans.

1. Introduction

Palliative care (PC) improves the quality of life of patients facing life-threatening illnesses, and that of their relatives, alleviating pain and other symptoms, as well as providing spiritual and psychological support from the moment of diagnosis to the end of life and during bereavement [1].
The World Health Organization [2], the European Palliative Care Association [3] and the PC strategy of the Spanish National Health System [4] recommend a shared model of palliative care that ensures care continuity. Therefore, this calls for the intervention of the various care resources: primary healthcare, hospital and centers specialized in PC (home- or hospital-based).
The care of patients with palliative needs, as well as the care of their relatives and/or caregivers, continues to pose important challenges. These include the scarce and unequal training of nursing professionals in PC at all levels of care [5,6,7]. Several studies confirm the need to address training deficits and to improve preparation to ensure high-quality care [8,9,10,11,12,13,14]. In the same vein, the European Association for Palliative Care (EAPC) [15] and the Spanish Association of Nursing in Palliative Care (Spanish acronym AECPAL) [16], recommend basic training in palliative care to be taught by all universities offering qualifications in nursing. Such basic training should impart the minimum knowledge required to provide effective care in the different clinical scenarios commonly found for these patients and their relatives at any level of care [17]. The AECPAL [16] defines five basic knowledge areas: principles of palliative care, communication skills, symptom management and specific care plans, coping with loss and death, and ethical and legal issues.
Besides this, care is based on acquired knowledge, which is reflected by derived actions or implementation. Pedagogy advocates the need to apply what has been learned to new contexts through other skills after a period of reflection and through a transfer process [18]. Moreover, from a bioethics perspective, a pedagogical teaching approach based on the transmission of knowledge and procedures is established, as well as a Socratic approach to change attitudes or behaviors [19].
In this vein, a study involving hospital nurses demonstrated that although half of them had basic training in PC, only about 15% actually applied this knowledge in practice (use of symptom assessment scales, application of non-pharmacological measures or patient participation in decision-making) [20].
The above would indicate that not only is it important to acquire knowledge, but also that this knowledge should reach patients and their relatives through practical application in clinical practice. Indeed, not only is it important to know “what nurses know”, but also “what they do or apply”.
The concern for palliative care-related issues has led to associated studies from various perspectives. Many instruments have been created and validated in Spanish to measure (jointly or separately) the knowledge, attitudes, skills and competencies on the subject, of different professionals (doctors, nurses and nursing assistants). Each of these professionals is considered within their specific discipline-related competencies, performance of their functions in the end-of-life care of dying patients and of their relatives and at different levels.
Some of the instruments attempt to approach the care provided by professionals to patients indirectly, assessing fear or competence on facing death, or attitudes towards patient care at the end of life. The Collet-Lester Scale and Bugen’s Coping with Death Scale measures the improvement in death [21,22,23].
Other instruments evaluating attitudes nursing trainees and professionals towards end-of-life patient care are Attitudes about End-of-Life Care Scale, the Death Attitude Profile-Revised (DAP-R), the Frommelt Attitude Toward Care of the Dying scale (FATCOD) or the Palliative Care Attitudes Scale (PCAS) [24,25,26,27,28,29].
Other scales focus on measuring the perceived self-efficacy competencies or self-reporting confidence and the perceived educational needs in PC, such as the Self-Efficacy in Palliative Care Scale [30,31] and the Self-reporting Confidence and Educational Needs in Hospice Care developed by Kwon [32].
Some scales have been created to measure attitudes and knowledge such as the Palliative Care Knowledge Test (PCKT) for general practitioners and nurses, the Palliative Care Attitude and Knowledge questionnaire (PCAK) for physicians and, more recently, the Palliative Approach for Nursing Assistants Questionnaire (PANA) that adds skills [33,34,35,36].
The Palliative Care Quiz for Nurses (PCQN) measures nurses’ knowledge in palliative care of three areas: philosophy and principles of PC, pain and symptoms control and the psychosocial aspects of care [37].
The Rotterdam MOVE2PC Scale assesses knowledge, attitudes, values and skills for the study of competencies and educational needs in PC of general nurses. This scale evaluates nurses’ opinions and subjective norms related to PC, potentially difficult situations in the final weeks or days of a patient’s life and knowledge [38].
However, the previous instruments aimed at nurses, evaluating knowledge, do so from the subjectivity of the respondent; according to their perception of how prepared they feel in a given situation. Furthermore, none of them evaluates the implementation of the knowledge they possess; therefore, the actual patient care provided is unknown.
These instruments usually take into account the common competences of the nursing discipline, but they do not usually take into account the tasks performed at the different levels of care and that, therefore, determine the type of care provided. The instrument should differ, depending on whether it is aimed at hospital nurses or primary and home healthcare nurses; based on their scope of care; and the possibility of monitoring family grief.
A validated instrument to assess theoretical knowledge and its application to clinical practice by primary or home healthcare nurses would enable specific training to be guided according to the shortcomings detected. This could have an impact on the provision of more effective palliative care for patients.
The purpose of the study was to develop and test the psychometric properties of the Investigación Cuidados Enfermeros/Investigation into Nurses’ Care Understanding of End-of-Life (INCUE) questionnaire, an instrument developed to assess the basic training needs in palliative care of primary and home healthcare nurses.

2. Materials and Methods

The INCUE questionnaire was developed based on a three-phase mixed-method design (Figure 1), based on the classical test theory and factor analysis models [39,40]. In the first phase, qualitative and quantitative methods and techniques were combined to delimit the framework or theoretical basis and generate the items [41,42]. In the second phase, the validity of the content and the internal consistency of the instrument were assessed. In the third phase, the psychometric properties of the instrument were tested by confirmatory factor analysis (CFA), to evaluate validity and reliability. According to Batista-Foguet, Coenders and Alonso the confirmatory perspective assigns specific indicators to specific dimensions. In this way, CFA models allow validity to be compared, adjusting a model that assumes it and diagnosing its goodness-of-fit (construct validation). In this type of model, each item saturates only on the factor-dimension of which it should constitute a valid indicator [39].

2.1. Phase 1: Theoretical Basis and Generation of Items

In the first step, a literature review about nursing competencies and training in PC was performed, objectives were set, the construct was defined, the dimensions and indicators to be measured were determined, and items and their response types were listed. The selection criteria (advanced PC training and having a minimum of 10 years’clinical experience) and number of experts were established, the evaluation template was designed, and the concordance between judges was calculated following the criteria of Escobar-Pérez and Martínez [42].
After reviewing the literature and the existing instruments, the construct was defined based on the five basic training areas in PC for nurses described by AECPAL [16]: principles of palliative care (A1); symptom management and specific care plans (A2); coping with loss and death (A3); communication skills (A4), and ethical and legal issues (A5).
The first panel of experts evaluated the results of the literature review and the proposed objectives. Using the Delphi method, they generated the items based on the document outlining the recommendations of the AECPAL for Nursing Degree training [43], which specifies the basic theoretical and practical knowledge required for the end-of-life care of dying patients and their relatives. This knowledge was considered as the minimum benchmark to care for these patients and their relatives effectively in the different clinical scenarios commonly faced by the primary or home healthcare nurse in undertaking his/her work duties.
The evaluation of theoretical knowledge and its application to clinical practice were established as objectives of each of the areas. Agreement was reached on the number of items required for each area (Table 1).
Subsequently, the first panel of experts established the selection criteria for the second panel of experts and the item evaluation sheets to assess the suitability, clarity and relevance of the items, as well as additional contributions. The panel of experts should comprise at least 10 nurses with advanced PC training and have a minimum of 10 years’ clinical experience.
In the second step, the first version of the questionnaire was submitted to expert judgment by Cronbach’s alpha test and Kendall’s W test, to evaluate its reliability, internal consistency and expert consensus. Kendall’s W test also compares the potential association between ordinal variables in related samples and tests the null hypothesis of lack of agreement between judges. We performed four Kendall tests, one for each criterion and one for the three criteria together (adequacy, clarity and relevance). Additional textual contributions were collected for content analysis and applied to the pilot questionnaire [44].

2.2. Phase 2: Pilot Study

In this phase, descriptive analyses were performed to analyze item comprehensibility using a Likert scale from 1 to 5, scoring from lowest to highest comprehensibility.
Different parameters were evaluated to determine the consistency of the items within each area. The KR20 coefficient was calculated for the items in the theoretical block, whereas the Cronbach’s alpha was applied to the practical items. The values of these coefficients range between 0 and 1 and are considered acceptable when they are equal to or greater than 0.7 in order to ensure scale reliability [45].

2.3. Phase 3: Validation

In the validation phase, various analyses were performed in order to assess the reliability and validity of the questionnaire responses. In the theoretical block they were analyzed according to the right/wrong answers of the respondents. To evaluate reliability, internal consistency was analyzed, calculating KR20 and Cronbach’s alpha coefficients for the items in the theoretical and practical parts, respectively [45]. Finally, the homogeneity index or item-total correlation of each item was calculated, indicating the correlation between the score in each item and the sum of the scores in the remaining items. To evaluate the construct validity of the instrument, CFA was performed using psych software [46] and lavaan R package [47].
As a previous step to this analysis and to verify the CFA suitability, the KMO measure of sampling adequacy was calculated [48]. In this coefficient, a value of less than 0.5 indicates that the correlation between the variables is not sufficiently significant; therefore, it is inadvisable to analyze the inter-variable relationships. Furthermore, the Bartlett sphericity test was performed to verify that the correlations between the variables are not null [49].
The most commonly used indices to assess the fit of a CFA model are the Comparative Fit Index (CFI), the Tucker Lewis Index (TLI) and the Root Mean Squared Error Approximation (RMSEA). In order to consider that a model has a good fit, the following values are recommended: CFI ≥ 0.9, TLI ≥ 0.95 and RMSEA < 0.08 [50].
As in the pilot phase, the data were initially subjected to descriptive analysis to assess item comprehensibility.
Statistical software R (R Foundation for Statistical Computing, Vienna, Austria.) was used for data analysis (version 4.0.2).

2.4. Sample and Data Collection

The first panel of experts was composed of eight nurses with advanced PC training, extensive clinical and university teaching experience; with connections to the research group through AECPAL.
The second panel of experts was formed by 10 palliative-care nurses from AECPAL.
In the pilot phase, 31 nurses were recruited and included through non-probabilistic sampling during the months of November and December 2020. These participants shared similar characteristics to those who are targeted by the questionnaire under validation. All of them met the inclusion criteria as nurses working in the primary or home healthcare area in Spain.
Finally, for the validation phase there is no consensus regarding the size of the sample size needed to apply a factor analysis. Some authors set different criteria that varies between five and 10 participants per item or a minimum of 300 [51], while others only consider that a minimum of 300 is needed in order to obtain reliable results [52,53], or that at least five individuals per item are counted on [54,55].
In this phase, 344 nurses were recruited. Only 339 met the inclusion criteria. These were the same as in the pilot phase. The sampling was non-probabilistic during January and February 2021.
Given the exceptional circumstances created by the state of alarm in Spain, data collection was conducted online via Google Forms. The great added difficulty for the online selection of participants was to carry out probabilistic sampling due to the voluntary self-selection of the participants. To avoid this bias, we used recruitment strategies through natural leaders, nursing forums and Spanish associations for primary nursing by means of a snowball technique through social media where the target was nursing professionals, highlighting those participants who adequately represent all the strata of the population under study. These same circumstances have been evidenced in similar studies [56,57,58]. The online self-report questionnaire contained a brief introduction, the objective of the study, inclusion criteria and the need for consent for participation, guaranteed anonymity, confidentiality and the possibility of withdrawal.

2.5. Ethical Considerations

The present study was reviewed and approved by the Drug Research Ethics Committee of the Hospital Universitario Dr. Peset (Research Project EAPCP19-V01 and CEIM code 11/20). Potential study participants were provided with a detailed description of the study and were assured of confidentiality. Written, informed consent was obtained from each participant. They were also informed of the voluntary nature of the study participation and completion without any negative consequences.

3. Results

The participants’ socio-demographic data are shown in Table 2.

3.1. Phase 1: Theoretical Basis and Generation of Items

3.1.1. Step 1

For the first version of the instrument, 67 items were drawn up (Table 1). Thirty-five items to assess theoretical knowledge (theoretical block), with dichotomous answers (yes, no, do not know—no answer). Thirty-two items evaluated the practical application or practical block (measured by a 5-point Likert scale: never, rarely, sometimes, frequently and always).

3.1.2. Step 2

Kendall’s W test for inter-judge concordance resulted in 0.421 for relevance, 0.508 for clarity, 0.484 for adequacy and 0.429 for all three criteria together. In addition, a p-value very close to <0.001 was obtained in all four tests. On this basis, the null hypothesis (Ho) was rejected, concluding that concordance was assumed in the judges’ assessment.
In the experts’ judgment, the analyses showed a Cronbach’s alpha 0.960 in relevance, 0.978 in clarity and 0.967 in adequacy. However, it was decided to remove items with low scores on relevance and adequacy, thus eliminating one item from the practical block on relevance (A5) and two items from the theoretical block on adequacy (A1 and A4). Items with low clarity scores were reformulated as a result of textual contributions from experts, through consensus. This affected one item from the theoretical block and another from the practical block in clarity (both from A4).
These modifications led to the second version of the instrument, with 33 items in the theoretical block and 31 in the practical block.

3.2. Phase 2: Pilot Study

The average comprehension of the items of the theoretical block was 4.69 and 4.76 for the practical block, with a maximum score of 5.
However, those items that obtained a score below 3 in comprehensibility by one of the subjects were reformulated by expert consensus. Consequently, the items in theoretical block 5 and 6 (A1), 10 and 12 (A2), 20 (A3) and 25 (A4) and practical block 53 (A4) and 63 (A5) were modified.
In this phase the KR20 coefficient for the theoretical block was 0.828.
Cronbach’s alpha for the practical block was 0.943.
These values indicated internal consistency to both the theoretical and practical part, giving rise to the third version of the questionnaire.

3.3. Phase 3: Validation

The mean comprehension score was 4.68 in the theoretical block and 4.78 in the practical block.
An average of 17 min was taken to complete the questionnaire.
In the reliability analysis to determine the internal consistency of each part or block of the questionnaire (theoretical and practical) items 7, 14 and 16 of the theoretical block were deleted as they did not present variability (all subjects answered the same option).
Some items were observed to have very low homogeneity indices (below 0.1). Then, the items with the lowest item-total correlation were eliminated one by one until all these indices were equal to or greater than 0.1. Following this criterion, items 6 and 22 of the same theoretical block were discarded. In the practical block all items showed reliability.
The KMO measure of sampling adequacy in the set of variables was 0.873.
Bartlett’s sphericity test provided sufficient statistical evidence to disprove that the correlation matrix was an identity matrix; accordingly factor analysis was performed in both blocks (χ2 = 8386, df = 1830, p-value = <0.001).
The model was constructed without the previously discarded items (6, 7, 14, 16 and 22). When constructing this model, it was observed that the estimated variance of item 3 was negative and its weight in the model factor greater than 1 (Heywood case), therefore this item was removed from the analysis. Item 19 presented multicollinearity with items 47 and 55 in the practical block; therefore, it was removed from the model. Item 62 in the practical block led to convergence problems in the model, and therefore it was also discarded.
Finally, after deleting these items, the resulting model presented a good fit. Again, we assessed the reliability of the constructs through internal consistency analysis. In this case, item 27 presented a homogeneity index lower than 0.1. Therefore, the CFA was discarded and repeated. After this, item 11 presented convergence problems and was removed from the model.
Subsequently, the panel of experts decided to withdraw item 28 to unify the evaluation criteria by areas of the instrument, even though it fulfilled statistical criteria.
Questionnaire reliability was 0.700 in the theoretical block (KR20 Index) and 0.941 in the practical block (Cronbach’s alpha).
Table 3 shows the load or weight of the items in the factors of the model without the deleted items and the final internal consistency and correlation between the items of the theoretical and practical block.
After the modifications described above, the model converges and shows an adequate fit, specifically CFI = 0.977, TLI = 0.977 and RMSEA = 0.05. The correlation between the two factors in the model is ρ = 0.63.

4. Discussion

The psychometric testing of the INCUE questionnaire indicated that the model defined by the researchers is adequate and presents a good fit, according to the criteria established in the CFA [39] regarding reliability, internal consistency and validity of the construct. Both theoretical and practical blocks demonstrated separate internal consistency. This property would make independent use feasible. However, the CFA considers both blocks together.
The reliability scores (KR20 0.700 and α 0.941) are similar to those obtained by other instruments in Spanish such as the PCQN (KR20 0.72 and α 0.67) [36], Rotterdam MOVE2PC (α 0,786) [37], EACP (α 0.807) [29], or the Self-Efficacy in Palliative Care Scale (α 0.944) [31], or that of other instruments developed and validated in English such as the PCKT (α 0.87) [33] or the three PANA questionnaires (α 0.69, α 0.81 and α 0.80) [38].
The INCUE questionnaire differs from the other instruments cited herein in that it evaluates knowledge and practical application of the same, giving similar importance to all areas of care [21,22,36]. What is more, the INCUE questionnaire evaluates the practical application of knowledge in the care of people with palliative needs, and the frequency (never, rarely, sometimes, frequently and always) with which this is performed. This is not assessed by any of the other instruments reviewed. Subjectivity of the respondent is avoided by eliminating questions as to whether he/she feels prepared [31] or placing them in a hypothetical care-related situation [38]. In this respect, a nurse may feel prepared to perform the care-related task but, nevertheless, classify the situation as difficult. The crux of the matter is whether he/she actually put this into practice, despite the complexity.
Furthermore, the INCUE questionnaire enables a correlation to be drawn between the theoretical block (knowledge or what we know) and its application in the practice of care (what we do), since this correlation is not always proportional [20]. Establishing in which areas primary or home healthcare nurses have shortcomings in knowledge or in its practical application would facilitate the identification of training needs and subsequent training pinpointing these areas. In this way, nurses could acquire the minimum or basic knowledge necessary for end-of-life care of patients and their relatives. In addition, shortcomings in practical application despite sufficient theoretical knowledge could be due to factors other than training (overload, lack of material resources, etc.) and would thus require assessment to rule out these potential causes.
Also, one desirable quality in an instrument is discrimination of participants. In this respect, the INCUE questionnaire differs from other instruments in that it exclusively targets nurses, adapting to the competencies of this discipline and the care they perform [33]. In the same vein, it is not aimed at nursing students. We understand that they are still in the process of acquiring knowledge and their clinical practice is not independent, but supervised.
In addition, the questionnaire targets only primary and home healthcare nurses in the Spanish context. Although the theoretical block is applicable to all nurses, regardless of where they practice their profession, the practical block is not applicable to hospital nurses. Only home healthcare nurses can monitor patients and those grieving at home and respond to items 32, 38, 39, 40 and 41 (Supplementary Materials Table S1).
Different scoring criteria were established for the theoretical and practical block, to reach the minimum standard necessary to effectively care for these patients and their relatives. In the first block, agreement was reached to obtain 75% of correct answers in the areas of four items (1 and 4) and 80% in the areas of five items (2, 3 and 5), allowing only one error per area. For the practical block, a minimum score of 18 points per area was established, out of a total of 24 potential points; also representing 75% of the total. The score given to each response was 0 points to “never“, 1 point to “rarely”, 2 points if they respond “sometimes”, 3 points to “frequently”, and 4 points to “always”. Any area in which the threshold score is not obtained will require training.
The final questionnaire consists of 23 items in the theoretical block and 30 in the practical block, and is effective and user-friendly due to its duration.
A few limitations should be taken into consideration in interpreting the results of this study. First, the validation procedure has been described in detail to provide full process information. The use of a panel of experts is criticized by some authors due to possible alterations in validity [59]. However, recommendations were followed with respect to the number of experts [60] on the double panel and the necessary methodological and statistical rigor measures were taken to interpret and apply their results correctly [42].
During the online selection of the sample the participants’ IP was not tracked, given that it does not guarantee the hindrance of a subject answering through two different IP directions. Different individuals can use the same IP in a professional context nonetheless.
Although the sample size in the psychometric tests was lower than that of some studies undertaken to develop and validate instruments [33], it was similar to others [17,36]. Moreover, it exceeded the number recommended by some authors [51,53,54,55], and that of other similar studies [30,35]; being sufficient for statistical analysis. Second, the questionnaire has been validated in the Spanish context and targets primary and home healthcare nurses, thus it is not applicable in other contexts or professional profiles. It would be interesting to validate the INCUE questionnaire in other care-related settings or contexts, once suitably adapted.

5. Conclusions

The INCUE questionnaire is suitable to assess knowledge in palliative care and its application in clinical practice of primary and/or community healthcare nurses and in the home environment. This instrument pinpoints specific shortcomings in the defined areas, identifying training needs. This facilitates specific training based on these needs, in order to acquire the minimum or basic knowledge necessary for the end-of-life care of dying patients and of their relatives.
Those responsible for the training of nurses need to objectively know the gaps and training needs when planning an effective training program. Comprehensive end-of-life care for patients and their relatives requires nurses to acquire and to exercise their skills.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/ijerph182010995/s1, Table S1: The INCUE questionnaire to assess primary and home healthcare nurses’ basic training needs in palliative care.

Author Contributions

Conceptualization, I.G.-S., E.C.-A., A.A.C. and M.L.D.l.R.E.; methodology, I.G.-S., E.C.-A., M.L.D.l.R.E., J.C.M. and S.F.P.; software, E.C.-A., M.L.D.l.R.E., J.C.M., J.B.L. and S.F.P.; validation, I.G.-S., E.C.-A., A.A.C., M.L.D.l.R.E., J.C.M., C.H.N., J.B.L. and S.F.P.; formal analysis, I.G.-S., E.C.-A. and M.L.D.l.R.E.; investigation, I.G.-S., E.C.-A., A.A.C., M.L.D.l.R.E., J.C.M., C.H.N., J.B.L. and S.F.P.; resources, C.H.N., J.B.L. and S.F.P.; data curation, I.G.-S. and E.C.-A. writing—original draft preparation, I.G.-S. and E.C.-A.; writing—review and editing, I.G.-S., E.C.-A., J.C.M. and M.L.D.l.R.E.; visualization, I.G.-S., E.C.-A., A.A.C., M.L.D.l.R.E., J.C.M., C.H.N., J.B.L. and S.F.P.; supervision, I.G.-S. and E.C.-A.; project administration, I.G.-S., E.C.-A. and A.A.C.; funding acquisition, I.G.-S., E.C.-A., A.A.C. and C.H.N. All authors have read and agreed to the published version of the manuscript.

Funding

This questionnaire is being developed as part of a project to pinpoint the training needs of primary care nurses in the field of palliative care at the Dr Peset Health Department in Valencia (Project EAPCP19-V01). This project has received funding under the first Call for R + D Grants in Nursing 2019 (UGP-19-258), from the Fundació per al Foment de la Investigació Sanitària i Bioèdica de la Comunitat Valenciana (Fisabio).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Drug Research Ethics Committee of the HOSPITAL UNIVERSITARIO DR. PESET (CEIM code 11/20).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Acknowledgments

The authors would like to acknowledge all the participants and experts involved in this study for their participation and contribution, especially to Paqui Corpas and Jordi Ripoll (Fisabio).

Conflicts of Interest

The authors declare no conflict of interest. The funders collaborated in the statistical analysis and interpretation of the data as methodological advisers.

References

  1. OMS. Cuidados Paliativos. Available online: https://www.who.int/cancer/palliative/es/ (accessed on 18 April 2021).
  2. Sepúlveda, C.; Marlin, A.; Yoshida, T.; Ullrich, A. Palliative care. J. Pain Symptom Manage. 2002, 24, 91–96. [Google Scholar] [CrossRef]
  3. Better Palliative Care for Older People. Available online: https://apps.who.int/iris/handle/10665/107563 (accessed on 15 April 2021).
  4. Ministerio de Sanidad Política Social e Igualdad. Estrategia en Cuidados Paliativos del Sistema Nacional de Salud. Available online: https://www.mscbs.gob.es/organizacion/sns/planCalidadSNS/docs/paliativos/cuidadospaliativos.pdf (accessed on 15 April 2021).
  5. González López-Valcárcel, B. Formación y Empleo de Profesionales Sanitarios en España. Un Análisis de Desequilibrios. Gac Sanit. 2000, 14, 237–246. [Google Scholar] [CrossRef] [Green Version]
  6. Gómez Batiste-Alentorn, X.; Amblás-Novellas, J.; Lasmarías Martínez, C.; Calsina-Berna, A. Manual de Atención Integral de Personas con Enfermedades Crónicas Avanzadas: Aspectos Clínicos, 1st ed.; Elsevier: Barcelona, Spain, 2017. [Google Scholar]
  7. Deravin Malone, L.; Anderson, J.; Croxon, L. Are newly graduated nurses ready to deal with death and dying? A literature review. Nurs Palliat Care. 2016, 1, 89–93. [Google Scholar] [CrossRef] [Green Version]
  8. Mallory, J.L. The impact of a palliative care educational component on attitudes toward care of the dying in undergraduate nursing students. J. Prof. Nurs. 2003, 19, 305–312. [Google Scholar] [CrossRef] [Green Version]
  9. Kwekkeboom, K.L.; Vahl, C.; Eland, J. Companionship and education: A nursing student experience in palliative care. J. Nurs Educ. 2005, 44, 169–176. [Google Scholar] [CrossRef] [PubMed]
  10. Mutto, E.M.; Errázquin, A.; Rabhansl, M.M.; Villar, M.J. Nursing education: The experience, attitudes, and impact of caring for dying patients by undergraduate Argentinian nursing students. J. Palliat. Med. 2010, 13, 1445–1450. [Google Scholar] [CrossRef] [PubMed]
  11. White, K.R.; Coyne, P.J. Nurses’ perceptions of educational gaps in delivering end-of-life care. Oncol. Nurs. Forum. 2011, 38, 711–717. [Google Scholar] [CrossRef] [PubMed]
  12. Prem, V.; Karvannan, H.; Kumar, S.P.; Karthikbabu, S.; Syed, N.; Sisodia, V.; Saroja, J. Study of nurses’ knowledge about palliative care: A quantitative cross-sectional survey. Indian J. Palliat Care. 2012, 18, 122–127. [Google Scholar] [PubMed]
  13. Marchán Espinosa, S. Afrontamiento del profesional de enfermería ante la muerte de pacientes en unidades de paliativos y oncología. Nure Investig. Rev. Científica De Enfermería 2016, 13, 1–12. [Google Scholar]
  14. Jors, K.; Seibel, K.; Bardenheuer, H.; Buchheidt, D.; Mayer-Steinacker, R.; Viehrig, M.; Xander, C.; Becker, G. Education in end-of-life care: What do experienced professionals find important? J. Cancer Educ. 2016, 31, 272–278. [Google Scholar] [CrossRef]
  15. De Vlieger, M.; Gorchs, N.; Larkin, P.; Porchet, F. Palliative nurse education: Towards a common language. Palliat. Med. 2004, 18, 401–403. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Codorniú, N.; Guanter, L.; Utor, L. Competencias Enfermeras en Cuidados Paliativos, 1st ed.; SECPAL: Madrid, Spain, 2013. [Google Scholar]
  17. Ross, M.M.; McDonald, B.; McGuinness, J. The palliative care quiz for nursing (PCQN): The development of an instrument to measure nurses’ knowledge of palliative care. J. Adv. Nurs. 1996, 23, 126–137. [Google Scholar] [CrossRef]
  18. Bransford, J.D.; Schwartz, D.L. Rethinking transfer: A simple proposal with multiple implications. Rev. Res. Educ. 1999, 24, 61. [Google Scholar]
  19. Feíto Grande, L. Ética y Enfermería; Universidad Pontifica de Comillas: Madrid, Spain, 2009; pp. 203–205. [Google Scholar]
  20. García Salvador, I. Capacitación Y Necesidades Formativas Básicas En Cuidados Paliativos De Enfermería En Hospital De Agudos; XI Jornadas SECPAL: Valencia, Spain, 2015. [Google Scholar]
  21. Lester, D. The collett-lester fear of death scale: The original version and a revision. Death Stud. 1990, 14, 451–468. [Google Scholar] [CrossRef]
  22. Bugen, L.A. Coping: Effects of death education. Omega 1981, 11, 175–183. [Google Scholar] [CrossRef]
  23. Río-Valle, J.S. Validación De La Versión Española De La Escala De Afrontamiento De La Muerte Y Del Perfil Revisado De Actitudes Hacia La Muerte: Estudio Comparativo Y Transcultural. Puesta En Marcha De Un Programa De Intervención. Ph.D. Thesis, Universidad de Granada, Granada, Spain, 2007. [Google Scholar]
  24. Sullivan, A.M.; Lakoma, M.D.; Block, S.D. The status of medical education in end-of-life care: A national report. J. Gen. Intern. Med. 2003, 18, 685–695. [Google Scholar] [CrossRef] [Green Version]
  25. Bermejo, J.C.; Villacieros, M.; Hassoun, H. Actitudes hacia el cuidado de pacientes al final de la vida y miedo a la muerte en una muestra de estudiantes sociosanitarios. Med. Pal. 2018, 25, 168–174. [Google Scholar] [CrossRef]
  26. Abu Hasheesh, M.O.; Al-Sayed AboZeid, S.; Goda El-Zaid, S.; Alhujaili, A.D. Nurses’ characteristics and their attitudes toward death and caring for dying patients in a public hospital in Jordan. Health Sci, J. 2013, 7, 384–394. [Google Scholar]
  27. Frommelt, K.H.M. Attitudes toward care of the terminally ill: An educational intervention. Am. J. Hosp. Palliat. Care 2003, 20, 13–22. [Google Scholar] [CrossRef] [PubMed]
  28. Kain, V.J. Pilot study of an instrument to measure attitudes to palliative care practice in neonatal intensive care nursing. Collegian 2007, 14, 16–20. [Google Scholar] [CrossRef]
  29. Ascencio Huertas, L. Adaptación en español de la escala de actitudes ante cuidados paliativos: Confiabilidad y análisis factorial. Psicooncologia 2016, 12, 367. [Google Scholar] [CrossRef] [Green Version]
  30. Mason, S.; Ellershaw, J. Assessing undergraduate palliative care education: Validity and reliability of two scales examining perceived efficacy and outcome expectancies in palliative care. Med. Educ. 2004, 38, 1103–1110. [Google Scholar] [CrossRef]
  31. Herrero-Hahn, R.; Montoya-Juárez, R.; Hueso-Montoro, C.; Martí-García, C.; Salazar-Blandón, D.A.; García-Caro, M.P. Cultural adaptation, validation, and analysis of the self-efficacy in Palliative Care Scale for use with Spanish nurses. Int. J. Environ. Res. Public Health. 2019, 16, 4840. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Kwon, S.H.; Yang, S.K.; Park, M.H.; Choe, S.O. Assessment for the needs to develop hospice training program for nurses. Korean J. Hosp. Palliat Care. 2008, 11, 147–155. [Google Scholar]
  33. Nakazawa, Y.; Miyashita, M.; Morita, T.; Ueda, M.; Oyagi, Y.; Ogasawara, T. The palliative care knowledge test: Reliability and validity of an instrument to measure palliative care knowledge among health professionals. Palliat Med. 2009, 23, 754–766. [Google Scholar] [CrossRef]
  34. Guevara-Valtier, M.C.; Santos-Flores, J.M.; Santos-Flores, I.; Valdez-Ramírez, F.J.; Garza-Dimas, I.Y.; Paz-Morales, M.A.; Gutiérrez-Valver, J.M. Conocimiento de enfermería sobre cuidados paliativos en centros de primer y segundo nivel de atención para la salud. Rev. CONAMED. 2017, 22, 170–173. [Google Scholar]
  35. Al-Ansari, A.M.; Suroor, S.N.; AboSerea, S.M.; Abd-El-Gawad, W.M. Development of palliative care attitude and knowledge (PCAK) questionnaire for physicians in Kuwait. BMC Palliat Care. 2019, 18, 49. [Google Scholar] [CrossRef] [Green Version]
  36. Karacsony, S.; Good, A.; Chang, E.; Johnson, A.; Edenborough, M. An instrument to assess the education needs of nursing assistants within a palliative approach in residential aged care facilities. BMC Palliat Care. 2019, 18, 61. [Google Scholar] [CrossRef]
  37. Chover-Sierra, E.; Martínez-Sabater, A.; Lapeña-Moñux, Y.R. An instrument to measure nurses’ knowledge in palliative care: Validation of the Spanish version of Palliative Care Quiz for Nurses. PLoS ONE 2017, 12, e0177000. [Google Scholar]
  38. Franco Pujante, L. Adaptación Transcultural Y Validación De La Versión Española De La Escala Rotterdam MOVE2PC Para Enfermeras. Master’s Final Project. Universitat de Valencia, Valencia, Spain. 2017. Available online: http://www.cuidarypaliar.es/wp-content/uploads/2016/12/TFM-con-portada.pdf (accessed on 13 January 2020).
  39. Batista-Foguet, J.M.; Coenders, G.; Alonso, J. Análisis factorial confirmatorio. Su utilidad en la validación de cuestionarios relacionados con la salud. Med. Clin. 2004, 122, 21–27. [Google Scholar] [CrossRef]
  40. Streiner, D.L.; Norman, G.R.; Cairney, J. Health Measurement Scales, 5th ed; Oxford University Press: Oxford, UK, 2015. [Google Scholar]
  41. Álvarez, R.R. Elaboración Y Validación De Un Cuestionario Multimedia Y Multilingüe De Evaluación De La Autoestima; Granada Editorial Universidad de Granada: Granada, Spain, 2008. [Google Scholar]
  42. Escobar-Pérez, J.; Martínez, A. Validez de contenido y juicio de expertos: Una aproximación a su utilización. Avances en Medición 2008, 6, 27–36. [Google Scholar]
  43. García Salvador, I.; Vallés Martínez, P.; Guanter Peris, L.; Utor Ponce, L.; Pérez Yuste, P.; López-Casero Beltrán, N.; Alonso, V.R.; Monfort, P.C.; Sierra, E.C.; Plaza, T.E.; et al. Recommendations About Training in Palliative Care in Nursing Degree Of The Spanish Palliative Care Nurses Association, 1st ed.; Inspira Network: Madrid, Spain, 2019; Volume 1, pp. 273–280. [Google Scholar]
  44. Bakieva, M. Diseño Y Validación De Un Instrumento Para Evaluar La Colegialidad Docente. Ph.D., Universitat de Valencia. Available online: https://www.uv.es/gem/gemeduco/publicaciones/Disenyo_y_validacion_de_un_instrumento_para_evaluar_la_colegialidad_docente_Tesis_doctoral.pdf (accessed on 3 March 2020).
  45. Nunnally, J.C. McGraw-Hill Series in Psychology. In Psychometric Theory, 2nd ed.; Mcgraw-Hill College: New York, NY, USA, 1978; pp. 9–17. [Google Scholar]
  46. Revelle, W.R. Psych: Procedures for Personality and Psychological Research. Available online: https://www.scholars.northwestern.edu/en/publications/psych-procedures-for-personality-and-psychological-research (accessed on 18 April 2021).
  47. Rosseel, Y. lavaan: An R Package for Structural Equation Modeling. J. Stat. Softw. 2012, 48, 1–36. Available online: https://www.jstatsoft.org/article/view/v048i02 (accessed on 6 March 2020). [CrossRef] [Green Version]
  48. Barbero García, M.I.; Vila Abad, E.; Suárez Falcón, J.C. Psicometria, 2nd ed.; Universidad Nacional de Educación a Distancia: Madrid, Spain, 2006. [Google Scholar]
  49. Bartlett, M.S. Tests of significance in factor analysis. Br. J. Stat. Psychol. 1950, 3, 77–85. [Google Scholar] [CrossRef]
  50. Kline, R.B. Principles and Practice of Structural Equation Modeling, 4th ed.; Guilford Publications: New York, NY, USA, 2015. [Google Scholar]
  51. Martínez-Arias, R. Psicometría: Teoría De Los Test Psicológicos Y Educativos; Síntesis: Madrid, Spain, 1995. [Google Scholar]
  52. Comrey, A.L.; Lee, H.B. A First Course in Factor Analysis, 2nd ed.; Lawrence Erlbaum: Hillsdale, NJ, USA, 1992. [Google Scholar]
  53. Snook, S.C.Y.; Gorsuch, R.L. Component analysis versus common factor analysis: A Monte Carlo study. Psychol. Bull. 1989, 106, 148–154. [Google Scholar] [CrossRef]
  54. Stevens, J. Applied Multivariate Statistics for The Social Sciences; Lawrence Erlbaum: Hillsdale, NJ, USA, 1992. [Google Scholar]
  55. DeVon, H.A.; Block, M.E.; Moyle-Wright, P.; Ernst, D.M.; Hayden, S.J.; Lazzara, D.J.; Kostas-Polston, E. A psychometric toolbox for testing validity and reliability. J. Nurs. Scholarsh. 2007, 39, 55–164. [Google Scholar] [CrossRef] [PubMed]
  56. De la Rica Escuín, M.; García-Navarro, E.B.; García Salvador, I.; De la Ossa Sendra, M.J.; Chisbert-Alapont, E. Acompañamiento a los pacientes al final de la vida durante la pandemia COVID-19. Med. Pal. 2020, 27, 181–191. Available online: https://www.medicinapaliativa.es/Acompanamiento-a-los-pacientes-al-final-de-la-vida-durante-la-pandemia-por-COVID-19534 (accessed on 14 November 2020).
  57. Rojas-Ocaña, M.J.; Araujo-Hernández, M.; Romero-Castillo, R.; Román-Mata, S.S.; García-Navarro, E.B. Nursing as a Sustainability Factor of the Health System during the COVID-19 Pandemic: A Qualitative Study. Sustainability 2020, 12, 8099. [Google Scholar] [CrossRef]
  58. Yu, H.; Huang, C.; Chin, Y.; Shen, Y.; Chiang, Y.; Chang, C.; Lou, J. The Mediating Effects of Nursing Professional Commitment on the Relationship between Social Support, Resilience, and Intention to Stay among Newly Graduated Male Nurses: A Cross-Sectional Questionnaire Survey. Int. J. Environ. Res. Public Health 2021, 18, 7546. [Google Scholar] [CrossRef]
  59. Lynn, M.R. Determination and quantification of content validity. Nurs. Res. 1986, 35, 382–385. [Google Scholar] [CrossRef]
  60. Hyrkäs, K.; Appelqvist-Schmidlechner, K.; Oksa, L. Validating an instrument for clinical super-vision using an expert panel. Int. J. Nurs. Stud. 2003, 40, 619–625. [Google Scholar] [CrossRef]
Ijerph 18 10995 g001 550
Figure 1. Investigación Cuidados Enfermeros/Investigation into Nurses’ Care Understanding of End-of-Life (INCUE) questionnaire developmental process.
Figure 1. Investigación Cuidados Enfermeros/Investigation into Nurses’ Care Understanding of End-of-Life (INCUE) questionnaire developmental process.
Ijerph 18 10995 g001
Table
Table 1. The INCUE questionnaire one item generation.
Table 1. The INCUE questionnaire one item generation.
AreasPart 1: Theoretical Knowledge
Theoretical Block (n = 35)		Part 2: Practical Application
Practical Block (n = 32)
Number of ItemsResponse TypeNumber of ItemsResponse Type
1. Principles of PC7Yes6Never
2. Symptom management and specific care plans7 6Rarely
3. Coping with loss and death8No6Sometimes
4. Communication skills7 6Often
5. Ethical and legal aspects6Do not know/no answer8Always
Abbreviations: Palliative Care (PC).
Table
Table 2. Sample socio-demographic data.
Table 2. Sample socio-demographic data.
VariableExpert Panel 1
(n = 8)		Expert Panel 2
(n = 10)		Pilot Phase
(n = 31)		Validation Phase
(n = 339)
Age (years), M ± SD49.6 ± 1.345.7 ± 1241.8 ± 1245.5 ± 11.2
Gender, n (%)
Female7 (75)9 (90)27 (87.1)280 (82.6)
Male1 (25)1 (10)4 (12.9)59 (17.4)
Maximum level of professional qualification
Doctorate2 (25)2 (20)1 (3.2)17 (5)
Nursing Specialization0 (0)0 (0)2 (6.5)53 (15.6)
Master degree6 (75)8 (80)5 (16.1)88 (26)
Diploma/Graduate0 (0)0 (0)23 (74.2)181 (53.4)
Current position
Reference Nurse in Palliative Care0 (0)10 (100)1 (3.2)45 (13.3)
Nurse and Center Coordinator0 (0)0 (0)2 (6.5)0 (0)
Nurse8 (100)0 (0)27 (87.1)208 (61.4)
Center Coordinator0 (0)0 (0)1 (3.2)33 (9.7)
Community case manager0 (0)0 (0)0 (0)53 (15.6)
Professional experience (years), M ± SD29.2 ± 2.220.7 ± 6.513.7 ± 9.1821 ± 11.7
Palliative-care training
Yes8 (100)10 (100)25 (80.6)292 (86.1)
No0 (0)0 (0)6 (19.4)47 (13.9)
Level of training in PC
Advanced (Master or PhD)8 (100)10 (100)0 (0)51 (15.0)
Intermediate (80–150 h)0 (0)0 (0)10 (32.2)92 (27.2)
Basic (25–80 h)0 (0)0 (0)15 (48.4)154 (45.4)
Don’t know/No answer0 (0)0 (0)6 (19.4)42 (12.4)
Abbreviations: Mean (M), Standard deviation (SD), hours (h).
Table
Table 3. Factors weighting of the items in the model and final block reliability measures.
Table 3. Factors weighting of the items in the model and final block reliability measures.
Theoretical Block (without the Questions 3, 6, 7, 11, 14, 16, 19, 22, 27 and 28)Practical Block (without Question 62)
ItemFactor WeightItem-Total CorrelationKR20 on Omitting the QuestionKR20 IndexItemFactor WeightItem-Total CorrelationCronbach’s Alpha
on Omitting the Question		Cronbach’s Alpha
10.560.1090.7000.700340.690.4150.9410.941
20.640.2420.693 350.760.7580.938
40.430.2630.690 360.800.7060.939
50.520.2550.692 370.710.6350.939
80.750.4320.677 380.690.6500.939
90.370.2060.695 390.680.5470.940
100.240.1190.705 400.740.6890.938
120.550.3480.681 410.610.6530.939
130.570.2900.688 420.600.6950.938
150.750.1710.698 430.490.3950.942
170.440.3040.686 440.810.6810.938
180.680.3390.686 450.650.7010.938
200.390.1480.703 460.830.7760.937
210.770.3570.680 470.840.6600.939
230.460.1620.697 480.660.6490.939
240.20.2120.697 490.730.6580.939
250.540.2720.689 500.640.6720.938
260.480.3840.678 510.520.5790.940
290.620.2130.695 520.360.5680.940
300.460.2230.694 530.680.4600.942
310.450.2180.694 540.690.6800.939
320.470.3390.683 550.760.5770.940
330.450.3510.681 560.800.7400.938
570.710.7820.938
580.690.5570.940
590.680.6630.939
600.740.6290.939
610.610.4290.941
630.600.3560.942
640.490.6050.940
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

García-Salvador, I.; Chisbert-Alapont, E.; Antonaya Campos, A.; Casaña Mohedo, J.; Hurtado Navarro, C.; Fernández Peris, S.; Bonías López, J.; De la Rica Escuín, M.L. Design and Validation of the INCUE Questionnaire: Assessment of Primary Healthcare Nurses’ Basic Training Needs in Palliative Care. Int. J. Environ. Res. Public Health 2021, 18, 10995. https://doi.org/10.3390/ijerph182010995

AMA Style

García-Salvador I, Chisbert-Alapont E, Antonaya Campos A, Casaña Mohedo J, Hurtado Navarro C, Fernández Peris S, Bonías López J, De la Rica Escuín ML. Design and Validation of the INCUE Questionnaire: Assessment of Primary Healthcare Nurses’ Basic Training Needs in Palliative Care. International Journal of Environmental Research and Public Health. 2021; 18(20):10995. https://doi.org/10.3390/ijerph182010995

Chicago/Turabian Style

García-Salvador, Isidro, Encarna Chisbert-Alapont, Amparo Antonaya Campos, Jorge Casaña Mohedo, Clara Hurtado Navarro, Silvia Fernández Peris, José Bonías López, and Maria Luisa De la Rica Escuín. 2021. "Design and Validation of the INCUE Questionnaire: Assessment of Primary Healthcare Nurses’ Basic Training Needs in Palliative Care" International Journal of Environmental Research and Public Health 18, no. 20: 10995. https://doi.org/10.3390/ijerph182010995

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop