Psychometric Evaluation of the Nursing Outcome Knowledge: Pain Management in People with Chronic Pain

Pain has a major impact on health and quality of life. Since the level of knowledge of painful conditions can influence how these are addressed and managed, assessing this knowledge in patients becomes crucial. As a result, it is necessary to have culturally adapted and validated instruments that specifically measure patients’ knowledge of chronic pain management. The objective of this study was to carry out the Spanish cultural adaptation and the validation of the outcome Knowledge: Pain Management of the Nursing Outcomes Classification (NOC) in patients with chronic pain, defined as extent of understanding conveyed about causes, symptoms, and treatment of pain. A three-stage study was designed: (1) translation and cultural adaptation through an expert panel, (2) content validation, (3) clinical validation. This study provides nurses with a Spanish version of this scale adapted to their context, as well as a set of structured indicators to measure patients’ knowledge about chronic pain. The results indicated that the culturally adapted Spanish version of the outcome Knowledge: Pain Management had a high level of content validity (CVI = 0.92), with 27 indicators being distributed between two factors. This version has been shown to be reliable in terms of inter-observer agreement (κ = 0.79) and internal consistency (α = 0.95). In conclusion, Knowledge: Pain Management has been shown to be reliable and valid to measure knowledge of chronic pain.


Introduction
Pain is a health issue that affects a large number of individuals [1,2]. In acute conditions, pain is commonly temporary. However, when pain lasts longer than 3 months it is considered to be chronic pain and has an important impact on health and quality of life [3,4]. Pain is understood as a complex experience shaped by a wide variety of biological, psychological, and social factors [3]. This complex combination of factors includes past experiences of pain, age, education, culture, ethnicity, and gender, all of which endow the individual with certain beliefs and knowledge about pain [5,6].
Healthcare professionals' knowledge of pain management and treatment has attracted considerable attention [7,8]. Conversely, patients' knowledge of their own resources and experiences with pain has had considerably less attention [9]. There is a need to assess and address gaps in this respect, as a significant proportion of the general public report having a lack of knowledge of chronic pain [10], which can influence how chronic pain is addressed and managed.
In this sense, there is evidence that knowledge about analgesic drugs is associated with adherence to analgesic therapy [11]. Caregivers having better knowledge of pain management is not only associated with them having fewer concerns about pain and medications, but also with their patients

Materials and Methods
A three-stage validation design was used for the outcome Knowledge: Pain management: (1) Spanish translation and cultural adaptation; (2) content validation; and (3) clinical validation.
Stage 1: The translation/back-translation method [29,30] was applied to the English version of the KPM (5th edition of the NOC) [31]. The label, definition, indicators, and measurement scale were translated into Spanish by a nurse with Spanish as her first language, who was an expert in nursing methodology. This translation was then compared with the Spanish versions of the KPM available in the 4th and 5th edition of the NOC [31][32][33]. A Spanish version was agreed upon and subsequently back-translated into English by a professional translator with English as his first language. This back-translation was submitted to the authors of the NOC at the Center for Nursing Classification & Effectiveness at the University of Iowa (Dr Swanson), who proposed a few modifications to the wording of some indicators. The first Spanish version was obtained as a result of this process.
Stage 2: A panel of 21 pain experts of Spanish nationality (19 nurses and two physiotherapists) assessed the content validity of the first Spanish version in two rounds of consensus using the Delphi technique [34]. The members of the panel had a mean amount of experience of 14.1 years in providing care to pain patients. Each expert independently scored the relevance of the indicators to the concept Knowledge about pain management, defined as the degree of understanding conveyed about causes, symptoms, and treatment of pain. The experts used a four-point Likert scale (1 = not appropriate, 2 = somewhat appropriate, 3 = moderately appropriate, 4 = completely appropriate), following the procedure proposed by Lynn [35] to calculate the content validity index (CVI). The same procedure was followed for the new dimensions proposed by the experts in the first round.
The CVI-indicator (CVI-i) was determined for each indicator, taking into consideration that, for six or more experts, this value should not be below 0.78 [35]. The modified kappa coefficient was also calculated for each indicator to assess the probability of random agreement among experts. For the overall score, the CVI-universal agreement (CVI-ua) and CVI-average (CVI-a) were calculated, considering CVI-a ≥ 0.80 to be an acceptable agreement [30,34,35].
Stage 3: An observational-longitudinal study was carried out to establish the reliability, validity, and sensitivity to change of the Spanish version of the KPM.

Setting
The study was conducted from March 2013 to April 2014 in 9 settings (primary care centers and hospitals) in the Andalusian Health Service in the provinces of Jaén (6) and Granada (3), Spain. Twenty nurses (16 women and four men) participated in the data collection. These nurses were working full time, had been working for more than 6 months in their respective healthcare centres, and had a mean experience of 20.7 years in caring for patients with chronic pain (Standard Deviation = 6.59). The nurses were working in the following units: Primary Care Team (50%), Oncology Impatient Unit (30%), Hospital Case Management (Advanced Practice Nurses) (10%), and Chronic Pain Unit (10%).

Sample
Data published by the authors of the validation of the NOC for use in people with chronic pain were used to estimate the size of the patient sample [36]. A minimum sample of 87 participants was deemed necessary assuming a kappa coefficient of 0.37, a 61% proportion of positive ratings for both observers, with a 95% confidence level. Patients were selected by non-probability sampling. Patients were invited to participate by collaborating nurses until a quota of 10 patients per centre was reached. The inclusion criteria were as follows: over 18 years of age; able to communicate in Spanish; and presence of chronic (cancer and non-cancer) pain for more than 6 months.

Main Research Variables
Knowledge: Pain management. The adapted Spanish version of KPM with 30 indicators, which was obtained after validation by experts, was used. These indicators measure the knowledge of how pain appears and is maintained, aspects related to medication and treatment, and various strategies and techniques for appropriate pain management. Twenty eight indicators were taken from the original English version [31] and a further two were proposed by the panel of experts: Multidimensional nature of pain (bio-psycho-social) and Benefits of physical exercise. The nurses assessed the patients and scored each indicator on a five-point Likert scale (from 1 = No knowledge to 5 = Extensive knowledge). Additionally, the option "Not applicable" was included, and the outcome Client satisfaction: Pain management was used to explore divergent validity.
Pain intensity. The Spanish version of the 11-point Numerical Pain Intensity Scale (NPIS) was used. The reliability and validity of this one-dimensional instrument are well-documented, being widely utilised for pain intensity assessment [37]. The aspects related to the experience of pain were measured using the Spanish version of a self-report questionnaire: Pain Coping Questionnaire (reduced version) (PCQ-R). The subscale "Seeking instrumental social support" was considered. This subscale assesses how individuals cope with pain by seeking advice, help, or information (problem-focused coping) on a 5-point Likert scale (0 = Never, 1 = Sometimes, 2 = Often, 3 = Very often, and 4 = Always). The psychometric properties of this questionnaire have been studied in the Spanish population. Internal consistency ranged from α = 0.75 to α = 0.94; test-retest reliability was r = 0.81, and convergent validity with other scales was as expected [38,39].
Ad hoc questions to patients. Table 1 shows the five questions posed to patients about self-perception about pain.

Data Collection
The principal investigator trained all collaborating nurses in the measurement of the outcome Knowledge: Pain Management and in the data collection procedure. Most of the interviews with primary care patients were conducted in the nursing offices of the healthcare centres, although some were conducted in the patients' homes. Inpatients were interviewed in a private area in the hospitalisation area. Patients in the Chronic Pain Unit were interviewed in the nurses' office of the unit.
An ad hoc questionnaire were used by the nurses to record the patients' demographic and clinical data, as well as their own demographic and professional data. Two assessments were conducted. In the first assessment, patients self-reported the NPIS, the PCQ-R, and the ad hoc questions about their knowledge and satisfaction. Two nurses then independently assessed the KPM, concealing their score from the other nurse (to measure inter-observer agreement). In the second assessment, conducted after a 20-30 day interval (to measure sensitivity to change), patients answered the ad hoc questions again and the KPM was reassessed by one of the nurses who conducted the first measurement.

Ethical Considerations
The project was approved by the Research Ethics Committee of Jaén, Spain. Nurses and patients were given detailed information on the project and signed an informed consent form. Participant anonymity and data confidentiality were preserved in compliance with current data protection laws in Spain.

Data Analysis
The data were collected in a spreadsheet (Microsoft Office Professional Plus 2013, Redmond, WA, USA) and analysed with SPSS (IBM ® SPSS ® statistics 19. IBM, International Business Machines Corporation, Armonk, NY, USA). Descriptive statistics were calculated. For nominal variables, frequencies and percentages were used. For continuous variables, means and standard deviations were used. Additionally, 95% confidence intervals were calculated for continuous variables.
An exploratory factor analysis (EFA) was performed using the principal components method with quartimax rotation to study the underlying conceptual structure of KPM. Sample adequacy was estimated with the Kaiser-Meyer-Olkin index and Bartlett's test of sphericity, with p > 0.05 indicating that the model is appropriate. The number of factors was determined using the parallel analysis method following Horn's criterion [40]. In the rotated factor matrix, indicators were considered to be saturated with values which were greater than 0.6 in one factor and with values which were smaller than 0.4 in the others.
Convergent and divergent validity was analysed using the multitrait-multimethod (MTMM) matrix [30,41]. In summary, Spearman's correlations were calculated between the KPM score and the following: (a) the score for the outcome Client satisfaction: Pain management (same method, different trait), to explore divergent validity; (b) the score for the subscale "Seeking instrumental social support" from the PCQ-R (different method, different trait), to explore divergent validity; (c) the scores for the three satisfaction questions: How satisfied are you with the care provided by the nurses in relieving your pain?, How satisfied are you with the treatment of your pain?, and How much have you managed to improve your pain with the treatment and care provided by professionals? (different method, different trait), to explore divergent validity; and (d) the scores for the two questions about knowledge of pain management: How much do you know about the causes, symptoms, and treatment of pain? and How well prepared are you to take care of yourself and to manage and control your painful condition? (different method, same trait), to explore convergent validity.
In order to assess reliability, internal consistency and inter-observer agreement were determined. Cohen's kappa coefficient with quadratic weighting was used to determine inter-observer agreement, considering kappa values ≤ 0.20 to reflect poor agreement; values of 0.21-0.40, mild agreement; 0.41-0.60, moderate agreement; 0.61-0.80, good agreement; and 0.81-1.00, excellent agreement. Internal consistency was calculated according to Cronbach's alpha coefficient, considering values > 0.80 to indicate a good level of consistency [34,41].
Sensitivity to change. The capacity of KPM to detect change was analysed by administering this outcome at two points in time separated by 20-30 days (baseline and final). Wilcoxon's rank-sum test was used to evaluate the mean agreement between the measurements, with statistical significance set at p < 0.05.

Stage 1. Translation and Cultural Adaptation
A Spanish adaptation of KPM, semantically equivalent to the English version, was obtained. The adapted version has modifications to the label, to 17 indicators, and to the measurement scale, compared to the 5th and 6th Spanish editions of the NOC [31][32][33].

Stage 2. Content Validation
After review by the panel of experts, a version with 30 indicators was obtained (28 from the original version; a further two were added). Overall, the CVI-ua was 0.33 and the CVI-a was 0.92. All indicators were rated as excellent with modified kappa scores ranging from 0.75 to 1.00 (Table 2). Thirteen indicators from the original version and two new indicators were eliminated due to the low agreement among the experts (with modified kappa scores ranging from 0.27 to 0.77).

Patients' Characteristics
The clinical validation study included 84 pain patients. Table 3 shows the patients' clinical and demographic characteristics. The most frequent nursing diagnoses (NANDA-I) were Chronic pain (42.9%) and Acute pain (11.9%). Patients were using different types of drugs (mean = 3.15; SD = 1.43), with scheduled non-opioid analgesics being more frequent (mean = 1.21; SD = 0.81). All 84 patients underwent a first (baseline) measurement (n = 84), and 44 patients had a second measurement (final). The mean interval between the initial and final measurements was 42.84 (SD = 19.35) days.
The overall mean score obtained through the outcome Knowledge: Pain Management was 3.24 (SD = 0.87) in the baseline measurement and 3.46 (SD = 0.77) in the final measurement. The lowest mean score was 2.52 (SD = 1.31) for the indicator "Potential medication interactions", and the highest mean score was 3.98 (SD = 0.88) for "Importance of informing health professional of all current medication". The scores for the 5 questions to patients about their knowledge and satisfaction with pain management are shown in Table 4. There is a slight improvement in the two questions about patients' knowledge and perceived readiness to manage this pain, but no improvement in the three questions about satisfaction with care. A preliminary EFA showed that three indicators, i.e., Safe use of prescribed medication (variance = 0.496), Effective heat/cold application (variance = 0.353), and Available community resources (variance = 0.279), presented low proportions of explained variance and low factor loading values. As a result, they were eliminated.
A second analysis with quartimax rotation showed a structure of 27 indicators distributed between two factors explaining 53.56% of the variance. The matrix of rotated factors shows their composition: (a) Knowledge about pain, made up of 25 indicators explaining 45.73% of the variance; (b) Non-prescription medication, made up of two indicators explaining 7.82% of the remaining variance (Table 5).

Multitrait-Multimethod Matrix
Analysis using MTMM makes it possible to assess the convergent and divergent validity of KPM (Table 6). For convergent validity, the KPM outcome score was moderately correlated with the two ad-hoc questions on patient knowledge: Q1 How much do you know about the causes, symptoms, and treatment of pain? (r = 0.40; p < 0.001); and Q2 How well prepared are you to take care of yourself and to manage and control your painful condition? (r = 0.39; p < 0.001).

Sensitivity to Change
No differences were found between the baseline and final scores for the outcome KPM (baseline score = 3.49; final score = 3.46 (Z = −0.68; p = 0.50)), nor for either of the two ad-hoc questions: How much do you know about the causes, symptoms, and treatment of pain? (Wilcoxon's Z = −1.42; p = 0.15) and How well prepared are you to take care of yourself and to manage and control your painful condition? (Wilcoxon's Z = −1.04; p = 0.29). However, in the analysis of indicators, statistically significant differences were observed in six of them between the baseline and final measurements: Strategies for preventive pain management (Z = 2.42; p = 0.01), Effective relaxation techniques (Z = 2.14; p = 0.03), Effective distraction (Z = 2.00; p = 0.046), Effective heat/cold application (Z = 2.40; p = 0.02), Benefits of ongoing self-monitoring of pain (Z = 2.43; p = 0.01), Available community resources (Z = 2.10; p = 0.03).

Discussion
This article report on the process to translate and adapt to Spanish context and culture the set of indicators included in the nursing outcome Knowledge: Pain Management. The psychometric properties (reliability and validity) of this outcome were tested in a sample of patients with pain. Nurses in Spanish-speaking contexts could use this adapted version of KPM to measure the degree of knowledge that their patients have about the causes, symptoms, and treatment of pain.

Cultural Adaptation
Following the principles of good practice [29,42], during the process of cultural adaptation we worked on the semantic and conceptual equivalence not only of the label and definition of the outcome, but also of its indicators and measurement scale. Since the KPM indicators are the specific elements that measure particular aspects of knowledge, their adaptation is especially relevant so that patients are able to understand what information is being requested from them and so that they can convey precise answers, and professionals are able to distinguish between ignorance and knowledge in the different dimensions that make up the KPM outcome. On the basis of the data from this study and other similar studies [43,44], this translation and cultural adaptation process results in a clearer and more comprehensible wording of the outcome definition and its indicators than that obtained by directly translating it from English into Spanish without expert assessment.

Content Validity
The content validity data suggest that the set of indicators is adequate for the conceptualisation of the KPM outcome in accordance with the recommendations made by Polit et al. [30], who proposed that CVI-a values greater than 0.90 indicate excellent content validity. The elimination of 13 indicators is a substantial reduction from the original version. Almost a third of the dimensions did not adequately represent the overall concept according to the group of experts, who even considered that some of them were not applicable in the care setting of this study, e.g., indicators Benefits of hypnosis (CVI-i = 0.27) and Benefits of biofeedback (CVI-i = 0.30), quite uncommon in the Spanish context. This invites reflection for two separate reasons. First, the KPM outcome has to contain the essential indicators that define it and adequately represent it to provide nurses with an operative list for their clinical work (the extensive initial list had 41 indicators). Secondly, it is necessary to conduct new content validity studies in other cultural settings to verify the content validity indices and the inclusion or exclusion of indicators resulting from this study.

Construct Validity
The two-factor structure proposed in the final model for the KPM outcome makes it possible to explain a significant percentage of the overall variance, although a fraction of the variance remains unexplained, so it is likely that other indicators may be added in the future. Further studies are required to verify the suitability of this structure in other clinical settings and types of pain patients. Factor 1, Knowledge about pain, includes a considerable number of indicators addressing various concepts of pain, such as treatment and improvement strategies (13), aspects related to medication (9), and, to a lesser extent, signs and symptoms associated with pain (3), which point to the complexity of this factor. Factor 2, Non-prescription medication, has only two indicators. The correlations of these indicators with the total are the lowest on the whole list, which suggests that factor 2 needs an improved definition of the overall set and/or its indicators. This distribution of outcome indicators into two factors may be useful for nurses when deciding which indicators to use in assessing patients' knowledge of different aspects related to pain and pain management, in line with other validation studies of other NOC outcomes [43][44][45].

Criterion Validity (Convergent and Divergent)
We found evidence of validity of convergent and divergent criteria for the KPM outcome. Convergent validity was verified drawing on the strong correlations with the specific questions about knowledge and ability to control pain, as well as on each of the two proposed factors. In addition, the KPM outcome is well differentiated from other related yet different concepts, such as satisfaction with pain management and ability to cope, as shown by the divergent validity analysis.
A more detailed analysis showed that some of the indicators referring to medication aspects had a weak correlation with the score on the ad hoc question about knowledge. At this point, it is pertinent to wonder whether the question was well configured in terms of content and scope for the population studied, as this suggests that knowledge about medication can be challenging for patients. The level of knowledge is associated with adherence to analgesic therapy [11], but knowledge may be insufficient by itself, as adherence may be influenced by other aspects such as motivation and behavioural skills [46], or by patients' attitudes and beliefs about medication [47].

Reliability
We found evidence of the reliability of KPM when used by nurses in patients with chronic pain. Inter-observer agreement was high for the overall set and most of the indicators. However, five indicators relating to medication had a low or moderate correlation [34], which may indicate that medication-related issues also pose challenges to nurses [48,49]. Internal consistency was high and there was adequate correlation between the total and each of the indicators, with no scores suggesting that any indicator should be eliminated [41,50].

Sensitivity to Change
It could not be established that KPM showed sensitivity to changes in patient knowledge in the overall score. Patients may not have experienced substantial changes in their knowledge which would explain why no significant changes were observed between the baseline and final scores. However, significant changes were observed in a group of indicators relating to various pain management and treatment strategies. In this study, the mean time interval between measurements was almost 43 days, longer than the interval in the study on the outcome Pain control (code 1605) [44], with 30-day intervals between measurements. Increasing the time between measurements may be the key to revealing sensitivity in populations of long-term pain patients. Studies involving an educational intervention may also provide further evidence on the sensitivity to change of the KPM outcome.
Although no validation studies have been identified with which to compare the psychometric tests in this study, the usefulness of KPM has been demonstrated in other studies. As such, KPM has been validated for certain nursing diagnoses, Lucena et al. [26] validated KPM using a group of experts for the following nursing diagnoses: Risk for Frail Elderly Syndrome and Frail Elderly Syndrome. KPM was included in the care plan of a patient with long-term left ventricular assist device (associated with the nursing diagnosis Acute pain) [27]. This approach to patients with acute (short-term) pain coincides with the small sample of patients in this study who presented with pain for more than one month. KPM was also considered useful in evaluating the evidence of quality of care provided by nurses in nursing homes in relation to the administrative rules measuring quality of care in the United States of America [28].

Limitations
A comprehension and cognitive equivalence test carried out with nurses, patients, healthy individuals, and even native speakers representing the target population (sex, age, level of education, diagnosis) may improve the cultural adaptation process and provide a better adjustment of the concepts used to the cultural characteristics of the given geographical area [42,51].
Homogeneity in the training of participating nurses cannot be ensured, as they were included in the study because they were willing to collaborate and this may, therefore, lead to an overestimation of reliability. Attempts have been made to control bias by training participants in the conditions of the study, by diversifying participation (from clinical settings to rural and urban health centres, hospitals, and specialised chronic pain units), and by seeking the participation of advanced practice nurses [52].
Patients were selected using convenience sampling and the sample size was relatively small. However, the representativeness of the study was improved by sampling at 9 different locations. Broader randomised samples and other additional care settings should be included to further increase knowledge of the psychometric properties of KPM. Some of the indicators included in the English version of this outcome were not included in the two-factor Spanish version, which may require further assessments. Additional studies are also necessary to increase the time between the baseline and the final measurements, seeking significant differences in the sensitivity to change of KPM and its indicators in patients with chronic pain.

Conclusions
The final version of the nursing outcome Knowledge: Pain Management, adapted to a Spanish context, includes 27 indicators. Evidence has been obtained of the reliability and validity of these indicators for use in nursing practice in Spanish-speaking contexts.
An internal structure of two factors, referred to as Knowledge about pain and Non-prescription medication, can be observed. This structure is novel and thus requires further research with larger patient samples, additional care settings, and acute pain scenarios. Being able to make an appropriate selection of indicators based on this two-factor structure may help nurses in their decision-making, especially the less experienced.
The present study highlights an interesting field of research. Improving the definition and conceptual configuration of the outcome KPM, presenting an operational list of indicators, and increasing the input of psychometric testing through further research will not only consolidate the reliability and validity of this outcome, but will also make it more useful in the complex process of caring for individuals with pain.
Author Contributions: P.L.P.-H. and J.C.B.-V. were responsible for the conception and design of the study, the collection and analysis of data, and the drafting of the manuscript.

Funding:
The present study was funded by the Fundación Pública Progreso y Salud of the Andalusian Public Health Service (Spain), PI-0543-2011. There is no conflict of interest.