Emergency Nursing-Care Patient Satisfaction Scale (Enpss): Development and Validation of a Patient Satisfaction Scale with Emergency Room Nursing

This study aimed to develop and validate an emergency nursing-care patient satisfaction scale to measure patient satisfaction with emergency room (ER) nursing. Patient satisfaction scales for ER nursing have been validated without considering the perspectives of the healthcare system or cultural background of the country. Moreover, although nursing care is changing with COVID-19, no scale has been specifically designed to assess patient satisfaction with ER nursing. The study population included patients who visited five ERs in Japan (March to December 2021) (n = 135). The rating scales were provided to patients who visited the ER and gave consent, and the patients were asked to reply. In the process of validating the scale, exploratory and confirmatory factor analyses of the construct and criterion validity were conducted. The confirmatory factor analysis results showed a factorial structure consisting of four factors. The domain and summary scores demonstrated good-to-excellent internal reliability (Cronbach’s range = 0.81–0.89). This patient satisfaction scale was designed and validated from the perspective of the Japanese healthcare system and cultural backgrounds. This scale may be useful for developing assessments and interventions to improve patient satisfaction with ER nursing.


Introduction
Patient response to healthcare services is one of the best ways to obtain information about the quality of healthcare [1]. In particular, patient satisfaction is an important indicator for evaluating service quality and improving healthcare services, and is a commonly used and valid indicator [2,3]. The field of medicine is divided into specialties, and specialized treatment and care are provided in each field. In measuring patient satisfaction, the development of satisfaction scales for each specialty was reported to help deal with specific problems [4]. In nursing, patient satisfaction is defined as the extent of the gap between patient expectations of ideal nursing care and their perception of the nursing care received [5]. We consider it important to use information obtained from patient satisfaction to improve care and enhance the quality of healthcare services.
Emergency nurses have multiple challenging responsibilities, including dealing with overstressed patients and their relatives, homeless and mentally ill patients, and victims [6]. In addition, emergency nurses play several roles, including triage [7], first aid [7], and (1) Development of items to measure patient satisfaction with ER nursing In this phase, we generated various items according to the different categories established by the CECSS [10] and the Patient Satisfaction Questionnaire Short Form (PSQ-18) [2]. The items were elaborated and written through four consensus meetings with consultations from experts in developing the measuring instruments. The research team consisted of eight experts: two certified nurses in emergency nursing, one certified nurse in operation nursing, three certified nurse specialists in critical care nursing, and a university professor of nursing in critical care [36]. The first version of the questionnaire was developed on the basis of the four dimensions of satisfactory care, with 34 items distributed among the four factors. Additionally, we conducted our review using the keywords "emergency care", "emergency nursing", and "patient satisfaction" and searched CINAHL, PubMed, Medline, and other databases in the field. As a result, 12 items were identified, and 46 items in six categories were generated.
(2) Content validation Previous studies have reported that it is desirable to reflect the opinions of experts in the field as well as patients and healthy people who are involved in the content validation process [37]. Content validation of the first questionnaire version was conducted by an expert panel of 10 healthy individuals who had visited an ER and 9 certified nurses in emergency nursing [38] (Table A1). We sent questionnaires to the panel of 19 people and asked them whether the survey items were valid. The survey items were assessed on a 4-point Likert scale ranging from 1 ("not at all important") to 4 ("very important"). We also asked about repetition, incomprehensibility, and ease of answering. Following the method proposed by Davis, the item-level content validity index (I-CVI) was calculated by dividing the number of experts who gave a rating of three or four for each item by the total number of experts [39]. Items with an I-CVI lower than 0.78 were eliminated [40]. Numerical codes were assigned to the completed forms to ensure confidentiality and anonymity of the questionnaires. The final version of the questionnaire was established after deleting five items on the basis of content validation research analysis and the results obtained from various consensus meetings between the research team and expert advisors. The 46 items included in the first version were reduced to 39 in the final version of the emergency nursing-care patient satisfaction scale (ENPSS).

Participants
The emergency medical care system in Japan is classified into three categories: primary emergency facilities that mainly treat patients who can return home without the need for hospitalization, secondary emergency facilities that mainly treat critically ill patients who require hospitalization, and life-saving emergency centers that treat critically ill patients who require advanced treatment [41]. In this study, patients who visited primary and secondary emergency facilities were included. The patients fulfilled the following criteria to participate in the validation study:

1.
Age ≥ 18 years at the time of recruitment; 2.
Capable of providing consent; 3.
Cognitive and physical capacity to complete self-administered questionnaires without the need for a proxy.
Participants were recruited from the emergency departments of five facilities in Japan between March 2021 and December 2021. The questionnaire was distributed to patients who visited the ER and met the above criteria. The research collaborator, an emergency nurse, asked each respondent to complete the questionnaire individually.
The sample size was targeted at >100 participants based on the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist, a guideline for scale development [37,42].

Survey components
The survey consists of four components. The first was a questionnaire on individual and institutional characteristics. The second part consisted of the number of visits to the ER, the time of day when they visited the ER, and the length of time they waited in the ER. The third component consisted of the 6-item EuroQol 5 dimensions 5-level (EQ-5D-5L) [43,44] to test criterion-related validity and the intensity of distress during the ER visit and overall satisfaction in the ER. The fourth component consisted of a satisfaction survey of the nurses in the ER. Instruments The EQ-5D-5L is a validated and standardized instrument that measures health-related quality of life (QOL) [43,44]. A Japanese version of the EQ-5D-5L is available [45]. The EQ-5D-5L consists of the following five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Each dimension has five levels: no problems, slight problems, moderate problems, severe problems, and extreme problems. Health status is represented in 3125 combinations, and each combination of answers can be converted into a QOL score, ranging from 0 (death) to 1 (perfect health), according to a Japanese value set [45]. We compared the scores from our study with those of a previously reported Japanese norm [46]. The EQ-5D-5L also uses a visual analog scale (VAS) ranging from 0 to 100, where 0 represents the worst imaginable health and 100 represents the best imaginable health.
To determine the content of the questions related to patient satisfaction with ER nursing, we first examined the available information from previous studies [47][48][49][50]. Second, we extracted content related to satisfaction with ER nursing. Third, on the basis of these contents, five certified nurses were interviewed. On the basis of these results, the following four items were adopted in this study, and each question was measured using the VAS with "strongly agree" as 100 and "disagree" as 0.
Satisfaction with the response of ER's physician; 3.
Intensity of distress at the time of ER visit; 4.
Satisfaction with the outcome of treatment in the ER.

Statistical Analysis
Descriptive statistics were derived for the analysis. Categorical data were expressed as numbers and percentages.
We examined some of the questionnaire items for possible exclusion according to the following criteria: items with a 20% rate or higher of "not applicable" [51] and items with an average score of 4.5, 1.5, or lower for each item. Furthermore, one of the items with a correlation coefficient of 0.7 or higher for each item was eliminated [52].
Exploratory factor analysis (EFA) using promax rotation and maximum likelihood extraction methods was conducted to determine the number and type of factors from 38 of the 39 survey items, excluding the question on overall satisfaction. EFA was conducted on the complete data for all 38 items at baseline. The factor solution from the EFA was based on the magnitude of the factor loadings for each item. On the basis of standard psychometric criteria, items with factor loadings of less than 0.35 were eliminated. The researchers assessed whether the elimination or retention of specific items was meaningful for assessing patient satisfaction.
On the basis of the results of the EFA, a confirmatory factor analysis (CFA) was used to evaluate factor solutions. The goal of the CFA was to evaluate the model fit of the factor structure using the root mean square error of approximation (RMSEA: where <0.09 is considered acceptable and <0.06 is considered excellent) and comparative fit index (CFI: where >0.9 is considered acceptable). As a result, we named each factor that represented various aspects of patient satisfaction with nursing in the ER.
The internal consistency reliability of the ENPSS was assessed using Cronbach's alpha. The estimates of reliability should exceed 0.70 (0.7 ≤ α < 0.8 is acceptable, 0.8 ≤ α < 0.9 is good, and 0.9 ≤ α is excellent) [53,54]. Construct validity was assessed using Pearson correlations of the EFA-yielded domains and the ENPSS summary score with the five validated questionnaires, namely, EQ-5D-5L (using the VAS), anxiety on EQ-5D-5L, distress at ER visit (VAS), confidence in physician (VAS), and satisfaction with treatment (VAS). We hypothesized that the ENPSS would correlate more with QOL [55] since satisfaction with treatment is associated with QOL.
Only questionnaires with complete data were included in the analysis, and there was no imputation of missing data. Statistical significance was set at p ≤ 0.05 (two-sided). Statistical analyses were performed using SPSS Statistics version 27 (IBM Corp., Armonk, NY, USA) and JMP Pro software version 15 (SAS Institute Inc., Cary, NC, USA).

Ethical Considerations
The protocol for this research project was approved by a suitably constituted Ethics Committee of Sapporo Medical University and conformed to the provisions of the Declaration of Helsinki, Approval No. 1-2-51. Informed consent was obtained from all the respondents. Participants were informed of the purpose and length of the survey, and their participation was voluntary. Consent was obtained from respondents by checking the box on the front page of the questionnaires that they understood the research explanation and agreed to participate according to Institutional Review Board recommendations.

Population
A total of 127 respondent surveys were included in the final analysis after excluding eight surveys with missing data. The characteristics of respondents are presented in Table 1. Patients who were still employed accounted for 66.1%, and those with underlying diseases accounted for 63.0% of the total. Fifty-two percent of the patients visited the ER during the day, and 40.2% visited the ER for the first time.

Selecting Items
A total of 127 participants (92%) had complete data on the ENPSS and constituted the population used for factor analysis; this was sufficient for previous EFA studies [37,42,56,57].
First, of the 38 question items, 4 items of which more than 20% were answered as not applicable were deleted. There were 31 pairs with correlation coefficients greater than 0.7. We eliminated one of the items of the pair, and 14 items were excluded. Second, factor analysis was conducted using the maximum likelihood method. In the EFA using promax rotation, one item with a factor loading of less than 0.35 was removed, and finally, 20 items were selected (Table 2). The EFA yielded 20 solution items loaded into four factors representing four domains: explanation and response (seven items), hospitality (six items), teamwork (three items), and symptom management (four items). The global satisfaction item was not included in the EFA, which constitutes the ENPSS-21 in English version (see Appendix B) and Japanese version (see Appendix C). This 21-item questionnaire's factor structure CFA (Figure 1) showed an acceptable fit: RMSEA = 0.1 (90% confidence interval = 0.08-0.11) and CFI = 0.9. EFA, exploratory factor analysis; ENPSS, emergency nursing-care patient satisfaction scale; CFA, confirmatory factor analysis; RMSEA, root mean square error of approximation; CFI, comparative fit index.

Internal Consistency Reliability
The internal consistency reliability (Cronbach's alpha) of the four domains of the ENPSS-21 ranged from 0.81 to 0.89 (Table 3), corresponding to good internal reliability.

Construct Validity
The correlations of the four domain scores of the ENPSS-21 showed that the EQ-5D-5L VAS (0.59-0.65, p < 0.01), EQ-5D-5L Anxiety (0.55-0.62, p < 0.01), confidence in the ER physicians (0.56-0.63, p < 0.01), satisfaction with the response of ER physicians (0.57-0.66, p < 0.01), and satisfaction with the outcome of treatment in the ER (0.51-0.62, p < 0.01) were in the moderate to high range. The correlation was lower for the intensity of distress at the time of the ER visit (−0.2-−0.29, p < 0.01). As hypothesized, the summary score of ENPSS-21 showed the strongest correlation with EQ-5D-5L (0.68) and satisfaction with the response of the ER physician (0.68). (Figure 2 and Table 4). Healthcare 2022, 10, x FOR PEER REVIEW 8 of 22 Figure 1. Confirmatory factor analysis. Figure 1. Confirmatory factor analysis.

Discussion
In this study, we developed a patient satisfaction scale focused on ER nursing, confirmed its validity and reliability, and concluded that it could be used in clinical practice. To the best of our knowledge, there are no reports on developing a patient satisfaction scale that focuses on ER nursing in Japan.
The factor analysis results were composed of four domains: "explanation and response", "hospitality", "teamwork", and "symptom management". This multidimensional structure is consistent with many reports on patient satisfaction analysis [2,10,24,58,59]. Furthermore, the specific dimensions obtained in this study are similar to those found in other scales [10,60], which we consider partially supportive of the construct validity of this tool.
A limitation when measuring patient satisfaction is that psychometric properties may not be reflected because cultural factors from different countries and regions are not adequately taken into account [61]. Translation of existing patient satisfaction measures, such as CECSS and others, may lead to differences in the perception of quality of care from the patient's perspective due to cross-cultural differences [62]. Because of the specific situation in ERs, where patients are more urgent than in general wards and require a variety of responses, a specific scale consistent with the culture of the country is considered essential. Therefore, a patient satisfaction scale in the ER that takes into account the Japanese cultural context was needed.
The ENPSS-21 domain "explanation and response" includes nurse-patient communication and provision of information. In order for the nurse to meet the needs of the patient, a natural and constructive relationship must be established [63]. Nurses can then provide counseling and guidance to patients to improve patient satisfaction [64]. In the ER, providing information and communication to patients is also an important factor in facilitating patient care and is an essential factor in patient satisfaction [65].
The second domain of ENPSS-21 is "hospitality". The nurse's concept of compassion and interpersonal relationships is an important element in understanding patient. The

Discussion
In this study, we developed a patient satisfaction scale focused on ER nursing, confirmed its validity and reliability, and concluded that it could be used in clinical practice. To the best of our knowledge, there are no reports on developing a patient satisfaction scale that focuses on ER nursing in Japan.
The factor analysis results were composed of four domains: "explanation and response", "hospitality", "teamwork", and "symptom management". This multidimensional structure is consistent with many reports on patient satisfaction analysis [2,10,24,58,59]. Furthermore, the specific dimensions obtained in this study are similar to those found in other scales [10,60], which we consider partially supportive of the construct validity of this tool.
A limitation when measuring patient satisfaction is that psychometric properties may not be reflected because cultural factors from different countries and regions are not adequately taken into account [61]. Translation of existing patient satisfaction measures, such as CECSS and others, may lead to differences in the perception of quality of care from the patient's perspective due to cross-cultural differences [62]. Because of the specific situation in ERs, where patients are more urgent than in general wards and require a variety of responses, a specific scale consistent with the culture of the country is considered essential. Therefore, a patient satisfaction scale in the ER that takes into account the Japanese cultural context was needed.
The ENPSS-21 domain "explanation and response" includes nurse-patient communication and provision of information. In order for the nurse to meet the needs of the patient, a natural and constructive relationship must be established [63]. Nurses can then provide counseling and guidance to patients to improve patient satisfaction [64]. In the ER, providing information and communication to patients is also an important factor in facilitating patient care and is an essential factor in patient satisfaction [65].
The second domain of ENPSS-21 is "hospitality". The nurse's concept of compassion and interpersonal relationships is an important element in understanding patient. The hospitality domain also included items, such as courtesy and personal appearance. In Japan, courtesy and personal appearance strongly influence patient satisfaction [66,67], and we consider these cultural factors unique to Japan.
The third domain of ENPSS-21 is "teamwork". Nurses are part of the healthcare team and are expected to collaborate with other healthcare professionals involved in patient care [63]. Moreover, nurses have an important task to fulfill as intermediaries between multiple healthcare professions. Organizational teamwork has been reported to be associated with satisfaction, and nurses need to practice in the best interest of the patient [68]. Therefore, it is considered a very important perspective for ER nurses to collaborate with ER physicians and co-medical staff to provide medical care.
The last domain of ENPSS-21 is "symptom management". Patients in the ER have a wide range of distress. Appropriate analgesia affects patient satisfaction [69]. The absence of physical pain increased patient satisfaction with nursing care [70]. Therefore, distress relief is considered by some patients to be equivalent to good nursing care [71]. Namely, symptom management is considered an indispensable item for providing care that is consistent with the needs of ER patients.
Moreover, this patient satisfaction scale was developed during the COIVD-19 pandemic. During the COVID-19 pandemic, healthcare professionals have reported that adequate infection control measures are important for reducing the risk of viral infection and patient anxiety about the virus [72]. The questionnaire items used in this study included items related to infection control among nurses, which may include an important domain of recent emergency care. In addition, the compassion and interpersonal relationships of nurses are important factors for patient satisfaction [25,73,74].
Cronbach's alpha for all domains of the ENPSS-21 scale was greater than 0.8. It was found that the ENPSS-21 had similar values to previous patient satisfaction scales, which were verified for internal consistency [59,75]. This means that each factor showed appropriate homogeneity.
Of all the hypotheses used for construct validity, the "EQ-5D-5" and "satisfaction with the response of ER physicians" were highly correlated with each of the four domains of the ENPSS-21 and summary score. Previous reports have shown an association between treatment satisfaction and QOL [55,58]. Patients with anxiety were also reported to be less satisfied with their healthcare [76], consistent with the results of this scale. In addition, satisfaction with physicians is related to overall satisfaction with healthcare [74], consistent with the hypothesis validation in this study. In contrast, a high ENPSS-21 score was not associated with the intensity of distress at the time of the emergency room visit. Although this was low compared to the intensity of distress at the emergency room visit in this study [71], it is consistent with the reported finding that VAS pain scores in the ER do not correlate with patient satisfaction [77].

Limitations
The current study has several limitations. First, test-retest reliability was not validated in this study. Having good test-retest reliability implies internal consistency of the test and ensures that the measurements obtained are representative [42]. In the future, a test-retest should be conducted to confirm reliability. Second, there are concerns about the time when data collection took place. In this study, data were collected during the COVID-19 pandemic period. As a result, the ER system was probably different from normal, which could have affected the assessment of patients. Third, this study used a minimum sample size of 100 for factor analysis from previous studies [37,42]. However, for CFA, a minimum sample of 150 is required [78]. This study did not meet that requirement, thus limiting the results of the analysis. In the future, it will be an issue to refine the items of the scale while taking the sample size into consideration.

Implications for Clinical Practice
ENPSS-21 is a brief, reliable, and valid instrument that can obtain information about patient satisfaction with ER nursing. The instrument has direct clinical utility for improving the quality of nursing care in the ER in Japan by providing a patient-centered perspective on satisfaction. The ENPSS-21 also helps address the weaknesses of the organization by measuring satisfaction regularly and comparing patient satisfaction levels relative to each other.

Conclusions
This study found that the ENPSS-21 was a robust measure of patient satisfaction, suggesting that it is possible to measure satisfaction with ER nursing. The ENPSS-21 is designed to focus on the nursing perspective of the ER. In creating the items, the ENPSS was designed to represent the emergency nursing care sought by patients on the basis of an extensive literature search and content validity. The ENPSS-21 consists of 21 items in four domains. It showed the highest correlation with the EQ-5D-5L and satisfaction with the response of ER physicians in criterion-related validity. By measuring patient-centered perspectives of satisfaction with ER nursing, factors lacking in each organization's ER can be identified and addressed to improve the quality of nursing care in the ER. Checking the box at the beginning of the questionnaire stating that they understood the study description and agreed to participate was considered consent.
Data Availability Statement: All data from this research have been included within the manuscript.
Appendix A   Table A1. Characteristics of expert panel, n = 19.

Appendix B. Nursing Emergency-Care Patient Satisfaction Scale-21 (ENPSS-21) English Version
The following questionnaire asks about the patient level of satisfaction with emergency room nurses. The goal of this questionnaire is to measure your level of satisfaction or dissatisfaction with the nurses on the basis of what you currently feel. Please place a check mark in the appropriate box for each question.
The following 20 questions relate to satisfaction with emergency room nurses.