Validity and Reliability of a Short Form of the Questionnaire for the Reflective Practice of Nursing Involving Invasive Mechanical Ventilation: A Cross-Sectional Study

The number of patients on ventilators is rapidly increasing owing to the coronavirus pandemic. The previously developed Questionnaire for the Reflective Practice of Nursing Involving Invasive Mechanical Ventilation (Q-RPN-IMV) for the care of patients on ventilators includes nurses’ thought processes as items. This study aims to develop a short form of the Q-RPN-IMV for immediate use in practice and to test its reliability and validity. A convenience sample of 629 participants was used to explore the factor structure using factor analysis. The test–retest reliability was assessed using the intraclass correlation coefficient (ICC). The study was a cross-sectional design instrument development study and was reported according to GRRAS guidelines. Q-RPN-IMV short form was divided into ventilator management and patient management. The ventilator management comprised 31 items organized into six factors. Cronbach’s alpha ranged from 0.82 to 0.91, and the ICC ranged from 0.82 to 0.89. The patient management comprised 27 items organized into five factors. Cronbach’s alpha ranged from 0.75 to 0.97, and ICC ranged from 0.75 to 0.97. The Q-RPN-IMV short form is a reliable and validated instrument for assessing care for patients on ventilators. This study was not registered.


Introduction
The essential skills required for nursing practice for patients on ventilator support include knowledge of ventilators, invasive techniques for preventing complications, and risk prediction for safe management [1,2].Therefore, management in the intensive care area is generally recommended for patients on ventilator support because it is well-equipped and has qualified experts to manage such patients [3].Recently, the number of patients with respiratory diseases due to aging has increased worldwide.Moreover, the number of patients with severe respiratory symptoms due to the spread of coronavirus disease 2019 (COVID-19) is rapidly increasing [4][5][6]; thus, the number of patients on ventilators during treatment has inevitably increased.In the current scenario of the ongoing pandemic, intensive care units cannot exclusively manage patients on ventilators.Therefore, the general ward continues to serve as an urgent management site for such patients [7][8][9][10].
Respiratory therapists in Europe and the United States are coping with the pressures of a growing number of patients suffering from infectious diseases; however, in the event of a pandemic, there could be a worldwide shortage of ventilators and specially trained respiratory therapists [11,12].General nurses from various backgrounds would therefore be forced to manage patients on ventilators and provide in-hospital and home care, including oxygen therapy, respiratory assistance, and ventilator therapy [13].Furthermore, even if nurses have little experience caring for patients on ventilators, they are expected to have the minimum knowledge and skills necessary to safely manage these patients.Therefore, nurses in charge of patients requiring ventilators carry a heavy burden because of a substantial amount of responsibility and potential lack of experience, frequently leading to negative emotions, such as anxiety, stress, and lack of self-confidence [9].
Despite this situation, inexperienced nurses currently provide ventilator education to other nursing staff using the methods of the facility.Furthermore, most nursing education programs generally solely focus on imparting knowledge on the individual skills of ventilator care [14].During the pandemic, it was difficult to ensure the quality and safety of nursing care for patients on ventilators because conducting group training in the hospital was impossible due to the increased number of patients.Therefore, with the rapid increase in the number of patients on ventilators, there is a need for an evaluation sheet that allows inexperienced nurses to provide safe nursing care to patients on ventilators and for nurses who practice on such patients to look back on their practice and self-evaluate it.
In a previous study, we developed the Questionnaire for the Reflective Practice of Nursing Involving Invasive Mechanical Ventilation (Q-RPN-IMV), a comprehensive, itemized measure of nursing practice of ventilator care by professional and certified nurses who are considered experts in fields related to ventilator care, including their thought processes [15].This questionnaire differs from previous checklists because it is not simply a checklist to determine whether self-care has been practiced.However, because ventilator care is complex and includes advanced-level practice items practiced by specialized and certified nurses, the number of items is notably large, and the so-called "Q-RPN-IMV long form" could not be used routinely by inexperienced nurses.Even for inexperienced nurses to safely manage ventilator-implanted patients and to be able to use the questionnaire immediately in clinical practice, the number of items in the questionnaire must be carefully selected and consolidated.Therefore, we carefully selected suitable items from the Q-RPN-IMV developed in previous studies and created a short form that can be easily used by inexperienced nurses.This study aims to develop a short form of the Q-RPN-IMV, based on the Q-RPN-IMV, for immediate use in practice and to test its reliability and validity.

Study Design
Scale development was based on the guidelines described in Scale Development: Theory and Applications and the Consensus-based Standards for the Selection of Health Status Measurement [16].Moreover, we followed the Guidelines for Reporting Reliability and Agreement Studies (GRRAS) [17].

Setting and Participants
Using convenience sampling, we selected clinical nurses who had experience caring for patients on ventilators.To meet the criteria for good methodology, a sample size of at least seven times the number of survey items and an absolute number of at least 100 are required [18].The first version of the questionnaire contained 71 items; therefore, the minimum sample size was 497.The purpose of this study was explained in writing and orally to the facility director to be surveyed, and permission was obtained before the start of the study.A questionnaire on ventilator care was distributed to clinical nurses working in wards where ventilators were used.Criteria for participation were full-time employment in a general hospital with at least 500 beds that was considered to offer group training on ventilators.Certified nurses, nursing managers, licensed practical nurses/nursing assistants, and midwives were excluded from the study.Nurses' responses to the questionnaire were collected from April 2020 to December 2021.The nurses were asked to post their responses in a collection box set up in the ward within 1 week after distribution.Furthermore, the participants were asked to answer a questionnaire on ventilator care after 10-14 days for test-retest reliability to examine the stability of the study.A filled-in questionnaire implied the nurses' consent to participate in the study [19].

Instrument
Participants answered a questionnaire consisting of the following three sections: (1) demographic characteristics, (2) the Self-Evaluation Scale for Nursing Involving Invasive Mechanical Ventilation, and (3) the educational needs assessment tool for clinical nurses.

Participant Demographic Characteristics
From the participants, we collected data on age, clinical career (years of work experience), number of patients on ventilators received to date, hospital affiliation, and presence or absence of certified nurse support.

Self-Evaluation Scale for Nursing Involving Invasive Mechanical Ventilation
To investigate the participants' nursing practice with patients on ventilators, we utilized the nursing practice process items related to ventilator care required by clinical nurses from the Q-RPN-IMV.The Q-RPN-IMV is a questionnaire that was itemized by observing the respiratory care for patients on ventilators practiced by certified nurses in previous studies, including their thought processes.Furthermore, the Q-RPN-IMV comprises 136 items: 26 observation, 66 judgment, and 44 implementation items.These items were relevant to the respiratory care of patients on ventilators by certified nurses.From these 136 items, the most essential were selected for immediate use by inexperienced nurses.
Content validity refers to whether the content of the scale adequately captures the constructs and requires expert judgment.The content validity index (CVI) is the most widely reported content validity approach in instrument development and can be calculated using Item-CVI (I-CVI) and Scale-Level-CVI (S-CVI).To verify content validity, the Delphi method was performed three times by 20 certified nurses, and the items were selected by more than 80% of the experts on all three occasions [20].Furthermore, those items were classified by an expert panel of certified nurses into two groups to make them easier to understand by less experienced nurses; items were classified into two groups, "ventilator management" and "patient management".After reviewing the items, they were classified into "ventilator management" and "patient management", and 71 items were adopted as proposal items.Thus, a ventilator evaluation scale consisting of 32 items for machine management and 39 for patient management was proposed for 71 items.Each item was rated on a 5-point Likert scale from 1 (not at all) to 5 (always) to determine the frequency of implementation.The resulting questionnaire was pretested on five nurses in a general ward, and the superficial validity of the proposed scale was examined by checking for unclear expressions and the time required for easy comprehension, even by inexperienced nurses.

Data Analysis
In the statistical analysis, questionnaires with missing items were excluded, and item analysis was performed first.Subsequently, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and criterion-related validity were performed to assess the scale's validity.Finally, internal consistency and reproducibility were examined to assess the reliability.All statistical analyses were performed using IBM SPSS version 28 and Amos 28 (IBM Corp., Armonk, NY, USA).

Item Analysis
Standard deviations and means for all items were estimated to detect ceiling and floor effects.

Validity Analysis
EFA was conducted to determine the factor structure supporting "ventilator management" and "patient management" of patients on ventilators.To assess the suitability of the data for factor analysis, the Kaiser-Meyer-Olkin (KMO) test for sampling adequacy and Bartlett's specificity test were performed [18].The number of factors was visually inspected using scree plots to identify the ideal number of potential factors and to determine if the KMO criterion of an eigenvalue ≥ 1 was met, and the proportion of variance explained by each factor [21].Maximum likelihood methods with promax rotation were then performed [22].Items with factor loadings < 0.4 were excluded [23].As a rule, the sum of the variance explained should be greater than 50% [24].EFA was conducted again with the removed items (Items with factor loadings < 0.4).The number of factors was determined to include all items with factor loadings of 0.40 or greater.For items with approximate factor loadings, a valid factor analysis was conducted using a covariance structure analysis, considering the item content included in that factor.
The CFA was performed to test the model fit.For the CFA, we used maximumlikelihood estimation and evaluated model fit.Model fit was tested based on the χ 2 , normed χ 2 (χ 2 /df), comparative fit index (CFI), and root mean square error of approximation (RMSEA) index used to assess the fit of the scale.A χ 2 /df value of ≤3 is considered adequate.Furthermore, the fit was considered good if the CFI value was >0.90 and the RMSEA value was <0.08 [25,26].

Reliability Analysis
Reliability was verified using Cronbach's alpha coefficients for the entire scale and each factor.To examine stability and verify the test-retest reliability, we determined the intraclass correlation coefficient (ICC) of the response scores to the scale obtained at the two-time points [19].ICCs were determined as <0.50 (poor), 0.50-0.80(fair to good), and >0.80 (excellent) [27].

Ethical Consideration
Participants were informed of the purpose and methods of the study and the risks and benefits of participation.In addition, participants were informed that their participation was voluntary, that they were free to drop out at any time, and that their privacy would not be affected.Data were collected anonymously using assigned IDs and were strictly controlled.Data handling and disposal procedures were also explained.In the interest of transparency, it was further explained that some of the findings would be presented in a public forum.Researchers provided participants with contact information.Ethical approval was obtained from the Institutional Review Board (IRB) Ethics Committee of the author's university (No. 696).All methods were performed in accordance with relevant guidelines and regulations.

Participant Characteristics
Table 1 presents the participant characteristics.Questionnaires were distributed to 945 clinical nurses working in general hospitals with more than 500 beds and with experience working with patients on ventilators, and 754 returned the questionnaires.Of the 754 returned questionnaires, 125 with >20% missing data were excluded, and 629 questionnaires were included in the analysis.The questionnaire collection rate was 79.8%, and the effective response rate was 83.4%.For the stability study, 157 questionnaires were collected, and 122 were analyzed after excluding 35 with missing data.The number of nurses with less than 5 years of experience was 212 (33.7%), which is higher than the percentage of nurses in Japan, but this is because the survey excluded managers.Approximately half of the participants were responsible for more than six ventilators.

Item Analysis
Each of the 71 items was analyzed.The mean individual item scores ranged from 3.03 to 4.89 (standard deviation 0.44-1.53),with no ceiling or bottom effects.

Factor Analysis
EFA was performed on managing ventilator items (32 items) and patient items (39 items) using the main factor method and promax rotation.
The following results were obtained according to the conditions for factor selection.

Ventilator Management
The KMO measure of sampling adequacy was 0.90.There were six factors based on eigenvalues ≥ 1 and the scree criterion.Items with a loading value of ≥0.4 were retained, and item 12 was deleted because the factor loadings were <0.4.The six factors covered the following roles: initial confirmation, artificial airways, alarm management, humidification management, emergency management, and airway fixation.The contributions of factors 1-6 were 32.29%, 10.79%, 6.03%, 5.36%, 4.91%, and 3.75%, respectively, and the cumulative contribution was 63.13%.The model fit index for the scale was calculated using CFA, and the results showed that χ 2 = 2058.1,χ 2 /df = 4.91, CFI = 0.844, and RMSEA = 0.079 (Table 2).The model fit was not optimal, although it was acceptable with respect to the factor contribution ratio.

Patient Management
The KMO measure of sampling adequacy was 0.90.There were five factors based on eigenvalues ≥ 1 and the scree criterion.Items with a loading value of ≥0.4 were retained, and item 12 was deleted because the factor loadings were <0.4.These five factors covered the following roles: prevention of complications, prevention of VAP, safe transfer, skin management, and post-transfer assessment.The contributions of factors 1-5 were 31.81%,12.87%, 6.13%, 5.44%, and 4.18%, respectively, and the cumulative contribution was 60.43%.The model fit index for the scale was calculated using CFA, and the results showed that χ 2 = 1723.9,p < 0.01, χ 2 /df = 5.49, CFI = 0.841, and RMSEA = 0.085 (Table 3).Factor contributions were acceptable.Model coefficients were not good, despite attempts to improve the model index as much as possible by removing items.χ 2 = 1723.9,p < 0.01, χ 2 /df = 5.49, CFI = 0.841, RMSEA = 0.085 Factor extraction method: main factor method, rotation method: Promax method with Kaiser normalization.Squared figures: factor loadings above 0.4.27 items from 39 initial items.
The following items had low loading and were excluded from the assessment: 37 For orally intubated patients, carry out oral care together with another staff member.42 Consider that tracheal secretions are affected by gravity.

Reliability Testing
The Cronbach's alpha coefficient for "Ventilator Management" was 0.91 at the scale level and 0.82, 0.82, 0.89, 0.87, 0.87, and 0.82 for factors 1-6, respectively (Table 2).In contrast, for "Patient Management", the coefficients were 0.91 at the scale level and 0.88, 0.86, 0.75, 0.97, and 0.77 for factors 1-5, respectively (Table 3).For reproducibility, the ICC intraclass correlation coefficient for factors, calculated using data from the 152 participants who returned their test-retest responses, ranged from 0.75 to 0.97 (Table 4).All factors of ventilator management show high reliability.Patient management factors 1, 2, and 4 showed high reliability.Factors 3 and 5 showed acceptable reliability.A lower ICC was predicted because the frequency of practice increased as nurses reflected on ventilator care in their nursing practice through retesting.

Discussion
The basis for this study was the Q-IMV-RPN, developed from the existing literature and observations of actual skilled nursing practice, which include their thought processes concerning the nursing practices required for ventilator care.The rating scale was developed by selecting essential items that should be used immediately by nurses in general wards.However, the nursing skills to be provided to patients with respiratory distress are complex.Therefore, for each machine and patient management, essential nursing practices were consolidated according to the nursing process, and a scale was created.
Mechanical management of ventilator care in a general hospital ward comprises six factors.These included essential information, such as setting alarms and damaging the ventilator circuit, which should be checked at regular intervals and personnel shifts.Factor 1, "initial confirmation", included confirming the forced ventilation mode setting, volume of single breaths, and respiratory frequency.Factor 2, "artificial airway", included an assessment to prevent incidents related to artificial airways that occur most frequently in general wards [28] and an assessment related to humidification.Factor 3, "alarm management", included an assessment of the patient's situation when the alarm went off.This is a crucial factor because even if alarms are set, it is impossible to prevent or manage abnormalities in the patient without being able to assess what is happening during the alarm [29].Factor 4 was related to the confirmation and implementation of "humidification management".Heating and humidification are essential assessment items for ventilator-associated pneumonia (VAP) prevention; however, they are also important factors, as inadequate management can lead to serious incidents, such as airway burns.Factors 5 and 6 included the confirmation and implementation items related to the artificial airway [30].Therefore, it is obvious that machine management for ventilator care includes the management of machine settings and alarms and the nursing practice process related to the management of the artificial airway, including heating and humidification.
Management of patients in a general hospital ward essentially comprises five factors.Factor 1, "complication prevention", consists of items for implementing necessary care, such as intraperiod suctioning, repositioning, and oral care, to prevent pneumonia and unscheduled extubation.We believe that the essential care items that should be implemented for patients using ventilators are concentrated in this category.Factor 2, "prevent VAP", comprises a collection of items describing the perspectives and rationale for preventing VAP, such as repositioning, oral care, and endotracheal suctioning.Therefore, it should be practiced continuously in conjunction with factor 1. Factor 2, the perspective of assessment, and factor 1, the implementation of safe care, can be regarded as practices to prevent complications [31].Factor 3, "transfer safely", includes items such as checking the connection and setting the ventilator.Many incidents related to ventilators in general wards involve unscheduled extubation during transfer, poor aftercare connection, and incorrect ventilator settings.Therefore, it is considered a crucial factor in safety management [28].Factor 4, "skin management", concentrates on items related to skin damage caused by the fixation of the artificial airway.Generally, great emphasis should be placed on preventing the unscheduled removal of artificial airways.Therefore, there is a high likelihood of skin damage due to immobilization.Tracheal tube fixation is essential for safe breathing.However, skin damage from fixation can prevent stable fixation from being maintained, increase patient distress, and reduce the patient's quality of life after extubation.Therefore, we value this factor as a perspective for implementing care while maintaining safety.Factor 5, "assessment post-transfer", consolidates items related to the assessment and the rationale for factor 3. Factors 3 and 5 together can be considered to pertain to safety management practice.Patient management for ventilator care focuses on nursing practices to prevent VAP and the unscheduled removal of the artificial airway.This is also expected to occur during the transfer of the patient from one location to another before and after care.The content pertains to nursing care aiming to prevent skin damage and improve safety management.
In Japan, nurses control the use of ventilators.However, this is not covered in basic nursing education.Instead, nurses are trained in the field of practice while working in hospitals.However, during the pandemic, the number of patients on ventilators increased.Moreover, there was no time to train nurses who could care for patients on ventilators because they were faced with urgent life-and-death situations.Therefore, we believe that the scale developed in this study is highly beneficial for inexperienced nurses to manage ventilators safely.Furthermore, nurses' ability to practice ventilator care can be assessed using this scale, which can further help evaluate the ability of wards to manage ventilators.Moreover, managers can evaluate which wards can safely manage patients on ventilators removed from the intensive care unit during trying times.It can also be used for objective evaluation by a third party to assess ventilator management capabilities and to help in education.This scale itemizes the nursing practice process of ventilator care that should be practiced by nurses, including thought processes.Therefore, it can benefit nurses with limited experience to safely practice ventilator care.Simultaneously, it can be used to assess the practice and educational impact of group education on ventilator care and which items should be taught effectively.Therefore, it is possible to evaluate the developmental stages of nurses using this scale.Furthermore, this scale can help hospital administrators evaluate which nurses can practice ventilator management and to what extent it can be used as a material to consider which area to move patients with ventilators.As there has been a rising need for human resources to provide ventilator care, it is believed that safe care for patients on ventilation can be provided using this system with the currently used checklist.
Regarding the item content, the items were narrowed down to the minimum necessary while verbalizing the practice of professional nurses, including their thought processes.Furthermore, the scale was divided into ventilator management and patient management so that routine nurses and inexperienced nurses could safely practice ventilator care.Therefore, this scale can be used to self-evaluate one's nursing practice and as an evaluation tool for others for educational purposes.In particular, we believe that, in cases where actual practical exercises or group training could not be conducted, including pandemics, the scale created in this study will allow even novices to practice the minimum necessary ventilator care and to reflect on the content of their practice.
However, this study had several limitations.First, the scale was developed only for Japanese nurses.Therefore, whether this scale can be used in other countries needs to be examined.Second, this study included only generalists working in Japanese hospitals with 500 or more beds.Therefore, future studies investigating its applicability to nurses working in smaller hospitals are needed.Third, if pediatric patients had participated in this study, it is highly likely that their attitudes and methods toward ventilator management would have changed.Therefore, the model as a scale is not optimal and needs to be improved.In the future, the number of items for each target nurse should be considered, and the structure should be balanced.In addition, it is necessary to create a scale that can be adapted according to the target patients.

Conclusions
The short form developed in this study is based on the Q-RPN-IMV, and the items are carefully selected to provide the minimum necessary content for nursing practice professionals to care for patients on ventilators.Furthermore, to facilitate use by generalist nurses, the scale is divided into thirty-one items with six factors related to ventilator management and twenty-seven items with five factors related to patient management to ensure reliability and validity.
Data are presented as n (%).IMV: invasive mechanical ventilation.

Table 3 .
Patient Management (Cronbach's α = 0.91).Use a suction tube insertion length that reaches about 1 to 2 cm from the tip of the tracheal tube, inserting until just before the bifurcation of the trachea.