Developing Nursing Standard Guidelines for Nurses in a Neonatal Intensive Care Unit: A Delphi Study

The purpose of this study is to develop nursing standard guidelines for nurses in a neonatal intensive care unit. The Delphi method was used in this study to elicit expert consensus. Thirteen experts who were nurses and pediatric adolescent specialists working in the neonatal intensive care unit participated in the study. In this study, 178 items were developed based on 5 nursing practice standards and 7 standards of professional practice. An additional 10 items were included based on observation in the neonatal intensive care unit. After expert validation, a final total of 184 items was developed. The standard guidelines for high-risk neonatal care developed in this study for practical clinical education in nursing are significant because they reflect the nursing practice standards in Korea and characteristics of nursing practice in the neonatal intensive unit.


Introduction
Protecting human health is a goal for all healthcare practitioners, and high-risk infants especially require greater care than other populations. This group is broadly defined as including: (1) the preterm infant; (2) the infant with special healthcare needs or dependence on technology; (3) the infant at risk because of family issues; and (4) the infant with anticipated early death [1]. The proportion of preterm infants who also have a low birth weight, defined as less than 2.5 kg, has increased steadily over time, rising from 3.3% in 1997 to 4.7% in 2007 and reaching 6.2% in 2018, with concomitant increases in high-risk infants-those categorized as in need of the neonatal intensive care unit (NICU) in South Korea (hereafter Korea) [2]. Immediately after birth, high-risk infants are admitted to the NICU, where they spend at least 24 h in the care of NICU staff, who play a critical role in their survival and eventual thriving [3]. NICU nurses provide various treatments and nursing care to newborns, including oxygen therapy, mechanical ventilation, maintenance of vital signs, nutritional supplementation, and infection prevention [4]. Notably, after birth, most newborns in the NICU experience various respiratory issues while transitioning to voluntary breathing, which may lead to emergency situations. In these situations, emergency nursing care from experienced nurses is also vital [5]. Moreover, NICU nurses also facilitate the interaction between the newborns and their parents and help the parents to learn their new parental roles [4]. Clearly, NICU nurses face the burden of achieving expertise in a range of complicated tasks spanning several domains in order to respond to the rapidly changing health conditions of newborns, and they are required to have excellent clinical decision-making skills to provide timely, accurate nursing care depending on the newborns' conditions [6]. Clinical decision-making skills in NICU nurses are one of those nursing competencies that increases the quality of nursing and exerts a positive influence on the treatment outcome in newborns [7].
It is also vital for the nurses to have standard guidelines for practice based on these assessments [8].
In clinical situations, standard guidelines help to improve nurses' competency. Moreover, in light of most Korean nursing students' limited clinical experiences with high-risk infants, such guidelines can be used as educational materials, providing systematic knowledge of this highly specific area of nursing [9].
At present, little work has been done on this issue in the Korean context. Jun (2000) [10] analyzed the workload of NICU nurses and divided their tasks into 16 direct nursing activities and 7 indirect activities. These activities, however, were categorized based on nursing tasks performed in specialized units other than NICU, as well as in more general units. Park (2007) [11] categorized neonatal nurse practitioners' tasks into 6 roles, 12 duties, and 46 tasks comprising 315 activities. However, with the exception of the management of high-risk infants in delivery rooms, the duties investigated by Park (2007) [11] involve care for healthy newborns. Moon (2006) [12] classified NICU nurses' tasks into 42 nursing behaviors for billing purposes, but the scope of tasks of NICU nurses was underestimated in this study. In terms of previous studies conducted in other countries, a study that developed neonatal nursing standards for Cambodia included five stages of nursing, health education, health promotion, continued education, and communication, but these are more limited compared to the range of tasks performed by Korean NICU nurses [8].
Because some Korean hospitals offer only limited pediatric clinical placements for nursing students so as to prevent infections in newborns, some students graduate without any clinical exposure to the NICU [13]; this makes it difficult for students to cultivate professional competencies through hands-on practical experience. When nursing students graduate to become new nurses without sufficient clinical exposure to the range of unique situations which occur in the NICU, they cannot respond to emergency clinical situations as well as if they had practical experience, and this is directly related to patient safety. In particular, nurses who lack experience detecting and quickly responding to emergency situations have been reported to delay emergency responses with high-risk infants [14]. Moreover, these nurses have been found to have a higher desire to change jobs due to decreased self-efficacy at work [15]. Therefore, in order to train competent nurses in an environment where nursing education for high-risk infants is limited, and to retain these nurses and facilitate their professional growth, it is necessary to develop standard guidelines on the tasks of NICU nurses and to use these guidelines in nursing education to form a foundation for practice.
To this end, the present study aimed to develop standard guidelines for NICU nurses and to obtain expert opinion on these guidelines in an attempt to provide basic data in clinical practice and to create evidence-based, targeted, relevant educational materials for nursing students to improve both their practice and their experience.

Study Design
The present study is a methodological study which aims to understand the tasks of NICU nurses and to develop standard guidelines for their practice and provision.

Study Participants
The main criterion for the selection of experts on the 13-member panel was to include people who had been recognized for their professional knowledge and experience. The criterion was operationalized as follows: (1) Those who had worked in the neonatal intensive care unit for more than 5 years, and had been active in medical institutions to date (2) Those who had undergone a neonatal intensive care course and had experience in developing and publishing books or papers related to neonatal intensive care (3) Persons with experience in developing related policies, educational programs, and conducting research as executives of neonatal intensive care-related organizations (4) neonatal intensive care workers of government or private organizations who had experience in developing and participating in related projects.

Study Stages
The process of this study is shown in detail in Figure 1.  The researchers reviewed the literature on standard guidelines for NICU nurses between 1 March and 15 April 2019. This literature review comprised a review of neonatal nursing standards suggested by the American Nurses Association (2004) [16] and the Australia College of Nursing (2019) [17], nursing standards from the Korean Nurses Association (2003) [18], a pediatric nursing textbook [19], a high-risk infant nursing textbook [20], previous studies on neonatal nursing [10][11][12], and job descriptions from hospitals.

Observations
At this stage, the researchers found it important to conduct a clinical field observation and a perusal of nursing records to see if any relevant tasks were overlooked during the literature review and job description. The researchers also investigated nursing work performed by NICU nurses through observation. For clinical observation, the researchers visited the director of nursing service at a general hospital located in G Province, explained the purpose of the study, and obtained approval for observation; the clinical observation was conducted on 15 and 16 April 2019. The observer was one trained individual who has a high understanding of the clinical practice being observed and nursing education, has completed a PhD in nursing, provided clinical lectures and training for pediatric nursing, and has nine years of clinical experience including experiences in NICU and pediatric settings. The observees were nurses who had at least 10 years of clinical experience with at least five years of experience in the NICU; observees were selected through consultation with the head nurse. Consent was obtained from all observees. Two nurses working in day and afternoon shifts on April 15 were observed, and one nurse working in the night shift on April 16 was observed. Moreover, in order to confirm whether any relevant items were omitted during observation, nursing records through the electronic medical system in the hospital were reviewed at the end of the observation periods.

Validation
The content validity of the items identified through clinical observation and literature review was confirmed by two nursing professors and one NICU head nurse working at a tertiary hospital in S City. The content validity in the expert panel was analyzed based on the content validity ratio (CVR) suggested by Lawshe (1975) [21].

Recruitment of the Panel of Experts
No clear rule exists for the number of panel experts and selection criteria in a Delphi study. However, because a Delphi study depends on expert opinions rather than direct data, the composition of the expert panel is crucial for its successful application. The previous studies suggested that at least 10 experts are needed and that small groups of experts (10 to 15) may yield useful findings [22,23]. The expert panel candidates in this study were NICU nurses working at three tertiary hospitals in S City and G Province in Korea, each of whom has more than five years of experience in the field, as suggested by Benner (1984) [24]. The following criteria were applied to select the panel nurses: those who are members of the Korean Association of Neonatal Nurses, have experience in conducting research on neonatal intensive care, and have completed an NICU nursing program; those with master's or PhD degrees in neonatal intensive care; those who are working as lecturers on neonatal intensive care in nursing programs at universities; those who are licensed to provide specialized care for pediatric and neonatal populations; and those who have experience drafting job descriptions at hospitals. Moreover, we also selected specialists responsible for neonatal intensive care at tertiary hospitals who have more than four years of experience in addition to research experience on neonatal intensive care and pediatrics. In total, 14 candidates were identified for possible inclusion in the study panel. The researchers reviewed the candidates' clinical experience in NICU and activity in expert groups and evaluated their independence, adequacy, quality, and responsibility. Of these 14, one nurse who had more than five years of experience in NICUs was excluded, as this person had moved to an adult unit more than two years ago. Finally, we selected 13 panel members.

Delphi Round 1
This study's Delphi survey was conducted between 30 May and 31 August 2019. In the first Delphi round, preliminary items for the standard guideline for NICU nurses identified through clinical observation and literature were presented, and the experts were asked to rate each item on a 5-point Likert scale. In the first Delphi round, in order to account for the experts' various opinions, the survey questions asked the experts to describe the work of NICU nurses; based on this information, items could be revised, deleted, or added.

Delphi Round 2
The second Delphi round was designed based on the results of the first Delphi round. The researchers analyzed the results of the first Delphi round and provided the mean, standard deviation, and inter-quartile range for each item in the second-round questionnaire, along with the responses of each expert in the first Delphi round. Through this process, the experts were able to compare their own responses to those of the other participants. This second Delphi round provided opportunities for the experts to reflect, revisit, and revise their opinions by comparing their own responses to the mean of all experts. If the experts' opinions deviated outside the inter-quartile range of the mean response of all experts, the experts were asked to describe why they evaluated the items as such. CVR, convergence, agreement, and coefficient of variation were calculated in the second Delphi round, as in the first Delphi round.

Delphi Round 3
The third Delphi round aimed to reach a consensus on nursing work standards based on the results of the first and second Delphi rounds. The results of the first and second rounds were analyzed, and the experts were asked whether the items with a CVR of and below 0.54 should be excluded or retained in the NICU nursing work standards, along with their reasoning for this response.
Delphi Round 4 In the fourth Delphi round, the NICU standard guideline developed through the third Delphi round was confirmed for content validity. Moreover, when clinical experts had opinions on the NICU nursing work standards, they were asked to provide these opinions.

Ethical Considerations
The present study was reviewed by the institutional review board of C University in Korea (1040271-201902-**-037), and the purpose of the study was explained to the participants prior to obtaining their written consent. The written consent outlined voluntary participation, anonymity, and scope of use of the results. The participants were reminded that they may withdraw their participation whenever they wished and that the study results will be used solely for research purposes. Consultation fees were provided to the expert panel after each round.

The Positive Coefficient of Experts
The positive coefficient of experts is the effective recovery rate of the expert consultation questionnaire and can reflect positive input from the experts [25].

The Degree of Expert Authority
The degree of expert authority was determined by two factors. First, that there is the expertise to make a judgment on the consultation, this represents the ability of the expert to judge the content of the coefficient (CA). A CA sum equal to 0, suggests that the impact of expert judgment is small, A CA sum equaling 0.5, means the influence degree of expert judgment is medium, and finally, a CA sum equaling 1.0, indicates that the influence degree of expert judgment is great. In this study, the experts used the terms "working experience," "theoretical analysis," "literature," and "intuitive sense" as the basis for judgments.
Second, the degree of familiarity of the experts on the inquiry content. (CS) [26] A Likert scale was used; very familiar (1 point), largely familiar (0.75 point), generally familiar (0.5 point), less familiar (0.25 point), and unfamiliar (0 point). The degree of familiarity of each expert was calculated statistically.
The degree of expert authority (CR) the coefficient (CA) and the degree of familiarity of the inquiry content (CS), were calculated thus: CR = (CA + CS)/2. Values greater than 0.7 were considered to be acceptable [27].

Degree of Expert Coordination
Based on this, the criteria for content validity was set as a CVR of 0.54, and items with a CVR at, or above 0.54 were considered to have adequate content validity. Moreover, the criterion for convergence was set below 0.50, and for agreement at 0.75 and above [28]. To achieve stability of items through expert agreement, coefficients of variation were calculated, and those with a coefficient of variation exceeding 0.8 indicated a need for re-evaluation [29].

General Characteristics of the Delphi Survey Expert Panel
The panel comprised 1 male (7.7%) and 12 female (92.3%) experts; participants included 1 physician (7.7%) and 12 nurses (92.3%). The mean age of the experts was 33.3 years, and the experts had an average 10.8 years of experience. Moreover, the experts had an average of 9.7 years of experience in the NICU. In terms of special licenses, three experts were International Board-Certified Lactation Consultants. The 13 experts thought that 5.4 years of clinical experience on average were necessary to achieve expertise as an NICU nurse (Table 1).

Positive Coefficient of Experts
In the first, second, third and fourth round of Delphi expert consultation, the recovery rate of the four rounds of expert inquiry questionnaires was 100%, and the effective rates were 100%.

The Degree of Expert Authority
The average CA = 0.78 and CS = 0.94 for the research experts. The average was CR = 0.86 for all the experts in this study.

Development of the Standard Guideline for NICU Nurses
In order to select items for the standard guideline for NICU nurses, this study first selected 178 preliminary items based on previous studies conducted on nursing standards in the NICU [10][11][12][18][19][20]. The selected items were classified based on the Korean Nurses Association after consideration of nursing education in Korea; nursing work standards comprised five domains, and professional performance standards comprised seven domains ( Table 2). Of the items observed in clinical practice, 12 items did not overlap with those found through the literature review and were thus included in the 190 preliminary items for the standard guideline. The content validity of the selected items was confirmed by two nursing professors and one head nurse working at the NICU of a tertiary hospital in S City. Blood sampling and foot stamping, which had content validity below 0.5, were excluded from the 12 items found through clinical observation; the other 10 items, including foley catheter insertion, enema, phototherapy, surgical patient care, umbilical cord care, and G-tube feeding, had CVRs of 1.0 and were retained. Moreover, shun syndrome, which was identified through the literature review, had a convergence below 0.5, suggesting that it does not reflect the Korean nursing environment. It was thus excluded. As a result, a total of 3 items were excluded from the initial 190 items; the remaining 187 items were selected for potential inclusion in the NICU standard guideline. In the NICU nursing standard guideline, the nursing work standard domain comprised data collection, diagnosis, planning, performance, and evaluation. This was based on the standard nursing activities outlined by the Korean Nurses Association.

Delphi Survey for the NICU Nursing Standard Guideline
The results of the first and second Delphi rounds are shown in Appendix A. In the first Delphi round, 21 items had CVRs below 0.54, a convergence of 0.50 or above, and agreement below 0.75, but the items had coefficients of variation above 0.8. In the second Delphi round, 14 items had CVRs below 0.54. The third Delphi round investigated whether it would be valid to exclude those 14 items; ultimately, 3 items with CVRs below 0.54 were excluded from the final guideline. The excluded items were: observe facial expression of mother/caregiver to identify feeling; demonstrate the use of manual breast pump; body weight and condition of the mother before and after delivery. A total of 184 were identified through the third Delphi round.
In the fourth Delphi round, all items had CVRs of 1.0. When asked for input on the standard guideline for NICU nurses developed in this study, the experts emphasized that systematic education should be provided in schools and in clinical practice based on the standard guideline. In particular, the experts made the following statements: "In real life, students can't touch the babies during their clinical placement. The caregivers react sensitively to students touching the babies . . . so students can't really do anything. When we accept new nurses, we expect them to know the basics, such as blood pressure, body temperature, and weight measurement . . . " (Expert 1) "I hope the new nurses will have sufficient knowledge of normal and abnormal from their training in school. We can't provide theoretical training in clinic due to a lack of time. In clinic, we have to provide care based on what we learned in school, so we should be well aware of the details." (Expert 2)

Discussion
The present study was intended to develop a standard guideline for NICU nursing through the Delphi method and to discuss future directions which may allow nursing students to obtain expert knowledge and skills in neonatal intensive care despite the limited clinical placements available to trainee nurses.
First, when experts on the panel offered their opinions on the proposed NICU standard guideline, they noted that they expected student nurses to be educated on basic nursing tasks, such as assessment, nursing diagnosis, planning, and performance, rather than professional tasks, such as evaluation, education, evidence-based nursing, cooperation, use of resources, and quality management. Recently, clinical experiences in nursing care for high-risk infants have been limited due to details of the clinical situations in Korea and the need to limit infection potential among high-risk newborns; therefore, nursing students have had little clinical exposure in the NICU [13]. To address this issue, it is crucially important for nursing educators to have access to basic data to prepare educational materials. In particular, clinical experts on the research panel emphasized that students should gain sufficient knowledge of nursing for high-risk infants at school. At a minimum, the experts agreed that student nurses should be able to identify normal and abnormal conditions in high-risk infants during any clinical placement, as it is often difficult to provide theoretical education in clinical situations. The main opinion of clinical experts was that there were no practical tasks that students could do in the current neonatal intensive care unit practice environment, so they wanted to sufficiently practice physical assessment and some technical skills in school. In particular, there is a lack of time to impart theoretical knowledge during clinical postings, which is why it is necessary to sufficiently learn theory in school and, based on that knowledge, observe clinical cases. The panel of experts also emphasized the importance of infection control and expected NICU nurses to learn about it thoroughly. On the other hand, aspects of NICU nursing that involve different systems and are hospital-dependent such as respiratory equipment, monitor operation, and hospital discharge, are expressed as areas that need to be strengthened in clinical education in the future. Further, the experts also suggested that schools should adopt other learning strategies to support students' skill development. For instance, although physical assessment is a basic task that is important for all nurses, students cannot directly touch newborns in clinical settings. Therefore, schools should strengthen objective structured clinical examination (OSCE), clinical performance/practice examination (CPX), standard patient (SP), and simulation education [30]. In addition, it can be seen that it is necessary to develop education programs including various scenarios, and to evaluate each one. This finding coincides with a previous study that applied virtual reality and blended simulation of asthmatic children [31] and found that various simulation systems, including high fidelity simulation, can improve nursing students' critical thinking, problem-solving, and clinical performance. On this basis, nursing educators should design clinical education programs which employ various evidence-based learning strategies when developing material on nursing for high-risk infants. Panel experts also pointed out the need for nursing education on the risks of infection, hypothermia, impaired gas exchange, and neonatal asphyxia, which are being emphasized in clinics, in addition to physical assessment. This aligns with previous studies emphasizing the importance of developing targeted scenarios and educational programs, such as neonatal CPR emergency nursing programs [32], simulation scenarios for apnea emergency management [33], and emergency airway management programs for NICU nurses [34] in order to improve the clinical competency of students. As most Korean studies have focused on developing scenarios for respiratory conditions, such as dyspnea and CPR, future studies should develop various cases and scenarios targeting areas of particular concern to the expert panel, such as hypothermia, impaired gas exchange, neonatal asphyxia, and risks of infection. These studies should provide education on using these materials and evaluate their effects on students' competencies. In addition, in order to meet student nurses' learning goals, future studies must consider the appropriate means of delivery for such scenarios.
Second, as a result of this study, the nursing standard guidelines for nurses in a neonatal intensive care unit consisted of nursing practice standards and professional performance standards.
The professional performance standards comprise ethics, performance evaluation, education, evidence-based practice and research, consultation, and operation, proper use of resources, and quality management. In contrast to a previous study conducted outside of Korea on nurses providing care for newborns [8]-which divided neonatal nurses' work standards into simple nursing stages and overlooked nursing practice domains-the present study has developed the proposed NICU standard guideline based on the nursing practice standard developed by the Korean Nurses Association, by applying professional performance standards particularly suitable for Korea. Another previous study conducted in Korea divided NICU nurses' work into direct and indirect nursing activities while focusing only on nursing work standards [10]. The present study also overcomes this limitation, adding items identified through clinical observation to reflect recent changes in nurses' work. The present study once again confirms that Korean nurses providing neonatal intensive care are required to perform a range of tasks from simple to highly complex, in addition to a range of emergency, professional, and educational activities and roles. As these competencies are not developed in a short period, it can be seen that a systematic step-by-step, education program is needed from the undergraduate level up to higher levels. According to a recent study on the ethical environment of neonatal intensive care unit nurses which targeted Korean nurses, neonatal intensive care unit nurses have higher levels of moral anguish than other departments, leading to higher work stress and turnover rates. The study suggests that education is necessary to overcome this problem [35]. Therefore, in the neonatal intensive care training course, in addition to theoretical education, case-based learning related to professional performance standards is required.
Third, the items excluded from the present study mostly concern breastfeeding. The results suggest that limited breastfeeding and tube feeding are more common in Korea, in contrast to other countries where breastfeeding is often provisioned in the NICU. In Korea, the mothers are trained on breastfeeding in postpartum care facilities and obstetrics clinics; it is, therefore, appropriate to exclude these tasks from an accounting of NICU nurses' main activities. Other tasks related to public and community engagement, such as "assess whether the family is able to take care of financial issues," had low CVRs in the first and second Delphi rounds. This may be due to the study context, as financial support and community engagement for high-risk infants are provided through administrative and clinical cooperation departments in Korea. However, the third Delphi round revealed that experts considered financial support assessment for high-risk infants and their families a part of NICU nursing professionals' repertoire. Bearing this in mind, future studies should investigate the roles of Korean NICU nurses in relation to specific professional performance domain items.

Conclusions
The present study used the Delphi method to develop a NICU nursing standard guideline for the Korean context and to confirm aspects of practice and professional skills that should be emphasized in clinical education for nursing students. This study is significant in that it developed a NICU nursing standard guideline upon consideration of the Korean clinical environment through clinical observation and literature review. Moreover, the proposed standard guideline can also be used as basic data when developing clinical guidelines for nurses in practice. Future studies should select specific domains and skills from the NICU nursing standard guideline that are expected to be demonstrated in practice by clinical experts, developing educational standard guidelines and materials based on these target areas.

Limitations of the Study
The present study is limited in that only clinical experts' opinions were considered in selecting the items assessed and ultimately included or excluded from the guidelines, determining which aspects were important to clinical placement and education on nursing for high-risk infants. To expand the relevance and applicability of this work, future studies should consult students who have had clinical placements in order to identify knowledge and techniques contained within the NICU nursing standard guideline that are difficult to obtain in clinical settings. This information can be used to develop and apply targeted educational programs to improve both nursing students' experience and professional standards of care in practice. Acknowledgments: In this section you can acknowledge any support given which is not covered by the author contribution or funding sections. This may include administrative and technical support, or donations in kind (e.g., materials used for experiments).

Conflicts of Interest:
The authors declare no conflict of interest.