A Hybrid Methodology for the Evaluation of Clinical Practice in Final-Year Nursing Students

(1) Background: Clinical practice constitutes a scenario where the student approaches reality. The pedagogical relationship that is built between the nurse, the tutors and the student becomes important. And this requires intentional and reflective accompaniment. The principal objective was to design a hybrid-learner-centered training model requiring reflection and acquisition of specific skills. (2) Methodology: This was a prospective observational study using an intentional sample of 87 students. A hybrid model based on a dynamic virtual forum and Individual Improvement Plan (IIP) was constructed, evaluated using a self-completed questionnaire with a Likert scale. (3) Results: A model of accompaniment to the practices was built that allows for unifying a work plan. A transversal activity IIP was designed. A discussion forum was incorporated for each subject tutor. The analysis of the questionnaire showed that learning assessment, tutorials, virtual forums, self-assessment and satisfaction statistically differed. (4) Conclusions: The model allows students to be accompanied to acquire skills, knowledge, and attitudes and to develop critical thinking, as well as to improve the teaching quality of the practices of the Curriculum of the Nursing degree and to achieve their own competences through student-centered methodologies. This study was not registered.


Introduction
Clinical practice occupies an important place in the nursing degree curriculum, and students in Spain must perform a total of 2300 h of practice during their four academic years, with most of this concentrated in the final year [1]. Practical subjects are where most theoretical and methodological knowledge transfer takes place, and where all scientific, relational, ethical, attitudinal and procedural learning must actually be deployed. These skills are based on the development of six axes, which constitute the basis on which the learning process of clinical practice is based: professional values, attitudes, behaviors and ethics of care; the scientific foundations of the biological, human and social sciences; critical, logical and creative thinking; information and knowledge management; clinical communication and relationship skills; and, finally, procedural clinical skills [2]. Clinical practice is meant to be an open, dynamic and changing space characterized by the complexity of clinical situations, and some studies [3,4] indicate that students may experience high levels of stress, depression and anxiety, especially as they approach the final stages of their degree. Pulido-Martos et al. [5] show that among stress-inducing factors are the perceived lack of knowledge and skills needed to care for patients or having to work under the close supervision of tutors. Given this reality, undergraduate training must offer strategies and accompanying resources that help students to carry out their clinical practice requirements in an enabling environment. In this context, the pedagogical relationship that is built between the nurse, the tutors and the student becomes important. Intentional and reflective Nurs. Rep. 2023, 13 1005 mentoring is required to guide the student's own reflection based on their experience and training and to help them develop analytical and critical thinking [6]. Based on the affirmation from Alfaro-Lefevre [7], it is not a method to be learnt, but a changing process, that needs skills, knowledge and some attitudes or dispositions; it is contextual, has a target and looks for self-improvement.
In this sense, the tutor must seek resources for dialogue around the significance of the act of caring, linking the results of students' reflective processes to future practice and encouraging reflective, scientific, dialogue-based and ethically competent actions. This learning, according to various authors [8], is generated when the student actively participates in its process, constructs meanings and elaborates proposals based on collaboration with both teachers and peers. Such learning thus occurs in situations that drive the student to develop their knowledge in a meaningful way. The aim for tutors is to make sense of each action and to link the results of this reflective process to future practice, i.e., to move from what to teach to how to teach, leading students to meaningful and deliberate learning. Schön [9] calls this process reflection in action. Although these skills are not necessarily innate, they can be learned through specific training, which is why nursing educational programs should contribute to the development of critical thinking skills. This is so much so that Zuriguel [10] proposes that educational institutions should structure syllabi, curricula, teaching methodologies and evaluation systems with the aim of integrating strategies that promote an educational environment that strengthens and develops these skill [11].
The type of learner, content to be studied, location of the participants in the process and electronic media available are all factors that will determine the model to be implemented. The literature consulted [11][12][13][14] shows that mixed learning models-known as "b-learning"-combine the best of virtual methods with face-to-face methods. However, in all of them, it is important to encourage the creation of a learning community-a space for the construction of knowledge where students and tutors can interact [15][16][17]. As a result, active methodologies such as forums, the flipped-classroom or conceptual maps are a strategic framework for student-centered learning, steering trainees toward action [18][19][20].
In the case of clinical practicums, it is difficult to find assessment tools that allow for an active methodology outside the classroom and that fit the philosophy of higher education. This study proposes a model based on these methodologies to accompany the student in their practicum and to be able to evaluate their competences. It is true that hybrid learning modalities are used in many countries, but in clinical practices outside the university, it is difficult to monitor the student. Therefore, this study proposes a hybrid model based on two axes: the theory-practice relationship and critical thinking. The model follows a constructivist theoretical approach, understanding knowledge as a constructive-creative process where students are expected to play an active role, promoting an enquiring attitude to research and developing what is meaningful to them. Thus, the main objective of this study was to design a hybrid-learnercentered training model requiring reflection and acquisition of specific skills, which could be integrated into the clinical practice requirements of the nursing degree. A secondary objective was to evaluate the satisfaction with the proposed model among students in the last year of their nursing degree.

Materials and Methods
A prospective observational study was carried out at the Faculty of Nursing in the University Rovira i Virgili of Tarragona (URV), Terres de l'Ebre Campus (Tortosa, Spain).

Sample
A total and intentional population of 87 fourth-year nursing degree students were selected. They were enrolled in three clinical practice subjects of the second block of the academic year, which corresponds to the Curriculum The subjects are divided into internship cycles of 5 weeks each, where in small groups, the students undergo 4 cycles of practices.

Ethical Considerations
The study obtained approval from the Pere Virgili Institute for Health Research's Drug Research Ethics Committee, with the code 121/2020. All participants were duly informed and signed the corresponding informed consent voluntarily.

Instruments
The tools that constituted the hybrid model of learning in clinical practice were ( Figure 1).

Sample
A total and intentional population of 87 fourth-year nursing degree students were selected. They were enrolled in three clinical practice subjects of the second block of the academic year, which corresponds to the Curriculum (2009): Geriatric Care, Critical Care and Psychiatry and Mental Health during the 2018-2019 and 2019-2020 academic years. The subjects are divided into internship cycles of 5 weeks each, where in small groups, the students undergo 4 cycles of practices.

Ethical Considerations
The study obtained approval from the Pere Virgili Institute for Health Research's Drug Research Ethics Committee, with the code 121/2020. All participants were duly informed and signed the corresponding informed consent voluntarily.

Instruments
The tools that constituted the hybrid model of learning in clinical practice were (Figure 1). A self-completed questionnaire (ad hoc): This questionnaire was created by the authors and was based on the specific needs of collecting short-term satisfaction with the hybrid model of learning. The questions of the ad hoc questionnaire were based on the models of three authors, Cabero Almenara [21], de Milenca Vilca Pozo et al. [22] and Pineda Herrero et al. [23]. Information was collected using the ad hoc questionnaire, which included a total of 50 questions on a Likert-type scale scored from 1 to 4 (1 = strongly disagree, 2 = disagree, 3 = agree, and 4 = strongly agree). The instrument included sociodemographic variables and satisfaction with the hybrid model. The different sections of the ad hoc questionnaire did not form a construct among themselves. Three aspects were asked, such as the functioning of the forum, assessment of the tutor and satisfaction. The last three questions consisted of a self-assessment. The instrument was supervised by a group of experts composed of professors from the psycho-pedagogical areas of the same university. The relevance, adequacy and clarity of the questions were assessed to obtain the opinion of the students in relation to the different parts of the model. Dynamic virtual forum: This was structured as the core of the model. The guidelines for the use, objectives and description of the activities were drawn up in advance. Training was given to the tutors who are facilitators of the forum. These are the academic professors A self-completed questionnaire (ad hoc): This questionnaire was created by the authors and was based on the specific needs of collecting short-term satisfaction with the hybrid model of learning. The questions of the ad hoc questionnaire were based on the models of three authors, Cabero Almenara [21], de Milenca Vilca Pozo et al. [22] and Pineda Herrero et al. [23]. Information was collected using the ad hoc questionnaire, which included a total of 50 questions on a Likert-type scale scored from 1 to 4 (1 = strongly disagree, 2 = disagree, 3 = agree, and 4 = strongly agree). The instrument included sociodemographic variables and satisfaction with the hybrid model. The different sections of the ad hoc questionnaire did not form a construct among themselves. Three aspects were asked, such as the functioning of the forum, assessment of the tutor and satisfaction. The last three questions consisted of a self-assessment. The instrument was supervised by a group of experts composed of professors from the psycho-pedagogical areas of the same university. The relevance, adequacy and clarity of the questions were assessed to obtain the opinion of the students in relation to the different parts of the model. Dynamic virtual forum: This was structured as the core of the model. The guidelines for the use, objectives and description of the activities were drawn up in advance. Training was given to the tutors who are facilitators of the forum. These are the academic professors coordinating the subjects and have been the key to developing the tool, as they served as reference points for the learner, be it to motivate, help and facilitate information or to redirect issues that arose and to keep the virtual environment alive.
The Individual Improvement Plan (IMP): this document was designed and structured (Table 1) based on a documentary analysis [24][25][26]. Evidence was collected to reflect the progress and objectives achieved by the students during their clinical placements [27,28]. The document allows for a grouping of improvement activities and interventions, coordinated and aggregated to avoid dispersion in their reflection [29]. In the introductory seminar of the practices, the document informed students about its structure, how to fill it in and its objective. At the end of the internship period, they will have to deliver it through Moodle: a virtual learning platform.

Design and Implementation
Three phases were established for implementing the hybrid model of student support during clinical placements: • Phase 1 "Diagnostics": A consolidated work plan with cross-curricular activities was designed. The Individual Improvement Plan (IMP) and support material were provided on the Moodle platform. • Phase 2 "Training": In the introductory seminar to the clinical placements and in the introductory seminar of the practices, the timetable of the model, that is, the dates of completion and delivery of the activities, was explained to the students. They also informed students of the IMP document, explaining its structure, how to complete it and the desired objectives. An asynchronous discussion forum was set up that would be run by the tutors of each subject for each internship period. At the end of each cycle, a face-to-face seminar was given where each student assigned a category code to their contributions to the forum and, in groups of 4 students, presented the results to the rest of the group by means of a concept map. Finally, they completed the model satisfaction questionnaire. • Phase 3 "Summary": Evaluation of the students' learning during the forum contributions was continuous and individual. This activity was linked to the IMP, where students incorporated all those aspects discussed in the forum and during their clinical practice experience, being evaluated at the end of each period by the tutor in charge.
SPSS version 27.0 was used to analyze the questionnaire results (Statistical software, IBM Corp., Armonk, NY, USA). The level of significance was set at p < 0.05 and p < 0.001. Descriptive statistics were carried out on the sociodemographic characteristics of the total subjects included, and 3 groups of clinical practice, with means and standard deviations. The distribution of the data was evaluated using the Shapiro-Wilk normality test. An ANOVA was used to compare the 3 groups of clinical practice.

Assessment of the Model
The implementation of the model resulted in a total of 261 IMPs: 87 for each subject. Each student delivered three IMPs, one for each subject. The sections of the Individual Improvement Plan document were based on reflection, action, evaluation, participation, personnel, usefulness and transfer [29].
In terms of participation across the six forums, there were a total of 1734 participations, with an average of 289 contributions per forum. In the Critical Care Unit placement, there were a total of 450 interventions with an average of 89 per forum. In the Geriatric Unit, there were 591 participations, with an average of 98.5. In the Mental Health Units, there were 693 participations with an average of 100 per forum. Finally, 30 concept maps were obtained, 13 for each subject.

Satisfaction Questionnaire: Analysis
A total of n = 158 (90.8%) self-completed questionnaires were obtained. Of the total, n = 111.5 (70.6%) of the students were women with a mean age of 21.18 years (SD = 2.186).
The analysis of the questionnaire showed mean scores above 3 points in almost all sections of the questionnaire, indicating a high degree of satisfaction in each of the questions. Likewise, when comparing the mean scores between the three clinical practice subjects, Geriatric Care, Critical Care and Psychiatry and Mental Health, significant differences were found. The first section collected the opinions in relation to the virtual learning environment and discussion forum: item 3 (p = 0.004), item 4, (p = 0.001), item 7 (p = 0.007), item 10 (0.008), item 11 (p < 0.001), and item 15 (p = 0.001). In the motivational tutor section of the forum: items 18 (p = 0.041), 23 (p < 0.001), 25 (p = 0.043), 28 (p = 0.001), and 29 (p = 0.016). Regarding overall satisfaction with the hybrid model, statistically significant differences were observed between the scores of item 45 (p < 0.001). Finally, no differences were observed in the scores of the self-assessment section (Appendix A, Table A1).

Discussion
The general results of the ad hoc questionnaire, in line with results from other studies [30], show that the forum and the virtual space were satisfactorily evaluated by the students.
We observed that the creation of an easily accessible virtual space, with no time restrictions and in small groups where experiences can be shared and explained, promotes debate and improves support by personalizing the supervision of students during their clinical training. This space is, therefore, ideal for recognizing and learning from previous experiences. Chan's study [31], which reviews strategies to improve critical thinking, concludes that the forum is a space that allows interaction. The tutor, who is constantly with the students during this learning process, is in charge of reassuring them, grounding their reflections and helping them grow. However, we also observed some points of disagreement between the groups. Regarding tutors, elements to be improved were helping with collaborative learning, contributing with more reference sources, giving individual feedback and synthesizing information. The literature [32] has already shown that one of the greatest challenges for teachers who propose a forum is to encourage participation and keep students motivated, with importance given to creating environments where students feel stimulated. Different studies [33,34] have shown that feedback is of central importance in online training processes, especially in the types of learning that require some degree of personalization.
The differences observed in this study suggest that student motivation is crucial for the tool to be used correctly and to achieve the objectives of the model. It is evident-and existing studies corroborate this [35]-that social motivations such as the desire to share knowledge, personal motivations such as recognition from colleagues and teachers, and technological motivations such as the usability and convenience of the platform play a key role in the success of discussion forums.
Motivation arises when what is being discussed gains relevance and when friendly and challenging spaces are created, which generate conceptual discussions that encourage participants to seek knowledge [36]. Therefore, it is important to highlight the role of the tutor as a person who galvanizes the platform while creating a safe space. The fact that this model is organized in small groups helps with students' motivation to share their experiences. It is a space to contribute to one's experiences with profound reflection, allowing a university education to shape competent, critical, creative and reflective students. The role of the teacher is to facilitate, guide and mentor students during their training, promoting their autonomy and ability to reflect on and regulate their own learning [37].
Thus, there is no doubt as to the importance of well-developed tutorial action and the high level of responsibility that falls on the tutor during clinical practice placements, where students have a real opportunity to learn in a professional context, to assess their level of mastery in real situations and to have the potential to both reflect and act on what has been learned in the university.
The practicum allows students to gauge their competencies and not only knowledge or abilities. But the competency approach in clinical practice, as Morales and Varela [38] warn, still requires collaboration, exploration and a clear construction of its premises to be able to develop and implement precise strategies for its teaching and evaluation. Armengol et al. [39] argue that this moment should be used as an opportunity for students to make individualized work plans, allowing them to acquire the skills they do not yet have and to strengthen the ones they already possess.
Along these lines, various initiatives have been developed to introduce active methodologies. However, Canalejas et al. [40] show that the methods used have often enhanced reproductive learning, rather than knowledge construction. Our proposed model is presented as a space for students to reflect on their lived experiences during clinical practice, allowing for a university education that trains competent, critical, creative and reflective students. Reflection here is understood as a means to improve professional action, not as an end in itself.
Combining active methodologies creates a model of semi-virtual assistance, a resource for the exchange of knowledge among participants that, in the form of an educational strategy, makes it possible to encourage critical thinking in the world of clinical practice. Other studies [41][42][43] also show that active methodologies are instruments that not only improve critical thinking, but also increase academic performance and motivation.
All together, the model is able to enhance students' reflections both during and on their clinical practice, acting as a guide that promotes critical thinking in a clinical practice environment where students often feel alone and do not perceive the relevance of the group. However, the role of the tutor needs to be further explored in order to achieve group enthusiasm, facilitating cooperation and collaboration in the learning process. This study presents a tool that can help students achieve learning outcomes in clinical training; it has great potential in the face of increasingly virtual settings that have brought about a change in the universal teaching model. Therefore, this study addresses a gap currently existing in the literature, and brings added value to the disconnect between the university professor and the undergraduate nursing student during clinical practice in health centers. Even so, the present study has some limitations, such as validating a questionnaire that incorporates a joint evaluation construct, replicating the multicenter study in other academic courses of the nursing degree in clinical practice subjects, as well as reasoning by means of a qualitative study the differences in the mean scores obtained between the different subjects observed.

Conclusions
Adapting to the European Higher Education Area (EHEA) implies a transformation in the way university education is conceived of and implemented; new ways of assessing competencies in clinical practice courses are needed to encourage student participation.
The need to be critical and reflective is considered both a challenge and a solution to the learning and teaching processes. The hybrid model of support is a good starting point from which to encourage nursing students to develop critical thinking; it inspires them to reflect, analyze and critique, thus creating a student-centered learning environment that favors collaborative learning where they can share knowledge with their peers.
The proposed model is based on a constructivist process: students start with the detection of a problem or situation; they reflect on it among their peers in a forum mediated by a tutor; they subsequently share concept maps; and, finally, they evaluate and search for solutions in their Individual Improvement Plans. This model is envisaged as an environment where students are actively involved in the learning process and not just passive participants. It is a space where they have the opportunity to analyze and reflect on professional practice; to understand the experience of clinical work; to be willing to change or relearn and accept responsibility for decisions made; and, above all, to aim for self-improvement. The model avoids a routine and mechanized experience of practical placements and responds to the new approach of increasingly virtualized universal education in the context of clinical practice.

Institutional Review Board Statement:
The study was conducted in accordance with the Declaration of Helsinki and approved by the Virgili Institute for Health Research's Drug Research Ethics Committee, with the code 121/2020.

Informed Consent Statement:
Written informed consent has been obtained from all subjects involved in this study to publish this paper.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.

Public Involvement Statement:
Final year nursing degree students participated in this study.

Guidelines and Standards Statement:
This manu-script was drafted against the the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines were followed in this observational study.

Acknowledgments:
We thank the students of the Universitat Rovira Virgili who have participated in the study.    Variables are presented as Mean ± standard deviation (SD) and ANOVA test is used to assess differences between groups. * p < 0.05; ** p < 0.001.