Efficacy of an Integrated Simulation-Based Education Approach to Train Non-Technical Skills in Medical Undergraduate Students

Abstract

Introduction: Non-technical skills (NTS) are essential to maintain best patient care and to enhance patient safety. Therefore, the early integration and implementation of non-technical skills (NTS) training during undergraduate medical education has not only been repeatedly emphasized but is crucial to ensure patient safety. However, there is a gap in knowledge on how to effectively convey and thereby enhance NTS in under- or postgraduate medical training. Although medical educators are eager to convey NTS, the best approach to do so is yet to be found. Next to didactic and teaching approaches, the design of the curriculum, in the sense of an integrated curricula, might have an influence in conveying NTS effectively. Therefore, we investigated if repetitive simulation-based training enhanced the NTS of medical undergraduates. Methods: 945 undergraduate students in their 1st, 2nd, 3rd and 4th year of medical studies who attended mandatory emergency simulation trainings at the medical faculty in Hamburg participated in this prospective observational study. The trainings are designed to align to a learning spiral (vertically integrated curriculum) and build upon each other, resulting in repetitive and advanced training. Each training has a preset training-specific standardized simulation scenarios. To determine progress in NTS performance, the NTS of the students were assessed during each training with the Anaesthesiology Students’ NTS (AS-NTS) taxonomy, which is composed of three dimensions. Differences in performance between the investigated cohorts of students were calculated with an analysis of variance (Welch’s ANOVA) and with a follow-up post hoc test (Games-Howell), and the significant differences were analysed. Results: The NTS of the undergraduate students improved with every ascending training and, hence, year of medical school. The overall NTS performance of the 4thyear undergraduate students differed significantly compared to the 1st, 2nd and 3rd year undergraduate students performances, F = 6.72 (4, 439.6), p ≤ 0.001. The greatest improvement was detected on dimension one of the AS-NTS (“Planning tasks, prioritising and problem-solving”) (F = 6.75 (4, 434.5), p ≤ 0.001), where 4th year undergraduates performed at a significantly higher skill level than the 1st, 2nd and 3rd year undergraduate students. On dimension two (“Teamwork and Leadership”) and three (“Team orientation”), significant differences were pronounced between the 4th year and 1st year undergraduate students (F = 3.56 (4, 442.9), p = 0.010; F = 4.50 (4, 441.0), p = 0.001). The lowest skill improvement was observed on dimension two. Conclusions: NTS are essential skills to complement technical skills and hereby ensure patient safety. The early longitudinal and vertical integration of NTS training in undergraduate medical curricula (learning spiral) is an effective approach to enhance and foster the NTS of future doctors.

1. Introduction

Since the landmark publication of the Institute of Medicine “To err is human” was published over two decades ago, NTS have been recognised as key elements to ensure patient safety and it has been emphasised that human error significantly leads to patient harm [1].

NTS are defined as “the cognitive, social and personal resource skills, that complement technical skills and contribute to safe and efficient task performance” [2]. NTS are composed of cognitive and interpersonal skills, which are usually acquired among the socialization process of every human and are therefore not homogeneously distinct [3]. It has been demonstrated that NTS do not improve with clinical experience [4]. Furthermore, NTS have traditionally resided outside formal technical education curricula and thus are described to be detached from medical knowledge [5]. Alas, the importance of NTS in terms of patient safety has been highlighted repeatedly, as 70–80% of fatal mishaps in medical high-risk settings involve human factors and hence, a lack of NTS [6].

In the mid 1980s, Gaba and his colleagues anticipated the importance of NTS in high-risk healthcare settings and implemented a pioneering Crew Resource simulation training for anaesthesiology [7]. This program was adapted from aviation (initially Cockpit Resource Management), where simulation-based licensing of NTS became obligatory [8,9] following the deadliest commercial plane crash in 1977,

Since the introduction of Crew Resource Management in anaesthesiology, the awareness of many other medical disciplines on NTS has been consolidated. Imperative efforts have been undertaken to reduce patient harm by minimizing human errors of healthcare providers [5]. These efforts reach from system approaches, amongst others, checklists published by the World Health Organisation [10,11,12] or special training programmes addressing and assessing NTS [5].

As human errors directly contribute to patient harm, early training of NTS is essential [13]. Therefore, it has been urged to integrate and implement NTS training already during undergraduate medical education [14,15,16,17]. Although there is no unified approach and certainly scope for further developments, many medical faculties tried to integrate NTS into their undergraduate curriculum [18].

Despite all the efforts in the field of under- and postgraduate training, the questions of how to convey and teach NTS effectively and how to achieve the greatest learning retention remain unanswered [19]. Simulation-based medical education (SBME) has been recognized as a propriate instructional design to train NTS [5,20]. However, evidence on the best and most effective way to teach and train NTS during SBME is inconclusive. Whilst focusing on didactic approaches and instructional designs during SBME, researchers have barely considered how an NTS training curriculum could be structured in order to convey NTS effectively [21,22,23,24,25,26,27].

Although plenty of research has emerged from the general field of medical curriculum development and many advantages of different curriculum designs have been suggested, translation of that evidence into the field of NTS training has hardly transposed. The reason for this phenomenon is the improvable cooperation and connection between the field of medical education experts and clinicians. While initially medical education was conducted by clinicians alongside their daily duties as a secondary task, the field of medical education has expanded, and has become its own specialty. However, clinical disciplines and medical education need more cooperation in order to achieve the best learning outcomes [28].

Medical education research suggests that an integrated curricula will lead to reinforced learning experiences by vertically and horizontally integrating learning contents [29,30]. A vertical integration means that learning contents are designed based on a learning spiral and therefore, next to repetition, each teaching unit deals with more complex learning contents [30]. The current definition of a vertical curriculum is stated as: “an educational approach that fosters a gradual increase of learner participation in the professional community through a stepwise increase of knowledge-based engagement in practice with graduated responsibilities in patient care” [31]. It is likely that the vertically (spiral) integrated education approach leads to improved learning outcomes [29,31]. This is reached through a stepwise engagement with the learning content, which creates a more mature and explicit understanding after each exposure [29]. However, it should be emphasized that a vertically integrated curriculum is far more complex that just repetitive training or education and, hence, it needs careful planning and adjustment, as well as a close liaison between curriculum experts and clinicians [29,32].

Based on the mentioned aspects, a possible approach to integrate NTS into the medical curricula is a vertically integrated curriculum [31]. A vertically integrated NTS curriculum would be designed based on a spiral (intended repetition), longitudinally integrating the teaching content [31,32]. Undergraduates are thereby gradually introduced to NTS. Translating such an NTS curriculum approach into practice would result in several trainings that build upon each other and, hence, repetitive exposure to SBME. The benefit of carefully designed repetitive NTS training during SBME (vertical integration) could lead to an acceleration of the learning cycle. Learning NTS or translating NTS into individual behaviour is detached from the classical learning approach and therefore, diligence will not necessarily lead to learning success [23]. Hence, avoiding NTS impeding factors could have the result of improved NTS. Such factors are increased perceived stress levels or the cognitive overload of caregivers [33,34,35]. The gradual and stepwise approach to teach NTS would reduce the cognitive load of the learners and make them familiar with stressful situations (emergencies) and would therefore result in reduced perceived stress and, hence, better NTS performance. Thereby, not only the learning cycle is accelerated but NTS impeding factors would be reduced.

However, there is a current paucity of evidence if repetitive exposure to SBME, in the sense of an integrated curriculum (solely repetition does not result in a vertically integrated curriculum), yields significant improvements in NTS performance.

Therefore, we aimed to prospectively investigate the effects of a vertically integrated curriculum in medical undergraduate education, with repeated exposure to SBME, on undergraduates’ NTS performance. We hypothesized that the integrated teaching approach would facilitate the learning cycle and therefore enhance NTS performance. For this purpose, we systematically and longitudinally assessed the NTS of undergraduates in different years of medical school during semester-specific emergency SBME trainings, which were attended in the same chronological order. The primary endpoint was the improvement of overall NTS performance. To detect if any skill component of NTS needs fostered training and special gratuity, the second endpoint was the improvement in NTS performance in different NTS dimensions.

2. Methods

2.1. Ethical Considerations

The local Ethics Committee of Hamburg was contacted with a detailed project description. The project was exempted from the need of approval, as paragraph 9 of the “Law of Healing Professions, Hamburg” (§ 9 des Hamburgischen Kammergesetzes für Heilberufe), § 15 Section 1 of the medical professional conduct (Berufsordnung für Hamburger Ärzt:innen) and, primarily, article 6 of the “Declaration of Helsinki” did not apply. Therefore, no necessity was seen for deliberation on the project (this study was a study with, but not on, humans). Written informed consent was obtained from each study participant.

2.2. Study Setting and Participants

This prospective cohort study was performed at the Department of Anaesthesiology in the University Medical Center of Hamburg-Eppendorf, Germany, during winter semester 2017/18 and winter semester 2018/19. Data were collected prospectively during undergraduate teaching units of the department.

The undergraduate anaesthesiology and emergency curriculum is based on a spiral curriculum model as described by Harden [32]. Therefore, undergraduates participate in nearly every year of their studies in compulsory emergency and anaesthesiology trainings. Each training has preset simulation scenarios, which are standardized and designed specifically and solely for each type of training. With each following training, the simulation scenarios require higher levels of TS and NTS. This study was observational and therefore the undergraduate emergency trainings were not changed.

Mannequins and high-fidelity simulators (Resusci Anne, Laerdal Medical AS, Stavanger, Norway) are used for the trainings, which are suitable for training technical skills such as endotracheal intubation, defibrillation or drug administration.

The trainings are attended in the described chronological order: The 1st year undergraduates participate in two trainings, “Basic Life Support” (BLS) and “Basic Emergency Medicine” (BEM). The 2nd year undergraduates participate in “Advanced Cardiac Life Support I” (ACLS I), the 3rd year undergraduates in “Advanced Cardiac Life Support II” (ACLS II) and at the end of their 3rd year or beginning of their 4th year of studies they participate in “Advanced Cardiac Life Support IIIa” and “Advanced Cardiac Life Support IIIb” (ACLS IIIa/b).

The duration of each training is between 120 and 150 min and due to its structure, each training offers a limited capacity for participants (12–17 undergraduates per training). Therefore, the undergraduates are divided into subgroups and assigned to their training by the Deans’ office before the start of each semester. The trainings are scheduled throughout the semester. Each training is conducted in the simulation centre of the Department of Anaesthesiology and starts with a brief seminar repeating theoretical aspects. Hereby, bias based on a lack of theoretical knowledge was ruled out.

Exemplary Operational Sequence of a Training

All instructors (medical educators who conduct the trainings) are experienced clinicians with extensive knowledge and training in emergency medicine, cardiac life support, medical education and rating NTS (by applying AS-NTS) during simulation trainings.

After the seminar, which is held by one instructor, the undergraduates are divided into smaller subgroups to rotate through the different simulation scenarios, which are conducted in different rooms. In each room, one instructor supervises the specific scenario, which is conducted by a smaller subgroup of three undergraduates. Thereof, the members of the smaller subgroups are randomly assigned to the roles for each scenario: One team leader (physician) and two nurses/paramedics.

The contents of the scenarios are different emergency situations with occurring or manifesting cardiac arrest.

Each scenario is conducted as following:

The undergraduates roleplay an emergency team (team leader (physician) and two nurses/paramedics) that is dispatched to a scene [36]. Prior to their “arrival” (actually entering the room in which the scenario is conducted), the emergency team receives a short briefing of why the emergency team has been alerted (the reason to call “911”), i.e., acute chest pain. After entering the room (scenario), the undergraduates should comply with standards of emergency care aligned with a real-life setting. The goals are to assess the situation by applying the SAMPLE scheme, to establish the monitoring and to follow the ADCDE algorithm. The SAMPLE and ABCDE are mnemonic acronyms to assess the patient’s medical situation and emergency condition [37]. When cardiac arrest occurs, resuscitation must be performed based on the Advanced Life Support algorithm. During the scenarios, therapeutic options are considered and established. The complexity of each medical scenario is based on the teaching unit (learning spiral). As an example, for the teaching units of ACLS II, the medical emergency team is called to a dialysis practice because a patient has chest pain. He has missed his dialysis appointment from the previous day. After a short communication with the undergraduates, cardiac arrest occurs. The scenarios and the events are fully formulated for the instructors.

After each scenario and before executing the next scenario, an instructor-led debriefing, including three conceptual phases, gathering, analysing and summarising, is conducted. The debriefings are designed in a conventional way, meaning that the role of the instructor during the debriefing is that of a teacher.

To rule out further bias that low NTS performance might be a result of unfamiliarity with the simulator or due to TS which have not been internalized, BLS, which is the first training, was not included in our analysis.

All medical undergraduate students in their 1st, 2nd, 3rd and 4th year (n = 958) of studies who attended the mandatory trainings during the study period were eligible to participate. Exclusion criteria was a preceding professional training in healthcare (e.g., paramedics). The undergraduates were informed about the study prior to the start of their semester. Participation was voluntarily and informed consent was attained from each study participant. The undergraduates ensured that information on the simulation scenarios would not be disseminated.

2.3. Assessment of the Outcome Measures: NTS Performance

During the simulation, NTS of the undergraduate partaking the role of the team leader (physician) were assessed by the instructor of each scenario. The German version of “Anaesthesiology students’ Non-Technical skills” (AS-NTS) [36] was used. The AS-NTS has been developed and validated in an elaborate process, which included anaesthesiologists, medical educators and psychologists all with extended expertise in the field of NTS [36]. Validity, feasibility and sufficient coverage of relevant NTS have been previously reported [36]. AS-NTS has proven its applicability in further studies to assess NTS.

2.4. AS-NTS Is Composed of Three Dimensions

• Planning tasks, prioritising and problem-solving,
• Teamwork and leadership,
• Team orientation.

Performance is rated on a five point Likert scale (1 = very good; 5 = very poor). To clarify the terms of a “good” or “poor” performance, an underlying skill structure is used to give behaviourally anchored rating examples.

The primary outcome measures were the summed scores of the dimensions, reflecting the overall performance. The second outcome measures were the scores of the dimensions, reflecting skill elements.

3. Statistical Analysis

Statistical analysis was performed with IBM SPSS Statistics (version 23.0, IBM Corp., Armonk, New York, NY, USA). Descriptive statistics were applied to calculate mean values, standard deviations and errors. For the primary and secondary outcome measures, differences in NTS performance of the cohorts were computed by conducting a one-way analysis of variance (ANOVA). A boxplot analysis was performed to detect outliners, a Shapiro–Wilk was calculated to determine the distribution of data and the homogeneity of variances was asserted using Levene’s test.

There were minor outliners, data were not normally distributed for each group (Shapiro–Wilk test, p < 0.001) and the homogeneity of variances was only partially given (Levene’s test, p < 0.05). Therefore, a Welch test was conducted as a more robust analysis and the post hoc analysis was calculated with the Games-Howell test [38].

4. Results

4.1. Participants and Assessment of NTS

All eligible undergraduates (N = 945) consented to participate within the study. Thirteen undergraduates were not eligible due to prior training in a healthcare profession.

There were no significant differences among the different cohorts regarding demographic characteristics (age, sex).

All rating sheets were complete, except three ratings of dimension one were missing and three rating sheets were not assignable.

Table 1. Number of AS-NTS ratings for each emergency training.

Training Unit	Number of Complete AS-NTS Total N = 939
BEM	140
ACLS I	156
ACLS II	220
ACLS IIIa	222
ACLS IIIb	201

Abbreviations. BEM = Basic Emergency Medicine; ACLS = Advanced Cardiac Life Support. AS-NTS = Anaesthesiology Students’ Non-technical skills.

gives an overview of the numbers of AS-NTS ratings for each emergency training, which were included in the final analysis.

4.2. Outcome Measures: NTS Performance

The level of NTS performance and its progress for each of the training units was analysed separately by conducting Welch’s ANOVA.

Generally, our results indicate that the undergraduates improved their NTS in every ensuing training, as shown in Figure 1. All undergraduates completed the trainings in the same order.

NTS performance for all teaching units and the Welch’s ANOVA results are shown in

Table 2. NTS performance and ANOVA results of the different trainings.

Non-Technical Skills (AS-NTS Score)	BEM	ACLS I	ACLS II	ACLS IIIa	ACLS IIIb	ANOVA
						Welch’s F (dfNum, dfDen)	p	η2
NTS_1	2.42 (1.04) A,B	2.19 (0.95) C	2.18 (0.95) D	2.06 (0.90) B	1.94 (0.70) ACD	6.75 (4, 434.5)	<0.001 *	0.047
NTS_2	2.19 (0.85) E	2.15 (0.89)	2.07 (0.91)	2.09 (0.88)	1.90 (0.78) E	3.56 (4, 442.9)	0.010 *	0.026
NTS_3	2.18 (0.83) F	2.12 (0.93)	1.99 (0.90)	2.00 (0.84)	1.84 (0.74) F	4.50 (4, 441.0)	0.001 *	0.033
Sum_Score	6.79 (2.33) G	6.45 (2.49) H	6.23 (2.51)	6.15 (2.32)	5.66 (1.90) GH	6.72 (4, 439.6)	<0.001 *	0.042

Note: * indicates a significant result, p < 0.01. df_Num indicates degrees of freedom numerator; df_Den indicates degrees of freedom denominator. Superscript letters indicate between which trainings the statistically significant differences were detected. Each letter is assigned to the corresponding training units that differed significantly in NTS performance. The scale of the NTS rating tool (AS-NTS) reaches from 1 (best rating) to 3 (worse rating). A decrease of the numeric AS-NTS score reflects an increase of NTS performance. Abbreviations: BEM = Basic Emergency Medicine; ACLS = Advanced Cardiac Life Support; NTS = Non-technical skills; NTS_1 = Dimension one of the AS-NTS: Planning tasks, prioritising and problem-solving; NTS_2 = Dimension two of the AS-NTS: Teamwork and leadership; NTS_3 = Dimension three of the AS-NTS: Team orientation; NTS_SUM = Summed score of all dimensions.

. The teaching units are depicted in the attended chronological order.

NTS performance differed significantly between the groups for the summed score (primary endpoint), as well as on all dimensions of the AS-NTS (secondary endpoint), as indicated in Table 2.

4.3. Primary Endpoint: Overall NTS Performance

The overall performance, which is reflected by the summed score of the AS-NTS, differed significantly between the 4th (ACLS IIIb) year and 1st year (p < 0.001) as well as between the 4th year and 2nd year (p = 0.009) trainings. The undergraduates improved their skills with a mean difference of 1.14 (95%-CI [0.48, 1.79]) points (AS-NTS sum_score) from their first to last, and with a mean difference of 0.80 (95%-CI [0.14, 1.40]) points from their second to last training.

4.4. Secondary Endpoint: NTS Performance of Each Dimension

4.4.1. Dimension One: Planning Tasks, Prioritising and Problem-Solving

The greatest improvement of NTS was observed in dimension one. Also, on this dimension, the NTS performance of the last training (4th year) was significantly better than the performance of undergraduates in their 1st, 2nd and 3rd year training. The undergraduates improved significantly from their 1st (BEM), to their 4th year (ACLS IIIa, p = 0.007; ACLS IIIb, p < 0.001) training, with mean differences of 0.363 (95%-CI [0.07, 0.66]) (ACLS IIIa) and 0.481 (95%-CI [0.20, 0.76]) (ACLS IIIb) AS-NTS score points.

Improvement from the 2nd (ACLS I) to 4th year training (ACLS IIIb) was 0.252 points (95%-CI [0.00, 0.50]), which is also significant (p = 0.047). Performance also improved significantly from the 3rd year training (ACLS III) to the last (4th year) training (ACLS IIIb) (p = 0.03; 0.237 (95%-CI [−0.46, −0.01] points).

4.4.2. Dimension Two: Teamwork and Leadership

On dimension two, the NTS performance differed significantly between the 4th year (ACLS IIIb) and 1st year training (p = 0.012; 0.291 (95%-CI [0.04, 0.54]).

4.4.3. Dimension Three: Team Orientation

On dimension three, significant differences in performance were found again between the 1st and 4th year trainings (ACLS IIIb) (p = 0.001; 0.335 (95%-CI [0.10, 0.58]), as well as between the 2nd and 4th year trainings (p = 0.023; 0.272 (95%-CI [0.20, 0.52]).

Our results indicate that students improved their NTS performance in every ascending training. The improvements were statistically significant in every dimension of the AS-NTS, as well as for the overall performance between the 1st year (BEM) and the last training (4th year, ACLS IIIb). On dimension one, improvement was statistically significant for every training compared to the last training (4th year). Furthermore, the greatest assessed (numeric) skill improvement occurred also on dimension one, which reflects “Planning tasks, prioritizing and problem-solving”. The lowest amount of improvement was found on dimension two. Both regarding the numeric improvement (AS-NTS score) and the number of trainings which differed significantly.

5. Discussion

In our prospective observational cohort study, we found that the NTS performance of medical undergraduates enhanced after repeated exposure to simulation-based emergency trainings. We assessed NTS on three dimensions of a validated rating tool (AS-NTS), which was specifically developed for undergraduate education [36]. On every dimension of the AS-NTS, skills improved between the first (1st year) and last (4th year of medical school) training. The greatest skill enhancement (score of AS-NTS) was detected for dimension one, “Planning tasks, prioritising and problem-solving”. Dimension one was also the only skill element for which improvement was statistically significant between the last training (ACLS IIIb) and the other trainings. The lowest skill enhancement was observed on dimension two, “Teamwork and leadership”.

Simulation-based training has been suggested to be an effective approach to address NTS and also, NTS have been identified as one of the five topics to be focused on during SBME for the goal of patient safety improvement [39]. Furthermore, SBME has proven to be superior to didactic NTS trainings [40] and therefore, many medical schools have implemented SBME to convey NTS [41,42] However, inconclusive evidence on the effectiveness of NTS-oriented SBME has emerged from the literature [22,23,24,25,26,27,43]. Therefore, a gap in knowledge on how to convey and teach NTS effectively and how to achieve the greatest learning retention needs to be filled [19]. For this purpose, a multidimensional approach and consideration from different angles are auspicious, condensing the cooperation between medical education and emergency experts. Hereby, exploring and optimizing each dependent variable (optimizing SBME elements), like the structure of the training or the debriefing techniques, as well as its way of incorporation into the medical curricula is essential. The curriculum design that defines the manner of repetition, intervals between the trainings and timepoints of training might also have an impact on the NTS learning cycle [22]. However, the actual concept of an integrated curriculum has not been considered in the mentioned investigations.

We introduced an integrated curriculum into our SBME teachings and confirmed our hypothesis that a vertically integrated NTS curriculum, which is based on a planned repetition of learning contents [31] and repetitive exposure to SBME, enhances NTS performance in medical undergraduate students.

There is a paucity of evidence on the efficacy of repetitive exposure to SBME regarding NTS improvement in undergraduate medical education. However, in the field of postgraduate training, Yee et al. demonstrated that exposure to two repetitive trainings enhanced NTS, whereas improvement from the second to the third training was absent [22]. As solely repetition does not result in a truly integrated curriculum, the additional NTS improvement might have been absent. Nevertheless, it has been indicated that the frequency and intervals of trainings might improve NTS performance [26].

NTS are detached from medical knowledge and therefore do not enhance with experience [4]. They are gained through the socialization process of every human [3]. Unlike factual knowledge, which is acquired through diligence, the enhancement of NTS performance requires the transfer of knowledge and behavioural patterns into the individual’s demeanour, which explains the complexity of its improvement [3]. Herein lies one explanation for the efficacy of repetitive NTS training that we found: the repetition caused an increased awareness of the necessary behavioural benchmarks, as well as internalization of those [44]. Also, through the repeated post simulation debriefings, factual knowledge related to NTS was conveyed and reflection was promoted [45,46]. Hereby, the active learning process was fostered and accelerated [47,48].

Although we can indicate that repetition of NTS training, designed through four years of medical school, enhanced NTS performance; due to a missing ceiling effect, further opportunities for improvement can be assumed. NTS improved the most and was statistically significant between the first and last (1st and 4th year undergraduates) training and as we did not detect statistically significant NTS skill improvement between the other preceding subsequent trainings, it can be assumed that the preceding trainings formed a foundation for the actual learning to take place. The process of NTS improvement is far more complex, reaching beyond the comprehension of classical learning.

Therefore, it would be revealing to evaluate the effect of further ascending trainings.

Interestingly, the undergraduates enhanced mostly on the first NTS dimension, which reflects “Task planning, prioritizing and problem solving”. Our findings can be explained from different perspectives: the first skill dimension is linked to situational awareness or conscientiousness on the emergency with which the undergraduate is confronted [36]. Conscientiousness is a personal trait [2] which increases with age [49]. As the undergraduates mature during the semesters, one explanation might be the more expressed conscientiousness and consecutively better performance on skill dimension one. Another explanation might lay in the assumption that emotional stress or a lack of technical skills negatively affect NTS performance [33,36]. A recent systematic review has highlighted specifically decision making to be impeded by external stressors [34]. Due to the repetitive exposure, familiarity with the emergency simulation surroundings increased. This, in turn, automatically reduced the stress that is caused by the situation itself. As a result, the NTS performance of the undergraduates was positively influenced, which holds true for all AS-NTS dimensions. This explanation can be substantiated and backed up by interactionism, a social psychology theory, which describes that in a situation, inter-individual interactions, as well as the environment (e.g., stress), influence peoples’ behaviour [50,51]. If assumed that familiarity with the training setting reduced undergraduates’ perceived stress, then traits might be more activated and the variability in their trait-expressive behaviour—which equals to enhanced NTS performance in our study—is plausible. [52]. Our explanations can be further endorsed by the cognitive load theory, which explains that the learning cycle is facilitated when extrinsic load (stress) is reduced [35]. The stress reduction leads to enhanced germane load which is the actual learning process composed of processing, construction and the automation of schemas (behaviours) [35]. In summary, the repetitive exposure functioned as an instructional design itself and accelerated the learning cycle.

We observed the lowest improvement from the first to last training, in comparison with the other dimensions, on dimension two (“Teamwork and Leadership”). Teamwork is based on antecedents and attributes which determine the occurrence and, hence, consequences (skill expression) of the concept [53]. In the setting of our study, they were either supported or not fulfilled. As antecedents and attributes are prerequisites for teamwork skills to be pronounced, the varying distinction of those brings further insight into the understanding of our results [53]. The antecedents of relevance are “information sharing” and “understanding of professional roles” [54]. It is not uncommon that the undergraduates, even if in different semesters, are on varying learning curves regarding their factual knowledge and situational awareness. Therefore, the sharing of information might not be as pronounced as it would be if the factual knowledge (e.g., the algorithm of the Advanced Life Support) was solidified. Based on possible insecurities, the sharing of information was impaired [53]. Furthermore, due to a lack of real-life experience, understanding of professional roles is obviously not distinct in undergraduates. The impaired pronunciation of antecedents hampered the occurrence of the concept and hereby restricted teamwork (dimension two).

The most relevant attribute of teamwork in the context of our study is shared decision making [54]. Shared decision making prerequisites on the one hand, a foundation to favour the decision, which is knowledge, and on the other hand, the shared mental models of the team [55]. Mental models are characterized by information which is gathered in a person’s mind to understand the external world [56]. Shared mental models of team members have proven to enhance team effectiveness and hence, teamwork [57,58] and are strengthened by team trainings [59,60]. However, the building of shared mental models was not directly focused on in the trainings. Therefore, the process of mental modelling took place automatically but with no acceleration. Together with the effects of the impaired antecedents, the lower but still significant distinction of skill performance on dimension two is plausible.

Several limitations of our study merit consideration. First, our study lacked a control group with other independent variables like a different SBME structure or varying exposure to SBME with other debriefing techniques. Therefore, it is not possible to determine due to which variable the NTS improvement occurred. Nevertheless, all variables are part of SBME and therefore contribute to NTS improvement. As the endpoint of our study was to explore if repetitive training in the sense of an integrated curriculum is an effective approach to convey NTS and maintain proficiency, this possible objection can be attenuated. This also justifies the inter-cohort comparison of our study, which disregarded the intra-individual variances. If the effectiveness of SBME elements were the endpoints, then repeated intra-individual measures and analysis of its variance would have been the appropriate design.

Concerns may arise that the comparison of AS-NTS performance of medical undergraduates in different years of medical school is debatable. However, as described by the authors, the assessment of NTS with the AS-NTS allows the comparison of different and repeated measurements at different time points of training. [36] Furthermore, the scenarios of each training are designed based upon the spiral curriculum, [32] resulting in a greater amount of necessary technical as well as non-technical skills for each proceeding training. Therefore, this curriculum structure allows the comparison of undergraduates’ NTS in different years of medical training.

To our best knowledge, we are the first to report the course of NTS development, assessed longitudinally over a period during medical school, using a validated tool which has been specifically developed for undergraduate education. Thereby, we did not focus on didactic interventions but answered the question of how a curriculum could be designed with respect to repetition. Although we can not determine the ideal interval between the trainings, one implication derives from our study: during the core phase of medical studies, one training of NTS per semester seems to be appropriate. A further implication is to convey the complex topic of NTS with a gradual, repetitive approach. This would foster undergraduates’ learning outcomes and promote their learning experience and, hence, behavioural changes. Although no significant skill improvement might be detected in subsequent trainings, the growing familiarity with emergency situations sets prerequisites for future skill improvement. We contemplate that a shift in paradigm in medical education seems necessary: the classical teaching approach, which includes that each learning unit is composed of learning goals that are directly measurable after the teaching or learning unit, seems not to be applicable in the context of NTS training.

6. Conclusions

The integrated curriculum which has been established in the field of medical education is an effective approach to effectively convey NTS.

An integrated NTS curriculum will lead to reinforced learning experiences by vertically and horizontally integrating learning content. The vertical integration means that learning content is designed based on a learning spiral and therefore, next to repetition, each teaching unit deals with more complex learning content. The integrated approach will lead to reduced emotional stress by increasing the familiarity (by repetition) of emergency situations. By circumventing emotional stress, the learning cycle the is accelerated, the NTS of undergraduates are improved and these future doctors will ensure safe and best patient care.

To realize such curriculum novelties and restructuring processes, knowledge from the field of medical education should be translated into clinical teaching and the development of the NTS curriculum and trainings should be based on expertise from the field of curriculum development. Thereby, close cooperation between the clinicians who conduct NTS trainings and experts in medical education is aspirational.

Further studies should investigate in one cohort the effects of such a curriculum design and thereby consider intra-individual differences. Furthermore, studies need to investigate how this cooperation might be realized at its optimal.