Assessing the Reactionary Response of High School Engineering Teachers Offering a Novel Pre-College Engineering Curriculum: Lessons Learned from the COVID-19 Pandemic

Abstract

The coronavirus (COVID-19) pandemic forced a rapid transition of K-16 education to remote and online learning in the final quarter of the 2019–2020 school year. The disruption was extreme for all teachers in K-12 but particularly for teachers involved in pilot programs, such as the NSF-funded Engineering for Us All (e4usa) project. This paper reports the key findings obtained through systematic data collection from a pilot cohort of high school teachers who adapted a brand-new engineering curriculum during the COVID-19 pandemic, students who experienced the adapted curriculum, and a new cohort of teachers who were tasked with teaching the updated curriculum.

1. Introduction

The novel coronavirus pandemic (COVID-19) forced a rapid transition of K-16 education to remote and online learning for the final quarter of the 2019–2020 school year. This extraordinary event created uncertainty and anxiety regarding the end of the school year and upcoming educational offerings that summer prior to the next school year [1,2,3,4]. Nine teachers in the pilot year of the NSF-funded Engineering for Us All (e4usa) project completed in-person professional development in the summer of 2019 prior to teaching the full curriculum, consisting of seven units in the pilot year [5]. Teachers were generally able to complete units 1–4 in the fall, as scheduled, when they had to radically change directions without warning, to complete units 5 through 7 via remote and online learning. The disruption was extreme for all teachers. In K-12 but particularly for those teachers who were in the pilot year of implementing a course that they had never taught before.

The teachers were geographically dispersed but located relatively near to the core partner universities: Arizona State University, University of Maryland, Morgan State University, Vanderbilt University, and Virginia Tech. This geographical diversity expanded for the Summer 2020 Professional Development (PD) experience, when a second cohort of teachers were onboarded for the 2020–2021 academic school year (see Figure 1). This diversity meant wider variations in local resources, regulations, structures, and student population among e4usa classrooms. The teachers themselves were also quite diverse in terms of gender, race, years of teaching, disciplinary expertise, and experience in engineering. This made the e4usa project an ideal situation for exploring the diverse impacts of the COVID-19 pandemic.

The curriculum for the NSF-funded e4usa initiative was intentionally designed ‘for us all’ [6]. The goal was to create an engineering mindset for all, even for students who may not have an engineering major at their sites. Although more engineers may be needed in our society, expanding the pool of engineers was not the goal of the e4susa curriculum. Instead, mindset was the focus, and exposure for all was the goal. This is distinctly different from other pre-engineering curricula designed to provide pre-engineering experience to students who are preparing to be engineering students at college. The curriculum has passed through its first few years of adoption, although the COVID-19 pandemic occurred in the midst of the pilot year. The pandemic forced the implementation of many unanticipated modifications during the pilot period. The curriculum’s initial design was based on the First-Year Engineering Classification Scheme [7], used to classify all possible content found in first-year, multidisciplinary Introduction to Engineering courses in general-admit (non-direct-admit) engineering programs. The curriculum provides progressively wider-ranging engineering design experiences relating to student fields of interest and real-world problems. The course objectives are organized into four major threads (Connect with Engineering, Engineering in Society, Engineering Professional Skills, and Engineering Design).

While most, if not all, pre-engineering curricula focus on “how” to engineer and “what” applied engineering looks like, the ‘why’ component differentiates the e4usa curriculum, with deep exploration of the design process and the intersection of engineering with society, the environment, and ethics. It is closely aligned with the engineering habits of the mind, as described by the National Academies [8]. These include (1) systems thinking, (2) creativity, (3) optimism, (4) collaboration, (5) communication, and (6) ethical considerations [9].

There was, rightfully so, a high level of frustration and negativity in the education world surrounding the disruption caused by COVID-19 on every level: from students, parents, teachers, and administration. Fortunately, following the e4usa project’s pilot year, the e4usa team were in constant contact with each teacher starting from the spring 2019 onboarding process. This initially involved an online community of practice, regular visits to teachers’ classrooms, and focus groups with students. As the curriculum was forced to go online, these efforts had to be revamped to supplement the new, online community of practice. This meant regular online videoconference meetings with teachers, open office hours (for students and teachers), the provision of design challenges for individual students to perform at home, and the setting up of online interviews with engineers to replace classroom visits. This complete contact throughout the year led to the establishment of excellent working relationships and a deep understanding of e4usa teachers and their classrooms before and during the COVID-19 pandemic.

When the disruption occurred, we quickly discovered that directives from local administrations lacked consistency within our cohort of teachers as they began to react and adapt to the new education climate [10]. These variations included attempts to continue with the curriculum via video conferencing software (a relatively untested technology at the time) with at-home projects, the implementation of a highly modified curriculum with modified home projects, and explicit instructions to discontinue teaching for the rest of the academic year. Additional difficulties emerged when students and teachers were challenged with inadequate technology at home and/or unreliable internet access, uncovering inclusion equity and access challenges [1,11,12,13]. These issues motivated a need to better understand the experiences of the e4usa teachers and their students, particularly as the team re-envisioned the onboarding process for a new cohort of approximately 30 teachers who were scheduled to begin their engagement with e4usa during the upcoming Summer 2020 PD experience.

This paper reports the key findings obtained through systematic data collection from a pilot cohort of high school teachers who adapted a brand-new engineering curriculum during the COVID-19 pandemic, students who experienced the adapted curriculum, and a new cohort of teachers who implemented the updated curriculum for the first time. The data collected from these three populations shed light on what drivers prompted decisions during the pandemic, what changes resulted from these decisions, what successes and failures resulted from these changes, how teachers and students reacted to and were affected by these changes, and what lessons can be gleaned to inform designers of professional development programs in the preparation of the next cohort of teachers.

The radical change to higher education required to address teaching during the COVID-19 pandemic was to “put it online” [14,15]. This somewhat straightforward approach was not so easily implementable at the K-12 level, which is locally driven and framed by potential inequities regarding access and inclusion [3,11,14]. The parties making the decisions for each school and/or school district on how to react to the pandemic, including closures, in theory, had to consider all stakeholders, including students, teachers, and parents. For some, the sudden conversion to a virtual format was a major hurdle to overcome. This change was a challenge even for seasoned teachers who had been using a curriculum for some time but had possibly never been exposed previously to online teaching techniques [16,17].

The added challenge for the pilot cohort of e4usa teachers was to adapt a curriculum being offered for the first time. This challenge was compounded by the fact that the new curriculum was designed to be hands-on and team-based. The existing network of schools and teachers established as part of the e4usa project presented an ideal scenario for studying school changes and student reactions in preparation for the next year of implementation. Therefore, the e4usa project offered a unique opportunity to capture the following:

• Changes in high school engineering education due to the COVID-19 pandemic;
• Ramifications of these changes and future occurrences in high school engineering education following the COVID-19 pandemic.

A proactive redesign of future professional development (PD) can leverage the lessons learned to build opportunities to affect teacher motivation, self-efficacy, sense of expectancy/value, and imposter syndrome. This work will dramatically benefit those involved in K-12 PD, providing guidance on how to approach similar disruptions in the future.

This research explored adaptations along with the challenges encountered when launching a pre-college engineering program in the midst of a pandemic. The ways in which educators delivered the e4usa curriculum while adopting alternate communication methods with their students were also studied. These necessary adaptations continued as the pandemic persisted, running in parallel with the continuous improvement efforts for the e4usa curriculum. The following questions were explored during this research:

• RQ1: How did the pilot-year e4usa teachers adapt and deliver the curriculum during the COVID-19 disruption?
• RQ2: What lessons were learned from curricular adaptations and the pilot professional development session, informing the design of an effective, online P.D.?
• RQ3: How did the COVID-19 pandemic affect the evolution of the e4usa curriculum?

2. Materials and Methods

To answer the research questions, a phenomenological approach was used to conduct detailed examinations of the teachers’ personal lived experiences and to gain insight into how they made sense of the phenomenon [12], offering the reward of “seeing meaning” into the heart of things [13]. This phase of the study utilized Interpretative Phenomenological Analysis (IPA) in an attempt to make sense of the participants’ efforts to make sense of what happened to them [7]. This approach was chosen to support the qualitative research with the intention of exploring lived experiences rather than experiences based on pre-existing underpinnings. The participants in this study had possibly experienced extraordinary emotional stressors due to their experience of the pivot. The announcements from school districts of another shift to remote teaching for the 2020–2021 academic year also surfaced as a source of emotional stress for educators.

Each of the nine pilot-year high school teachers who taught the e4usa curriculum during the 2019/2020 AY were invited to participate in the study. The purposive sampling approach was used because this non-randomized group had common experiences, as each member was a pilot-year teacher and taught the e4usa curriculum during the spring 2020 COVID-19 disruption. The chosen population were interviewed by facilitators of the e4usa project team. Due to existing social distancing policies, the interviews were conducted virtually on the Zoom platform in a password-protected network and recorded using Otter.ai to transcribe the data before upload to Dedoose for coding. Upon completion of the data collection for each participant, the interview transcripts were analyzed using Dedoose. The first round of inductive coding was completed, and this first pass searched for emerging themes.

3. Results

The results and findings were classified into the following categories:

• The challenges faced by teachers as a result of COVID-19;
• Pre-PD focus group to identify teachers’ needs;
• Modifications to PD in response to COVID-19;
• Online PD training;
• Challenges with online PD training;
• Curriculum revision.

3.1. The Challenges Faced by Teachers as a Result of COVID-19

The interviews conducted with the participating teachers, and the phenomenological analysis which followed revealed that they were attempting to adapt in real time to the challenges posed by the COVID-19 pandemic with little guidance. This experience is similar to what almost every educator encountered as a result of the pandemic [9]. The combination of the COVID-19 pandemic challenges coupled with the teaching of a new curriculum meant that significant progress was halted. Most teachers were only able to complete the first three or four units out of the planned seven units available in the first year.

Some of the challenges identified through teachers’ interviews included the adaptation of communication with students, student motivation (grades and student engagement), digital equity (laptops and internet access), and project successes (alternate projects) [9]. Furthermore, grading and attendance were challenging for many of the teachers. One of the teachers commented, “In Pennsylvania here, our governor, sort of in part of the decree, said that no student could fail on account of the COVID outage, which got to kids very quickly, and that had a pretty big impact”.

The teachers had to deal with the sudden transition to online learning as well as challenges regarding student motivation. These challenges were further compounded by many districts ruling that no student should be allowed to fail [9]. As a result of this new rule, many students stopped participating or even attending class altogether.

In addition, communication with students, whether via Zoom or email, was challenging and not very effective. Many teachers also felt pressure to be online and available at any time of day. As one of the teachers interviewed commented,

“Long story short, I would say that once we adapted Canvas and started teaching online, we found that even though we gave them the technology, many of them were disengaged and then were either watching siblings or just not coming to school at all. Additionally, then if they did do it, they just did it on their phones. So, I know that the lesson we learned is how—was the lack of engagement and how we’re gonna try to overcome that because technology was provided, hotspots were there, it’s just they were not engaged in learning.”

3.2. Pre-PD Focus Group to Identify Teachers’ Needs

There were two focus group sessions before and after the PD. Each session was divided into multiple groups of five to eight teachers who met in breakout rooms. The focus group data were analyzed with an inductive approach, as outlined in Miles et al. [18].

The findings from the pre-PD focus group data suggest that the teachers were anxious about being well-equipped in order to effectively teach a new engineering curriculum in a virtual learning environment. One teacher summed up their expectations in a pre-PD focus group: “However, what I am expecting is to just get an insight of what’s expected of me as the program progresses into the year and see how it’s going to work. So far, I am enjoying doing the projects that we’ve been hammering. So, I am just—get some insight, more insight on what’s expected of me.”

Another teacher shared: “So I am happy to chime in there. My hope, I do not know if my expectation, my hope is that by the end of the professional development I will have a much better, clearer view in my mind of what the curriculum is going to look like for our kids and how we will go about implementing it. I think, you know, as we were talking, the challenge next year is that this is not gonna be a normal school year for any of us, regardless of what happens on campus. Therefore, I am anxious, a little bit, to figure out how we take whatever e4usa has planned and how we modify that and make it within what the constraints of the new rules of the game are gonna be for us all next year”.

In addition, some of the teachers expressed a desire for project ideas that students could work on from home using readily available household materials.

3.3. Modifications of PD in Response to COVID-19

The introduction of a new course in an area in which teachers have little or no training called for an innovative professional development model [9]. The initial PD effort was a 2-week, in-person model, and it was highly successful. The training was very hands-on, thereby enabling many of the new teachers to investigate engineering themselves. They were also able to discover the societal impact of engineering design and attempted many of the hands-on activities that their students would work on. The teachers also investigated some of the ethical issues in engineering, such as implicit bias and imposter syndrome.

The COVID-19 pandemic forced a drastic change in the PD delivery modality. While the PD was designed specifically as a ‘cohort model’, the configurations of ‘group identity’ based on geographic location became less appropriate because of group migration and groups becoming less homogenous. With the proliferation of online communities, communities became linked less by location and more by common interests and goals. The proliferation, accessibility, and affordability of global communication systems resulted in the development of many new learning communities at a faster rate and in an easier manner [8]. Such a drastic change required more than moving to video-based instruction. Hands-on, in-person group activities were replaced with individual activities or family-based activities that incorporate teamwork. The results were also shared online.

As a result of the COVID-19 restrictions, the summer 2020 PD was held online [9]. This was quite unlike the initial 2019 PD, which was onsite and cohort-based. Campus closures and severe restrictions on travel necessitated a complete redesign of the existing, in-person PD model. Four PD sessions were scheduled as week-long, on-campus working sessions to onboard the second cohort of teachers. The e4usa team worked with the new cohort of teachers while capturing feedback from the pilot teachers to counter the deficiencies often associated with hastily designed PD efforts. Early plans included the packaging and mailing of materials for activities prior to the PD, thereby allowing teachers to simulate the hands-on portion of the in-person summer PD training.

Teachers were required to complete activities individually before collaborating with other teachers prior to a synchronous module facilitated by the e4usa PD team, master teachers (a subset of teachers who taught in the pilot year of the program), and local university faculty liaisons. This added cohort of e4usa teachers who were uniquely situated to engage with pilot teachers as they prepared to become master teachers, as well as incoming teachers as they became new e4usa teachers.

Two versions of the e4usa summer PD were developed, called the sprint and marathon versions. The two versions provided educators with the flexibility to complete the PD within a week or over several weeks. One benefit of the “marathon” version was that it allowed educators to spread the course content over two meetings a week for a total of four weeks. In addition, there were greater expectations to engage asynchronously through discussion boards and homework submissions in order to maximize flexibility in the marathon version. On the other hand, an anticipated benefit of the “sprint” version was that it minimized the “spill over” into the rest of the teachers’ summer schedule by limiting the PD to just one week. The sprint version consisted of two hours each day as a “group study hall” where teachers worked together with staff support to complete the same amount of work as that carried out in the marathon version. Both versions were offered over Zoom, with asynchronous content delivered and assignments collected via the Canvas LMS platform, similar to what was accomplished in 2019.

3.4. Online PD Training

The first cohort of teachers were exposed to the activities as their students would experience them: in groups (or teams, depending on the timing of the activity in the curriculum), with opportunities to share with fellow teachers. For those teachers participating remotely in the online curriculum, during the pandemic, kits with all of the necessary materials and activity instructions were mailed to the teachers’ homes in advance. These teachers were then asked to work with family or friends to complete the curricular activities (as local health guidelines and restrictions would accommodate) and be prepared to share their experiences during synchronous online sessions. In fact, this served as a possible model of how to engage students learning remotely in the upcoming year. Teachers were highly receptive to the mailed kits and found the online sessions to be very informative, according to their post-professional development feedback. The activities received consistently positive feedback from teachers, including some who had remained skeptical and concerned during the PD period. One teacher commented: “I thought actually getting to do the activities was very helpful, making the time in the curriculum for that, because when I ask my kids to do this I can reflect back on my own experience”. Another teacher mentioned: “However, I think that having us build the stuff as we went, I mean that really helps, because that really immerses you in the activities that you’re going to have to have them do. So that made me feel a whole better about what it is and what we’re doing, what we’re trying to achieve. So, for me it was very beneficial”.

3.5. Challenges with Online PD Training

The online environment was not conducive to building relationships and community with the e4usa staff and peer group for some teachers. In the sprint version of the PD, a number of teachers also reported that the pace was too quick to allow for all of the information to be digested. Nonetheless, teachers indicated that they generally felt confident teaching the e4usa curriculum at the end of the PD period. These findings also helped to inform the next professional development session offered.

In addition, the mentoring support provided by the returning teachers proved to be beneficial in addressing some of the challenges. A new resource for incoming teachers in the second cohort of the PD was the returning teachers, who were asked to serve as mentors. Several of these mentors had participated in the online PD in the previous years; thus, the incoming teachers were able to hear their experiences. This was extremely valuable, as the first cohort of teachers had experience with teaching the curriculum face-to-face for the first fall semester pre-COVID-19 and also with using a variety of remote learning strategies for the spring semester of the academic year during COVID-19.

Overall, the teachers, along with the e4usa team, adapted as necessary through the pivot to alternative methods of teaching and learning. The professional development sessions provided teachers with not only the necessary understanding of, and exposure to, the curriculum but also the resources required to teach in a rapidly changing online environment. The PD also provided teachers with experience of the course through the eyes of the student. This was designed to generate an appreciation of the experiences and challenges that their own students would likely face in a remote learning environment.

3.6. Curriculum Revision

We found the feedback from the first cohort of teachers to be extremely valuable and instrumental during the first curriculum revision. Changes beyond those necessary to offer the curriculum online were implemented. First, it should be noted that the faculty who designed and offered the PD updated the content of the previous PD based on this feedback; furthermore, by this time, the PD team also had significantly more experience to draw from as they designed new activities. Only a draft of the curriculum was published during the initial PD. The curriculum itself was later improved with feedback from teachers and thorough internal and external reviews, so that the curriculum presented in the second PD was also far more stable and improved. For instance, to help teachers to better prepare for the semester in advance, a master classroom supply list of the curricular activities was created. Definitions of keywords were added at the beginning of each unit. This provided an additional resource for the teachers and also additional content for teachengineering.org (the portal through which the teachers accessed the curriculum) when the curriculum became accessible on their website. Each lesson and each activity were examined, each course outcome (CO) was reviewed, and a gap analysis was performed. A set of finalized student learning outcomes (SLOs) and COs was the result [19].

4. Discussion

The key lessons learned from this study are summarized below.

Flexible Online PD Options: The COVID-19 pandemic forced a drastic change in the PD delivery modality. This change required much more than simply moving to video-based instruction. Hands-on group activities that would have been performed with fellow teachers were replaced with individual or family-based activities that could be performed at home, and the results were shared over the internet. In addition, the introduction of two versions of the e4usa summer PD provided educators with the flexibility to complete the PD within one week or over several weeks. The perceived benefit of the “marathon” version was that it spread the content over two meetings a week for a total of four weeks. Within the marathon version, there were greater opportunities and expectations to engage asynchronously through discussion boards and homework submissions so as to maximize flexibility. An anticipated benefit of the “sprint” was that it minimized the “spill over” into the rest of the teachers’ summer schedule by limiting the PD to just one week; however, teachers thought that the one-week model was simply too compact for a brand-new curriculum.

Importance of Participants’ Input for Improvement of the PD: Feedback from the first cohort of high school teachers was extremely valuable and instrumental for the first curriculum’s revision. First, it is important to consider that the PD faculty revised the previous PD based on this feedback; the team also had significantly more experience on which to base the activities. During the initial PD, the curriculum was in draft form. The curriculum was improved throughout the year with the teacher feedback and thorough internal and external reviews, so that the curriculum available in the second PD was much improved, having evolved from the original pilot curriculum. Some teachers indicated that they struggled to build relationships and community with the e4usa staff and peer group in the online environment. A number of teachers undertaking the sprint version of the PD also found the pace too quick to allow for the information to be absorbed. Teachers indicated feeling confident in teaching the e4usa curriculum after completing the PD. These findings also helped to inform the next professional development session offered.

Mentoring Support: In addition, the mentoring support provided by the returning teachers proved to be beneficial for the second cohort. Returning teachers served as mentor teachers, providing a new resource for incoming teachers. Many served roles in the online PD and were able to share their experiences from previous years with the incoming cohort. Overall, the teachers, along with the e4usa team, adapted as necessary through the pivot to alternative methods of teaching and learning. Professional development sessions provided teachers with the necessary understanding of the curriculum and the resources required to teach online. It also allowed teachers to experience the course through the eyes of the student and to appreciate the kinds of experiences and challenges that their students would face in the remote learning environment.

5. Conclusions

Some of the key lessons learned from this study can help to inform the educational community in the face of future disruptions. First, educators are resilient and have a tendency to pivot when necessary and when given the freedom to do so. For instance, although the first cohort of e4usa teachers faced the need to abruptly pivot to an online modality, each individual teacher expressed the opinion that if they were to be faced with this challenge again, they would still choose to teach the e4usa curriculum. Secondly, student engagement during the pivot to online learning did come with some challenges. Although the teachers were creative and provided multiple opportunities for communication including Zoom office hours and Google Classroom, the students still struggled to engage. This lesson provides an opportunity to further explore how we can utilize communication platforms that may be more student-centered. Finally, regardless of the disruption, the continuous improvement plans for the e4usa curriculum continued. The teachers continued to provide feedback which informed the evolution of the curriculum. When the e4usa curriculum was first developed, there was no intent for the course to be an online course. In fact, the pandemic created an opportunity for project kits to be developed, packaged, and delivered to the students, as well as opportunities for the teachers as they completed their online professional development [5]. This may represent a silver lining to what initially appeared as a hindrance when the world was faced with a pandemic.

6. Limitations

This study specifically addressed the experiences of a set of teachers teaching a new engineering curriculum for the first time when a pandemic unexpectedly disrupted the academic year. The results of this study are important, as they help us to understand the potential challenges in designating professional development for teachers and the effects of unexpected disruptions on innovative curricula. Further, the study provided data for consideration when designing a curriculum. Finally, it offers hope in the event of other unexpected disruptions, showing that teachers can pivot and adapt as necessary.