Co-Adapting a Reflective Video-Based Professional Development in Informal STEM Education

Abstract

Traditional professional development for informal educators often relies on brief, lecture-based sessions that reinforce familiar teaching practices. In collaboration with 27 informal learning organizations across four years, we attended to the need for authentic, long-term professional development through the co-adaptation and co-refinement of a reflective video-based cycle. This focus on supporting informal educators to productively attend, interpret, and respond to youths’ experiences with failure while engaged in science, technology, engineering, and mathematics (STEM) related activities. This paper describes how and why organizations co-adapted components of the professional development failure cycle to support the operational constraints of their organizations while addressing the specific needs of their informal educators. Co-adaptations addressed the following: (a) educators’ discomfort and vulnerabilities, (b) time constraints, (c) staff turnover, and (d) lack of tools. These adaptations refined the development of a reflective video-based professional development framework that informal STEM institutions can adopt to equip educators with strategies and build a supportive community, helping youths navigate and learn from failure.

1. Introduction

Informal science, technology, engineering, and mathematics (STEM) educators play a critical role in shaping youths’ learning experiences, yet professional development (PD) opportunities for these educators are often short-term, lecture-based, and misaligned with the hands-on, inquiry-driven nature of informal learning environments (Bevan & Xanthoudaki, 2008; Robinson, 2019; Tran et al., 2019). Many PD opportunities, such as two-hour workshops or conference sessions, may reinforce familiar teaching practices like knowledge transmission rather than supporting informal educators on how to foster youths’ deeper engagement with STEM practices and concepts. While research supports the value of sustained, immersive PD that encourages inquiry, hands-on engagement, and experimentation (Grenier, 2009; Nawab et al., 2021; Robinson, 2019; Sancar et al., 2021; Taylor et al., 2008; Tran et al., 2013), informal educators often lack access to these types of experiences.

One promising yet underutilized approach to professional development in informal learning environments is the integration of reflection and video analysis. As argued by Tran (2021) and Mohamed et al. (2022), reflection is one of the most formidable ways to transform education through a process of self-growth and understanding. It is often situated within moments that can be characterized as troubling or puzzling, and yet constrained by what one can “see” based on their prior experiences, beliefs, values, and assumptions. As such, reflection—the process of analyzing and synthesizing experiences to bring clarity and coherence to complex situations (Clarà, 2015)—has gained prominence in museum education (Moore et al., 2020; Tran & Halversen, 2021). Research situated within museum contexts highlights how engaging in reflective practices has the potential to support museum educators in developing a common language, exploring innovative instructional ideas and strategies (Allen & Crowley, 2017; Grabman et al., 2019), deepening their understanding of equity issues (Archer et al., 2024), and enhancing their confidence in facilitating visitor learning experiences (Grabman et al., 2019; Moore et al., 2020).

Incorporating video into these practices offers educators an opportunity to reflect-on-action (Schön, 1983)—to observe and reflect back on their teaching methods, potentially revealing aspects that might otherwise go unnoticed (Hollingsworth & Clarke, 2017). As noted by Sherin (2004), “Video allows one to enter the world of the classroom without having to be in the position of teaching in-the-moment” (p. 13). Sharing and reflecting on videos as part of professional development can lead to positive outcomes, such as fostering a sense of community among educators (Amador et al., 2023; Simpson et al., 2024a), offering new perspectives on teaching practices (White et al., 2021), and providing strategies to improve instruction and facilitation of learning opportunities (Amador et al., 2023; Chen et al., 2020; Hollingsworth & Clarke, 2017). Furthermore, while watching videos of others may be more comfortable (Wager et al., 2023), utilizing videos from their own learning environments may be more effective in motivating educators to adopt new instructional practices, as these contexts are more familiar and relatable (Beisiegel et al., 2018; Seidel et al., 2011). However, while these approaches have been widely studied in classroom settings, their application in informal learning environments—particularly museums—remains underexplored.

Therefore, in this study, we examined how 27 museums across three cohorts co-adapted and refined a reflective video-based professional development model—originally designed for mathematics classroom teachers (Borko et al., 2015)—to better support informal educators in responding to youths’ failure moments during STEM activities. Specifically, we highlight how museums tailored the PD cycle to fit the logistics of their organizations and the unique needs of their educators. We intentionally examined failure in STEM-related activities within the professional development cycle as failure typically has a negative connotation, yet is an important component of being a STEM learner (Marks & Chase, 2019) and developing as a STEM professional (Simpson & Maltese, 2017). Informal learning environments, like museums, play a unique role in positively shifting youth’s relationship with failure within a low-stakes and safe space. Informal educators’ views of and experiences with failure inform how they respond to youths’ failure moments (Simpson et al., 2025; Zarrinabadi & Afsharmehr, 2022), and in some cases, informal educators’ perspectives of failure moments are framed as something to fear and to avoid (Simpson et al., 2024b). For example, researchers, Lottero-Perdue and Parry (2017), found that many educators of youths in their study associated failure with negative connotations (e.g., giving up, not trying) and that youths should not be allowed to fail.

The findings from this study provide insights into how informal STEM institutions can adapt and implement sustained, reflection-driven PD that supports educators to productively attend, interpret, and respond to youths’ experiences with failure while engaged in STEM focused activities. The significance of these adaptations has led to a reflective video-based professional development that other informal STEM institutions can utilize in providing approaches and a community for their informal educators to support youths when experiencing failure.

2. Overview of Professional Development Cycle

Our collaborative team of educational researchers and practitioners co-adapted two professional development models grounded in the field of mathematics: the Mathematics Leadership Preparation model to support the professional growth of teacher leaders in their school and/or school district and the Problem-Solving Cycle model to support the professional growth of practicing teachers (Borko et al., 2015). These research-based models incorporate core components of effective PD such as a focus on instructional practices, opportunities to engage with colleagues in an active and collaborative community setting, learning by doing, reflecting on their practice, and ongoing support (e.g., Desimone, 2009; Dunst et al., 2015; Garet et al., 2001; Loucks-Horsley et al., 2009). As noted by Borko and colleagues (2015) these two models are adaptable and focused on specific problems of practice identified by teachers. Therefore, we co-adapted and co-adopted the model within informal museum learning environments specific to supporting youths’1 experiences with failure in STEM-focused tasks. In this paper, we attend to the co-adaptation and co-adoption of the Problem-Solving cycle, which we termed as the Making Failure-Solving Cycle (MFSC) model, through our collaborative efforts with lead facilitators (i.e., those who would lead the MFSC cycle with their educators—train-the-trainer model). The MFSC consisted of a series of four interconnected workshops designed to enhance informal educators’ pedagogical practices around youths’ experiences with failure during STEM-focused activities within their organizations (see Figure 1). Further, organizations were provided flexibility as to when to engage in the professional development cycle, with the majority engaged in the cycle during the summer sessions. Below we discuss how we, as a research team, initially presented the cycle to Cohort 1.

The goal of MFSC Workshop 1 (Engage) was to engage informal educators in a specific activity that they were expected to facilitate with youths. The purpose of this workshop was to engage in the activity as learners, to develop an understanding of the knowledge and skills required to facilitate the activity, and to develop empathy for the learner. Additionally, participants considered times in which youths may experience failures, including how the available material, tools, and human resources might lead to experiences with failure. The goal of MFSC Workshop 2 (Plan) was to discuss and develop pedagogical moves to productively address youths’ failures before, during, or after they occur, particularly within the activity from Workshop 1. For example, informal educators could decide to conduct a failure brainstorming session at the beginning of the activity to develop a community generated list of ways to handle failures.

Prior to MFSC Workshop 3, informal educators were asked to videotape 1–2 days of the STEM-focused activity. They then watched and reflected on the videos with a focus on ways in which they handled and/or addressed youths’ failures. They selected a 3–5 min video clip to share with the other informal educators during the third and fourth workshops. The use of videos was intentional as it afforded participants an opportunity to engage in a reflective, collaborative endeavor around shared experiences and to discuss ideas for improvement (e.g., Sherin, 2004). Prior to sharing and watching video clips in MFSC Workshops 3 and 4, as an education team, they were to develop ground rules for communication norms that enabled challenging yet supportive discussions around the video clips, as well as creating an atmosphere of openness and trust. In MFSC Workshop 3 (Analysis), informal educators shared their clip with a focus on analyzing and reflecting upon their pedagogical approaches to youth’s failures during their making program. Educators were challenged to think deeply about the role they played in engaging with the learners’ ideas and goals, and the support provided (or not). MFSC Workshop 4 (Reflect & Improve) was similar to the previous workshop in that video clips were utilized to foster a productive discussion. However, the focus of this workshop was how pedagogical moves are at times dependent on the individual youths’ reactions to failures and how to shift resilience and perceptions of failure in such a short time frame. As our prior research indicated, youths’ responses to failure are based mostly on the individual experiencing the failure (Simpson et al., 2018, 2019), which may also be dependent on a variety of factors (e.g., age, gender, race/ethnicity, dispositions, etc.). Organizations had the option of combining MFSC Workshop 3 and MFSC Workshop 4. After the initial MFSC model, each informal educator was expected to set pedagogical goals in terms of interacting with youths around experiences with failures, as well as to develop supports needed to obtain their goals.

3. Materials and Methods

This study was informed by qualitative research methods, utilizing both natural observations of recorded meetings (Mulhall, 2003) and interviews with individual museum organizations (Tavory, 2020), which allowed the research team to examine and understand how and why museum organizations adapted and refined the professional development cycle. Our collaborative work with six museum organizations in Cohort 1 spanned two years of the project. We met virtually once a month for 1 to 1.5 h. These meetings were framed to allow for group connections (e.g., icebreaker questions), discussion of an issue or topic specific to the project (e.g., video recording procedures, development of observation tool), development of workshops, and time to reflect on the process. In our second cohort (i.e., Cohort 2), our collaborative work with 13 informal organizations spanned one year and focused on establishing the MFSC model as a sustainable video-based, reflective professional development model in museum learning environments. We hosted six virtual meetings and three summer “office hour” sessions to guide the lead educational team at each organization in adapting and implementing the MFSC model, as well as giving opportunities to ask questions and share insights. They were also provided with access to a project website that included more information and resources about each workshop and the video recording and selection process. As such, Cohort 2 was provided with more structure to the MFSC model, based on the co-development of the model from Cohort 1. The workshops within the model were the same, but the nuances in how to implement the various workshops and video recording and selection process were more streamlined and organized. Lastly, in the final year, we partnered with eight additional museum organizations. This group, Cohort 3, was asked to test out the website as a stand-alone resource in implementing the professional development cycle as co-adapted and co-developed by Cohort 1 and Cohort 2. We held a one-hour introductory meeting with interested organizations, followed by three 30–45 min follow-up meetings with each individual organization to discuss their progress.

3.1. Organization and Participant Information

Purposive sampling was employed to recruit and select museum organizations that espoused failure as part of their mission, and were willing to engage their informal educators within the professional development cycle (Campbell et al., 2020). In addition, the STEM activities within the organizations were typically created to encourage iterative refinement, meaning a youth’s initial designs would often fail and need to be revised. The 27 partnering organizations spanned multiple regions of the United States, as well as Canada, and also varied in size and focus (see Figure 2). The figure highlights the cohort and number of informal educators involved in the project for each partnering organization. More information regarding each organization and their implementation of the MFSC model can be found on our project website (https://sites.google.com/binghamton.edu/failureinmaking/home).

Additionally, across the partner organizations, we worked with 51 lead facilitators (i.e., teacher leaders). Their role was twofold: (1) to collaborate with the research team on co-adapting and co-adopting the MFSC model, and developing and tailoring resources to their specific organization, and (2) to implement the MFSC model with a group of informal educators facilitating a STEM-focused activity within their organization. Over the three years of the project, approximately 160 informal educators participated in the professional development cycle. Years of experience of informal educators ranged from 1 month to 23 years.

3.2. Data Source

To address the purpose of this study, the main data source was our collaborative virtual meetings across the three years (see

Table 1. Overview of cohorts and objective.

Year, Cohort	Number of Organizations	Data Source and Total Time	Overarching Objective
Year 1, Cohort 1	6	16 group meetings 18 h 30 min	Co-adapt the original PD cycle grounded in math education.
Year 2, Cohort 1	5	8 group meetings 10 h 26 min	Implement and co-refine the working PD cycle from Year 1.
Year 3, Cohort 2	13 (Group of 7 and Group of 6)	8 group meetings each 14 h 47 min	Implement and co-refine the working PD cycle from Year 2.
Year 4, Cohort 3	8	3–4 individual meetings per organization ~13 h	Implement the PD cycle from Year 2 using the project website. Support changes to the website.

). In Year 1, we met with six organizations as part of Cohort 1 sixteen times for approximately 18 h 30 min. Topics discussed in these meetings included the following: (a) professional development check-ins (e.g., what has been working well or working not so well); (b) developing a working definition of failure; (c) our prior research on failure; (d) how to facilitate video discussions with informal educators; and (e) how to shift professional development to be more long-term and reflective. In Year 2, we met with five of the original six organizations in Cohort 1 eight times for approximately 10 h 26 min. As this was our second year together, the meetings shifted to sharing questions, challenges, and mutual support around the common topic—supporting informal educators’ approaches to youths’ experiences with failure through professional development opportunities that include video-based reflections.

In Year 3, we expanded our efforts to collaborate with 13 new organizations. Cohort 2 met 16 times (eight times with two different groups) for approximately 14 h 47 min. Topics included but were not limited to the following: (a) identifying where failure shows up in their organization, (b) providing an overview of Workshops 1 and 2, (c) overviewing the video recording and video clip selection process, (d) discussing youths’ failure moments and ways to respond, and (e) sharing and receiving feedback on their professional development plans. The final meeting at the end of Year 2 (i.e., Cohort 1) and Year 3 (i.e., Cohort 2) included a final presentation from each organization. In this meeting, they shared the following information: (a) definition of failure in their programs/museum, (b) the professional development sequence/process they utilized, (c) what the team found useful in the project (e.g., impacted their work), and (d) any reflections on what they learned from interacting with others in the cohort. In Year 4, unlike the first two cohorts, Cohort 3 did not engage in teamwork, discussions, or peer-to-peer support across organizations. Rather, they joined a one hour-long virtual orientation meeting to introduce the project and the website and then independently ran their professional development based on the website resources. Each organization met one-on-one with a research team member 2–3 times for around 30 min each meeting. These meetings were structured to report what work had been done and to answer any questions from the lead facilitators.

3.3. Data Analysis

Data analysis was guided by Braun and Clarke’s (2021) principles of thematic analysis and occurred in four phases: (1) taking notes following or during each virtual meeting, (2) condensing those notes into a summary for each organization, (3) summarizing across workshops, and (4) examining patterns across workshops and organizations. In the first phase, Author 1 re-watched the video recordings and took notes of moments when lead facilitators discussed their approaches to the different workshops and video recording and selection process, the reasons for their decisions, and the reflections on things that worked well and did not work well. These notes were transferred to a chart format for each partnering organization (see Online Resource 1 for an example from Cohort 1, Year 2).

In phase two, Author 4 and Author 5 condensed the notes for each organization from Cohort 1 into a summary highlighting how each organization adopted and/or adapted the PD model. They met to discuss similarities and differences between each organization in Cohort 1, creating an initial set of codes focusing on patterns they noticed within each component of the cycle (e.g., Workshop 1, video recording), as well as across each of the six organizations from Cohort 1. Examples of initial codes included the following: engage in making as learners, addressing youths’ failures, creating clips, and norms. Author 4 and Author 5 carried out a similar process for Cohort 2, finding additional ways that the 13 organizationsadopted and/or adapted the PD model. For example, building a community was a new code that highlighted how organizations implemented activities and practices to get to know one another as an educational team, as well as build a shared understanding around failure in their organization. Following the analysis of Cohort 2, the authors re-examined the patterns in Cohort 1.

In phase three, Author 4 and Author 5 looked for similarities in codes within each component of the cycle (e.g., Workshop 1, video recording) across Cohort 1 and Cohort 2. In phase four, the coding was transferred to an excel sheet where the authors examined patterns based on frequency. During this phase, authors noticed inconsistencies when topics were discussed within the PD cycle and condensed codes as a result. For example, some organizations considered when youths may experience failure in the STEM activity during Workshop 1 while others discussed this in Workshop 2. These were originally coded separately, one code for Workshop 1 and another for Workshop 2, but these were condensed into a single code as whether the activity happened was more important than when it happened. Further, in phase four, we considered which patterns were adapted from the original professional development cycle presented above, and, if possible, why they were adapted.

Data from Cohort 3 were used to supplement and expand the findings around reflective practice. Author 4 summarized the tools and methods each organization used across their professional development sequence. These organizations diverged from the first two cohorts in terms of how they implemented reflective practice; thus, these summaries were compared to the prior analysis paying close attention to how reflection was used in the PD.

4. Results

In this section, we describe how and why organizations across each cohort co-adapted components of the professional development failure cycle categories, particularly while engaging in the cycle during one of their busiest times, the summer season. These include co-adaptations to address the following: (a) educators’ discomfort and vulnerabilities, (b) time constraints, (c) staff turnover, and (d) lack of tools. Pseudonyms for individuals and organizations are used to maintain their anonymity.

4.1. Educators’ Discomfort and Vulnerabilities

Within the context of the professional development failure cycle, lead facilitators co-adapted components of the video recording, selecting, and sharing process to hedge educator’s discomforts and vulnerabilities around feelings of being evaluated and judged from their peers and supervisors through a reflective video-based professional development. For example, as noted by a lead facilitator at STEM Learning Museum (Cohort 2), “We got Lori involved in this as a staff member because she’s the one that wants things to be so perfect. And she’s not uncomfortable with her own methods, but she refuses to let other people help her” (14 July 2023). Therefore, a “new staff member jumped into this project due to [Lori’s] uncomfortability being video recorded” (27 October 2023). Broadly, this quote exemplifies the “scary” nature of this process for educators, as well as how lead facilitators were attentive to educators’ feelings and discomforts. Lori continued to be a part of the professional development workshops, but due to feelings of discomfort, she was not engaged in her professional growth through video-based self-reflection and discussions with her peers around her own failure-based instructional moves with youths who visit their museum. Similarly, for some partnering organizations in Cohort 3, lead facilitators and educators’ discomfort led to alternative approaches to the video recording process itself, namely incorporating reflective practices void of videos. Exploration Station, for example, asked educators to write down their reflections and interactions with guests during failure moments. These documented moments and experiences were discussed as a collective in their final workshop. World of STEM conducted live observations followed by individual debriefs with educators. As noted by one of the educators, “Being able to debrief and talk about what happened and getting support from another staff member helped me push through the summer”.

Other ways lead facilitators hedged educators’ concerns was through sharing clips of themselves in Workshops 3/4. The reasoning for this decision is captured well in the following quote from a lead facilitator at the Museum of Awe:

We also made sure Aurora and I [lead facilitators] were also being filmed so that it wasn’t just us critiquing other people that they were also turning around and critiquing videos of us. And we were filming ourselves so that we could get in the mindset of nobody’s perfect. Everybody has room for improvement. And everybody has failure moments. So I think just the trust that was built, knowing that this was like an all around experience, for everyone. We were all sitting in the same seat … and thinking about it as a universal educator kind of moment, you know, this is hard to watch.

(Cohort 1, 27 October 2021)

As noticed in this quote, this co-adaptation was to build a culture of inclusivity and vulnerability. For Children’s Museum of Curiosity, lead facilitators shared clips of themselves to not only encourage educators to be vulnerageous (vulnerable and courageous), but also to establish norms (e.g., person in video speaks first; do not talk about facilitator not in the room). Another approach from The STEM Collaborative (Cohort 1) and City Science Museum (Cohort 2) was to offer one-on-one meetings with their supervisor prior to Workshops 3/4. The intent of these meetings was to ease educators into the video sharing process; to discuss and reflect upon two clips—one effective clip and one non-effective clip—as well as plan for the type of feedback they would want from their peers upon sharing their clips in Workshops 3/4. For Museum of Science and Wonder (Cohort 2), their approach included watching video clips of themselves on their own, but they did not share or watch video clips as a group during the final workshop. Instead, Workshop 3/4 was grounded in a 15-question survey completed prior to the workshop. Questions included the following: (a) Why was it helpful to watch yourself on video? Why was it not?; (b) How would you imagine video self-reflection working other places/programs in the museum?; and (c) Did you feel personal growth happening? How and where?

Moreover, lead facilitators co-adapted the professional development through including staff beyond those involved in the immediate project. The intent was to shift educator’s discomfort with failure, as well as their practices around failure, on a larger scale. For instance, the Children’s Exploration Zone (Cohort 2) shared and discussed video clips from four educators with their entire staff. At Museum of Engaging Science (Cohort 2), the four educators involved in the project did a final reflection presentation to their full team about their thoughts on the professional development failure cycle process and what they learned.

4.2. Time Constraints

Lead facilitators often made changes to the PD cycle due to time constraints; in particular, scheduling issues such as time for everyone on the project team to come together, time for educators to watch and select video clips, and time before seasonal staff left for the summer. As stated by a lead facilitator from Museum of Awe (Cohort 1), “Because we had little flexibility in the way that we did things knowing that the camp day is so just so overwhelming” (27 October 2021). This sentiment was echoed by others, such as Little Human’s Science Museum (Cohort 2), “In summer, we are open seven days, so it’s hard to find time to pull everybody at once” (10 April 2023).

As such, one adaptation was to implement Workshop 3 and Workshop 4 as a single video-based reflection session. As noted by a lead facilitator at the Museum of Awe (Cohort 1), “We followed pretty closely with it [initial PD cycle] besides the Workshop 3 and the Workshop 4. We only had one debrief meeting” (27 October 2021). This was attributed to too much flexibility and not enough time for self-reflection during the summer months; time got away from them as an educational team. Following Cohort 1, this became an adopted component of the PD cycle—one workshop—Reflect and Improve. Similarly, all but one partnering organization engaged in the professional development failure cycle only once, shifting the cycle from an iterative cycle grounded in reflection and improvement (see Figure 1) within a summer season to one that was more linear (see Figure 3), where educators “carried” the experience of their professional development failure cycle to another context (e.g., preschool, exhibit). As such, this co-adaptation spanned several months. As noted by lead facilitators from The STEM Collaborative (Cohort 1), they “prefer[red] a shorter time period between when recorded to watch recording to talking through recordings; it was like 4–5 months from June to September” (8 October 2021). This time span impacted how organizations talked about things. At the beginning of Workshop 3/4, City Science Museum (Cohort 2) stated they “recapped Workshop 1 and Workshop 2 due to time in between, about 3 months” (25 October 2023).

Conversely, Museum of Engaging Science (Cohort 2) implemented the video-based reflection component of the professional development failure cycle several times within a short time frame—four times, once a week over a one month period (see Figure 4). Two educators recorded each other for about one hour on Tuesday morning, and two other educators recorded each other for about an hour on Wednesday morning. Educators were provided time to review their video the afternoon of their recording. On Thursday, they would spend 1–1.5 h sharing and reflecting on one another’s video clip (i.e., Workshop 3/4). This cycle was done within the same activity as opposed to a cycle grounded in different STEM activities within their organization.

Further, time constraints informed who video recorded, as well as who reviewed and selected the video clips for further reflection. As noted above, this was originally to be done by informal educators; however, many lead facilitators co-adapted these two components of the professional development failure cycle by taking a more active role. There were several variations in who video recorded, from a lead facilitator popping in to turn on the video recording device and then leaving (Museum of Awe, Cohort 1; Exploration Lab, Cohort 2) to lead facilitators remaining in the space to video record interactions (Okay Science Museum, Cohort 2; Nature & Science Museum, Cohort 2) to a pair of educators in the space together, with one facilitating the visitor experience and one managing the recording (Museum of Engaging Science, Cohort 2).

There was also variation in how the video clips for Workshop 3/4 were reviewed and selected. Often, educators intended to select their own clips, but again, the following quote highlights how time due to staff issues impacted how the organizations implemented this process: “We reviewed everyone’s footage and cut it into clips that demonstrated the facilitator helping a student through a failure moment. We hoped to get the educators to do this but lack of staff and sickness prevented it” (The STEM Collaborative, Cohort 1, 14 October 2022). As this quote also emphasized, one co-adaptation involved lead facilitators picking out 3–7 short video clips for educators to review and pick 1–2 clips to share with their peers during the final workshop (The STEM Collaborative, Cohort 1; Waterfront Science & Nature Museum, Cohort 2; Creative Reuse Center, Cohort 2). For educators at Waterfront Science & Nature Museum, they “selected clips of facilitation moves that were successful and one that could be improved” (27 October 2023). Another co-adaptation included lead facilitators selecting clips to show and reflect upon in Workshop 3/4 (Innovator, Cohort 1; Exploration Lab, Cohort 2; Franklin Observatory, Cohort 3). One last co-adaption was for someone outside the project to document and “thoughtfully select” failure moments (Children’s Museum of Curiosity, Cohort 2):

We did not want to add to their [educators] workload in terms of watching their own clips. But we did not ask if they wanted to do this because we knew they would want to watch their clips, which would add to their workload and anxiety.

(4 October 2023)

4.3. Staff Turnover

Staff turnover was another reason why lead facilitators made co-adaptations to the professional development failure cycle. It is common for informal learning environments such as museums to hire seasonal staff in the summer, as well as struggle with retention of permanent staff (Museums Moving Forward, 2022). Co-adaptations were intended to build a sense of community among the educators involved in the project. One method was to include community building icebreaker activities prior to delving into the concept of failure. One activity was Duct Tape Draw suggested by Imaginary (Cohort 1). In a team of 3–4 educators, the group worked together to draw an image (e.g., smiley face) or write a word (e.g., fail) using one marker controlled by each member of the team with only a strip of duct tape. During final presentations, many organizations from Cohort 2 highlighted the use of such activities, particularly in their first workshop. See Figure 5 as an example from Little Human’s Science Museum.

Another approach was to define/conceptualize failure, personally and professionally. As noted by Museum of Exploration (Cohort 2), “We used our workshops, because we had new staff, it was what is failure in making...we helped define this definition [of failure] together” (27 October 2023; italics added for emphasis). Art & Culture Institute (Cohort 1) began Workshop 2 by asking educators to “tell us about your failures in the last month, what’s been going on, what’s a moment of failure happened, just as a check in” (18 June 2021). These personal failure moments (e.g., missing an interview, backing out of a puppet show, reacting negatively) led to a discussion on a living definition of failure for their team. As posed, “Based on what we shared, what are some ideas/notions/definitions we have about failure? How do our identities/backgrounds/contexts (e.g., familial, cultural, systemic) affect our ideas of failure?” Professionally, these discussions were more general, not grounded in the STEM activity as part of the project. Waterfront Science & Nature Museum (Cohort 2), for example, created a poll of eight various statements about failure for educators to vote on which quote resonated with them (see Figure 6). The goal was to develop a shared understanding of failure, but through the discussion they determined there was not one definition that fully encompassed failure at Waterfront Science & Nature Museum.

4.4. Lack of Tools

Due to the novelty of this professional development for museum educators, as well as the focus on supporting youths’ failure moments, many tools were developed and adapted. We highlight a few of the tools that were utilized the most across all cohorts. One was the development of embracing failure scenarios for Workshop 1 and/or Workshop 2 (The STEM Collaborative, Cohort 1). These scenarios were created to discuss what individual educators would do when they or youths experienced a failure moment. The scenarios were developed based on actual events and activities within the organization. Another popular tool co-adapted for Workshop 1 and/or Workshop 2 was referred to as Marvin’s Swamp. The Innovator (Cohort 1) utilized this game with their youths to talk about failure, which became a game to talk about failure amongst educators as part of the professional development cycle. This game requires individuals to work as a team and learn from other’s mistakes to help accomplish a common goal. The trick is that you can only use non-verbal actions to communicate with one another. See Online Resource 2 for instructions.

Further, the Innovator developed a problem-solving flowchart based on their analysis of video data (see Figure 7). Because the goal of the MFSC was to facilitate a conversation in Workshop 1 or Workshop 2 around how educators would respond to a student when they expressed a fear of failure or fear of messing something up, the flowchart provided educators with possible pedagogical moves to support youths through their fears. Similar to scenarios, organizations were encouraged to use this flowchart as a basis to create their own.

City Science Museum (Cohort 2), for example, adapted this flowchart and used it as a “template for creating scenarios based on the STEM activity for this project. And we brainstormed together what our responses might be” (25 October 2023). This was done through posting instructional moves, but also in clarifying what was meant by “no” and “yes” within the flowchart (see Figure 8).

Another widely utilized tool was an observation form developed by Museum of Awe (Cohort 1). This document was not only for individuals reviewing videos but also for framing the reflective practice when sharing videos in Workshop 3/4. As illustrated in Figure 9, this observation tool focused informal educators on viewing a video through three possible failure buckets or lenses.

Another unique tool was the use of connection cards, a tool co-adapted by Museum of Awe (Cohort 1) to create connections between and among educators during Workshop 3/4. These cards included phrases such as (a) I would have done that too!; (b) Oooh, I feel your pain.; (c) You failed like a champ!; and (d) I totally see what you were thinking/saying. Educators were encouraged to give connection cards to one another, particularly the focal educator, when reviewing and reflecting on their videos.

Lastly, Cohort 1 co-developed different ways to collect data from youths regarding their experiences with failure and getting “unstuck”. These data can serve as an additional reflection tool throughout the PD cycle and/or within Workshop 3/4 as part of video reflections. The Innovator (Cohort 1) developed a concept map where youths were asked to draw lines to words or phrases that they did during STEM-activities. These included the following: (a) make connections, (b) solve problems, (c) think creatively, (d) learn from failures, (e) discover, (f) share, (g) try, and (h) wonder. Youths were also asked to write a sentence or two about why they circled the items they circled (see Figure 10 for an example).

The New England Museum of Science and Museum of Awe co-developed an “unstuck” poster with the instructions: “Create a list of what to do when you are stuck”. As a cohort, they suggested this be created by the youths and educators at the beginning of the STEM activity. When created by youths and educators at the beginning of a STEM activity, this poster can become a living document that can be added to throughout the activity, whether this be a drop-in session, a workshop, or a week-long camp. In a week-long camp, youths can add a dot at the end of the day to note which strategies were used—e.g.,use a different color dot each day (see Figure 11). We encourage readers to visit our website for additional tools co-created and co-adapted by organizations in Cohort 1.

5. Discussion

In this paper, we documented how and why museums across the United States and Canada co-adapted an initial professional development cycle to support informal educators in their interactions with youths when experiencing failure in STEM-related activities. The results also highlighted how a professional development that originated in mathematics education (Borko et al., 2015) is adaptable to other contexts. Lead facilitators adapted the professional development cycle to accommodate the logistics of their organizations, as well as the unique needs of their informal educators, to integrate reflection as an approach to increase educators’ growth and professional competencies round facilitating youth’s failure moments during STEM activities. Co-adaptations addressed four problem areas that led to the successful implementation of the professional development cycle: (a) educators’ discomfort and vulnerabilities, (b) time constraints, (c) staff turnover, and (d) lack of tools. As such, these areas were highlighted as difficulties in integrating reflection and reflective practices in museum contexts (Tran, 2021). For example, dedicated time for museum educators to reflect during the busy workday and with limited number of staff is an often-cited challenge to address (Martin, 2021; Tran, 2021). However, the co-adaptations of lead facilitators in this study not only attended to these barriers, but led to a reflective video-based professional development that other informal STEM institutions can utilize in providing approaches and developing a community for their informal educators to support youths when experiencing failure, a construct often feared by educators and youths alike (e.g., Lottero-Perdue & Parry, 2017; Simpson et al., 2024b). This professional development cycle further addresses the ongoing argument to support informal educators’ growth through a sustained experience within authentic contexts (Nawab et al., 2021; Robinson, 2019).

Prior scholarship also supports professional development that provides opportunities to engage educators in the intersection of reflection and video analysis (e.g., Hollingsworth & Clarke, 2017; White et al., 2021). For example, Amador et al. (2023) found that participation in video clubs not only supported educators’ pedagogical practices such as groupwork and questioning, but also highlighted the value of collaborative reflection in that educators learned from one another. Based on our prior research, we saw evidence of this with our cohorts as the reflective video-based professional development impacted informal educators by fostering their self-reflection around failure and creating a collaborative community of educators (Simpson et al., 2024a). For example, we saw evidence of the impact of the professional development on educators’ language use and support of learner agency (Simpson et al., 2025). Informal educators in our cohorts talked about how they took time to intentionally examine the language they used around failure, becoming more aware of its meaning and significance, which allowed them to more purposefully use language to support their goals/purpose. As another example, informal educators gained new failure pedagogical “success” moves in their toolkit such as taking a step back, assigning roles, and validating youths’ feelings around failure (Simpson et al., 2025, 2024a). In addition, we have anecdotal evidence of how the professional development topic of failure extended beyond the project team to conversations across and changes within the organization. For example, the Okay Science Museum (Cohort 2; 12 July 2023) described how conversations of failure became part of a ukulele course provided for staff. This was specifically initiated by an educator who engaged in the professional development failure cycle. As another example, the following quote from an organization from Cohort 3 illustrates the possible impact of this professional development beyond the educational team:

In our organization as a whole, we have continued to look into other ways to encourage a growth mindset and acceptance of failure as a part of the workplace. We are interested to see how the industry at large addresses failure and find ways within our organization to improve our processes by embracing failure, taking risks, and trying new things. This mindset has allowed us to have more vocabulary and tools to accomplish organizational goals.

(6 November 2024)

Such actions were grounded in the reflective video-based professional development, which strengthens the notion of reflection as an approach to transform educational practices, organizational structures, and self-understanding as educators (Mohamed et al., 2022; Tran, 2021).

Based on this study and our collaboration with these 27 informal learning organizations, we provide a few recommendations when integrating video-based reflections to develop informal educators’ growth and development. One, be aware of potential barriers (e.g., educators’ discomfort and vulnerabilities, time constraints) when designing professional development opportunities, but also be willing to co-construct and co-adapt plans before and/or during implementation of the professional development. This was also suggested by Wager et al. (2023), “[a] more purposeful approach would include sitting with teachers to take their advice and develop the plans” (p. 157). Two, be open to scaffolding the professional development process based on capacity and needs. For example, include one-on-one chats or coffee house office hours prior to Workshop 3/4. As another example, it is likely that many organizations are not yet equipped or comfortable with video-based reflections, and they need an entry level way to start and/or continue building upon this process. This may be grounded in observational data, journaling, or audio recorded reflections to drive Workshop 3/4. Three, be willing to “push” educators outside of their comfort zone, namely video-based reflections, as the outcomes seem to outweigh the initial discomforts and vulnerabilities. Finally, be playful when discussing and implementing the professional development. For example, the Museum of Awe (Cohort 1) presented Workshop 3/4 as a Failure Party, “…a celebration of what [they] did all summer” (14 October 2022). In throwing the party, they provided food, drinks, and party favors.

Limitations

First, the researchers’ interpretation of the results could introduce bias. To mitigate bias during data analysis, investigator triangulation was employed (Denzin, 1984), which involved multiple researchers analyzing and interpreting the data. As stated by Fusch et al. (2018), “Bias is mitigated by different investigators observing the same data who may not agree on its interpretation” (p. 22). In particular, Author 4 and Author 5 were not involved in the conceptualization of the professional development cycle, nor active participants in the co-adaption and co-refinement of the cycle with Cohort 1. Second, our data source across the three cohorts were grounded in the lead facilitators’ memories and reflections. This could be viewed as introducing social desirability bias, where participants tend to give responses that they perceive as socially acceptable, desirable, or favorable. Further, our data sources did not include analysis of the actual professional development sessions. As a research team, we included strategies to minimize social desirability bias, such as detailing the purpose of the study and how confidentiality and anonymity would be maintained, establishing rapport with participants, providing assurances, probing for more information and examples to support their responses (Bergen & Labonté, 2020), and building a community among organizations through sustained collaboration.

6. Conclusions

This study highlights how museums across the United States and Canada co-adapted a professional development (PD) cycle, originally designed for mathematics education (Borko et al., 2015), to support informal educators in facilitating youths’ engagement with failure during STEM-related activities. Through iterative adaptations, lead facilitators addressed key challenges, including educators’ discomfort with failure, time constraints, staff turnover, and the lack of tools, creating a reflective video-based PD model tailored to their unique contexts. These adaptations not only enhanced educators’ professional growth but also fostered a community of practice focused on helping youths reframe failure as an opportunity for learning. The findings underscore the potential for the transferability of structured PD frameworks across disciplines and settings, offering a model for other informal STEM institutions. This work contributes to the broader efforts to provide sustained, authentic support for informal educators and positions the PD cycle as a tool for advancing equitable, high-quality STEM learning experiences for youths. Ultimately, the study provides actionable insights for scaling reflective, adaptable professional development to meet the evolving needs of informal education.