Next Article in Journal
Many-Dimensional Model of Adolescent School Enjoyment: A Test Using Machine Learning from Behavioral and Social-Emotional Problems
Previous Article in Journal
Correlation between High School Students’ Computational Thinking and Their Performance in STEM and Language Courses
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Education Sciences
Volume 13
Issue 11
10.3390/educsci13111102
education-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Adoption of Pedagogical Innovations: Social Networks of Engineering Education Guilds

by
Darby Riley
1,
Kaitlin Mallouk
1,*,
Courtney Faber
2,† and
Alexandra Coso Strong
3,†
1
Department of Experiential Engineering Education, Rowan University, Glassboro, NJ 08028, USA
2
Department of Engineering Education, University at Buffalo, Buffalo, NY 14260, USA
3
School of Universal Computing, Construction, and Engineering Education, Florida International University, Miami, FL 33199, USA
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Educ. Sci. 2023, 13(11), 1102; https://doi.org/10.3390/educsci13111102
Submission received: 19 September 2023 / Revised: 20 October 2023 / Accepted: 23 October 2023 / Published: 1 November 2023
Browse Figures
Review Reports Versions Notes

Abstract

:
This work investigates how innovations propagate through two professional networks (guilds): the Kern Entrepreneurial Engineering Network (KEEN) and the Consortium to Promote Reflection in Engineering Education (CPREE). Previous research has demonstrated that the adoption of pedagogical innovations is supported by the socialization of the innovation among potential adopters. In this work, we use social network analysis to explore the impact of professional connections on innovation adoption. Our research questions are: (1) How does overall social structure differ between guilds? (2) How do measures of social network structures relate to innovation adoption? A survey was distributed to members of KEEN and CPREE to capture the interactions respondents had while adopting the guild’s innovation. Social networks were generated for each guild and each respondent. These networks were analyzed to identify relationships between social network measures and the frequency of use of the innovation. Responses to open-ended questions were analyzed using thematic coding. The guilds’ overall structures impacted the formation and structure of distinct clusters/cliques, but these differing structures did not appear to affect sustained adoption. Individuals’ ego networks demonstrated a weak negative correlation between the frequency of adoption and the individual’s ego network density. Our results imply that having a diverse network exposes instructors to more ideas or allows them to see one idea from many perspectives leading to a higher likelihood of adoption.

1. Introduction

Engineering education as a field contributes a significant amount of time, energy, and resources towards the development of new and innovative pedagogy [1,2,3]. Numerous studies examining best practices in classrooms are published with the American Society for Engineering Education (ASEE) and other venues each year, yet few of these innovations ever see widespread use in actual engineering classrooms [1,2]. This might be because the traditional publication-to-classroom pipeline follows a “dissemination paradigm”—developers trust that instructors looking for new or improved teaching methods will seek them out, and no special efforts needing to be made to ensure adoption occurs [4,5,6]. While this approach may work for some, many innovations published in this manner will never see widespread use in engineering classrooms. By contrast, under the “propagation paradigm”, developers of new and innovative pedagogy make an effort to work with potential adopters throughout the development and implementation process and beyond [4,7]. Again, while this approach may work for some, it is still not a universal solution to ensuring the widespread adoption of evidence-based pedagogy.
The engineering education “guild” presents a combined approach: both publishing studies, lesson plans, and other standalone pieces under the dissemination paradigm, while also hosting workshops, encouraging collaboration between colleagues, and even partnering with universities to actively aid in the adoption process [8,9]. This guild approach is reflected in the two engineering education guilds explored in this study: the Consortium to Promote Reflection in Engineering Education (CPREE) and the Kern Entrepreneurial Engineering Network (KEEN). CPREE’s mission was to encourage faculty to incorporate reflection into their engineering courses [10]. This mission was funded by the Leona M. and Harry B. Helmsley Charitable Trust from 2014 to 2018 and was carried out by Drs. Jennifer Turns and Cindy Atman from the University of Washington. CPREE consisted of 12 partner institutions, each with a principal investigator who was responsible for understanding how reflection was already being used in their campus’ classrooms and producing field guides about those reflection activities to be shared with the engineering education community more broadly [10]. KEEN is a network of over 50 institutions and has been in existence since 2005 [11]. Its directive is to instill an entrepreneurial mindset (EM) in all engineering undergraduate students in the United States. With its funding from the Kern Family Foundation, KEEN has advanced its directive through grants to network schools, a platform for social engagement and sharing classroom innovations, and a variety of community-building activities, including an annual national conference and faculty development workshops.
Prior work has examined how the approaches to propagation used by CPREE and KEEN align with the evidence-based propagation techniques [8]. Using the Designing for Sustained Adoption Instrument (DSAAI) as a guide [12], the authors found that CPREE and KEEN used many of the approaches highlighted in the instrument, but that there were some high-impact practices used by CPREE and KEEN that were not captured by the DSAAI. These practices included providing funding for adoption efforts, public recognition of faculty innovators, and mutual accountability.
Other work in the field of educational change has identified the social aspects of changemaking—such as mutual accountability, community building, collective learning, and so on—as critical to lasting pedagogical innovation adoption [13,14,15,16]. Previous studies have found that instructors who discuss teaching practices with other faculty are more likely to adopt new teaching strategies [17,18,19]. When given the opportunity to observe their colleagues teaching, change becomes even more likely [20]. Coordinated changes to teaching methods (such as through instructor teams or departmental teams) give instructors a greater feeling of ownership over and investment in their classrooms [21,22], while also ensuring that change is perpetuated even as faculty may turn over [22,23]. Much of this work has been performed in the context of teaching Communities of Practice [16,24] and co-teaching [25]. However, the unique social structure of guilds such as KEEN and CPREE has not yet been examined.
KEEN and CPREE were chosen as case studies for several reasons: both guilds are focused on engineering education, an area of national importance; both guilds were funded by charitable organizations, providing a blueprint for future philanthropic engagement; and both have been shown to use evidence-based practices for encouraging pedagogical change, which can be implemented by any party with similar goals. This paper uses social network analysis (SNA), supported by the analysis of qualitative survey responses, to investigate the ways in which pedagogical innovations propagate through relationships between guild members, and how these innovations might travel beyond the guild’s reach through instructor collaboration. We do this by responding to the following research questions: (1) How does the overall social structure of KEEN differ from that of CPREE? (2) How do measures of social network structures relate to faculty’s adoption of the innovation?

2. Methodology

This study aimed to generate two social networks—one for CPREE’s members, and one for KEEN’s members. To this end, two surveys were distributed via Qualtrics, each aimed at members of one of the aforementioned guilds. CPREE members were identified using CPREE’s website, which lists the PI and CoPI at each partner institution [10]. For KEEN, we shared the survey with the KEEN Leaders at each partner institution [11] and requested that they share the survey with their colleagues.

2.1. Survey Instrument

Faculty members responding to the survey gave their name, indicated how frequently they made use of the guild’s innovation (for CPREE members, reflection; for KEEN members, entrepreneurial mindset) in their classrooms on a Likert scale, and responded to three open-ended prompts regarding their adoption process. At the end of the survey, participants were given the opportunity to name up to ten people they knew who also used the innovation, as well as indicate the ways in which the two had interacted (this person taught me about reflection, I taught this person about reflection, we collaborated on reflection-related pedagogy, or other). The provided names were then added to the distribution list as a form of snowball sampling. A screen capture of this question as it appeared in the survey can be seen below (Figure 1). The questionnaire can be found in its entirety in the Supplementary Materials.

2.2. Ego Network Creation

From this information, the research team could construct the instructor’s ego network—an image of an individual’s direct social connections, and the direction of those connections [26]. For example, if instructor A noted that instructor B taught them about reflections, the connection between them would be directional, traveling from instructor B to instructor A. As ego networks were constructed, the names provided by survey participants were used to create a guild-wide adjacency matrix of collaborators, which was then used to generate a visual representation of guild interactions using ORA-LITE social network visualization software [26]. Figure 2 shows the visual difference between an ego network and a social network. In addition to the information provided in the survey, the researchers utilized publicly available data from the ASEE Papers on Engineering Education Repository (PEER) database [27], as well as card authorship on KEEN’s website EngineeringUnleashed, to generate more social connections. Each instructor who responded to the KEEN survey (n = 23) was located on EngineeringUnleashed [11], and co-authors (as indicated by authorship of KEEN cards—lesson plans/concepts relating to entrepreneurial mindset published to EngineeringUnleashed) were added to the overall network as bi-directional collaborators. Similarly, each instructor who responded to the CPREE survey (n = 25) was searched in ASEE PEER [27], and co-authors on papers explicitly related to reflection pedagogy were added to the network as bi-directional collaborators.
The survey item “How often do you make use of assignments relating to [innovation] in your classroom(s)?” both confirmed innovation adoption and allowed network nodes to be sized by the frequency of innovation use (larger nodes use the innovation more frequently, while smaller nodes use the innovation less frequently). A complete list of survey questions used to generate the social networks can be found in Table 1.

2.3. Social Network Analysis

Overall social networks were primarily analyzed through cluster analysis and echo-chamberness. This work uses the Leiden algorithm [28], an iterative process which subdivides a large community into smaller sub-communities known as clusters. A ‘cluster’ is defined as a group within a larger community which demonstrates greater density (i.e., more interconnections between individuals) relative to other groups or individuals near it [28]. This procedure is useful for examining the overall structure of a knowledge-sharing community, as it can identify sub-communities where information or ideas might become “stuck”. This quality of social clusters can be quantified as echo-chamberness. The echo-chamberness (EC) of a cluster can be calculated as follows:
E C = r d 3
where r is the reciprocity [29] of the network (the ratio of bi-directional edges and the total number of edges in the network), and d is the density of the network (the ratio of existing edges to total possible edges in a network) [30].
Ego networks were characterized through more individually focused measures of centrality and density. In-degree and out-degree centrality capture the directionality of an individual’s ego network—in this case, is information regarding innovations mostly coming in or going out? In-degree and out-degree centrality are measured as a ratio of the number of links pointed into (for in-degree) or out of (for out-degree) a node to the total number of links in the network [29]. Density reflects the interconnectedness of an individual’s ego network. In other words, it measures how many of an individual’s direct connections are connected to one another [31]. This is calculated as a ratio of the number of reported connections in an ego network to the total number of connections possible in that ego network [29].
Cluster analysis was used to examine the overall structures of CPREE and KEEN. Measures of density and centrality for each individual were correlated with the reported frequency of use of the innovation in the classroom. All network analysis processes (cluster analysis, density, and centrality calculations) were performed in ORA-LITE. All statistical analyses of network metrics were performed in SPSS. The interpretation of these analyses are additionally supported by responses to open-ended questions in the survey.

2.4. Qualitative Data Analysis

To provide a context and nuance to the quantitative methods described above, the participant responses to three open-ended questions in the survey were analyzed. These open-ended questions were as follows:
  • Of the resources listed above, which was the most influential in helping you integrate reflections (entrepreneurial mindset) into your classroom(s)?
  • What was your process for integrating reflection (entrepreneurial mindset) into your classroom?
  • What advice do you have for others integrating reflections (entrepreneurial mindset) related assignments into their classroom(s)?
To analyze the responses to these questions, a code book was generated and then applied to the responses by a group of five coders. The code book was developed using inductive, descriptive methods and the codes and their definitions were agreed upon in an in-person meeting after each of the five coders had individually identified prominent themes in the responses [32]. The codebook appears in the Appendix A, Table A1.

3. Results and Discussion

The distributed surveys received a combined total of 48 complete responses—23 from KEEN members and 25 from CPREE. The previously described, publicly available data (the ASEE PEER repository and KEEN’s EngineeringUnleashed) were used to add more collaborators in a limited fashion, leading to a KEEN network of 58 individuals (of approximately 4900 total members) and a CPREE network of 131 individuals (of approximately 500 total participants).

3.1. RQ1: How Does the Overall Social Structure of KEEN Differ from That of CPREE?

The social networks pictured below were generated using snowball sampling. As such, they represent only a fraction of the complete guilds, and so many larger-scale structural analyses (which mathematically assume a complete network) would be inaccurate. However, a qualitative examination of smaller scale “clusters” in each guild—these being cliques or social groups which are algorithmically determined to be separate from other social groups—is appropriate.
Using standard network analysis tools, we show that the communities detected for KEEN and CPREE have different structures, which reflect the intentions of the organizations themselves. Additionally, the structure of the two organizations’ networks suggests that CPREE members may have had several routes for support, whereas KEEN members tend to have a single “expert” in their immediate network acting as a primary knowledge disseminator.

3.1.1. Leiden Algorithm Community Detection

A total of 9 distinct groups (n = 58) were detected in the KEEN network (Newman modularity 0.72), and a total of 15 distinct groups (n = 131) were detected in the CPREE network (Newman modularity 0.72). Further details can be found in Table 2 below. Both networks were found to be highly modular (meaning the communities detected by the algorithm demonstrate a high level of separation from one another and that the algorithm was generally successful) [28].
In KEEN’s social network, many clusters (e.g., 3, 4, 5) have similar structures: one or more larger, central nodes, surrounded by smaller nodes in a starburst shape (Figure 3). KEEN’s average subgroup density is 0.21, indicating that, in any individual cluster, approximately 21% of the total possible connections were reported. This, again, indicates a lack of interconnectedness amongst KEEN participants—smaller nodes (less frequent users) are gathered around larger nodes (more frequent users), but do not often form connections with each other or continue to pass knowledge down. Many of these large central nodes represent KEEN leaders—members of the KEEN leadership council, recipients of a KEEN award, department leaders at partnered institutions, and so on. This pattern implies a mentor–mentee style of collaboration, wherein community leaders pass knowledge directly to other members of the group—an often-touted method of successful propagation [33,34].
More than anything else, KEEN members advised attending KEEN-sponsored workshops as a primary method for learning about EM. KEEN members also noted events—overwhelmingly workshops—as especially influential, useful, and supportive during their adoption process. One instructor stated that “The emphasis on deliverables [in the KEEN workshop] helped me to clarify my goals, processes, and outline materials to share with the community”, highlighting the importance of the structure workshops provide. Another instructor noted that their process consists primarily of “following along with the advice of the KEEN workshop”. This well-defined structure places KEEN resources—workshops, community leaders, written materials, etc.—at central points in the organization’s overall network of knowledge and collaboration. Previous research found that KEEN prioritized defining and outlining EM in their own words [8], thereby building a foundation of shared language for its members and positioning themselves and their community leaders as authorities on the topic. As such, the most central members of KEEN’s network—those at the center of starburst clusters—are all KEEN community leaders, workshop coordinators, and/or “rising star” awardees. This pattern of mentor expertise is apparent in one participants’ answer to the third open-ended question in the survey about how to get started with EM: “I think it is helpful to have someone who is familiar with KEEN and EM to help to guide the development of EM activities at first. This allows you to make sure that you do not get off-track”. This, again, implies that structure is not only helpful, but necessary as an instructor begins to incorporate EM into their classroom. In previous research, communities of practice with similar prescribed mentor–mentee structures have led to similar social networks: peer mentors become community leaders, tying otherwise disparate clusters to one another through “bridging ties” [24,35].
By contrast, CPREE’s clusters do not fit any single pattern (Figure 4). While some of the outlying, disconnected groups have a similar starburst shape to many of KEEN’s, the groups at the center of the image have a great variety of shapes, many of which are more densely interconnected. CPREE’s average subgroup density is 0.34, indicating that, in any individual cluster, approximately 34% of the total possible connections were reported—higher than KEEN’s average density of 0.21. CPREE as a guild has a more decentralized structure than KEEN, and it would appear that this non-hierarchical organization is reflected in the resulting social network—clusters 1, 3, 4, and 6, in particular, have unusual structures consisting of a mix of individuals collaborating in a variety of combinations which can be seen by the many interconnecting lines in clusters 1 and 4 and the multiple hubs of collaboration in clusters 3 and 6. This decentralized social network structure could result from the lack of prescribed mentor–mentee relationships—a cornerstone of KEEN’s organizational structure. While CPREE did not necessarily present itself as the authority on reflection, it did tend to communicate change largely through partnering with institutions and selecting one or more instructors at these institutions to become peer mentors [8]. This less-structured approach to collaboration also supports propagation, though in a different manner—groups with decentralized leadership more frequently promote independence and creativity [36]. Additionally, CPREE has been inactive as an organization since 2018 [8]. It is possible that the social network has evolved to include more interconnections since CPREE activities ceased.
CPREE members’ responses de-emphasized workshop events, instead noting mentors and colleagues as their most influential resources in the adoption process. The mentors mentioned by name in responses—all notable CPREE community leaders—still tend to occupy the central positions in their clusters and in the network overall, similar to KEEN’s network [24,35]. However, when describing their process in adopting reflection activities into their classroom, CPREE members most frequently described collaborating with peers. One instructor explained their approach to iteration as follows: “I would try something, then see a problem, then ask colleagues… about ways to address the problem”. Another stated that “interaction with my colleagues in CPREE provided me with a lot of helpful feedback to make [reflection] work well”. As mentioned above, previous research has shown that communities of practice with prescribed mentors often organize themselves into clusters connected by “bridging ties”—mentors are connected to one another, but most mentees are only connected to their direct mentors [24,35]. This bridging ties structure is reflected in the KEEN network, but is less apparent in the CPREE network. It is likely that, when a community is purposefully decentralized, networks of knowledge and collaboration also become decentralized—rather than viewing particular group members or sources of information as preferred or “correct”, support can come from many different directions [24,35,37].

3.1.2. Echo-Chamberness

A Kendall’s tau-b correlation analysis was conducted to evaluate the strength and direction of the relationship between the echo-chamberness of an individual’s cluster and the frequency of use of the innovation. This test was selected due to the non-normality of the continuous variable echo-chamberness.
The relationship between an individual’s cluster score and frequency of use of the pedagogical innovation was found to be not statistically significant (p = 0.68). The correlation analysis found a negligible positive correlation between echo-chamberness and frequency, (r = 0.05, 95% Bootstrap CI [−0.24, 0.31], (p = 0.68, n = 48. The effect size for this analysis was r2 = 0, 95% Bootstrap CI [−0.06, 0.10], indicating that 0% of the variance between echo-chamberness and frequency is shared in this data, with replications possibly finding a small positive or small negative correlation. Post hoc power analysis suggests that the test was underpowered ( 1 − β = 0.06) with a small sample n = 48.
This analysis indicates that, based on available data, there is no relationship between the echo-chamberness and frequency of use of guild-supported innovations. The lack of a relationship implies that the insularity, density, and reciprocity of clusters has no bearing on whether a faculty member adopts a guild innovation. While many survey responses mentioned working with colleagues, these responses never imply that colleague-to-colleague interactions are structured, insular, or exclusive in any way [37]. From this, it would appear that the attributes of clusters/communities are less important than the attributes of someone’s immediate social circle—in other words, their ego network. As such, the following section explores ego network measures.

3.2. RQ2: How Do Measures of Social Network Structure Relate to Faculty’s Adoption of the Innovation?

While the larger social networks previously discussed can help provide a clearer understanding of how guilds operate on a broad scale, the impact and influence of guilds on individual instructors is better examined at an individual scale. The ego networks of individual survey respondents were generated and then analyzed using typical measures of ego networks: In-Degree and Out-Degree Centrality and Density.

3.2.1. In-Degree and Out-Degree Centrality

To investigate the impact of directional relationships on the ego networks, we measured their in-degree and out-degree centrality. Centrality is a measure of network position, identifying influential or otherwise highly ranked members based on their connectedness to others in the network on a scale of zero to one. More central or influential nodes have a centrality measure closer to one, while the centrality of less influential nodes will be closer to zero. In-degree centrality specifically measures relationships directed into the node, while out-degree centrality measures relationships out of the node. In this case, in-degree centrality measures information being learned by an individual, while out-degree centrality measures information being taught by an individual. In-degree and out-degree centrality were calculated for each node to one degree (meaning only considering a node’s direct connections). As these analyses are carried out on an individual level, members from both guilds were combined for a larger sample size.
The normality of the continuous variables in-degree centrality and out-degree centrality were checked and found to be non-normal. Thus, a Kruskal–Wallis test was conducted to compare in-degree and out-degree centrality between individuals with select levels of innovation adoption: faculty who used the innovation “never”, “rarely”, “sometimes”, “often”, and “constantly and consistently”. Due to a lack of “never” (n = 0) and “rarely” (n = 1) responses, these categories were discounted. No statistically significant difference was found between the remaining groups based upon the in-degree (p = 0.24) or out-degree centrality (p = 0.31). This might be due to the small sample size (n = 47) underpowering the analysis. However, it appears that for both in-degree and out-degree centrality, participants who responded “often” were the most central, with “sometimes” and “constantly and consistently” having little to no difference from one another.
Assuming that no true difference exists between the “sometimes” and “constantly and consistently” groups, this result indicates that neither having many mentors nor being an active mentor alone increases the likelihood that an individual will adopt an innovation more permanently into their classrooms. This is somewhat at odds with the common adage that teaching something leads to a deeper understanding of the concept [38,39]. In fact, survey responses were less likely to mention or advise establishing mentor–mentee relationships—instead, members of both CPREE and KEEN emphasized the influential power of collaboration: “conversations with other faculty help me to build skills”, one instructor stated. Another noted that “interaction with my colleagues in CPREE provided me with a lot of helpful feedback to make it work well”. When prompted to recommend resources to others, instructors pointed to workshops and other events as sources of information—“Attend education related conferences and see what you might be able to adapt from other areas”—but recommended peer support as implementation continued—“Start small and get a lot of feedback early on in the process”, or, more directly, “I would also recommend committing to a collaborative EM project in a co-taught or multiple section course so that you have others to bounce your ideas off of”. The survey responses indicate, then that, while mentor–mentee relationships may be influential, they do not have any direct effect on the likelihood of sustained adoption. Previous research has found that a sense of belonging in a community—whatever form that community takes—can be a powerful motivator in undertaking new behaviors [40,41]. However, there may be other factors or qualities of an individual’s network that do have a direct impact on their frequency of use of the pedagogical innovations.

3.2.2. Clustering Coefficient Density

Another descriptive measure of an individual’s ego network is the density. This measure captures the “interconnectedness” of a network on a scale of zero to one. If all of an individual’s direct connections are connected to one another, their ego network density will be equal to one. If none of an individual’s direct connections are connected to one another (the previously described starburst shape), their ego network density will be equal to zero.
A Pearson’s r correlation analysis was conducted to evaluate the strength and direction of the relationship between ego network density and frequency of use of the innovation. The normality of the continuous variable density was checked and found to be within range.
While the relationship found was not statistically significant (p = 0.34), the correlation analysis found a weak negative correlation between network density and frequency of innovation use (r = −0.15, 95% Bootstrap CI [−0.42, 0.18], p = 0.34, n = 47). The effect size for this analysis was r2 = 0.02, 95% Bootstrap CI [−0.18, 0.03], indicating that 2% of the variance between density and frequency is shared in this data. This is a small effect size, and the results indicate that replications are likely to find a similar effect. Post hoc power analysis suggests that the test was underpowered ( 1 − β = 0.17) with a small sample n = 47.
Despite the small sample size, this analysis indicates that those with a less dense ego network (meaning fewer interconnections between collaborators) use the pedagogical innovations slightly more often than those with denser ego networks. Though this, at first, seems counter-intuitive—it does, after all, oppose the driving force behind densely interconnected communities of practice [15,16,24]—looking to the open-ended responses provides context for this result. Many responses describe needing to approach the innovation from multiple perspectives, or experiment with it in a variety of ways, before the innovation becomes a permanent fixture in the faculty member’s classroom. For example, one faculty member describes their process as gathering “a suite of potential ideas” before presenting these to colleagues for feedback. Another stated that “My best advice is to experiment with different forms of [the innovation] and run with what works best for the students”. Other mentions of finding the “right” resources, or finding an effective method through trial and error, were frequent throughout the open-ended responses. It might be that, while having many mentors/sources of information alone does not improve the frequency of use in the classroom, having many different sources of information does. As one instructor stated, “many EM resources exist, its [sic] just a matter of connecting with the right people/organizations and finding them”.
This brings us back to the dissemination and propagation paradigms; one of the fundamental issues Froyd et al. identified under the dissemination paradigm was a tendency for instructors to reject innovations out of hand based on a perceived “fitness” to their classroom or institution [4]. If, however, instructors are presented with more versions of the innovation, and more methods for adopting it into their classrooms, it is more likely that they will encounter a version or method with a clear and obvious fit. In practice, this approach is similar to the use of divergent thinking in engineering design: it centers flexibility, innovation, and the value of multiple perspectives in the process of developing and designing solutions to problems [42,43,44]. Instructors who are able to view or participate in this divergent process appear to be less likely to reject the concept before trying it. It is possible, then, that getting started is one of the most difficult parts of the process—but perhaps not only due to a lack of classroom fitness.
When prompted to give advice to other faculty members looking to adopt EM/ reflections into their classrooms, responses overwhelmingly focused on pushing past this perceived barrier to entry: “Just give it a try”, “start small and to simply try”, “Start small and get a lot of feedback early on in the process”, or even “First, DO IT”. This illuminates an additional barrier to entry: a resistance to change in general. Previous research has found that the greater a faculty member’s resistance to change, the more barriers they perceive to educational change [45]. Additionally, encouraging collaboration and “collective sensemaking” is one of the most effective methods in combating this resistance [46].
It is clear that engineering education guilds are already in a position to provide collaborative support to those who perceive high barriers to entry. However, it may also benefit these guilds to ensure that even their most active members continue to experiment and communicate with others who have different perspectives on or experiences with the innovation in question—particularly if long-time members reach out to non-members with these new ideas and approaches. By continuing to experiment and provide documentation on low-stakes classroom activities, guilds stand a higher chance of introducing instructors to the “right” version of the innovation for their classrooms, thereby reducing the barriers to entry.

4. Limitations and Future Work

We were unable to generate a full network for KEEN or CPREE, due to the weakness of the snowball sampling method and low response rates. The low response rate also left the statistical analyses underpowered, though trends of potential interest were noted and supported by the qualitative analyses. Even with these limitations, the purpose of this paper is not to present widely generalizable findings, but rather to share how the structure of organizations affects the social networks of participants in those organizations and the possibility of a relationship between guild participants’ social networks and their adoption of new pedagogical practices.
Future work will explore the specific faculty development offerings of KEEN and the aspects of those offerings that contribute to a faculty member’s likelihood of making change to their pedagogical practice. There is also an opportunity to explore the social networks of faculty who participate in other, similar, organizations, and the impact of those connections on a variety of faculty characteristics (e.g., the use of evidence-based pedagogical practices, career satisfaction, and overall wellbeing).

5. Conclusions

Despite the amount of effort put towards developing and publishing innovative methods of teaching, these evidence-based practices still rarely find widespread use in classrooms. Engineering education guilds present a unique way of combating this problem by combining dissemination (the traditional publication route) and propagation (an active method of communicating and aiding the adoption of innovative practices) approaches to pedagogical change, though their processes remain understudied. This work aims to address the lack of research regarding the social aspect of engineering education guilds’ practices through social network analysis. These results found that the guilds’ overall structures (hierarchical vs. decentralized) impacted the formation and structure of clusters which arose through inter-colleague collaboration. However, the attributes of these clusters did not appear to have an effect on sustained adoption. In addition, while the guild members’ mode of engagement (teaching the innovation vs. learning the innovation) did not appear to affect the frequency with which they used the adoption in their classroom, a weak negative correlation between frequency and ego network density was observed. This may indicate that individuals who are exposed to an innovation from a variety of differing perspectives have a higher likelihood of adopting the innovation and using it consistently in their classrooms. The responses to the open-ended questions support this hypothesis. These results can guide further exploration of the unique ways in which engineering education guilds approach changemaking, as well as present evidence supporting efforts to continually expose guild members to innovation from new or unique perspectives.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/educsci13111102/s1.

Author Contributions

Conceptualization, K.M.; methodology, K.M. and C.F.; validation, D.R.; formal analysis, D.R. and K.M.; investigation, D.R. and K.M.; writing—original draft preparation, D.R.; writing—review and editing, D.R., K.M., A.C.S. and C.F.; visualization, D.R.; supervision, K.M., A.C.S. and C.F.; project administration, K.M., A.C.S. and C.F.; funding acquisition, K.M., A.C.S. and C.F. All authors have read and agreed to the published version of the manuscript.

Funding

This material is based upon work supported by the National Science Foundation under Grant No. 1927268.

Institutional Review Board Statement

The study was approved by the Institutional Review Board of Rowan University (#PRO-2020-61, 23 October 2020).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Due to privacy restrictions, data for this study will not be made publicly available.

Acknowledgments

We, the authors, wish to thank our advisory board members, Jeffrey Froyd, Jonathan Hilpert, and Julia Williams for their guidance and Noor Aulakh, Hannah Corbin, Alexander Herrman, Shahir Mollah, and Conor Peterson for their analysis of the qualitative data.

Conflicts of Interest

Author Kaitlin Mallouk is a faculty professional development consultant for the Kern Family Foundation, an organization that may be affected by the research reported in the enclosed paper. These interests have been fully disclosed to the journal, and approved plans through Mallouk’s university are in place for managing any potential conflicts arising from the consulting involvement.

Appendix A

Table A1. Qualitative codebook.
Table A1. Qualitative codebook.
CodeDefinition
EventsResponse describes attending structured events hosted by the guild such as workshops or conferences
Consuming the LiteratureResponse describes using literature—including formal sources such as textbooks and journal articles and informal sources such as websites or blog posts—to learn about the innovation
MentorsResponse describes interacting with community mentors or other individuals knowledgeable in the innovation
MenteesResponse describes mentoring other members of the community
CollaborationResponse describes collaborative discussions or other interactions with faculty members that do not have a mentor–mentee structure
IterationResponse describes the adoption process as iterative and needing constant adjustments
Getting StartedResponse describes the early stages of the adoption process and/or encourages others to take action
Personal ReflectionResponse describes using personal, independent reflection to improve classroom practices related to the innovation
ReframingResponse describes modifying existing classroom practices or coursework to include the innovation, rather than starting from scratch

References

  1. Borrego, M.; Henderson, C. Increasing the use of evidence-based teaching in STEM higher education: A comparison of eight change strategies. J. Eng. Educ. 2014, 103, 220–252. [Google Scholar] [CrossRef]
  2. Felder, R.M.; Brent, R.; Prince, M.J. Engineering instructional development: Programs, best practices, and recommendations. J. Eng. Educ. 2011, 100, 89–122. [Google Scholar] [CrossRef]
  3. Melsa, J.L.; Rajala, S.A.; Mohsen, J. Creating a culture for scholarly and systematic innovation in engineering education. J. Eng. Educ. 2009, 98, 209. [Google Scholar] [CrossRef]
  4. Froyd, J.E.; Henderson, C.; Cole, R.S.; Friedrichsen, D.; Khatri, R.; Stanford, C. From dissemination to propagation: A new paradigm for education developers. Chang. Mag. High. Learn. 2017, 49, 35–42. [Google Scholar] [CrossRef]
  5. Khatri, R.; Henderson, C.; Cole, R.; Froyd, J. Successful propagation of educational innovations: Viewpoints from principal investigators and program. In Proceedings of the AIP Conference Proceedings; American Institute of Physics: College Park, MD, USA, 2013; Volume 1513, pp. 218–221. [Google Scholar]
  6. Khatri, R.; Henderson, C.; Cole, R.; Froyd, J.E.; Friedrichsen, D.; Stanford, C. Characteristics of well-propagated teaching innovations in undergraduate STEM. Int. J. STEM Educ. 2017, 4, 2. [Google Scholar] [CrossRef]
  7. Besterfield-Sacre, M.; Cox, M.F.; Borrego, M.; Beddoes, K.; Zhu, J. Changing engineering education: Views of US faculty, chairs, and deans. J. Eng. Educ. 2014, 103, 193–219. [Google Scholar] [CrossRef]
  8. Mallouk, K.; Strong, A.; Riley, D.; Faber, C. How Engineering Education Guilds are Expanding our Understanding of Propagation in Engineering Education. J. STEM Educ. Innov. Res. 2022, 23, 38–49. [Google Scholar]
  9. Riley, D.; Mallouk, K.; Strong, A.; Faber, C. Adoption of Pedagogical Innovations: Resource Networks of Engineering Education Guilds. In Proceedings of the 2021 IEEE Frontiers in Education Conference (FIE), Lincoln, NE, USA, 13–16 October 2021; pp. 1–5. [Google Scholar]
  10. Consortium to Promote Reflection in Engineering Education (CPREE), C. CPREE. About Us. Available online: http://cpree.uw.edu/about-us/ (accessed on 7 June 2023).
  11. KEEN. About—Engineering Unleashed. Available online: https://engineeringunleashed.com/about (accessed on 7 June 2023).
  12. Stanford, C.; Cole, R.; Froyd, J.; Friedrichsen, D.; Khatri, R.; Henderson, C. Supporting sustained adoption of education innovations: The designing for sustained adoption assessment instrument. Int. J. STEM Educ. 2015, 3, 1. [Google Scholar] [CrossRef]
  13. Cross, K.J.; Mamaril, N.; Johnson-Glauch, N.; Herman, G. Building cultures of collaboration that promote instructional change. Stud. Eng. Educ. 2021, 2, 1–18. [Google Scholar] [CrossRef]
  14. Dancy, M.; Lau, A.C.; Rundquist, A.; Henderson, C. Faculty online learning communities: A model for sustained teaching transformation. Phys. Rev. Phys. Educ. Res. 2019, 15, 020147. [Google Scholar] [CrossRef]
  15. Gehrke, S.; Kezar, A. The roles of STEM faculty communities of practice in institutional and departmental reform in higher education. Am. Educ. Res. J. 2017, 54, 803–833. [Google Scholar] [CrossRef]
  16. Pitterson, N.; Allendoerfer, C.; Streveler, R.; Ortega-Alvarez, J.; Smith, K. The importance of community in fostering change: A qualitative case study of the Rigorous Research in Engineering Education (RREE) Program. Stud. Eng. Educ. 2020, 1, 20–37. [Google Scholar] [CrossRef]
  17. Lane, A.K.; Skvoretz, J.; Ziker, J.; Couch, B.; Earl, B.; Lewis, J.; McAlpin, J.; Prevost, L.; Shadle, S.; Stains, M. Investigating how faculty social networks and peer influence relate to knowledge and use of evidence-based teaching practices. Int. J. STEM Educ. 2019, 6, 28. [Google Scholar] [CrossRef]
  18. Ma, S.; Herman, G.L.; West, M.; Tomkin, J.; Mestre, J. Studying STEM faculty communities of practice through social network analysis. J. High. Educ. 2019, 90, 773–799. [Google Scholar] [CrossRef]
  19. Macdonald, R.H.; Beane, R.J.; Baer, E.M.; Eddy, P.L.; Emerson, N.R.; Hodder, J.; Iverson, E.R.; McDaris, J.R.; O’Connell, K.; Ormand, C.J. Accelerating change: The power of faculty change agents to promote diversity and inclusive teaching practices. J. Geosci. Educ. 2019, 67, 330–339. [Google Scholar] [CrossRef]
  20. Neal, J.W.; Neal, Z.P.; Atkins, M.S.; Henry, D.B.; Frazier, S.L. Channels of change: Contrasting network mechanisms in the use of interventions. Am. J. Community Psychol. 2011, 47, 277–286. [Google Scholar] [CrossRef]
  21. Elrod, S.; Kezar, A. Increasing student success in STEM: Summary of a guide to systemic institutional change. Chang. Mag. High. Learn. 2017, 49, 26–34. [Google Scholar] [CrossRef]
  22. Reinholz, D.L.; Apkarian, N. Four frames for systemic change in STEM departments. Int. J. STEM Educ. 2018, 5, 1–10. [Google Scholar] [CrossRef]
  23. Marbach-Ad, G.; Ziemer, K.S.; Orgler, M.; Thompson, K.V. Science Teaching Beliefs and Reported Approaches within a Research University: Perspectives from Faculty, Graduate Students, and Undergraduates. Int. J. Teach. Learn. High. Educ. 2014, 26, 232–250. [Google Scholar]
  24. Mestre, J.P.; Herman, G.L.; Tomkin, J.H.; West, M. Keep your friends close and your colleagues nearby: The hidden ties that improve STEM education. Chang. Mag. High. Learn. 2019, 51, 42–49. [Google Scholar] [CrossRef]
  25. Henderson, C.; Dancy, M.; Niewiadomska-Bugaj, M. Use of research-based instructional strategies in introductory physics: Where do faculty leave the innovation-decision process? Phys. Rev. Spec. Top.-Phys. Educ. Res. 2012, 8, 020104. [Google Scholar] [CrossRef]
  26. McCarty, C.; Lubbers, M.J.; Vacca, R.; Molina, J.L. Conducting Personal Network Research: A Practical Guide; Guilford Publications: New York, NY, USA, 2019. [Google Scholar]
  27. ASEE for Engineering Education (ASEE). ASEE PEER Document Repository. Available online: https://peer.asee.org (accessed on 7 June 2023).
  28. Traag, V.A.; Waltman, L.; Van Eck, N.J. From Louvain to Leiden: Guaranteeing well-connected communities. Sci. Rep. 2019, 9, 5233. [Google Scholar] [CrossRef]
  29. Wasserman, S.; Faust, K. Social Network Analysis: Methods and Applications; Cambridge University Press: New York, NY, USA, 1994. [Google Scholar]
  30. Krackhardt, D.; Stern, R.N. Informal networks and organizational crises: An experimental simulation. Soc. Psychol. Q. 1988, 51, 123–140. [Google Scholar] [CrossRef]
  31. Watts, D.J.; Strogatz, S.H. Collective dynamics of ‘small-world’networks. Nature 1998, 393, 440–442. [Google Scholar] [CrossRef] [PubMed]
  32. Miles, M.B.; Huberman, A.M.; Saldaña, J. Qualitative Data Analysis: A Methods Sourcebook; SAGE Publications Inc.: Thousand Oaks, CA, USA, 2019. [Google Scholar]
  33. Katz, J.; Henry, M. Turning Professors into Teachers: A New Approach to Faculty Development and Student Learning; ERIC: Brussels, Belgium, 1988.
  34. Schnader, A.L.; Westermann, K.D.; Hanes Downey, D.; Thibodeau, J.C. Training teacher-scholars: A mentorship program. Issues Account. Educ. 2016, 31, 171–190. [Google Scholar] [CrossRef]
  35. Ma, S.; Herman, G.L.; Tomkin, J.H.; Mestre, J.P.; West, M. Spreading teaching innovations in social networks: The bridging role of mentors. J. STEM Educ. Res. 2018, 1, 60–84. [Google Scholar] [CrossRef]
  36. Dee, J.R.; Henkin, A.B.; Pell, S.W. Support for innovation in site-based-managed schools: Developing a climate for change. Educ. Res. Q. 2002, 25, 36. [Google Scholar]
  37. Grunspan, D.Z.; Holt, E.A.; Keenan, S.M. Instructional communities of practice during COVID-19: Social networks and their implications for resilience. J. Microbiol. Biol. Educ. 2021, 22, 10–1128. [Google Scholar] [CrossRef]
  38. Carberry, A.R.; Ohland, M.W. A Review of Learning-by-Teaching for Engineering Educators. Adv. Eng. Educ. 2012, 3, 2. [Google Scholar]
  39. Cortese, C.G. Learning through teaching. Manag. Learn. 2005, 36, 87–115. [Google Scholar] [CrossRef]
  40. Chiu, T.K.F.; Chai, C.-s. Sustainable Curriculum Planning for Artificial Intelligence Education: A Self-Determination Theory Perspective. Sustainability 2020, 12, 5568. [Google Scholar] [CrossRef]
  41. Sparks, C.; Dimmock, J.; Lonsdale, C.; Jackson, B. Modeling indicators and outcomes of students’ perceived teacher relatedness support in high school physical education. Psychol. Sport. Exerc. 2016, 26, 71–82. [Google Scholar] [CrossRef]
  42. Cropley, D.H. Promoting creativity and innovation in engineering education. Psychol. Aesthet. Creat. Arts 2015, 9, 161. [Google Scholar] [CrossRef]
  43. Passow, H.J.; Passow, C.H. What competencies should undergraduate engineering programs emphasize? A systematic review. J. Eng. Educ. 2017, 106, 475–526. [Google Scholar] [CrossRef]
  44. Runco, M.A.; Acar, S. Divergent thinking as an indicator of creative potential. Creat. Res. J. 2012, 24, 66–75. [Google Scholar] [CrossRef]
  45. Gratz, E.; Looney, L. Faculty resistance to change: An examination of motivators and barriers to teaching online in higher education. Int. J. Online Pedagog. Course Des. (IJOPCD) 2020, 10, 1–14. [Google Scholar] [CrossRef]
  46. Meyer, A.; Hartung-Beck, V.; Gronostaj, A.; Krüger, S.; Richter, D. How can principal leadership practices promote teacher collaboration and organizational change? A longitudinal multiple case study of three school improvement initiatives. J. Educ. Chang. 2022, 24, 425–455. [Google Scholar] [CrossRef]
Education 13 01102 g001
Figure 1. Question from Qualtrics survey eliciting social network data.
Figure 1. Question from Qualtrics survey eliciting social network data.
Education 13 01102 g001
Education 13 01102 g002
Figure 2. There are two types of networks referenced in this work: (a) An ego network, displaying only an individual and their direct connections. (b) A social network, displaying the interconnections of an entire community. Ego network (a) is highlighted in blue. In each type of network, the colored dots represent instructors who use the innovation in some capacity, while the lines represent working relationships between instructors.
Figure 2. There are two types of networks referenced in this work: (a) An ego network, displaying only an individual and their direct connections. (b) A social network, displaying the interconnections of an entire community. Ego network (a) is highlighted in blue. In each type of network, the colored dots represent instructors who use the innovation in some capacity, while the lines represent working relationships between instructors.
Education 13 01102 g002
Education 13 01102 g003
Figure 3. KEEN social network. The colored dots represent instructors who use entrepreneurial mindset in some capacity, while the lines represent working relationships between instructors. Color coding and numerical indicators define Leiden groups (from cluster/clique analysis). Sizes indicate how frequently each instructor uses entrepreneurial mindset in their classroom (self-report, scale of 1–5). Note that the smallest node size indicates non-response—this instructor was mentioned on a survey, or located through publicly available data, but did not respond to the survey and so could not provide a frequency rating.
Figure 3. KEEN social network. The colored dots represent instructors who use entrepreneurial mindset in some capacity, while the lines represent working relationships between instructors. Color coding and numerical indicators define Leiden groups (from cluster/clique analysis). Sizes indicate how frequently each instructor uses entrepreneurial mindset in their classroom (self-report, scale of 1–5). Note that the smallest node size indicates non-response—this instructor was mentioned on a survey, or located through publicly available data, but did not respond to the survey and so could not provide a frequency rating.
Education 13 01102 g003
Education 13 01102 g004
Figure 4. CPREE social network. The colored dots represent instructors who use entrepreneurial mindset in some capacity, while the lines represent working relationships between instructors. Color coding and numerical indicators define Leiden groups (from cluster/clique analysis), and sizes indicate how frequently each instructor uses reflection in their classroom (self-report, scale of 1–5). Note that the smallest node size indicates non-response—this instructor was mentioned on a survey, or located through publicly available data, but did not respond to the survey and so could not provide a frequency rating.
Figure 4. CPREE social network. The colored dots represent instructors who use entrepreneurial mindset in some capacity, while the lines represent working relationships between instructors. Color coding and numerical indicators define Leiden groups (from cluster/clique analysis), and sizes indicate how frequently each instructor uses reflection in their classroom (self-report, scale of 1–5). Note that the smallest node size indicates non-response—this instructor was mentioned on a survey, or located through publicly available data, but did not respond to the survey and so could not provide a frequency rating.
Education 13 01102 g004
Table 1. Survey questions used in social network generation.
Table 1. Survey questions used in social network generation.
Survey ItemResponse Options
How often do you make use of assignments relating to [innovation] in your classroom(s)?Never/Rarely/Sometimes/Often/Always
We would appreciate the name of anyone with whom you have discussed/learned about [innovation]Open-ended response
How did you collaborate with [collaborator]They taught me about [innovation]/I taught them about [innovation]/We collaborated on [innovation]-related pedagogy/Other
Table 2. Leiden algorithm community detection results.
Table 2. Leiden algorithm community detection results.
Kern Entrepreneurial Engineering Network (KEEN) (n = 58)Consortium to Promote Reflection in Engineering Education (CPREE) (n = 131)
Sub-Groups815
Modularity0.720.72
Large Groups (3+ members)710
Dyads (2 members)25
Members in Largest Sub-Group1031
Members in Smallest Sub-Group54
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Riley, D.; Mallouk, K.; Faber, C.; Coso Strong, A. Adoption of Pedagogical Innovations: Social Networks of Engineering Education Guilds. Educ. Sci. 2023, 13, 1102. https://doi.org/10.3390/educsci13111102

AMA Style

Riley D, Mallouk K, Faber C, Coso Strong A. Adoption of Pedagogical Innovations: Social Networks of Engineering Education Guilds. Education Sciences. 2023; 13(11):1102. https://doi.org/10.3390/educsci13111102

Chicago/Turabian Style

Riley, Darby, Kaitlin Mallouk, Courtney Faber, and Alexandra Coso Strong. 2023. "Adoption of Pedagogical Innovations: Social Networks of Engineering Education Guilds" Education Sciences 13, no. 11: 1102. https://doi.org/10.3390/educsci13111102

APA Style

Riley, D., Mallouk, K., Faber, C., & Coso Strong, A. (2023). Adoption of Pedagogical Innovations: Social Networks of Engineering Education Guilds. Education Sciences, 13(11), 1102. https://doi.org/10.3390/educsci13111102

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop