Next Article in Journal
Children’s Right to Belong?—The Psychosocial Impact of Pedagogy and Peer Interaction on Minority Ethnic Children’s Negotiation of Academic and Social Identities in School
Previous Article in Journal
Facilitating Student Understanding through Incorporating Digital Images and 3D-Printed Models in a Human Anatomy Course
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Education Sciences
Volume 11
Issue 8
10.3390/educsci11080382
education-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Macrosystems EDDIE Teaching Modules Increase Students’ Ability to Define, Interpret, and Apply Concepts in Macrosystems Ecology

by
Alexandria G. Hounshell
1,*,
Kaitlin J. Farrell
2 and
Cayelan C. Carey
1
1
Department of Biological Sciences, Virginia Tech, 926 West Campus Drive, Blacksburg, VA 24061, USA
2
Odum School of Ecology, University of Georgia, 140 E. Green Street, Athens, GA 30602, USA
*
Author to whom correspondence should be addressed.
Educ. Sci. 2021, 11(8), 382; https://doi.org/10.3390/educsci11080382
Submission received: 18 June 2021 / Revised: 21 July 2021 / Accepted: 22 July 2021 / Published: 26 July 2021
(This article belongs to the Section Higher Education)
Browse Figures
Versions Notes

Abstract

:
Ecologists are increasingly using macrosystems approaches to understand population, community, and ecosystem dynamics across interconnected spatial and temporal scales. Consequently, integrating macrosystems skills, including simulation modeling and sensor data analysis, into undergraduate and graduate curricula is needed to train future environmental biologists. Through the Macrosystems EDDIE (Environmental Data-Driven Inquiry and Exploration) program, we developed four teaching modules to introduce macrosystems ecology to ecology and biology students. Modules combine high-frequency sensor data from GLEON (Global Lake Ecological Observatory Network) and NEON (National Ecological Observatory Network) sites with ecosystem simulation models. Pre- and post-module assessments of 319 students across 24 classrooms indicate that hands-on, inquiry-based modules increase students’ understanding of macrosystems ecology, including complex processes that occur across multiple spatial and temporal scales. Following module use, students were more likely to correctly define macrosystems concepts, interpret complex data visualizations and apply macrosystems approaches in new contexts. In addition, there was an increase in student’s self-perceived proficiency and confidence using both long-term and high-frequency data; key macrosystems ecology techniques. Our results suggest that integrating short (1–3 h) macrosystems activities into ecology courses can improve students’ ability to interpret complex and non-linear ecological processes. In addition, our study serves as one of the first documented instances for directly incorporating concepts in macrosystems ecology into undergraduate and graduate ecology and biology curricula.

1. Introduction

Over the past decade, macrosystems ecology has emerged as an important subdiscipline of ecology and environmental biology that examines how populations, communities, and ecosystem processes interact across multiple spatial and temporal scales [1,2]. Conducting macrosystems ecology research requires managing, visualizing, and analyzing heterogeneous, complex datasets in addition to using advanced computational and modeling techniques e.g., [3,4,5]. As macrosystems concepts and approaches become increasingly integrated into ecological research, ecology training programs need to evolve to prepare students to conduct macrosystems-scale science [6]. However, many skills commonly used to conduct macrosystems research, including analyzing long-term and high-frequency time series data and using ecosystem simulation models, are rarely taught in ecology curricula in the U.S. [7,8].
As the field of macrosystems ecology has grown, three concepts have emerged as integral to macrosystems-scale research: cross-scale interactions, teleconnections, and macro-scale feedbacks [1,5]. Cross-scale interactions are defined as processes that occur at one scale (e.g., global) that interact with processes at another scale (e.g., local or regional; [9]). Teleconnections are phenomena that link remote regions via cause-and-effect relationships, such as the impacts of El Niño and other global climate patterns on ecosystems worldwide [10]. Finally, macro-scale feedbacks refer to processes occurring at different scales that can either amplify or diminish each other [1]. These three processes can drive non-linear ecological dynamics, highlighting the importance of incorporating macrosystems processes into our understanding of how ecological systems may respond to anthropogenic forcing at different temporal and spatial scales e.g., [5,9].
Identifying macrosystems patterns and processes at local, regional, and continental scales often requires collecting and analyzing long-term and high-frequency data for multiple ecological variables. When collected across regions and continents, in situ sensor data collected on temporal scales from minutes to decades can be used to both distinguish spatial and temporal patterns and identify local, context-dependent ecological processes e.g., [11,12]. Research networks such as the National Ecological Observatory Network (NEON) and the Global Lake Ecological Observatory Network (GLEON), have facilitated the collection and use of sensor data across broad temporal and spatial scales [13,14], which is enabling rapid advances in the field of macrosystems ecology, e.g., [15,16,17,18,19].
Researchers can use NEON and GLEON data to develop and run ecosystem models to better understand macrosystems phenomena, as well as explore the effects of complex anthropogenic forcing, including climate and land-use change that occur across multiple temporal and spatial scales [20,21,22]. Ecosystem simulation models, which use linked equations to represent interacting ecosystem processes [23], can improve our understanding of ecological interactions within and among ecosystems and can be used to make predictions about how ecosystems may change in the future, e.g., [20,24]. For example, Thomas et al. (2017) analyzed 35 years of eddy-covariance sensor and other data to model both local and regional forest responses to drought, elevated carbon dioxide, and higher nutrient conditions expected in the future [22]. Studies such as these highlight the utility of pairing ecosystem simulation models with long-term and high-frequency sensor data to understand ecological patterns and processes across multiple spatial and temporal scales.
Despite the growth of multi-scale and data-intensive ecological research sensu [25], macrosystems ecology concepts and approaches are rarely taught in ecology and biology courses [7,26,27,28]. This gap may be due in part to the challenges associated with teaching complex concepts such as scale [26,28], which in ecology is generally defined as both the spatial and temporal dimension of a process [29,30]. Consequently, ecology and biology students may be missing key training in concepts and approaches that they will need to tackle complex environmental challenges in their future careers [6,25,31]. In addition, ecosystem responses to human activities are complex, non-linear, and driven by feedbacks that span local, regional, and continental scales, requiring managers and policymakers to make challenging decisions about how best to sustain ecosystem services [32]. Understanding how to apply macrosystems concepts to inform ecosystem management under multiple climate and land use scenarios requires training that integrates data management and analysis, ecosystem modeling, decision support, and team science [25,33,34]; skills that all students need, regardless of their future careers.
To address this gap in ecology and biology training, we developed four stand-alone teaching modules as part of the Macrosystems EDDIE (Environmental Data-Driven Inquiry and Exploration) program (Table 1) that can be readily adapted and implemented by instructors. Each module addresses a key concept in macrosystems ecology (i.e., cross-scale interactions, teleconnections, or macro-scale feedbacks) through the lens of freshwater ecology and uses high-frequency and long-term data from NEON and/or GLEON lakes to run ecosystem simulation models in the R software environment [35,36]. Each module simultaneously teaches the core concepts of macrosystems ecology, and the quantitative and computational skills needed to conduct cutting-edge ecological research and can be easily implemented in a wide range of ecology and biology classrooms that otherwise might not cover concepts in macrosystems ecology as part of their curricula.
We assessed the efficacy of the Macrosystems EDDIE modules as tools for teaching macrosystems ecology concepts through the use of pre- and post-module assessments. Our focal question was: What is the effect of integrating hands-on modeling and data analysis activities in ecology courses on students’ ability to define, interpret, and apply concepts in macrosystems ecology? We focused on students’ ability to define, interpret, and apply macrosystems concepts across dimensions of both knowledge and cognitive processes (terms hereafter italicized) [37,38,39,40]. These three processes (define, interpret, apply) are key components of higher order thinking, as defined by different pedagogical frameworks [37,38,39,40]. While we acknowledge that define, interpret, and apply are simplifications of complex cognitive processes, they represent different scaffolded components of learning at a high level. For example, in the initial stages of comprehension, students must first be able to define key concepts in their own words. This builds toward the ability to interpret data and visualizations to understand concepts, and finally, to apply knowledge by using the newly-learned concepts to solve new problems or accomplish new tasks.

2. Materials and Methods

2.1. Macrosystems EDDIE Modules

Each Macrosystems EDDIE module consists of a short (1 to ~3-h), self-contained suite of scaffolded activities that instructors can adapt to meet the needs of their lecture or laboratory class. Macrosystems EDDIE modules teach macrosystems ecology concepts within the four-dimensional ecology education framework (4DEE), as adopted by the Ecological Society of America [41]. While the modules use freshwater ecology examples to teach macrosystems ecology, modules were specifically designed to be broadly applied to general ecology and biology courses, as well as more specialized courses. Thus, each module includes PowerPoint slides that introduce key concepts in limnology, freshwater ecology, and aquatic biology, but can be customized by instructors with additional background knowledge that may be needed for their students. Within each module, pairs of students begin by selecting a NEON or GLEON lake from a list of sites and then complete modeling activities that build from simple (activity A) to more complex (activities B and C). Because of the modules’ flexible structure, instructors can tailor the activities to match their classroom’s needs. For example, instructors of introductory courses may choose to teach module activities A and B in a class and then assign activity C as homework or omit certain activities, while instructors of more advanced courses may choose to assign activity A for pre-class homework and complete activities B and C in class. Modeling activities and associated worksheets and handouts follow the 5Es of learning (engagement, exploration, explanation, expansion, and evaluation) [35,42] and are designed to foster hypothesis-driven exploration of ecosystem model outputs following [43]. Modules generally increased in complexity from Module 1 to Module 4, though none of the modules have pre-requisites and modules can be used in any order as stand-alone activities.
Each module includes: (1) Microsoft PowerPoint slides for instructors to introduce the module’s focal macrosystems ecology concepts and modeling activities, as well as key concepts in freshwater ecology; (2) an editable Microsoft Word student worksheet that prompts students to make hypotheses about different model scenarios, analyze model results, and determine whether their hypotheses were supported; and (3) an annotated R script that guides students through exercises using pre-packaged ecosystem models configured for each lake ecosystem. All modules were developed assuming no prior R or modeling experience for either students or instructors and can be accessed via the Macrosystems EDDIE website (www.MacrosystemsEDDIE.org; accessed on 23 July 2021) or the Environmental Data Initiative (EDI) repository [44,45,46,47].

2.2. Module Implementation and Assessment

We assessed the effectiveness of the four Macrosystems EDDIE modules in 23 classrooms at 17 colleges and universities and one conference workshop (see Supplementary Material, Table S1). Modules were primarily taught in undergraduate classrooms in freshwater/aquatic ecology or biology courses (n = 11), with other courses ranging from introductory environmental science to environmental informatics and global change ecology. Due to the wide range in classroom size (ranging from 3 to 39 students), course type, and student and instructor experience levels (see Supplementary Material, Table S1), analyses were conducted on the overall pool of students (n = 319) and were not separated by institution or classroom. Since we include students from mixed-enrollment (undergraduate plus graduate) and graduate classrooms in our dataset and analysis and are unable to separate their anonymized responses, we focus the discussion on biology and ecology curricula broadly and do not specifically distinguish between undergraduate or graduate responses. Each classroom taught a single module (Module 1, 2, 3, or 4) during a semester, except for three classrooms that taught two modules during a semester (see Supplementary Material, Table S1). For the three classrooms that taught two modules, we focused on data from the first module taught to be consistent with the other single-module classrooms.
Pre- and post-module assessments were administered within 2 weeks before and after module instruction, respectively, to evaluate changes in the students’ ability to define, interpret, and apply macrosystems ecology concepts. Pre- and post-module assessment questions were identical to each other and had a similar structure among the four modules. The assessments included both self-assessment questions designed to evaluate students’ self-perceived knowledge of macrosystems ecology and quantitative assessments designed to evaluate students’ knowledge of macrosystems ecology by defining key words, interpreting complex data visualizations, and applying concepts in macrosystems ecology to new problems.
The pre- and post-module assessments included four types of questions to assess students’ growth. All pre- and post-module assessments were administered online, with no time limit or restrictions on materials that could be used to answer the questions. Assessments were not graded but were used by some course instructors for extra credit if the student completed both pre- and post-module assessments. First, we used multiple-choice questions where students ranked their perceived proficiency and confidence using high-frequency and long-term data and their self-perceived knowledge of macrosystems ecology on a Likert-type scale from 1 (low, “Not at all familiar”) to 5 (high, “Extremely familiar”) (see Supplementary Material, Table S2). Second, free-response questions were designed to assess students’ ability to define macrosystems ecology concepts based on the module they were completing (see Supplementary Material, Table S3). Third, multiple-choice questions evaluated students’ ability to interpret data visualizations from model output related to the module’s focal concepts (see Supplementary Material, Table S4). Finally, a free-response question evaluated students’ ability to apply macrosystems ecology concepts to describe the approach they would use to predict lake ecosystem responses to climate and/or land-use change in the year 2099 (see Supplementary Material, Table S5). The latter three types of questions enabled us to move beyond self-perceptions of knowledge to quantify students’ growth in response to the modules.
We analyzed the pre- and post-module assessments using standard methods [48,49]. For the define and apply questions, we used a standardized coding method [48] to classify qualitative responses and Wilcoxon signed-rank tests to compare coded pre- and post-module responses (Appendix A). We also used Wilcoxon signed-rank tests to compare pre- and post-module responses for the self-assessment and interpret questions (Appendix A). Three interpret questions were excluded from analysis due to the ‘ceiling effect’ (correct pre-module responses >85%), which limits opportunity for evaluating growth [49,50] but are reported in the supplementary information (Figure S1). Statistical significance was defined a priori as α = 0.05 using p-values based on a normal approximation. For each metric (proficiency, confidence) and assessment question (high-frequency data; long-term data), effect sizes were calculated as Z/√n. All students consented to participate in the study per our Institutional Review Board (IRB) protocol (Carleton College IRB #0002470). Additional details on module implementation and quantitative assessment are included in Appendix A.
There was a wide range in the number of student responses per question (see Supplementary Material, Table S7), and the assessment questions varied among modules. A total of 319 students who consented to participate in the study completed some portion of either the pre- or post-module assessment. For our analyses, however, a student needed to complete both the pre- and post-module response for a specific question to be included in the analysis. In addition, we asked instructors to teach modules that best aligned with their course; as such, the total number of classrooms testing each module varied (see Supplementary Material, Table S1). Thus, given differences in student response rates to individual questions and the numbers of students completing each module, the total number of students used in the analysis of each question varied (see Supplementary Material, Table S7). Second, the types of questions included in the assessments for each module (Module 1–4) varied slightly. While we generally sought to keep assessment questions consistent among modules, some changes were made in subsequent modules as we learned from ongoing module testing. By aggregating results across modules, we feel our analyses are applicable to the overall module goals and to a range of classroom sizes, course levels, and institutions.

3. Results

3.1. Increased Familiarity with Macrosystems Ecology and Greater Confidence Using Data

Based on the self-assessment questions, students’ median self-reported knowledge of macrosystems ecology significantly increased after completing a Macrosystems EDDIE module (p < 0.001, effect size = 0.66; Figure 1, see Supplementary Material, Table S8). On average, students described their pre-module knowledge of macrosystems ecology as ‘Slightly familiar, I have heard of macrosystems ecology, but cannot elaborate’ (median Likert score of 2), and their post-module knowledge as ‘Somewhat familiar, I could explain a little about macrosystems ecology’ (median Likert score of 3). There was also a significant increase in students’ self-reported confidence and proficiency using both high-frequency and long-term data after completing a module (p < 0.001, effect size = 0.60 and 0.58, respectively; Figure 2, see Supplementary Material, Table S8). Students’ median self-reported proficiency and confidence working with long-term data was higher than for high-frequency data both before and after module use, but students exhibited greater gains in their proficiency and confidence working with high-frequency data after module completion (see Supplementary Material, Table S8).

3.2. Hypothesis-Driven, Hands-On Activities Promote Macrosystems Ecology Learning

In addition to comparing students’ self-perceived knowledge of macrosystems ecology, as reported above, we also directly evaluated students’ ability to define, interpret, and apply key concepts in macrosystems ecology before and after module use. Overall, the proportion of students able to correctly define “macrosystems ecology” significantly increased (p = 0.03, effect size = 0.22) from 7.4% to 16.8% after completing a module (Figure 3, see Supplementary Material, Table S9). Moreover, significantly more students also correctly defined “cross-scale interactions” and “teleconnections” (p = 0.004, effect size = 0.29 and p = 0.02, effect size = 0.21, respectively) following the completion of modules on these topics. The proportion of students able to correctly define macro-scale feedbacks also increased, from 11.9 to 19.1%, however, this increase was not statistically significant (p = 0.35, effect size = 0.14). These define questions specifically sought to capture students′ understanding of the multiple interacting scales across which ecological processes occur (see Supplementary Material, Table S3). Specifically, for the macrosystems ecology definition question, there was a significant increase post-module in student responses noting that macrosystems ecology inherently contains processes that occur across multiple temporal and/or spatial scales (10.5% to 23.2%, Figure S2).
A second skill necessary to conduct macrosystems ecology is the ability to create and interpret data visualizations [25], which is also an important component of several pedagogical frameworks used to commonly assess student comprehension e.g., [37,38,39,40]. All students completing Macrosystems EDDIE modules create multiple visualizations in R to interpret their ecosystem simulation model output and assess whether their initial hypotheses were supported. After completing a module, there was generally an increase in students’ ability to correctly interpret data to answer questions about macrosystems ecology-related concepts (Figure 4, see Supplementary Material, Table S10, Figure S1), though only one question showed a statistically significant increase. Below, we offer several reasons potentially driving these results.
First, the questions removed due to the ceiling effect (>85% correct pre-response questions) suggest that students might already have been familiar with interpreting graphical output prior to module use, indicating that students were able to appropriately reason correct responses based on a given graphical output without prior knowledge of the concepts (i.e., concepts in macrosystems ecology). We thus speculate that the questions removed due to the ceiling effect may have been testing students’ ability to interpret graphical output, as opposed to being able to interpret new macrosystems ecology concepts represented. We also acknowledge that some of the interpret questions may have required knowledge of freshwater ecology concepts that some students may not have been familiar with prior to completing a Macrosystems EDDIE module (i.e., in an introductory ecology course). Thus, gains post-module could have resulted in part from an increased knowledge of concepts in freshwater ecology and not necessarily from an increased understanding of macrosystems ecology concepts (e.g., Module 1, Question 1). Finally, we recognize that gains in student comprehension following module use could be attributed to students’ ability to use logical reasoning to arrive at the correct answer (i.e., process of elimination; interpretation of graphical output, etc.) rather than an increased comprehension of macrosystems ecology. While we acknowledge these caveats, we nonetheless think these questions represented an important component of our pre- and post-module analysis to assess whether there was evidence for increased comprehension of macrosystems ecology concepts following module use, especially given the inherent challenges of assessing student growth across a range of diverse classrooms and experience levels.
Finally, students were significantly more likely to apply macrosystems approaches to describe how they would predict changing ecosystem responses to future land use and/or climate change after completing a Macrosystems EDDIE module. The responses to this question suggest a greater understanding of the complexity of ecosystem responses and the need for more sophisticated computational and modeling techniques to predict future changes. For example, significantly more students described the use of simulations, models, and/or forecasts in their responses (‘model’; p < 0.001, effect size = 0.48; Figure 5, see Supplementary Material, Table S11), while statistically fewer student responses included the use of “trend lines” (‘trend’; p < 0.001, effect size = 0.25; Figure 5, see Supplementary Material, Table S11). The increase in student responses that described using ecosystem models suggests an increased understanding of the complexity associated with predicting ecosystem processes and the need to use more sophisticated techniques (e.g., simulation modeling) over more simplistic linear trends and empirical models, which can fail to account for the complexity of ecosystem responses. Similarly, there was a significant increase in the percent of student responses that described assessing multiple lakes or multiple land-use and/or climate change scenarios (‘many’; p < 0.001, effect size = 0.36; Figure 5, see Supplementary Material, Table S11), versus just one ecosystem or scenario. However, there was no change in the percent of responses that included ‘data’ (p = 0.61, effect size = 0.03), indicating many students already recognized the importance of collecting data to answer questions in macrosystems ecology prior to module use.

4. Discussion

Taken together, results from the define, interpret, and apply assessment questions demonstrate that short (1–3 h) inquiry-based, hypothesis-driven, ecosystem modeling exercises, such as those in the Macrosystems EDDIE modules, can help to introduce ecology and biology students to macrosystems ecology. In each Macrosystems EDDIE module, students are introduced to ecosystem simulation modeling, which enables students to develop and test their own hypotheses about ecological phenomena using model scenarios. By explicitly asking students if their hypotheses are supported by their modeling results as part of the module activities, students are encouraged to critically examine their own understanding of macrosystems ecology. Considering that only a few students in our study had been introduced to macrosystems ecology prior to our study—based on discussions with instructors and the low proportion of correct definitions in the pre-module assessments (Figure 3)—the significant growth in student understanding we observed provides support for a short, self-contained, module-based approach to integrate complex macrosystems ecology concepts into undergraduate curricula.
To the best of our knowledge, Macrosystems EDDIE is one of the first formalized educational programs to incorporate macrosystems ecology concepts directly into ecology and biology curricula. Our study indicates that pre-packaged modules such as those in the Macrosystems EDDIE program may be a relatively easy and efficient way for instructors to formally incorporate new concepts into their classrooms, compared to developing their own lectures and/or lab materials. Due to the flexible nature of the modules, all three module activities can be completed within one 3–4 h lab period or three 60-min lecture periods, depending on student experience level and course structure. In addition, modules were developed assuming no prior knowledge of R or ecosystem modeling for either the instructors or students, allowing implementation in a variety of classroom settings and skill levels. While multiple modules can be incorporated within curricula, our assessment results demonstrate that even single-module use leads to significant gains in student comprehension of macrosystems ecology in addition to increases in students’ self-perceived knowledge and confidence using high-frequency datasets. Our module assessments included students from mixed-enrollment (undergraduate plus graduate) or graduate classrooms, although the majority of our respondents were enrolled in undergraduate-only classes, and previous work on other modeling-based modules suggests undergraduates may exhibit greater learning gains than graduate students through such activities [51].
Multiple recent studies have highlighted the challenges of teaching concepts and processes that span multiple spatial and/or temporal scales in undergraduate science courses [26,28], a key component of the sub-discipline of macrosystems ecology. Here, we demonstrate how hands-on activities embedded into ready-to-use teaching modules can be used to successfully teach students ecological concepts that span multiple temporal and spatial scales. Similar to this study, Styers et al. (2021) observed gains in students’ understanding of cross-scale biodiversity processes following the use of hands-on teaching modules that analyzed NEON data collected across multiple spatial scales [28]. Taken together, our study and Styers et al. (2021) highlight the utility of hands-on activities that analyze real environmental data to teach students complex concepts spanning multiple scales, an important component of macrosystems ecology [28].

Module Limitations and Potential Improvements

The student assessments highlight potential improvements for future module use and assessments. First, we note that there was a wide range in the number of paired pre- and post-assessment responses across modules due to a varying number of classrooms that completed the different modules, in addition to a wide range in student experience level and institution type, which likely impacted our results. While questions were specific to the macrosystems ecology concepts contained in each respective module (i.e., cross-scale interactions, teleconnections, or macro-scale feedbacks), the complexity and structure of the questions were designed to be comparable across modules (Tables S3–S5). By aggregating responses across modules, we feel that the results from this study can be broadly applied to a wide range of institution types, classroom sizes, and student experience levels, and are broadly applicable across all modules. While this assessment design allowed for comparisons across modules, future improvements could be made to ensure similar classroom sizes and student response rates across all modules for better comparisons, as well as recruiting a more representative distribution of institutions.
Similarly, we were unable to parse out individual classroom experiences in our analysis, as each classroom varied across multiple axes including institution type, classroom level, instructor experience, classroom size, and course focus/goals. Thus, we did not have enough statistical power to compare individual classrooms. Future studies should focus on recruiting classrooms that may be similar on one metric (i.e., size) but differ on another (i.e., classroom level) to determine differences in module effectiveness among classrooms. More granular studies that parse the effectiveness of module elements based on student- and classroom-specific characteristics could highlight ways that module activities could be customized for different teaching contexts. Finally, a longer-term study is needed to quantify students’ lasting growth in comprehension and skills, especially in courses that use multiple modules throughout a semester. While our results suggest short-term gains in student comprehension following single-module use, we did not include longer (>2-week) post-module assessments to assess longer-term (months to years) student comprehension.
In addition to the improvements for module assessment, student assessments and informal instructor feedback also highlighted potential module improvements. Specifically, the process-based model used in the four Macrosystems EDDIE modules can effectively demonstrate the non-linear, interacting processes that occur in lake ecosystems through the simulation of multiple ecosystem variables, represented by linked equations [21]. However, because of the model’s complexity, students are unlikely to fully understand its mechanistic underpinnings during a 1–3 h module. A simpler ecosystem model that allows students to understand and manipulate individual model components would enable additional engagement and interaction, as highlighted by several instructors during informal post-module reflections.

5. Conclusions

Here, we demonstrate the effectiveness of Macrosystems EDDIE modules to teach undergraduate students’ concepts in macrosystems ecology while also increasing students’ self-reported confidence and proficiency using high-frequency and long-term data. Importantly, the skills (i.e., high-frequency and long-term data analysis; ecosystem modeling) gained during module use can be broadly applied to fields outside of ecology and can enhance students’ abilities to use large data sets in many emerging fields and career paths both within and beyond STEM. In addition, incorporating hands-on activities into undergraduate and graduate classrooms has been shown to lead to increased gains in student comprehension [52]. Moreover, an increasing number of studies have highlighted the importance of incorporating active learning into traditional, lecture-based courses to help close the achievement gap between historically underrepresented and historically overrepresented students, e.g., [53,54,55].
Importantly, our work highlights one of the first sets of teaching modules designed to integrate macrosystems ecology concepts and approaches into ecology and biology classrooms. As highlighted by recent studies [26,28], students need a greater understanding of complex and non-linear ecological interactions that occur across multiple temporal and spatial scales. While teaching concepts in ecology that span multiple scales can be challenging, students’ greater understanding of the complexity of ecological systems after completing a module suggests that short-term (1–3 h) modeling activities, such as those embedded in each Macrosystems EDDIE module, can be an effective approach for helping to prepare students to tackle global environmental challenges that occur across multiple temporal and spatial scales.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/educsci11080382/s1, Table S1: Description of classrooms assessed; Table S2: Likert-type scales for student pre- and post-module self-assessment questions; Table S3: Define questions for pre- and post-module responses and coding criteria; Table S4: Interpret questions for pre- and post-module responses for the four modules; Table S5: Questions for the apply assessments; Table S6: Coding criteria for apply questions; Table S7: Number of paired student responses for each module and each assessment question; Table S8. Statistical results for student self-reported proficiency and confidence using high-frequency and long-term data and their self-reported knowledge of macrosystems ecology; Table S9: Statistical results for define questions; Table S10: Statistical results for interpret questions; Table S11: Statistical results for apply questions; Figure S1: Results from interpret questions including those removed from analysis due to the ceiling effect; Figure S2: Percent of students for the define questions that included elements of “ecology”, “effect”, and “scale” thematic bins.

Author Contributions

Conceptualization, C.C.C.; methodology, C.C.C., K.J.F., and A.G.H.; module development, C.C.C., K.J.F., and A.G.H.; module testing, C.C.C., K.J.F., and A.G.H.; formal analysis, C.C.C., K.J.F., and A.G.H.; data curation, C.C.C., K.J.F., and A.G.H.; writing—original draft preparation, A.G.H.; writing—review and editing, C.C.C., K.J.F., and A.G.H.; visualization, K.J.F., C.C.C., and A.G.H.; supervision, C.C.C.; project administration, C.C.C., K.J.F., and A.G.H.; funding acquisition, C.C.C. All authors have read and agreed to the published version of the manuscript.

Funding

Macrosystems EDDIE is supported by the National Science Foundation (NSF) DEB-1702506 and DEB-1926050, and leverages additional support from CNS-1737424, DEB-1753639, DBI-1933016, and DBI-1933102.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board (IRB) of Carleton College (IRB # 0002470, 29 August 2017).

Informed Consent Statement

Informed, voluntary consent was obtained from all subjects involved in the study.

Data Availability Statement

All teaching module materials are available in the Environmental Data Initiative (EDI) repository (Carey et al. 2018; Carey and Farrell, 2019; Farrell and Carey, 2019; Carey et al. 2020). The human subjects assessment data presented in this manuscript cannot be publicly archived, per our Institutional Review Board protocol.

Acknowledgments

We thank all of the instructors and students who enthusiastically tested Macrosystems EDDIE modules. We thank Kristin O’Connell from the Science Education Research Center (SERC) of Carleton College for assistance with qualitative data analyses as well as Ellen Iverson, Cailin Huyck Orr, Monica Bruckner, and Sean Fox from SERC for providing Macrosystems EDDIE programmatic and web support. We also thank members of the Reservoir Group at Virginia Tech for helpful comments to improve the manuscript. In addition, we thank two anonymous reviewers and the handling editor for their feedback which greatly improve the manuscript.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Appendix A

Appendix A.1. Assessment Description

First, the pre- and post-module assessments included multiple choice questions ranking students’ perceived proficiency and confidence using high-frequency and long-term data on a Likert-type scale from low (1) to high (5) and self-reported knowledge of macrosystems ecology, from ‘not at all familiar’ (1) to ‘extremely familiar’ (5) (see Supplementary Material, Table S2). Such self-assessments are valuable tools for students to gauge their own understanding and how it changes over time, as well as motivate further learning [56].
Next, students completed free-response questions designed to assess their ability to define macrosystems ecology concepts based on the module they were completing; specifically, cross-scale interactions and macrosystems ecology (Module 2), teleconnections (Module 3), and macro-scale feedbacks (Module 4, see Supplementary Material, Table S3). Module 1 did not include a define question as this module did not focus on a specific concept in macrosystems ecology but instead was designed to provide a broad introduction to the modeling and computational techniques used in macrosystems ecology. In addition, we removed the define macrosystems ecology assessment question from subsequent modules (Modules 3 and 4) to limit survey fatigue and the keep the assessments <10 min.
Following the define questions, a series of multiple-choice questions evaluated students’ ability to interpret data visualizations. The graphs and corresponding data interpretation questions were developed to represent a similar level of complexity across modules (see Supplementary Material, Table S4). These questions were designed to assess students’ ability to interpret ecosystem simulation model output as related to the module’s focal concepts. The data interpretation questions within each module were scaffolded to increase in difficulty from question 1 (least difficult) to question 3 (most difficult). We excluded questions from analysis in which >85% students answered the pre-module assessment correctly because of the limited ability of those questions to evaluate students’ learning (by the presence of the ‘ceiling effect,’ following [49,50]).
Finally, students completed a free-response question that evaluated students’ ability to apply macrosystems ecology concepts, including the use of ecological simulation models and high-frequency and long-term data, by asking them to describe how they would predict lake ecosystem responses to climate and/or land-use change in the year 2099 (see Supplementary Material, Table S5). As with the define question, we did not include an apply question for Module 1 as this module did not focus on a specific concept in macrosystems ecology.

Appendix A.2. Student Assessment Analysis

We used Wilcoxon signed-rank tests to compare the mean Likert scores among paired pre- and post-module student responses for their perceived proficiency and confidence using high-frequency and long-term data, and for self-reported knowledge of macrosystems ecology following single module use. Responses were aggregated across modules and only included paired pre- and post-module student responses for a given question. Statistical significance was defined a priori as α = 0.05 using p-values based on a normal approximation. For each metric (proficiency, confidence) and assessment question (high-frequency data; long-term data), effect sizes were calculated as Z/√n.
For data interpretation (interpret) questions, the percentage of students who answered each question correctly was compared in the pre- and post-module assessments and analyzed for significance using Wilcoxon signed rank-tests.
The qualitative responses for the define and apply free-response questions were coded in two steps using a provisional coding method [48]. The first step identified themes in student responses that were used to develop consistent coding criteria that could be applied across responses for each individual question. The second step applied the coding criteria to all pre- and post-module student responses. This process resulted in a database of student responses that were consistently coded for the presence or absence of identified themes (details below). For the define and apply questions, pre- and post-assessment responses were aggregated and randomized prior to coding
During step one of coding, two Macrosystems EDDIE coordinators preliminarily reviewed student responses to each assessment question and noted emerging themes and keywords to develop an initial codebook. After developing the initial codebook, a set of ~10% of the pre- and post-module responses for each question were randomly selected. This set of responses was then coded independently by two Macrosystems EDDIE coordinators for each question. Coding responses were then compared, coding disagreements resolved, and the codebook further refined to best characterize student responses.
During step two, student responses for each individual module and question were divided among and coded by two Macrosystems EDDIE coordinators using the refined codebook. As during the codebook development, all identifying student, course, and survey timing (pre- or post-module) information was hidden and all responses were randomly ordered based on a random number generator. This two-phase process resulted in coded student responses for each question of presence or absence for each thematic bin. Question responses that were left blank or included ‘I don’t know’ were excluded from further analysis.
For the define questions (Modules 2, 3, 4), coding was separated into thematic bins by module and question (see Supplementary Material, Table S3). ‘Correct’ student answers included elements from each thematic bin. All other answers were coded as ‘incorrect.’ Wilcoxon signed-rank tests were then used to compare the percentage of correct answers pre- and post-module. We further divided responses by each thematic bin (i.e., ‘eco’ (ecology), ‘effect’, ‘scale’ for macrosystems ecology definition; see Supplementary Material, Table S3) which each student answer included, allowing us to assess if students’ answers increased in complexity pre- and post-module use. For the apply questions (Modules 2, 3, 4), responses were aggregated and coded across all modules using a common set of thematic bins (see Supplementary Material, Table S6). Wilcoxon signed-rank tests were used to compare the percentage of responses that included each thematic bin in pre- and post-module responses. For the define, interpret, and apply questions, results include students who completed either one or two modules. All statistical analyses were conducted in R 4.0.3 [57].

References

  1. Heffernan, J.B.; Soranno, P.A.; Angilletta, M.J.; Buckley, L.B.; Gruner, D.S.; Keitt, T.H.; Kellner, J.R.; Kominoski, J.S.; Rocha, A.V.; Xiao, J.; et al. Macrosystems ecology: Understanding ecological patterns and processes at continental scales. Front. Ecol. Environ. 2014, 12, 5–14. [Google Scholar] [CrossRef] [Green Version]
  2. Fei, S.; Guo, Q.; Potter, K. Macrosystems ecology: Novel methods and new understanding of multi-scale patterns and processes. Landsc. Ecol. 2016, 31, 1–6. [Google Scholar] [CrossRef] [Green Version]
  3. Dahlin, K.M.; Ault, T.R. Global linkages between teleconnection patterns and the terrestrial biosphere. Int. J. Appl. Earth Obs. Geoinf. 2018, 69, 56–63. [Google Scholar] [CrossRef]
  4. Zarnetske, P.L.; Read, Q.D.; Record, S.; Gaddis, K.D.; Pau, S.; Hobi, M.L.; Malone, S.L.; Costanza, J.M.; Dahlin, K.; Latimer, A.M.; et al. Towards connecting biodiversity and geodiversity across scales with satellite remote sensing. Glob. Ecol. Biogeogr. 2019, 28, 548–556. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Soranno, P.A.; Wagner, T.; Collins, S.M.; Lapierre, J.; Lottig, N.R.; Oliver, S.K. Spatial and temporal variation of ecosystem properties at macroscales. Ecol. Lett. 2019, 22, 1587–1598. [Google Scholar] [CrossRef]
  6. Farrell, K.J.; Weathers, K.C.; Sparks, S.H.; Brentrup, J.A.; Carey, C.C.; Dietz, M.C.; Foster, J.R.; Grayson, K.L.; Matthes, J.H.; SanClements, M.D. Training macrosystems scientists requires both interpersonal and technical skills. Front. Ecol. Environ. 2021, 19, 39–46. [Google Scholar] [CrossRef]
  7. Weathers, K.C.; Groffman, P.M.; Van Dolah, E.; Bernhardt, E.; Grimm, N.B.; McMahon, K.; Schimel, J.; Paolisso, M.; Maranger, R.; Baer, S.; et al. Frontiers in Ecosystem Ecology from a Community Perspective: The Future is Boundless and Bright. Ecosystems 2016, 19, 753–770. [Google Scholar] [CrossRef] [Green Version]
  8. O’Reilly, C.M.; Gougis, R.D.; Klug, J.L.; Carey, C.C.; Richardson, D.C.; Bader, N.E.; Soule, D.C.; Castendyk, D.; Meixner, T.; Stomberg, J.; et al. Using Large Data Sets for Open-Ended Inquiry in Undergraduate Science Classrooms. BioScience 2017, 67, 1052–1061. [Google Scholar] [CrossRef] [Green Version]
  9. Soranno, P.A.; Cheruvelil, K.S.; Bissell, E.G.; Bremigan, M.T.; Downing, J.A.; Fergus, C.E.; Filstrup, C.T.; Henry, E.N.; Lottig, N.R.; Stanley, E.H.; et al. Cross-scale interactions: Quantifying multi-scaled cause–effect relationships in macrosystems. Front. Ecol. Environ. 2014, 12, 65–73. [Google Scholar] [CrossRef] [Green Version]
  10. Kitzberger, T.; Brown, P.M.; Heyerdahl, E.K.; Swetnam, T.W.; Veblen, T.T. Contingent Pacific-Atlantic Ocean influence on multicentury wildfire synchrony over western North America. Proc. Natl. Acad. Sci. USA 2007, 104, 543–548. [Google Scholar] [CrossRef] [Green Version]
  11. Solomon, C.T.; Bruesewitz, D.A.; Richardson, D.C.; Rose, K.C.; Van de Bogert, M.C.; Hanson, P.C.; Kratz, T.K.; Larget, B.; Adrian, R.; Babin, B.L.; et al. Ecosystem respiration: Drivers of daily variability and background respiration in lakes around the globe. Limnol. Oceanogr. 2013, 58, 849–866. [Google Scholar] [CrossRef] [Green Version]
  12. Keenan, T.F.; Migliavacca, M.; Papale, D.; Baldocchi, D.; Reichstein, M.; Torn, M.; Wutzler, T. Widespread inhibition of daytime ecosystem respiration. Nat. Ecol. Evol. 2019, 3, 407–415. [Google Scholar] [CrossRef]
  13. NRC. NEON: Addressing the Nation’s Environmental Challenges; The National Academies Press: Washington, DC, USA, 2004. [Google Scholar]
  14. Weathers, K.C.; Hanson, P.C.; Arzberger, P.; Brentrup, J.; Brookes, J.; Carey, C.C.; Gaiser, E.; Gaiser, E.; Hamilton, D.P.; Hong, G.S.; et al. The Global Lake Ecological Observatory Network (GLEON): The evolution of grassroots network science. Limnol. Oceanogr. Bull. 2013, 22, 71–73. [Google Scholar] [CrossRef]
  15. Hanson, P.C.; Weathers, K.C.; Kratz, T.K. Networked lake science: How the Global Lake Ecological Observatory Network (GLEON) works to understand, predict, and communicate lake ecosystem response to global change. Inland Waters 2016, 6, 543–554. [Google Scholar] [CrossRef]
  16. Li, D.; Stucky, B.J.; Deck, J.; Baiser, B.; Guralnick, R.P. The effect of urbanization on plant phenology depends on regional temperature. Nat. Ecol. Evol. 2019, 3, 1661–1667. [Google Scholar] [CrossRef]
  17. Marcé, R.; George, G.; Buscarinu, P.; Deidda, M.; Dunalska, J.; de Eyto, E.; Flaim, G.; Grossart, H.-P.; Istvanovics, V.; Lenhardt, M.; et al. Automatic High Frequency Monitoring for Improved Lake and Reservoir Management. Environ. Sci. Technol. 2016, 50, 10780–10794. [Google Scholar] [CrossRef] [PubMed]
  18. Weintraub, S.R.; Flores, A.N.; Wieder, W.R.; Sihi, D.; Cagnarini, C.; Gonçalves, D.R.P.; Young, M.H.; Li, L.; Olshansky, Y.; Baatz, R.; et al. Leveraging Environmental Research and Observation Networks to Advance Soil Carbon Science. J. Geophys. Res. Biogeosci. 2019, 124, 1047–1055. [Google Scholar] [CrossRef] [Green Version]
  19. Seyednasrollah, B.; Young, A.M.; Li, X.; Milliman, T.; Ault, T.; Frolking, S.; Friedl, M.; Richardson, A.D. Sensitivity of Deciduous Forest Phenology to Environmental Drivers: Implications for Climate Change Impacts Across North America. Geophys. Res. Lett. 2020, 47. [Google Scholar] [CrossRef]
  20. Becknell, J.M.; Desai, A.R.; Dietze, M.C.; Schultz, C.A.; Starr, G.; Duffy, P.A.; Franklin, J.F.; Pourmokhtarian, A.; Hall, J.; Stoy, P.C.; et al. Assessing Interactions Among Changing Climate, Management, and Disturbance in Forests: A Macrosystems Approach. BioScience 2015, 65, 263–274. [Google Scholar] [CrossRef] [Green Version]
  21. Hipsey, M.R.; Bruce, L.C.; Boon, C.; Busch, B.; Carey, C.C.; Hamilton, D.P.; Hanson, P.C.; Read, J.S.; de Sousa, E.; Weber, M.; et al. A General Lake Model (GLM 3.0) for linking with high-frequency sensor data from the Global Lake Ecological Observatory Network (GLEON). Geosci. Model. Dev. 2019, 12, 473–523. [Google Scholar] [CrossRef] [Green Version]
  22. Thomas, R.Q.; Brooks, E.B.; Jersild, A.L.; Ward, E.J.; Wynne, R.H.; Albaugh, T.J.; Dinon-Aldridge, H.; Burkhart, H.E.; Domec, J.-C.; Fox, T.R.; et al. Leveraging 35 years of Pinus taeda research in the southeastern US to constrain forest carbon cycle predictions: Regional data assimilation using ecosystem experiments. Biogeosciences 2017, 14, 3525–3547. [Google Scholar] [CrossRef] [Green Version]
  23. Cuddington, K.; Fortin, M.J.; Gerber, L.R.; Hastings, A.; Liebhold, A.; O’Connor, M.; Ray, C. Process-based models are required to manage ecological systems in a changing world. Ecosphere 2013, 4, 1–12. [Google Scholar] [CrossRef]
  24. Lauenroth, W.K.; Canham, C.D.; Kinzig, A.P.; Poiani, K.A.; Kemp, W.M.; Running, S.W. Simulation Modeling in Ecosystem Science. In Successes, Limitations, and Frontiers in Ecosystem Science; Pace, M.L., Groffman, P.M., Eds.; Springer: New York, NY, USA, 1998. [Google Scholar]
  25. Hampton, S.E.; Jones, M.B.; Wasser, L.A.; Schildhauer, M.P.; Supp, S.R.; Brun, J.; Hernandez, R.R.; Boettiger, C.; Collins, S.L.; Gross, L.J.; et al. Skills and Knowledge for Data-Intensive Environmental Research. BioScience 2017, 67, 546–557. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Cheek, K.A.; LaDue, N.D.; Shipley, T.F. Learning About Spatial and Temporal Scale: Current Research, Psychological Processes, and Classroom Implications. J. Geosci. Educ. 2017, 65, 455–472. [Google Scholar] [CrossRef] [Green Version]
  27. Michener, W.K.; Jones, M.B. Ecoinformatics: Supporting ecology as a data-intensive science. Trends Ecol.Evol. 2012, 27, 85–93. [Google Scholar] [CrossRef] [Green Version]
  28. Styers, D.M.; Schafer, J.L.; Kolozsvary, M.B.; Brubaker, K.M.; Scanga, S.E.; Anderson, L.J.; Mitchell, J.J.; Barnett, D. Developing a flexible learning activity on biodiversity and spatial scale concepts using open-access vegetation datasets from the National Ecological Observatory Network. Ecol. Evol. 2021, 11, 3660–3671. [Google Scholar] [CrossRef]
  29. Turner, M.G.; Gardner, R.H. Landscape Ecology in Theory and Practice: Pattern and Process; Springer: New York, NY, USA, 2015. [Google Scholar]
  30. Schneider, D.C. The Rise of the Concept of Scale in Ecology. BioScience 2001, 51, 545. [Google Scholar] [CrossRef] [Green Version]
  31. Balch, J.K.; Nagy, R.C.; Halpern, B.S. NEON is seeding the next revolution in ecology. Front. Ecol. Environ. 2020, 18, 3. [Google Scholar] [CrossRef] [Green Version]
  32. IPCC. Climate Change 2014: Mitigation of Climate Change. Contribution of Working Group III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change; Edenhofer, O., Ed.; Cambridge University Press: Cambridge, UK; New York, NY, USA, 2014. [Google Scholar]
  33. Cheruvelil, K.S.; Soranno, P.A. Data-Intensive Ecological Research Is Catalyzed by Open Science and Team Science. BioScience 2018, 68, 813–822. [Google Scholar] [CrossRef]
  34. Farley, S.S.; Dawson, A.; Goring, S.J.; Williams, J.W. Situating Ecology as a Big-Data Science: Current Advances, Challenges, and Solutions. BioScience 2018, 68, 563–576. [Google Scholar] [CrossRef] [Green Version]
  35. Carey, C.C.; Farrell, K.J.; Hounshell, A.G.; O’Connell, K. Macrosystems EDDIE teaching modules significantly increase ecology students’ proficiency and confidence working with ecosystem models and use of systems thinking. Ecol. Evol. 2020, 10, 12515–12527. [Google Scholar] [CrossRef]
  36. Farrell, K.J.; Carey, C.C. Power, pitfalls, and potential for integrating computational literacy into undergraduate ecology courses. Ecol. Evol. 2018, 8, 7744–7751. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Anderson, L.W.; Krathwohl, D.R. (Eds.) A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives; Longman: New York, NY, USA, 2001. [Google Scholar]
  38. Biggs, J.B.; Collis, K.F. Evaluating the Quality of Leaning: The SOLO Taxonomy (Structure of the Observed Learning Outcome); Academic Press: New York, NY, USA, 1982. [Google Scholar]
  39. Fink, D.L. Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses, 2nd ed.; John Wiley & Sons Inc.: San Francisco, CA, USA, 2007. [Google Scholar]
  40. Marzano, R.J.; Kendall, J.S. The New Taxonomy of Educational Objectives, 2nd ed.; Corwin Press: Thousand Oaks, CA, USA, 2007. [Google Scholar]
  41. Klemow, K.; Berkowitz, A.; Cid, C.; Middendorf, G. Improving ecological education through a four-dimensional framework. Front. Ecol. Environ. 2019, 17, 71. [Google Scholar] [CrossRef] [Green Version]
  42. Bybee, R.W.; Taylor, J.A.; Gardner, A.; Van Scotter, P.; Powell, J.C.; Westbrook, A.; Landes, N. The BSCS 5E Instructional Model: Origins and Effectiveness; A Report Prepared for the Office of Science Education and National Institutes of Health; BSCS: Colorado Springs, CO, USA, 2006. [Google Scholar]
  43. Gilbert, J.K.; Justí, R. Modeling-Based Teaching in Science Education; Springer: Cham, Switzerland, 2016. [Google Scholar]
  44. Carey, C.C.; Aditya, S.; Subratie, K.; Figueiredo, R.J.; Farrell, K.J. Macrosystems EDDIE Module 1: Climate change effects on lake temperatures. Environ. Data Initiat. 2018. [Google Scholar] [CrossRef]
  45. Carey, C.C.; Farrell, K.J. Macrosystems EDDIE Module 2: Cross-scale interactions. Environ. Data Initiat. 2019. [Google Scholar] [CrossRef]
  46. Farrell, K.J.; Carey, C.C. Macrosystems EDDIE Module 3: Teleconnections ver 4. Environ. Data Initiat. 2019. [Google Scholar] [CrossRef]
  47. Carey, C.C.; Farrell, K.J.; Hounshell, A.G. Macrosystems EDDIE Module 4: Macro-Scale Feedbacks ver 1. Environ. Data Initiat. 2020. [Google Scholar] [CrossRef]
  48. Miles, M.B.; Huberman, A.M.; Saldana, J. Qualitative Data Analysis: A Methods Sourcebook, 4th ed.; SAGE Publications Inc.: Thousand Oaks, CA, USA, 2020. [Google Scholar]
  49. Vogt, W.P. Dictionary of Statistics & Methodology: A Nontechnical Guide for the Social Sciences, 3rd ed.; SAGE Publications Inc.: Thousand Oaks, CA, USA, 2005. [Google Scholar]
  50. Terwee, C.B.; Bot, S.D.; De Boer, M.R.; Van Der Windt, D.A.; Knol, D.L.; Dekker, J.; Bouter, L.M.; De Vet, H.C. Quality criteria were proposed for measurement properties of health status questionnaires. J. Clin. Epidemiol. 2007, 60, 34–42. [Google Scholar] [CrossRef] [Green Version]
  51. Carey, C.C.; Gougis, R.D. Simulation Modeling of Lakes in Undergraduate and Graduate Classrooms Increases Comprehension of Climate Change Concepts and Experience with Computational Tools. J. Sci. Educ. Technol. 2017, 26, 1–11. [Google Scholar] [CrossRef] [Green Version]
  52. Freeman, S.; Eddy, S.L.; McDonough, M.; Smith, M.K.; Okoroafor, N.; Jordt, H.; Wenderoth, M.P. Active learning increases student performance in science, engineering, and mathematics. Proc. Natl. Acad. Sci. USA 2014, 111, 8410–8415. [Google Scholar] [CrossRef] [Green Version]
  53. Haak, D.C.; HilleRisLambers, J.; Pitre, E.; Freeman, S. Increased Structure and Active Learning Reduce the Achievement Gap in Introductory Biology. Science 2011, 332, 1213–1216. [Google Scholar] [CrossRef] [PubMed]
  54. Ballen, C.J.; Wieman, C.; Salehi, S.; Searle, J.B.; Zamudio, K.R. Enhancing Diversity in Undergraduate Science: Self-Efficacy Drives Performance Gains with Active Learning. CBE-Life Sci. Educ. 2017, 16, 1–6. [Google Scholar] [CrossRef] [PubMed]
  55. Theobald, E.J.; Hill, M.J.; Tran, E.; Agrawal, S.; Arroyo, E.N.; Behling, S.; Chambwe, N.; Cintrón, D.L.; Cooper, J.D.; Dunster, G.; et al. Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proc. Natl. Acad. Sci. USA 2020, 117, 6476–6483. [Google Scholar] [CrossRef] [Green Version]
  56. Ma, W.; Winke, P. Self-assessment: How reliable is it in assessing oral proficiency over time? Foreign Lang. Ann. 2019, 52, 66–86. [Google Scholar] [CrossRef]
  57. Team, R.C. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2020. [Google Scholar]
Education 11 00382 g001 550
Figure 1. Students’ self-reported knowledge of macrosystems ecology increased after completing a Macrosystems EDDIE module. Percent of responses at each level of familiarity are based on paired pre- and post-module student assessments (n = 216). Lines between bars represent changes in individual responses pre- and post-module use (see Supplementary Material, Table S8 for statistical results).
Figure 1. Students’ self-reported knowledge of macrosystems ecology increased after completing a Macrosystems EDDIE module. Percent of responses at each level of familiarity are based on paired pre- and post-module student assessments (n = 216). Lines between bars represent changes in individual responses pre- and post-module use (see Supplementary Material, Table S8 for statistical results).
Education 11 00382 g001
Education 11 00382 g002 550
Figure 2. Students’ self-reported proficiency and confidence using high-frequency and long-term data significantly increased after module use. Questions used a Likert scale from 1 (low) to 5 (high). Jittered points represent individual students; boxplots represent the median response (dark horizontal line) while the boxes represent the interquartile range (IQR; 25th and 75th percentile), the whiskers correspond to 1.5x IQR; n indicates number of students with paired pre- and post-responses (see Supplementary Material, Table S8 for statistical results).
Figure 2. Students’ self-reported proficiency and confidence using high-frequency and long-term data significantly increased after module use. Questions used a Likert scale from 1 (low) to 5 (high). Jittered points represent individual students; boxplots represent the median response (dark horizontal line) while the boxes represent the interquartile range (IQR; 25th and 75th percentile), the whiskers correspond to 1.5x IQR; n indicates number of students with paired pre- and post-responses (see Supplementary Material, Table S8 for statistical results).
Education 11 00382 g002
Education 11 00382 g003 550
Figure 3. The percentage of student responses that correctly defined macrosystems ecology, cross-scale interactions, teleconnections, and macro-scale feedbacks increased after module use. Stars (*) indicate statistically significant (p < 0.05) increases between pre- (grey) and post-module (blue) assessments; n indicates number of paired pre- and post-module student responses for each definition (see Supplementary Material, Table S9 for statistical results).
Figure 3. The percentage of student responses that correctly defined macrosystems ecology, cross-scale interactions, teleconnections, and macro-scale feedbacks increased after module use. Stars (*) indicate statistically significant (p < 0.05) increases between pre- (grey) and post-module (blue) assessments; n indicates number of paired pre- and post-module student responses for each definition (see Supplementary Material, Table S9 for statistical results).
Education 11 00382 g003
Education 11 00382 g004 550
Figure 4. Percentage of correct responses to interpret questions for all four modules generally increased after module use. Questions for which >85% of students selected a correct response in the pre-module assessments were omitted from the analysis. Questions increased in complexity from questions 1 to 3 (see Supplementary Material, Table S4); n indicates number of paired pre- and post-module student responses; stars (*) indicate statistically significant (p < 0.05) increases between pre- and post-module assessments (see Supplementary Material, Table S10 for statistical results).
Figure 4. Percentage of correct responses to interpret questions for all four modules generally increased after module use. Questions for which >85% of students selected a correct response in the pre-module assessments were omitted from the analysis. Questions increased in complexity from questions 1 to 3 (see Supplementary Material, Table S4); n indicates number of paired pre- and post-module student responses; stars (*) indicate statistically significant (p < 0.05) increases between pre- and post-module assessments (see Supplementary Material, Table S10 for statistical results).
Education 11 00382 g004
Education 11 00382 g005 550
Figure 5. The percentage of student responses to the apply questions that contained thematic keywords. Responses including ‘many’ or multiple model runs and ‘model’ or the use of ecosystem simulation models increased after module use, while ‘trend’ or the use of linear trendlines to estimate responses in the future decreased. We also report changes in pre- and post-module responses that included ‘data’ or the use to data to predict how lake ecosystems will change in the future. Student responses were aggregated across modules (modules 2, 3, and 4; n = 232 total). Stars (*) indicate statistically significant (p < 0.05) differences between pre- (grey) and post-module (blue) assessments (see Supplementary Material, Table S11 for statistical results).
Figure 5. The percentage of student responses to the apply questions that contained thematic keywords. Responses including ‘many’ or multiple model runs and ‘model’ or the use of ecosystem simulation models increased after module use, while ‘trend’ or the use of linear trendlines to estimate responses in the future decreased. We also report changes in pre- and post-module responses that included ‘data’ or the use to data to predict how lake ecosystems will change in the future. Student responses were aggregated across modules (modules 2, 3, and 4; n = 232 total). Stars (*) indicate statistically significant (p < 0.05) differences between pre- (grey) and post-module (blue) assessments (see Supplementary Material, Table S11 for statistical results).
Education 11 00382 g005
Table
Table 1. Module titles and descriptions for developed Macrosystems EDDIE modules. All modules are published and available in the Environmental Data Initiative repository (EnvironmentalDataInitiative.org; accessed on 23 July 2021).
Table 1. Module titles and descriptions for developed Macrosystems EDDIE modules. All modules are published and available in the Environmental Data Initiative repository (EnvironmentalDataInitiative.org; accessed on 23 July 2021).
Module 1: Climate Change Effects on Lake Temperatures (Carey et al., 2018)
Students set up an ecosystem simulation model for a GLEON lake, then “force” the model with a climate scenario of their own design to assess how climate change will impact lake thermal structure. Students then use a distributed computing tool to run hundreds of different climate scenarios for their lake and examine tipping points in lake responses. While this module is not directly associated with a specific macrosystems concept, it is designed to introduce students to the modeling and computational techniques used in macrosystems ecology.
Module 2: Cross-Scale Interactions (Carey and Farrell 2019)
Students set up an ecosystem simulation model for a GLEON lake, then “force” the model with climate and land-use change scenarios and evaluate how regional and local drivers, respectively, interact across spatial scales to affect lake water quality.
Module 3: Teleconnections (Farrell and Carey 2019)
Students set up an ecosystem simulation model for a GLEON or NEON lake, then “force” the model with El Niño scenarios to compare how different lakes respond to a global teleconnection. Students test their hypotheses for how global drivers influence regional weather and interact with local lake characteristics to affect water temperature and ice cover.
Module 4: Macro-scale Feedbacks (Carey et al., 2020)
Students set up an ecosystem simulation model for a GLEON or NEON lake, then “force” the model with climate scenarios to examine how greenhouse gas emissions from different lakes may either increase or decrease in the future, thereby creating a feedback that either amplifies or diminishes global climate change.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Hounshell, A.G.; Farrell, K.J.; Carey, C.C. Macrosystems EDDIE Teaching Modules Increase Students’ Ability to Define, Interpret, and Apply Concepts in Macrosystems Ecology. Educ. Sci. 2021, 11, 382. https://doi.org/10.3390/educsci11080382

AMA Style

Hounshell AG, Farrell KJ, Carey CC. Macrosystems EDDIE Teaching Modules Increase Students’ Ability to Define, Interpret, and Apply Concepts in Macrosystems Ecology. Education Sciences. 2021; 11(8):382. https://doi.org/10.3390/educsci11080382

Chicago/Turabian Style

Hounshell, Alexandria G., Kaitlin J. Farrell, and Cayelan C. Carey. 2021. "Macrosystems EDDIE Teaching Modules Increase Students’ Ability to Define, Interpret, and Apply Concepts in Macrosystems Ecology" Education Sciences 11, no. 8: 382. https://doi.org/10.3390/educsci11080382

APA Style

Hounshell, A. G., Farrell, K. J., & Carey, C. C. (2021). Macrosystems EDDIE Teaching Modules Increase Students’ Ability to Define, Interpret, and Apply Concepts in Macrosystems Ecology. Education Sciences, 11(8), 382. https://doi.org/10.3390/educsci11080382

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop