To investigate the three research questions of this paper, we first examine how specifically the teaching and learning process functioned with regards to integrating the modeling practice into the water quality unit using the modeling tool. Afterwards, we focus on students’ developing models, combined with their responses to prompts for why they specified the relationships between variables the way they did. Lastly, we present our analysis of students’ responses to pre- and post-unit survey questions about their metamodeling knowledge.
4.1. Modeling Practice Elements Integrated in the Water Quality Unit
The first research question (How can the elements of the modeling practice be integrated in a unit using the modeling tool?) was investigated by examining the observations notes and video recordings of the modeling lessons together with the reflection notes made by the class teacher, as the third author of this article. In the following section, we describe the learning and modeling process, as carried out in one of the classes. All other classes were taught in a similar way by the same teacher.
After student groups collected two pieces of water quality data, pH and temperature, the teacher probed students to assess their prior understandings of what scientific models were and their ideas on the purpose for constructing models. Building from student responses, a discussion followed that models are tools that assist scientists with explaining and predicting phenomena because they illustrate relationships between variables [7
]. The teacher then informed student groups that they would collaborate to create a model of the stream system based on their current knowledge; they would represent the various variables that had been investigated so far and that the students had learned influence water quality. Students were informed that their models were working models
, meaning that their models would develop over time. As they collected more water quality data they would add to and modify their models. On the classroom board, during the whole-class instruction, the modeling elements were written and discussed: plan
, and share
. On one occasion, when referring to these elements, the teacher said:
“What we’ve been doing now, we’ve been planning [points at the word ‘plan’ on the board]. That’s the first part of thinking about a model, to plan what should be in the model. And we want to think about the various variables. What is it that we want to model? What is it that we want to use to be able to predict or to explain?”
After this, the teacher introduced students to the modeling program, SageModeler, by modeling a simple, non-science example where the various components of the program were illustrated. This allowed the students to see how the tool worked. Students were then asked what the various factors (variables) for their model would be if they were to build a model to address the unit’s driving question based on what they knew so far. Students accurately responded that the variables would be the water quality measures that they had conducted on the stream (pH, temperature, and water quality) and the factors that influenced the water quality measures. The teacher informed students that the modeling tool included no content—no science ideas—and that they would need to create a model that portrayed their own science ideas. Students would be responsible to plan, build, test, revise, and share their models that accurately modeled the stream phenomenon. In order to assist students in determining if their models worked the way that they thought they should work, students were provided with the following guidelines:
Students were instructed to set each of their independent variables to the best possible conditions of a healthy stream and then run a simulation of their models by producing an output graph to determine if the water quality (the dependent variable) was set to the highest possible water quality;
Students were instructed to test portions of their models (pH and water quality or temperature and water quality, for example) to systematically verify if those parts of the model ‘worked’. This meant students checked if the relationship between these variables was scientifically appropriate in that it matched the data the students had collected or the information they had learnt about in class beforehand;
Students were instructed to select and set one independent variable on their models while keeping all other variables set at the best condition for water quality and then predict the impact on the overall water quality of the selected variable. They then had to test their model to see if their prediction and the results matched.
If any of the three model testing procedures worked differently than the student groups expected, students were asked to further evaluate their models by considering the following questions:
Do I need to evaluate and revise the relationships in my model by evaluating the relationship type setting (more and more, less and less, etc.)? Are my relationships accurate?
Do I need to rethink my science ideas?
Students were asked to evaluate if their model was complete and accurate or if their current thinking was problematic. As part of the process, students would both share their models and critique classmates’ models to get feedback from their peers and the teacher so that their models would provide a more complete and scientifically appropriate explanation. While working on their models, the teacher rotated between student groups to provide support and specific guidance. Some groups were asked to share their models with the entire class. In the following lesson, the teacher provided the students with an opportunity to reflect on the initial models they constructed in the previous lesson. In the beginning of this lesson, the teacher reminded the students about the elements of the modeling practice and asked them to test their current models to see if they work the way they would like them to work before adding the new variables:
“How many of you finished? [several students raise hands] … so what we want to do today, finish that first... Now today, if we get finished with this, and remember, so we plan, we build, we test...you remember we test it, how many of you tested it? [several students raise hands]. OK, you played around with the stuff, you got a data table. OK, so we make sure we test those today and revise it if it doesn’t quite work the way we want it towork, right?”
After a few minutes of discussion about testing and revising the model, the teacher instructed the students to add the new variables to their models once they felt confident about having the appropriate model:
“So we have to ask ourselves, Does this work? If you get to the way that it works, the way you think it works, then now we can expand our model ... you are going to plan, and build it, you are going to test it, you are going to see if it runs the way you think and then you will revise it.”
A second iteration of modeling occurred following data collection of a third water quality measure, conductivity, which represents the amount of dissolved solids. Students expanded their initial models to include all three water quality measures. A similar process was followed; students planned, built, tested, revised, and shared their models.
Several lessons later, after collecting dissolved oxygen and turbidity data from the stream, students had a third and final iteration that resulted in a model that reflected five water quality tests. Because it was the end of the semester, most groups did not have time to include turbidity in their models, simply due to time constraints. At the beginning of this model revision, the teacher reviewed what was done so far with the models and what students would be expected to add during this final revision. However, in this lesson, the teacher did not ask the students to test and revise their current models before adding the new variables:
“So we have been using this modeling program ... we’re trying to model relationships between various water quality measures, right? And the health of the stream and its organisms. And you have in there right now pH, temperature, and conductivity and what we are going to do today is add dissolved oxygen.”
After adding the dissolved oxygen variable, the teacher discussed with the students adding more connections between the variables in their models. Following this discussion, students were given time to add additional cross-variable relationships to their models:
“You have created a very complex model of a very complex system, right? The water quality is a really complex system. And for lots of you, you have your four water quality measures all connected to here [points at water quality variable in an example model projected on the screen], which is very important, it affects our relationships. But then there is also some of these kinds of relationships [cross referencing with her hands] that even make the model more complex. Right? So if we are really going to model the complexity of this phenomenon we also have to look for relationships in between.”
Summary: Students developed a model of a stream system, growing over time in quality and complexity, as new water quality data were collected. All the three observed iterations of modeling included the modeling practice elements: (i) building an initial model with the pH, temperature and water quality variables; (ii) adding the conductivity variable, and (iii) adding the dissolved oxygen variable. Following each new data collection, students were asked to revise their models by performing the following elements: Testing their existing models for accuracy and completeness, planning what variables to add to their model, adding those variables and defining their relationships with the other variables in the model, testing the model for expected outcome, and revising their model if necessary. Students were directed to receive feedback from their classroom peers during the modeling process and had several opportunities to share their models with other groups and with the entire class.
4.2. Development of Students’ Models and Metamodeling Knowledge
To address the second research question, ‘How did students’ modeling practice develop throughout the unit with regard to constructing, using, evaluating, and revising models?’, models developed by 11 groups were analyzed with a focus on how students included the appropriate variables and relationships at each modeling iteration cycle. Six of the 11 groups had complete and accurate models at the end of each of the modeling cycles. In the following, we present types of incompleteness found in the remaining five groups’ models as an umbrella term to signify that an element of a student’s model had not reached its required final form, as expected by the curriculum. Broadly, we distinguish two aspects under this umbrella term that occur throughout this article: We speak of inaccuracy when an element is present, but it may be specified in a less-than-ideal way with regards to using the model for making predictions or explanations of the observed phenomenon. We speak, more specifically, of incompleteness, if a model element is missing (i.e., a relationship or variable). We observed several different types of incompleteness in the models of five of our 11 groups (Table 1
From the 11 groups of students that produced all three models during the lessons, three representative groups were taken for qualitative in-depth analysis. The first two cases represented a type of modeling: Group E produced fully complete models with all accurate variables and relationships in all models, representing the five groups that were able to accurately complete the task of building and revising their model in all modeling lessons. Group C had one inaccurate definition of relationship between variables that was revised in the next revision. This group is one of the two groups that were able to identify and revise prior inaccuracy in their models in a later modeling lesson and hence revise elements of their model in the narrow sense of its meaning (see theoretical background). In the third case, Group A’s model represented an exceptional case of a group that went beyond the expected system to be modeled.
We report findings for these three cases concerning the development in students’ models across the water quality unit (RQ 2) in the following section, as well as students’ answers to the metamodeling questions in the pre- and post-surveys about the nature and uses of models and modeling (RQ 3).
4.2.1. Case 1: A complete Model (Group E)
Student model. Five of the 11 analyzed groups had a complete and accurate model throughout the unit with regards to the variables expected at the three modeling cycles, amongst them Kara and Sandy (here and in all following cases, students’ real names have been replaced by pseudonyms) in group E (Figure 3
). In the initial model, they included high level causal reasoning for both water quality measurements, as written in their prompt text boxes. For the relationship between pH and water quality, they wrote “Because as the pH of the stream gets farther away from neutral, organisms start dying.
”, and for the relationship between thermal pollution and water quality they wrote, “Because as the temperature difference increases, the water quality gets worse. For example, algae grows more in warm water and causes more thermal pollution. Then fish start to die.
” While the students did not provide exactly how they believed algae growth contributes to the warming of water, these responses can be counted as causal-mechanistic and also include an example to demonstrate the effect on specific species. Including an example was only found in this group’s reasoning. In their reasoning for the first model revision, the group wrote the following explanation for the relationship between conductivity and water quality: “Because as the conductivity increases, the quality of the water gets worse and worse.” This is a low-level response because it is only repeating the chosen relationship without providing causal reasoning. Similar low-level reasoning was also provided in the third modeling cycle, where they wrote the following explanation for the relationship between dissolved oxygen and water quality: “Because as the dissolved oxygen increases, the water quality will rapidly get better, but then it will increase by less”.
Metamodeling knowledge. While students’ models from virtually all groups showed substantial progress across the water quality unit, students’ answers to the metamodeling questions about the modeling practice showed fewer and less pronounced changes.
We observed three dominant conceptions widely used amongst all participating students. All of these can be seen in the answers of group E. First, the most characteristic response was that students described models as a simple visualization, often seen as something used to show things that are otherwise hard to perceive due to their size (scaling) or with the goal to share information with others.
Quote 1: “Scientists use models to show/describe phenomena that their peers may need to see.” [Sandy, pre-survey, question 3]
Second, many students in both pre- and post-surveys reported that they needed to pay close attention to the model’s accuracy and to it being ‘up-to-date’. Students did not specify what they meant by up-to-date, but it is likely that they were referring to the variables they added into their models as they progressed through the unit. As the third prominent feature, students indicated that a good model has to be simple [3
]. Notably, students in group E transgressed these rather basic conceptions and added that models need to be parsimonious (Quotes 2 and 3) in that they focus only on the essential concepts involved in the process. A positive surprise was here that Sandy even realized that models are, in fact, not
accurate (Quote 4):
Quote 2: “A scientific model should be clear with no unneeded parts. It should also be up to date on all current information.” [Kara, pre-survey, question 4]
Quote 3: “You need to think about all of the scientific ideas and concepts involved, our model should be accurate, and the best models are simple and don’t have unnecessary parts.” [Kara, post-survey, question 4]
Quote 4: “I think about how […] not everything will be exact.” [Sandy, pre-survey, question 4]
Students in group E showed development from pre- to post-survey in terms of their perception of model uses: Kara, in the pre-survey, focused on showing, understanding or explaining information. In the post-survey, she indicated models can be used to explore how scientific ideas work (unlike what scientists use models for) or, to explain difficult issues or to test predictions (like scientists do). Seen in the context of all participating students in the unit, conceptions about using models for predictions were rare. Similar to Kara, Sandy started (pre-survey) with a focus on using models for representation and on specific aspects of designing the model (Quote 5). The student then shifted, in the post-survey, to a focus on using models to explore interactions (Quote 6).
Quote 5: “I think about how there may be a need for a key, the scale […]” [Sandy, pre-survey, question 4];
Quote 6: “We could use scientific models to model how things affect our stream.” [Sandy, post-survey, question 2]
Students in group E did not show consistent distinctions between how they (as students) may use models versus how scientists use them. In the pre-survey, students used almost the same conceptions for both scientists and themselves; in the post-survey, the students from group E indicated that students can use advanced practices of modeling (i.e., test predictions, focus on interactions).
Summary. In Case 1 (Group E), the students’ model could account as a high level explanation of the stream’s water quality. It was characterized by accurately showing all relevant variables and the relationships between them throughout the three modeling cycles. They also demonstrated examples of high level causal reasoning for defining the relationships between variables in their initial model. Students’ conceptions about the nature of models and modeling included typical lower-level conceptions that we also observed across multiple other participating students (e.g., models as visualizations). However, group E extended these simple conceptions by sophisticated and rare conceptions regarding the inaccuracy of all models and the possibility to use models for making predictions.
4.2.2. Case 2: An incomplete model revised in later modeling iterations (Group C)
Five of the 11 groups in our sample showed a total of nine instances of incomplete (e.g., missing or unconnected variable, undefined relationship) or inaccurate (e.g., inaccurate labels or relationships) models (see Table 1
). Of the five instances that we observed specifically in students’ initial model and in the first model revision, only two were revised in the following model submission, thus falling into the category of a model (element) revision in the narrow sense. For these two cases, we assume that students appropriately tested, evaluated, and revised their models and may have just ran out of time to complete their models. Rick and Ron of group C were one of these groups.
The students produced an incomplete initial model, in which the relationship between temperature and water quality was defined as ‘vary’ and was drawn as an increasing and then decreasing relationship (Figure 4
b). Their explanation for this relationship provided a basic reasoning: “When temperature changes
, the water quality changes.
” The relationship between pH and water quality was also basic and not causal: “Because when the pH changes the water quality changes.
In the first model revision, the students changed the relationship between temperature and water quality to be a decreasing relationship, therefore improving their model to be more accurate. They also added the conductivity variable to their model.
In their third model, Rick and Ron added the dissolved oxygen variable and accurately defined the effect of it on water quality. They added an explanation to this relationship that did not include any causal reasoning: “D.O.
[dissolved oxygen] is good so it will cause water quality to increase
” (Figure 4
Metamodeling knowledge. In group C, Rick was one of only very few students in our unit to indicate that models are never perfect (Quote 7, see also Quote 4 above), that there are competing models (Quote 8) and that they constantly change (Quote 9):
Quote 7: “[…] All models should be simplistic. All models have right and wrong factors.” [Rick, pre-survey, question 1]
Quote 8: “They [models] are simple, and there can be many correct models about one phenomenon.” [Rick, post-survey, question 1]
Quote 9: “[…] They are constantly changing, so if you have to change it, you’re ok. […].” [Rick, pre-survey, question 4]
Students in this Group showed some development from pre- to post-survey with students’ understanding becoming more refined. For instance, Rick went from an understanding that scientists (Q3) use models to “represent and test things” to a more specific understanding that they use models to “test different theories on scientific phenomena.” The same student indicated before the unit only that change is a regular process in modeling (Quote 9 above), while specifying at post also when the model should be changed:
Quote 10: “You might have to change it, if your theory doesn’t work, that’s when you revise it […].” [Rick, post-survey, question 4]
Unlike in Case 1 (Group E), students in Case 2 (Group C) seemed to see differences between how they as students use models versus how scientists use models. Both students appeared to associate more advanced aspects of modeling with scientists compared to how students use models:
Quotes 11: [Rick, pre-survey, questions 2&3];
Students: “[…] to scale something.”
Scientists: “[…] represent and test things”;
Quotes 12: [Ron, post-survey, questions 2&3];
Students: “To see what happens in a smaller version.”
Scientists: “[…] show other people what will happen in a smaller version.”
Summary. Students in Case 2 (Group C) stood out by being one of only two groups that revised an earlier incomplete model in the narrow sense of the meaning. Their reasons for defining the relationships between variables in their model did not prove to be of high level causal reasoning. While they appeared to distinguish between how models can be used by scientists versus students, they specified their understanding from pre- to post-survey and expressed rare and advanced conceptions referring to competing models and model revisions.
4.2.3. Case 3: A Complete Model with Additional Variables (Group A)
Student model. Of the 11 groups with all three model submissions, Group A was a particularly interesting case because the students provided a model that would not be considered to confirm to the expectations in the strictest sense (i.e., with regards to the expected variables at the different modeling iterations). However, the students’ model can neither be considered to have strictly incomplete or incorrect sections.
The students, Dave and Zack, constructed a complete and accurate initial model. They had the expected variables and relationships of temperature and pH affecting water quality. They provided a low-level response for explaining the effect of temperature on water quality that just repeated the defined relationship: “The higher the temperature the lower the water quality,” but did not write any response for the pH and water quality relationship.
In the first model revision, Dave and Zack accurately added the conductivity variable to their model, and added a low-level response to its effect on the water quality that describes the type of relationship but not the causal reasoning of it: “First conductivity is fine for a little bit, but then it starts lowering the water quality by a lot.” However, in this modeling cycle, they also added another variable that was not expected to be directly connected to the water quality: bacteria. In their explanation for this relationship they wrote: “The bacteria takes in a lot of the oxygen in the water making it unsuitable for organisms to live in it.” This response provides causal reasoning for this relationship. They decided to directly connect the bacteria variable to water quality and not to the mediating variable, oxygen level in the water, that was mentioned in the students’ response and was added to the model as ‘dissolved oxygen’. In the second model revision, Dave and Zack did not change the relationship between the bacteria and water quality variable. A reason for this may be linked to real-life experiences by students and the related preconceptions, where students learn that bacteria presence in water is ‘bad’ in a direct way (i.e., without mediating variables) for humans or other organisms. In addition, the teacher’s focus on the upcoming model revision lay on integrating dissolved oxygen as a variable, hence probably distracting students from going back and working on the bacteria variable.
The students also added dissolved oxygen as a variable before they were expected to (first model revision, Figure 5
b). While they did not connect this variable in their first model revision before this variable was formally required in class, they did go back to it in their second and final model revision and connected the variable accurately, even though they did not provide an explanation for this relationship (Figure 5
c). Even though they were triggered to do so, this may be considered an example for a model revision in the narrow sense, i.e., going back to revise a model element that was entered in an earlier model version.
Metamodeling knowledge. Like many other students in our study, both students in Case 3 (Group A) indicated—both during the pre- and post-surveys—that models are used for scaling (Quote 13) and visualization (Quote 14):
Quote 13: “If we can’t see something from the naked eye, you can make a model to see what it looks like up close.” [Dave, pre-survey, question 2]
Quote 14: “[...] a model of something that is being studied. It can be visualized and created to model almost anything.” [Zack, post-survey, question 1]
Similarly and connected to this conception, the students in case 3 put their focus on basic aspects of modeling that might indicate the students’ modeling practice to be more model-of
], instead of conceptualizing their modeling as a way to test or generate knowledge (model-for
]). For example, the students mentioned that models have to always be correct (Quote 15) and that they have to pay attention to the specifics of model construction and design (Quote 16):
Quote 15: “So that they are always correct.” [Dave, post-survey, question 4]
Quote 16: “What is [going to] represent what.” [Zack, pre/post-surveys, question 4]
(Note: This likely refers to pictures students chose for variables in their models).
Unlike in Case 1, students in Case 3 did not attribute additional, more advanced aspects of model usage to students as model users. However, Group A associated more advanced levels of model uses to scientists. For Zack, the understanding of model uses became also more refined from pre- to post-survey:
Quote 17: “They [scientists] use models to explain a theory.” [Dave, pre-survey, question 3]
Quote 18: “Scientists use scientific models to predict what things could turn out as.” [Zack, pre-survey, question 3]
Quote 19: “To hypothesize on how somethings works.” [Zack, post-survey, question 3]
Summary. In Case 3 (Group A), students produced an interesting model that showed they tried to integrate their own ideas into their models. The students provided mixed results in terms of their ability to go back and revise earlier aspects of their model. While their reasons for defining the relationships were of low-level causal reasoning, the students’ conceptions about modeling were focused both on basic (e.g., models have to be always correct) and advanced uses of models, while the latter was associated more with scientists’ uses of models.