Using Critical Integrative Argumentation to Assess Socioscientific Argumentation across Decision-Making Contexts

Abstract

Socioscientific issues (SSI) are often used to facilitate students’ engagement in multiple scientific practices such as decision-making and argumentation, both of which are goals of STEM literacy, science literacy, and integrated STEM education. Literature often emphasizes scientific argumentation over socioscientific argumentation, which involves considering social factors in addition to scientific frameworks. Analyzing students’ socioscientific arguments may reveal how students construct such arguments and evaluate pedagogical tools supporting these skills. In this study, we examined students’ socioscientific arguments regarding three SSI on pre- and post-assessments in the context of a course emphasizing SSI-based structured decision-making. We employed critical integrative argumentation (CIA) as a theoretical and analytical framework, which integrates arguments and counterarguments with stronger arguments characterized by identifying and refuting counterarguments. We hypothesized that engaging in structured decision-making, in which students integrate multidisciplinary perspectives and consider tradeoffs of various solutions based upon valued criteria, may facilitate students’ development of integrated socioscientific arguments. Findings suggest that students’ arguments vary among SSI contexts and may relate to students’ identities and perspectives regarding the SSI. We conclude that engaging in structured decision-making regarding personally relevant SSI may foster more integrated argumentation skills, which are critical to engaging in information-laden democratic societies.

1. Introduction

Throughout the past several decades, there has been an increasing focus on improving public STEM literacy, described as the knowledge of integrated STEM disciplines and the skills required to use STEM concepts in solving complex real-world problems [1,2], and science literacy, defined by the National Research Council (NRC) as “knowledge and understanding of scientific concepts and processes required for personal decision making, participation in civic and cultural affairs, and economic productivity” [3]. Both STEM and science literacy require that learners understand a variety of STEM content (e.g., concepts within life sciences, physical sciences, geosciences, mathematical sciences, social sciences, computers, information technology) and be proficient in scientific practices (e.g., evidence evaluation, argumentation, communicating information). Further, considerable overlap exists when we consider the goals of both STEM and science literacy; both paradigms emphasize that learners should ultimately be equipped to solve complex problems, make well-informed decisions, and effectively participate in our democratic society. Thus, we can view the goals of STEM and science literacy as goals of integrated STEM education as a whole. Importantly, all of these outcomes are predicated on the assumption that these skills transfer, such that students apply skills learned in formal and informal classrooms to novel contexts [4], with the ultimate goal of skills transferring to real-world contexts.

Socioscientific issues (SSI) provide valuable real-world contexts for learners to engage with STEM content while integrating knowledge from multiple STEM disciplines, engage in STEM practices, and practice STEM-informed decision-making ultimately contributing to their development of STEM and science literacy [5,6,7]. Indeed, SSI-based instruction has been identified as a promising avenue for integrated STEM instruction that “elevates the purpose to include STEM literacy for all citizens regardless of their future participation in a STEM career” [5]. Specifically, SSI instruction provides an opportunity for learners to make decisions that consider STEM knowledge within social, political, and economic contexts [8]. SSI-based instruction has been shown to support critical learning objectives, including science content [9], reasoning skills [10], understanding of the nature of science [11], and socioscientific reasoning [12]. Romine et al. (2020) posited that SSI-based instruction emphasizing specific competencies may foster the transfer of such competencies into novel contexts [12]. They drew upon transfer theory [13] to predict that engaging learners in SSI-based instruction may enable learners to employ socioscientific reasoning (SSR) in SSI contexts that were not used in the course (i.e., SSR competencies would transfer from the context in which they were experienced to novel contexts) [12,13]. Results demonstrated that SSR competencies did transfer to SSI scenarios not addressed in the course. Based on prior work [14], the authors attributed transfer of SSR competencies to course characteristics including drawing upon students’ prior knowledge and experiences, repeated engagement in SSI-based decision-making and scientific practices toward mastery of these skills, and substantial discussions regarding the complexities of decision-making, the relevance of scientific knowledge and values in SSI, and considering the perspectives of multiple stakeholders [12].

While SSI instruction provides opportunities for practicing many STEM literacy and science literacy skills with a goal of transfer across SSI contexts, this article will focus on two related practices–SSI argumentation and SSI decision-making. Previous literature has not clearly defined these practices nor often distinguished them from each other. Indeed, researchers have asserted that decision-making itself necessitates argumentation to evaluate claims and reach a final decision [15,16], emphasizing the connection between decision-making and scientific argumentation. Given this connection, it is important to explore the distinction between decision-making and scientific argumentation.

Within science education, argumentation is broadly described as the overarching scientific practice of “evaluating knowledge in the light of evidence” [17,18,19]. Within this broad description, there is undoubtedly a great deal of variety in how scholars operationalize this practice; we draw upon the work of Jiménez-Aleixandre (2008) to describe argumentation as a context in which learners are “active producers of justified knowledge claims and efficient critics of others’ claim[s]” [20]. In other words, we consider argumentation to be a scientific practice in which learners, at minimum, use evidence to (a) evaluate knowledge based on specific criteria, (b) consider and critique various claims, and (c) develop an evidence-based claim or explanation for a problem or question (e.g., an argument). The outcome of argumentation, then, is an argument.

In contrast, we define decision-making as a multi-step process in which students are presented with a problem, set specific criteria that a solution should meet, identify several potential solutions, employ scientific data, knowledge and practices to analyze evidence regarding the potential solutions, and ultimately decide upon one solution to the problem [21,22,23]. We consider decision-making to be one of many types of argumentation; this aligns with evidence-based decision-making being described as one of four argumentation contexts described by Jiménez-Aleixandre et al. (2014), with the others being constructing and evaluating evidence-based explanations, critically evaluating others’ claims, and interpreting and explaining data observed in laboratory contexts [17,18,22]. Within the multi-step decision-making process, there are also opportunities for learners to engage in the argumentation types of constructing and evaluating evidence-based explanations [17,18] as they use scientific and other forms of evidence to evaluate how potential solutions meet values-based objectives. However, in this work, we focus on the final decision as an argument that is a result of the entire decision-making process including a trade-offs analysis, which addresses the question about what should be done to resolve an SSI. Additionally, we characterize a high-quality decision as one that depends on the quality of the process by which it is made, relies on scientific and other kinds of evidence to reduce uncertainty about the world, and is based on tradeoffs around personal priorities; thus, scientific information and values hold separate but important roles in the decision-making process [24,25,26].

In summary, the process of decision-making asks students to integrate multidisciplinary perspectives, consider tradeoffs among valued criteria or objectives and organize relevant information to determine potential outcomes of decisions [24,27,28]. Through engaging in the decision-making process, learners are engaging in a type of scientific argumentation [18], which is a crucial component of critical thinking, decision-making proficiency and science literacy [19,29,30,31]. Based on the positive outcomes of SSI-based instruction described above and our synthesis of decision-making and scientific argumentation, we suggest that engaging in SSI-based decision-making, a type of scientific argumentation, can be a valuable tool for STEM educators to foster learners’ STEM and science literacy in integrated STEM contexts. Further, open and spirited deliberations on multifaceted issues are commonplace in our society with both information and misinformation readily available, which makes the development of argumentation skills and transfer of such skills across STEM disciplines and issues contexts especially critical for engaging in 21st-century democracies [32]. This study brings together literature in SSI, argumentation, decision-making, and science literacy, and is framed by the following research questions: (1) Do learners’ socioscientific argumentation skills change after completion of a course using a structured decision-making framework to assess alternative solutions to SSI? (2) Do learners’ socioscientific argumentation skills transfer from an SSI discussed in class to a novel SSI? Below, we explore the bodies of literature that inform our work.

2. Theoretical Framework

2.1. Socioscientific Argumentation

Literature in argumentation has established the importance of learners explaining and defending their own claims in collaborative settings, as well as evaluating the claims and explanations of their peers to come to a consensus, as a means to developing argumentation skills [33,34]. However, fewer studies have investigated socioscientific argumentation [32,35,36], which can be distinguished from scientific argumentation in that the learner considers social and scientific factors instead of solely constructing explanatory conclusions or descriptive frameworks as in scientific debate. To foster learners’ development of socioscientific argumentation skills, we draw upon literature in science education and the decision sciences to engage learners in structured decision-making (SDM). SDM is grounded in normative decision-making models designed to minimize the effects of cognitive biases during the decision-making process [27] and explicitly models how learners can find, analyze, and use scientific evidence to make informed decisions about SSI [24,37]. We employ an SDM process, outlined in Figure 1 and more thoroughly described in [24], to guide learners in identifying alternatives to address the SSI, setting measurable objectives informed by their values and priorities to assess the impact of each alternative, and considering the tradeoffs of each alternative based on the objectives. Engaging in SDM provides learners with experience considering various objectives regarding an SSI, considering the tradeoffs of different solutions across multiple stakeholders, and integrating scientific knowledge with social dynamics to select a preferred solution to the SSI. Therefore, we suggest that engaging in SDM may facilitate learners’ development of socioscientific argumentation skills. Below, we summarize several studies that inform our work and that have investigated the quality of socioscientific arguments.

The dominant framework in argumentation is Toulmin’s (1958) model, referred to as Toulmin’s Argumentation Pattern (TAP); although TAP is not explicitly designed for socioscientific argumentation, it forms the basis of many other argumentation frameworks, and warrants a brief discussion. TAP classifies arguments into six idealized components: a claim, or the assertion about the issue; data, or the facts used to support the claim; warrants, which justify the relevance of the data to the claim; backings, which support the use of the warrant to connect the claim and the data; qualifiers, or the conditions under which the warrant is relevant; and rebuttal, which addresses the opposing view [33,38,39]. Claims, warrants, data, and backings are classified as field-invariant, or not varying based on context, yet TAP also asserts that what qualifies as an appropriate claim, warrant, datum, or backing is field-dependent [38,39]. Critics of TAP have argued that that this overlap complicates the assessment of arguments [33,38,40], leading to many scholars modifying the framework (e.g., [41,42]) or using another framework entirely.

Many frameworks derived from TAP assess arguments by numerically scoring arguments (e.g., 1–5 as in [41,43], 0–5 as in [42], 0–4 as in [44,45,46,47]), with higher scores indicating the presence of more argument components, typically some combination of data, warrants, backings, and rebuttals. While distilling complex arguments into a numerical score provides a holistic view of the quality of the argument, it does not identify any variability in the components of the argument; for example, how learners employ rebuttals in their arguments, the type of data used to support the claim, the way social objectives are incorporated into the argument, etc. Several other argumentation frameworks have been used as coding schemes to assess specific aspects of learners’ socioscientific arguments, which we briefly explore below.

In a study investigating the relationship(s) between content knowledge, morality, and SSI argumentation, Sadler and Donnelly scored learners’ verbal arguments for three criteria: position and rationale, rebuttal, and multiple perspective-taking [48]. The criterion of position and rationale echoed argumentation frameworks inspired by TAP in that the authors sought to identify the extent to which learners established a position on an SSI scenario and supported that position with justifications such as data, warrant, and backings [48]. Similarly, the criterion of rebuttal specifically addressed the inclusion of a counterposition (i.e., a position different from their own) and rebutting that counterposition [48], echoing the rebuttal component of TAP. The third criterion, multiple perspective-taking, related to the moral considerations of SSI and evaluated how learners considered and included perspectives other than their own in their arguments [48]. This third criterion connects to the social objectives that students need to incorporate when evaluating SSI arguments. The scores from these three criteria were ultimately summed to give each argument a score from 0 to 6, with results finding no relationships among the variables measured, potentially due to the difficulty in quantitatively measuring complex constructs such as morality [48].

Christenson and Chang Rundgren (2015) proposed a framework that incorporates aspects of TAP to assess learners’ socioscientific arguments, in which arguments are assessed for claims and justifications (justifications include data, warrants, and backings) [49,50]. In their framework, justifications are organized hierarchically into pros or cons, then further divided into statements relating to conceptual knowledge regarding the issue or statements of values, which they described as broad moral principles or statements of personal preferences [49]. Analyzing students’ values statements connects to social objectives that play an important role in SSI argumentation. They described higher quality arguments as those including a claim, justifications on both sides of the issue (i.e., pro and con), and justifications including both values and relevant content knowledge [49]. They developed this framework with the goal of being easily implemented by teachers wishing to assess learners’ arguments, rather than their earlier and more complex SEE-SEP model [49,51]. The SEE-SEP model integrates six dimensions of SSI (sociological/cultural, environmental, economic, science, ethics/morality, and policy) and knowledge, values, and personal experiences [51]. Learners’ arguments were not assessed for quality, but rather coded based on their use of SSI dimensions relating to knowledge, values, and experiences, ultimately finding that learners invoked their own values as reasons supporting their argument more frequently than knowledge or personal experience [51].

Other scholars have evaluated socioscientific arguments based on the skills demonstrated or resources drawn upon during argumentation. Rundgren et al. (2016) assessed learners’ individual written arguments in response to a question asking learners to evaluate the decision of the Swedish government to adopt an exemption from EU regulations regarding environmental toxins in seafood [52]. Arguments were analyzed for the presence of five skills of informal argumentation: making claims; providing supporting reasons; presenting counterarguments; recognizing qualifiers of the claim; and evaluating arguments [52]. Again, arguments were not explicitly assessed for quality, although results demonstrated that while the learners made different claims regarding the Swedish government’s decision, all (n = 7) accounted for counterarguments and limitations in their arguments [52]. Argumentative resources have also been a topic of study in socioscientific argumentation; Jafari and Meisert (2021) classified argumentative resources that learners invoked when constructing written arguments before and after a dialogic argumentation intervention that provided learners with an SSI context, background on the SSI, and scaffolded their construction of arguments on two sides of the issue [53]. They specifically categorized learners’ argumentative resources as normative (i.e., moral principles and personal preference) or fact-based (i.e., consequences of potential outcome, current circumstances of stakeholders involved) [53]. After the argument intervention, they observed an increase in learners drawing upon moral principles and both types of fact-based resources and a decrease in learners drawing upon personal preferences in the post-arguments [53]. Notably, the authors’ coding scheme defined personal preferences as “personal interests as well as other peoples or other groups’ interests,” which aligns with the socioscientific competency of perspective-taking and importance of considering multiple perspectives [12,48]. The authors noted that the increased use of moral principles was appropriate in light of the complexity associated with SSI, but in their discussion, described personal preferences as “rather unspecified resources” and “simple and vague […] offer little potential for verification beyond the individual” [53]. However, they also acknowledged the importance of learners expressing their personal preferences in decision-making and argumentation, which suggests a challenge in recognizing how students successfully integrate personal preference (i.e., multiple perspectives and perspective-taking) into arguments alongside moral principles and fact-based resources.

Finally, in another recent study, learners were provided with a description of an SSI, asked their stance on the issue, and then asked to create any combination of social, ethical, economic, scientific, and ecological arguments to support their position [54]. Individual arguments were scored based on three categories: first, the number of arguments in each category listed above; second, the number of argument components present in each argument (supportive statements, counterarguments, and rebuttals); and third, the diversity of argument components and arguments [54]. Arguments were then classified on a scale from 0 to 4, with learners who provided multiple types of arguments, included all argument components, and employed these argument components in multiple types of arguments scored as having higher-quality arguments [54]. Results found that students’ epistemic beliefs and prior knowledge predicted higher-quality arguments, counterarguments, and rebuttals, as well as greater diversity of arguments, with higher-quality arguments typically including ethical, economic, and scientific components [54].

Taken together, the studies discussed above provide insight on learners’ construction of socioscientific arguments, particularly that they: incorporate multiple perspectives into arguments [48]; use their own values and content knowledge as justification in constructing arguments [49,50]; draw upon both normative and fact-based argumentative resources [53]; and construct arguments based on different domains of knowledge and stakeholders [54]. These studies also demonstrate that instructional interventions (e.g., Jafari and Meisert’s (2021) SSI lesson) can prompt learners to address counterarguments [52] and incorporate scientific evidence alongside moral principles in their arguments [53]. It is also evident that the field of SSI education lacks a clear consensus on what constitutes a high-quality socioscientific argument, as well as an instrument or framework used to assess such arguments. While TAP is widely used, the drawbacks of this framework have been extensively discussed in prior literature (e.g., [38,41,42]). Other frameworks focus on narrow aspects of socioscientific arguments (e.g., [52] assessing informal argumentation skills, [53] classifying argumentative resources), and those that do look at an argument holistically (e.g., [42,43,44,45,46,47]) often distill each individual argument into a single numerical score, losing the opportunity to assess how learners use various argumentative components and resources to construct socioscientific arguments. This demonstrates the need for researchers to explore other argumentation frameworks for evaluating socioscientific arguments, particularly those that provide opportunities for learners to incorporate personal priorities, values, and social factors into their arguments. Employing such novel argumentation frameworks may provide more insight on how we collectively define a high-quality socioscientific argument, assess such arguments, and build learners’ skills in constructing such arguments.

2.2. Critical Integrative Argumentation

We approach SSI argumentation through the theoretical lens of Nussbaum’s critical integrative argumentation (CIA) [55]. As previously discussed, the dominant argumentation model in science education is Toulmin’s Argumentation Pattern [39], though its drawbacks include that it does not provide standards for evaluating the strength of arguments [29,38,41] and does not have clear mechanisms for incorporating a weighing of social or personal priorities and values, which are inherent components of SSI. In contrast, CIA notes when students provide a claim and support their claim with evidence, but also when students pay attention to reasoning on the opposing side as their own claim (counterarguments) and address said counterargument [55]. Stronger arguments are characterized by identifying counterarguments, and further strengthened by refuting these counterarguments. CIA identifies three types of refutations: basic refutations, which directly refute the conclusion of a counterargument or otherwise explain that the counterargument is flawed [56]; design claims, which consider both sides of a claim and integrate both sides into a new solution that circumvents, reduces the severity of, or avoids a problem entirely; and weighing, which integrate the costs and benefits, or tradeoffs, of both sides of a claim [55,56]. Design claims and weighing are both opportunities for students to attend to the social or personal priorities and values inherent in SSI. Employing CIA as a framework for analyzing socioscientific arguments allows for a holistic assessment of the argument (i.e., do learners provide a claim, evidence, counterarguments, and various refutations), while describing and identifying these specific types of refutations provides a more fine-grained perspective on individual arguments and may allow researchers to identify how values and personal priorities are present in such arguments, as opposed to many of the TAP-inspired frameworks discussed above that combine all types of refutations into the category of “rebuttals.” Therefore, this study uses CIA as a novel framework for analyzing learners’ socioscientific arguments and a potential avenue for the SSI education community to collectively define and assess such arguments.

2.3. Decision-Making and Critical Integrative Argumentation

As discussed above, this work draws upon literature in the decision sciences to engage learners in SSI-based structured decision-making (SDM; see Figure 1). SDM requires that learners identify potential solutions to an SSI, set measurable objectives that a solution should address, evaluate evidence regarding the impact of each potential solution on the objectives, consider the tradeoffs of each potential solution, and ultimately integrate scientific, social, economic, ethical, and ecological perspectives to select a preferred solution. In previous work, instruction using SDM has shown gains in many areas within STEM literacy, including socioscientific reasoning competencies (i.e., complexity, perspective-taking, inquiry, and skepticism) [12], learner self-efficacy in finding and applying scientific evidence [24], use of explicit scientific evidence to support claims [37], awareness of multiple potential solutions to SSI [37], civic engagement attitudes [57], and social justice, interpersonal and problem-solving, and political awareness skills [57].

We suggest that engaging in SDM may foster another critical skill in STEM literacy, specifically learners’ construction of integrated socioscientific arguments as defined by CIA. We see considerable overlap between SDM and CIA; namely, the goal of CIA is to weigh evidence regarding the plausibility of possible explanations, solutions, or scientific models [56], which aligns strongly with practices of decision-making in that high-quality decision-making is characterized by learners comparing multiple solutions, weighing them by considering tradeoffs in desired outcomes, and determining which solution is most plausible based upon the objectives set forth [24,58]. In the course in which this study was conducted, learners become “experts” on specific potential solutions for the SSI topic, then engage in dialogic argumentation to evaluate the impact of each potential solution on the desired objectives and reach consensus on how well each potential solution meets each objective [24,58]. In doing so, they become familiar with several potential solutions, the evidence supporting and refuting the plausibility of each solution, and the tradeoffs of each solution. These argumentation sessions provide learners with opportunities to build their argumentation skills, such that they may demonstrate growth in making and supporting claims, identifying counterarguments, and providing basic refutations and/or integrated refutations to those counterarguments. Therefore, we predict that instruction emphasizing evidence-based decision-making, in which learners are prompted to integrate multidisciplinary perspectives on an SSI and consider tradeoffs of various solutions, may foster stronger integrated socioscientific arguments, such that learners are more likely to identify and refute counterarguments after instruction and transfer these skills to novel SSI contexts. Given that STEM literacy endeavors to prepare learners to solve complex problems, make well-reasoned decisions, and effectively participate in democratic society, SDM and CIA provide valuable contexts in which learners may develop skills to meet these overarching goals.

3. Research Questions

Do learners’ socioscientific argumentation skills:

• change after completion of a course using a structured decision-making framework to assess alternative solutions to SSI?
• transfer from an SSI discussed in class to a novel SSI?

4. Methods

4.1. Participants and Setting

This study used a mixed methods approach to answer the research questions listed above. Specifically, we used both quantitative and qualitative analyses to assess students’ argumentation skills in a data transformation design [59]. The data transformation design was appropriate for our research because it allowed us to analyze qualitative data, transform it into quantitative data suitable for statistical analysis, and then return to the qualitative data to explore the patterns observed in the results of the quantitative analysis [59]. The specific quantitative and qualitative analyses are described in Section 4.2 and Section 4.3 below. This study was conducted in the context of an introductory, integrated STEM/multidisciplinary, large-enrollment course with learning objectives emphasizing science-informed decision-making, information literacy, and systems thinking [24,57]. This course is required for all students in the College of Agricultural Sciences and Natural Resources at the University of Nebraska-Lincoln, with both STEM (two-thirds of students) and non-STEM (one-third) majors. Approximately 600 students take this course throughout the academic year, with about 120 students in each course section. The course includes explicit instruction on cognitive biases that compromise decision-making (e.g., confirmation bias, availability heuristic, anchoring) and uses a structured decision-making (SDM) process described above (Figure 1) to guide learners’ decision-making around socioscientific issues. During the course, the SSIs of plastic pollution, water conservation, and an SSI of learners’ choosing for a final project served as a backdrop to practice the learning outcomes. Argumentation is not explicitly taught, although the SDM process supports learners in creating an argument that incorporates scientific evidence to determine potential outcomes and weigh the outcomes of multiple alternative solutions, which is essentially integrating refutations across multiple solution claims.

Data were collected in a section of the course that was taught online in the spring semester of 2021. Ninety students completed both assessments and consented for their coursework to be used for research. Demographic data for the students in this study were not collected; however, the population of students within the College of Agricultural Sciences and Natural Resources during the spring of 2021 were 54% female, 46% male; 78% white, 11% nonresident alien, 5% Hispanic, 2% two or more races, and <1% Asian, <1% Black or African American, and <1% Native groups; and 30% first-generation college student. All research was conducted in accordance with University of Nebraska-Lincoln IRB approval (#20140813907EP).

4.2. Argumentation Task

To assess students’ socioscientific arguments, we developed a qualitative argumentation task administered as a pre- and post-assessment that provided background information about both SSI employed in the course (plastic pollution and water conservation) and the SSI of wind energy as a transfer task, and asked students to evaluate a given alternative solution to the SSI. For plastic pollution and water conservation, the given alternative was one of the alternative solutions used in the course during the SDM process. The text of each prompt can be found in

Table 1. Prompts for pre- and post-assessment of socioscientific argumentation.

SSI	Argumentation Prompt
Plastic pollution	There is an increasing amount of plastic in our oceans. Currently between 5 to 14 million tons of plastic enter the ocean each year, which is projected to increase. The World Economic Forum estimated that if we continue at this rate, plastics in the ocean will outweigh fish pound for pound by 2050. Plastics enter the ocean through rivers after being discarded in the environment, especially in locations without trash management facilities. Plastics in our waterways are a problem because they pose health hazards to birds, fish, turtles, and other wildlife, and may even ultimately impact human health. Single-use plastics (plastic bags, packaging, straws, cups, etc.) are used only once and thrown away, however they are inexpensive for consumers and companies who use them. Packaging (one type of single-use plastic) accounts for nearly half of all plastic waste generated globally, and most of it never gets recycled or incinerated. Given this issue, should we ban single-use plastics in the United States?
Water conservation	Nebraska irrigates approximately 10 million acres for agricultural production. That is more than any other state in the U.S., and more than every country (except Mexico). Some areas in Nebraska restrict groundwater irrigation for agriculture by giving farmers an allocation (a set amount of water that they can use over a certain number of years), however most areas in Nebraska have few to no restrictions on groundwater use. The groundwater is from the Ogallala Aquifer, which extends across Nebraska, Kansas, Oklahoma and Texas, and has been depleted by about 9% in general. In Nebraska, the aquifer is much deeper than in other states, and there are some areas in Nebraska that experienced no depletion and others, like southwestern Nebraska, that have seen significant aquifer declines since the 1970′s. If the aquifer is depleted, it may take thousands of years to replenish naturally through rainfall. Farmers in Nebraska widely rely on groundwater irrigation for consistent yields. Across the entire Great Plains, groundwater irrigated farmland contributes about $1.6 billion to the economy. Given this issue, should we further restrict irrigation for agriculture in Nebraska?
Wind energy (transfer task)	Electricity in Nebraska is sourced mainly from coal (55%), with smaller percentages of power coming from wind (20%) and nuclear power (19%). Nebraska ranks 14th highest in the United States in the amount of wind power it produces, although it still has more wind energy potential, and ranks as one of the top states for undeveloped wind energy. When coal is burned for power, it produces toxins such as sulfur dioxide and greenhouse gases, which contribute to global warming and human health impacts such as asthma and heart disease. Generation of electricity from wind does not produce these emissions. Concerns about wind farms include impacts on wildlife, including birds and bats, with potential harm to endangered species. Wind turbines may also produce a lot of noise, disrupting nearby homeowners and impacting property values. The state of Nebraska has invested $80 million in wind farm construction and receives $12 million per year in tax revenue from private wind farms. The placement of wind turbines is regulated in most Nebraska counties with limits on how close they can be located to home or property lines. Some counties have very large distance requirements which can discourage wind development. Given this issue, should Nebraska reduce restrictions on wind farm placement?

. Following each prompt, students were asked to provide a stance on the given alternative, then respond to the following sub-questions. In each sub-question, “the given alternative” stands in for the alternatives proposed in the prompt.

• Give as many reasons as you can for why we should, or why we should not, implement [the given alternative]. Please justify your responses with reasoning and evidence.
• Besides [the given alternative], are there other solutions to the issue that you can think of?
• Are these other solutions better or worse solutions than [the given alternative]? Explain why they are better or worse.

The structure of these sub-questions was intended to provide scaffolded opportunities for students to first identify their stance on the SSI, then (a) provide reasons and evidence to support their stance, which could integrate counterarguments and/or basic refutations, (b) consider other possible solutions to the SSI, thus potentially identifying counterarguments and/or fostering the development of design claims, and (c) compare possible solutions to one another by examining the tradeoffs of each solution (i.e., weighing).

The pre-assessment was completed before the start of instructional material in the course, with the post-assessment given as part of a required final that was awarded points for completion. Both assessments were administered using Qualtrics. At the conclusion of the semester, responses were downloaded, deidentified, and randomly reordered for analysis. The responses to all three sub-questions were combined to create a single unit of analysis.

4.3. Qualitative Coding

We conducted multiple rounds of deductive coding using qualitative content analysis [60] with the goal of applying a critical integrative argumentation framework (CIA) [40,55] to describe students’ responses in the context of socioscientific argumentation. Responses to each SSI were coded separately; the plastic pollution task was the first to undergo coding procedures. Responses were first coded for the presence or absence of six argument components present in the CIA framework: claim, reasons, counterargument, basic refutation, integrated refutation-design claim, and integrated refutation-weighing (described in

Table 2. Qualitative coding scheme developed and used in this study, adapted from [40].

Code	Indicators
Claim	Gives a yes or no response to multiple-choice question regarding given alternative
Reasons	Proposition supporting the final claim (a claim needs to be stated to have a reason). May include reiterating a reason given from question text, initiating an independent line of argument, or offering evidence in the form of specific cases, examples, statistics, and/or citations.
Counterargument	Reason on the other side of the issue than the final claim, for example, a consequence of opposite value. Counterarguments require that students have a claim (rather than selecting “I don’t know”)
Basic refutation	An argument for why a counterargument is flawed or less applicable, not integrated in that it does not address a premise. Acknowledges and evaluates counterarguments, but ultimately rejects them rather than integrating them into the argument. Argues that the conclusion of the counterargument is false.
Integrated refutation	Integrated refutations may be either design claims or weighing refutations, as described below. Design claim: Addresses a counterargument by designing a solution. Would include any supporting reasons. (Counterargument could be implicit if it is a premise explicitly in the question stem’s text). The proposed solution preserves the benefits of an alternative while reducing the negative consequences of a counterargument. The design claim should clearly state how the proposed solution can mitigate negative consequences of counterargument. Weighing: A refutation (i.e., negative assessment of an argument) that weighs something against something else. Weighing could include moral values, amounts, or probabilities. (Counterargument could be implicit if it is a premise explicitly in the question stem’s text). Design claims and weighing refutations may be considered implicit if they address a counterargument that is a premise explicitly stated in the question’s text; when they address a clearly identifiable counterargument, they are considered explicit.

). These coding categories were used in previous work (e.g., [40,55,56]), and we synthesized the descriptions of these categories to create a comprehensive coding scheme. For example, [40] described basic refutations as “An argument for why a counterargument is flawed or less applicable and not falling into another category,” (p. 1541) while [56] used the general term refutation to describe statements that “could be used to show that the conclusion of the counterargument is false or that the counterargument is somehow flawed. This strategy […] is considered the least integrative strategy” (p. 447). By synthesizing these various terms and descriptions, this comprehensive and detailed coding scheme can be more easily applied to various argumentation contexts, including SSI contexts integrating multiple fields within STEM such as our own.

Responses that did not have a claim present (i.e., did not answer “yes” or “no” to the question) were scored as all argument components being absent, given that arguments are dependent upon staking a claim regarding an issue [55]. After coding a subset of responses (n = 25), the coders met to discuss and refine the coding scheme. After a few iterations, we recognized that students often incorporated both types of integrated refutations, design claims and weighing, into the same response. In these cases, their responses were coded as having both types of integrated refutations, leading to the total number of integrated refutations being artificially inflated by some students having both present. Therefore, we combined the design claim and weighing categories into one “integrated refutation” category. In this iterative process, we also identified responses that contained an integrated refutation without explicitly stating a counterargument, but referencing a counterargument related to one of the premises contained in the prompt. Thus, we coded these as valid integrated refutations and noted when counterarguments were implicit (referencing a premise) or explicit (stated a clearly identifiable counterargument). Table 2 describes the final version of the coding scheme used in this study.

After fully developing the coding scheme, two coders independently coded 25 student responses for the plastic pollution argumentation prompt and met to compare and discuss codes. Interrater reliability (IRR) was calculated using Cohen’s kappa (κ) due to the binary nature of the codes as presence or absence of each argument component [60]. However, Cohen’s kappa is susceptible to a phenomenon called the prevalence problem, which causes kappa values to be unrepresentatively low when the distribution of observed codes is skewed to one category or the other (i.e., presence is coded far more than absence or vice versa) [61,62]. We observed this in our data set; for example, when calculating IRR for a group of 25 responses, the coders agreed that basic refutations were absent in 24 of the 25 responses with one disagreement, yet IRR was calculated as κ = –2.78×1015. In another group of 25 responses, both coders agreed that basic refutations were present in 3 responses and absent in 21 responses with one disagreement; in this case, IRR was calculated as κ = 0.834. Due to these inconsistencies, we chose to report in text an adjusted kappa that corrects for the prevalence problem [61,62,63]; we provide Cohen’s kappa and percent agreement for the first group of responses and all responses in Table S1. For the first group of 25 responses to the plastic pollution prompt, IRR demonstrated substantial agreement above κ = 0.60 in all coding categories [64,65], with κ_claims = 1.00, κ_reasons = 0.76, κ_{counterarguments} = 0.92, κ_basic = 0.76, and κ_integrated = 0.68. All disagreements were resolved through discussion, which also served to further clarify the coding scheme. This was repeated for another set of 25 responses at a time until all 180 responses were coded. Overall IRR for all plastic pollution responses was κ_claims = 1.00, κ_reasons = 0.911, κ_{counterarguments} = 0.689, κ_basic = 0.756, and κ_integrated = 0.623, demonstrating substantial agreement with disagreements resolved through discussion. Exemplars from the plastic pollution responses were added to the codebook (see Table S2) to further support shared interpretation of the coding scheme.

After coding the plastic pollution argumentation task, the coding process was repeated for the wind energy transfer prompt, although these responses were coded in groups of 30 rather than 25. Again, coding began with both coders independently coding the first group of responses and meeting to compare and discuss codes. IRR remained high for each coding category: κ_claims = 1.00, κ_reasons = 1.00, κ_{counterarguments} = 0.867, κ_basic = 1.00, and κ_integrated = 0.933. All disagreements were resolved through discussion. This was repeated for each group of 30 responses until all 180 responses were coded. Overall IRR for all wind energy responses was κ_claims = 1.00, κ_reasons = 0.978, κ_{counterarguments} = 0.867, κ_basic = 0.856, and κ_integrated = 0.789, with disagreements resolved through discussion, and showing an increase in all coding categories compared to the plastic pollution responses.

Based on the consistently substantial IRR on both the plastic pollution and wind energy responses and the increase in overall IRR between the plastic pollution prompt and wind energy prompt, coding for the water conservation prompt began with both coders independently coding 20% (36) of the responses to determine initial IRR. IRR remained high for this prompt as well: κ_claims = 1.00, κ_reasons = 0.944, κ_{counterarguments} = 0.889, κ_basic = 0.833, and κ_integrated = 0.889. Therefore, the remainder of the responses to the water conservation prompt were coded by one coder.

4.4. Quantitative Analysis

Given that the goals of this study were to (a) identify changes in students’ argumentation skills over time and (b) if changes were present, determine if they transferred to a novel SSI context, we conducted a quantitative analysis of the frequency of each argument component between the pre- and post-assessment. We used the Related-Samples McNemar Change test with a binomial distribution for this analysis because of the binary nature of the data (i.e., responses were coded for the presence or absence of each argument component) and the dependence of the pre- and post-assessments. This accounts for students who began the semester with strong argumentation skills (i.e., proposing integrated refutations) and maintained those skills, while a chi-square or binomial frequency test of significance would look solely at the frequency of each argument component in the overall sample [66]. This analysis was conducted for each argument component in each SSI prompt using SPSS Statistics for Mac Version 28.0. To complement the data transformation research design, we used the results of the quantitative analysis to identify significant changes or other noteworthy trends in the data, and subsequently returned to the qualitative data to identify relevant student responses to illustrate these trends [59].

5. Results

5.1. Plastic Pollution

The plastic pollution argumentation task asked students if single-use plastics should be banned in the United States, then to provide reasons for their response, suggest other solutions to the issue, and evaluate if the alternative solutions were better or worse than banning single-use plastics. The number of students in favor of, opposed to, or undecided regarding a ban on single-use plastics is shown in

Table 3. Student stances on the plastic pollution argumentation task in the pre- and post-assessment responding to “Given this issue, should we ban single-use plastics in the United States?” (full question text in Table 1).

	Pre-Assessment: In Favor of Ban	Pre-Assessment: Opposed to Ban	Pre-Assessment: Undecided	Post-Assessment Totals
Post-assessment: in favor of ban	36	7	13	56
Post-assessment: opposed to ban	7	18	3	28
Post-assessment: undecided	2	2	2	6
Pre-assessment totals	45	27	18

. Fifty percent of students (45/90) were in favor of banning single-use plastics in the United States on the pre-assessment; on the post-assessment, 36 of these students did not change their stance, while 7 changed their stance to being opposed to banning single-use plastics and 2 changed their stance to being undecided. Thirty percent of students (27/90) were opposed to banning single-use plastic in the United States on the pre-assessment; on the post-assessment, 18 students maintained this stance, 7 students changed their stance to being in favor of banning single-use plastics, and 2 changed their stance to being undecided. Finally, 20% of students (18/90) were undecided on the pre-assessment; on the post-assessment, 13 students were in favor of banning single-use plastics, 3 students were opposed to banning single-use plastics, and 2 remained undecided. In total, 37.78% of students (34/90) changed their stance on the plastic pollution task between the pre- and post-assessment.

While students’ responses varied greatly in their use of argumentation components, many of the responses in favor of banning single-use plastics referenced impacts on wildlife, the number and mass of single-use plastic in oceans, and the low percentage of single-use plastics that are recycled. For example, one student responded:

It is more harmful for everyone in the long run if we continue to use single use plastic. In order to prevent even more intense restrictions or crisis in regards to our environment, simply removing these types of plastics from everyday use will be simpler in the long run. Additionally, if the entire country gets on board with the idea of not using these plastics, it will be fairly easy to shift around these changes in society because it will become the norm. Since the biggest issue in regards to plastic getting into the ocean revolves around it getting into other waterways first, another option to help the pollution would be to create filters that skim the surface of waterways to collect the plastic floating in them. It would be a complicated process with a lot of roadblocks and issues to work around such as how can people and machinery still use these waterways without disrupting the filter system. A positive would be that for most people it wouldn’t affect their daily lives, but it would be an expensive project for the country to go through. Additionally, it doesn’t address the source of the issue like banning single use plastics instead it just deals with the consequences. [pre-assessment]

Students who were opposed to banning single-use plastics frequently cited economic concerns, a lack of suitable alternatives to single-use plastics, and the convenience of single-use plastics. For example, one student responded:

Overall, I don’t think we should ban single-use plastics. I think there’s a way to limit the amount we use but banning them altogether would take millions of people out of their jobs. It would also not solve the problem altogether because of the trillions of pieces that are already in the ocean. [Other solutions would be] definitely just different incentives when promoting plastic use. Cleaning up projects would also be another solution because of the large amounts in the ocean. Biodegradable plastics was an alternative discussed but they could only work in certain climate conditions that are not normally found in the ocean. Besides biodegradable plastics which is a worse idea, I think others altogether wouldn’t be as effective but could still help. [post-assessment]

After scoring the responses as described above, the Related-Samples McNemar Change test was conducted to determine if the frequency of any argument components significantly changed between the pre- and post-assessments. There was a statistically significant difference in the use of claims (p = 0.012) and reasons (p = 0.011) between the pre- and post-assessment task, while the use of counterarguments, basic refutations, and integrated refutations did not show significant change (

Table 4. Counts of each argumentation component and results of Related-Samples McNemar Change test on the pre- and post-assessment plastic pollution argumentation task (n = 90). * indicates statistical significance at α = 0.05.

	Pre-Assessment	Post-Assessment	Test Statistic	Significance
Claim	72	84	6.050	0.012 *
Reasons	67	81	6.500	0.011 *
Counterargument	28	32	0.237	0.626
Basic refutation	8	16	3.063	0.077
Integrated refutation	27	29	0.029	0.864

). Although the number of basic refutations present increased from 8 to 16, which appears to be a large increase, the crosstabulation table generated from the Related-Samples McNemar Change test showed that 70 out of 90 students did not provide a basic refutation on either assessment, 4 students provided a basic refutation on both assessments, 12 provided a basic refutation on only the post-assessment, and 4 students provided a basic refutation on only the pre-assessment. Thus, the details provided by the crosstabulation table provides important context regarding how individual students’ responses changed.

Although there was no significant change in the frequency of integrated refutations between the pre- and post-assessment, examining the integrated refutations provided sheds some light on how students identified and addressed counterarguments to their claim. While analyzing the quality of integrated refutations was beyond the scope of our research questions in this study, we noted qualitative differences in the types of design claims proposed; many pre-assessment integrated refutations included design claims on the individual level, as shown below.

I don’t think we should ban single use plastics because there are so many companies that can’t afford to buy more expensive plastics and materials. One solution that I can think of is reusable cups and discounts. For example, allow people to buy a $1 reusable cup at restaurants and gas stations and provide a rewards program or discount if the cup is used. These are better because people will have an option to use something that is reusable and businesses won’t go broke trying to purchase a more expensive material. [pre-assessment]

On the post-assessment, many integrated refutations included macro-level design claims such as implementing a tax on plastic production or promoting the use of biodegradable plastics. For example, the response below explains that they are in favor of implementing a ban on single-use plastics in the United States, provide reasons for this stance, identify the counterargument that eliminating single-use plastics would likely be an unpopular policy, and posits a design claim of transitioning to biodegradable plastics, which they state are safer for humans and the environment and still allow single-use plastics to be used.

Yes [we should ban single-use plastics] because of the amount of pollution already on this earth and the evidence that has been given for how it effects [sic] the earth. The plastic pollution has evidence of significant negative effects on human health. [Banning] single use plastics would reduce the amount of plastics and would in turn positively impact human health. I think that there are solutions for plastic pollution like biodegradable plastics. They are able to keep the use and be environmentally safe and also safe to human health because they are biodegradable. I think that biodegradable is better because people do not want to completely get rid of the use of plastics. So this way the pollution around the world would be able to be solved. Along with this it would be able to be implemented because people are very innovative and want to do the best for the effects on the earth and human health. [post-assessment]

Similarly, while the frequency of basic refutations did not significantly differ between the pre- and post-assessment, we did note that basic refutations on the pre-assessment tended to contain assumptions about the impact of a ban (e.g., “Banning them all will not stop it. People will still get plastics as they do with anything else that is already banned”) rather than citing evidence to refute the claim that banning single-use plastics in the United States would be effective. However, on the post-assessment, many of the basic refutations included evidence supporting their claim that a ban on single-use plastics in the United States would not address the issue of global plastic pollution, as shown in the response below.

While the United States doing our part to cut back on plastic pollution would help the overall picture, most of the plastic waste on Earth comes from other countries that don’t have waste management facilities. With that being said, this would just cause a rift in society while not really helping the overarching issue, which is global plastic pollution, not American plastic pollution. [post-assessment]

Some responses containing this common basic refutation also provided design claims and considered tradeoffs of different solutions to the problem of plastic pollution, as shown in the response below.

While single-use plastics comprise a large portion of the plastic responsible for plastic pollution, banning single-use plastic in the United States will likely be largely unsuccessful in addressing the plastic pollution problem because the 90% of all plastic pollution arrives in the ocean through a mere ten rivers. None of these rivers are in the US. While good plastic management is important in the US, the plastic pollution occurring around the world is primarily from foreign countries with little to no waste management systems. Thus, banning single-use plastic would not address the heart of the issue. [Another solution would be] implementing a tax on plastic-producing corporations to generate funds for creating waste management facilities/procedures in foreign countries. This alternative cuts right to heart of the issue by addressing the largest source of the current plastic pollution issue. Thus, this alternative significantly outperforms banning single-use plastic in the United States. One major downfall to this alternative is that American plastic companies are paying for the poor waste management of other countries. I do not think plastic companies should pay for other people’s ill use of their products just because the companies could afford to. [post-assessment]

This student refuted the assumption that banning single-use plastics in the United States would sufficiently address plastic pollution, then proposed a design claim of taxing plastic-producing companies and using those funds to create and maintain waste management infrastructure in countries that are major sources of plastic pollution. They also considered the tradeoffs of this solution, namely that US-based companies would essentially be funding waste management in other countries. Thus, it is worth noting that although the statistical analysis did not detect significant changes in the overall frequency of counterarguments, basic refutations, and integrated refutations, some students did provide more sophisticated refutations to the plastic pollution task on the post-assessment as shown above.

5.2. Water Conservation

The water conservation argumentation task asked students if agricultural irrigation should be further restricted in Nebraska, then to provide reasons for their response, suggest other solutions to the issue, and evaluate if the alternative solutions were better or worse than restricting agricultural irrigation. The number of students in favor of, opposed to, or undecided regarding further restricting agricultural irrigation in Nebraska is shown in

Table 5. Student stances on the water conservation argumentation task in the pre- and post-assessment responding to “Given this issue, should we further restrict irrigation for agriculture in Nebraska?” (full question text in Table 1).

	Pre-Assessment: In Favor of Restrictions	Pre-Assessment: Opposed to Restrictions	Pre-Assessment: Undecided	Post-Assessment Totals
Post-assessment: in favor of restrictions	16	8	14	38
Post-assessment: opposed to restrictions	13	21	7	41
Post-assessment: undecided	1	3	7	11
Pre-assessment totals	30	32	28

. In total, 33.33% of students (30/90) were in favor of further restricting agricultural irrigation in Nebraska on the pre-assessment; on the post-assessment, 16 of these students did not change their stance, while 13 changed their stance to being opposed to further restricting agricultural irrigation and 1 changed their stance to being undecided. Additionally, 35.56% of students (32/90) were opposed to further restricting agricultural irrigation in Nebraska on the pre-assessment; on the post-assessment, 21 students maintained this stance, 8 students changed their stance to being in favor of further restricting agricultural irrigation, and 3 changed their stance to being undecided. Finally, 31.11% of students (28/90) were undecided on the pre-assessment; on the post-assessment, 14 students were in favor of further restricting agricultural irrigation in Nebraska, 7 students were opposed to further restricting agricultural irrigation, and 7 remained undecided. In total, 51.11% of students (46/90) changed their stance on the water conservation task between the pre- and post-assessment.

Many responses in favor of further restricting water for agricultural irrigation in Nebraska referenced the importance of preserving water for the future, concerns about aquifer levels, and the amount of water that Nebraska uses to irrigate. For example, one student responded:

Restricting irrigation for agriculture isn’t ideal but it will likely be a necessity if we are to preserve water for the future. Overusing our water reserves will lead to a future where we might not even have water for human use, let alone agriculture, so to ensure that we still have a stable supply of water far into the future we should step in now to make sure that not too much is used. [Other solutions are] more efficient irrigation systems could be developed, or we could engineer new supplies of water. More efficient irrigation systems sound good, but in practice they just lead to farmers using more water because it is cheaper to water the same amount of land. Using engineering, we could alter the flow and supply of water in certain areas that need it, and this is probably the best solution. [post-assessment]

Students who were opposed to further restricting water for agricultural irrigation frequently cited the importance of irrigation to agricultural yields, farmer economics, and the domestic and international food supply. For example, one student explained that although a large amount of water is being used, that water is critical to agricultural production, as shown below.

None of this water from the Ogallala Aquifer being used for irrigation is being wasted. It is all being used to create other resources that our country needs and cannot survive without, which is the production of agriculture. Agriculture is the base of almost every product that we use, the amount of things that corn is used for is essential for the way we live life. In addition, we need to figure out how we are going to feed the growing population and production in Nebraska relies on irrigation. [Other solutions could be] practices that improve irrigation water management; crop residue and tillage management; nutrient and pesticide management; grazing systems; and wetland restorations. I think these solutions are better for farmers and production, but are not as effective just because restricting irrigation is a larger way to conserve and you can see the results. [pre-assessment]

Other students addressed the issue at a more local scale, considering current management practices and the livelihood of farmers in Nebraska, such as the response below.

I do not believe that we should restrict this because people need food to survive, and if the crops are not good, farmers lose their crops which means they lose their food. It has also been studied and found that farmers and people in general are likely to reduce water use if they are left alone and shown the problem. If they can see that new technology can reduce the water usage while increasing crop production, they will buy this technology and reduce their water usage. I think that one solution would be to show farmers that new technology that reduces the amount of water used can actually increase the farmers yield and profit. Studies have found that new technology can increase profit while using less water. I believe that this is better because it is leaving it up to the farmer to decide. Most farmers would see that this reduces the water usage, which costs them less, and actually can increase crop yield which makes them more money. [post-assessment]

After scoring the responses as described above, the Related-Samples McNemar Change test was conducted to determine if the frequency of any argument components significantly changed between the pre- and post-assessments. There was a statistically significant difference in the use of claims (p < 0.001) and integrated refutations (p = 0.003) between the pre- and post-assessment task, while the use of reasons, counterarguments, and basic refutations did not show significant change (

Table 6. Counts of each argumentation component and results of Related-Samples McNemar Change test on the pre- and post-assessment water conservation argumentation task (n = 90). * indicates statistical significance at α = 0.05.

	Pre-Assessment	Post-Assessment	Test Statistic	Significance
Claim	62	79	10.240	<0.001 *
Reasons	60	71	3.448	0.063
Counterargument	19	22	0.211	0.648
Basic refutation	4	10	1.786	0.180
Integrated refutation	15	32	8.828	0.003 *

). Based on the observed significant change in integrated refutations, we totaled the number of integrated refutations that contained an implicit counterargument (referencing a premise in the question) and an explicit counterargument (stated a clearly identifiable counterargument which was coded as a present counterargument) in

Table 7. Number of integrated refutations identified in responses to the water conservation task employing implicit and explicit counterarguments.

	Pre-Assessment (n = 15)	Post-Assessment (n = 32)
Implicit counterargument	4	13
Explicit counterargument	11	19

. On the post-assessment, nearly half (43.75%) of the integrated refutations contained an implicit counterargument, compared to only about one-quarter (26.67%) containing an implicit counterargument on the pre-assessment.

Integrated refutations varied widely in the stance that students took on the issue, the evidence used to support their claim, and the design claims and weighing incorporated into their response. On the pre-assessment, integrated refutations often proposed vague design claims (e.g., “We could probably come up with ways to spread our water source across the fields. That would help lessen our water usage but not take away people’s incomes”), suggested design claims that students explicitly stated were less effective than restricting agricultural water use (e.g., “Another solution could be to set water restrictions such as limiting the amount of time people can water the lawns or turning the water off when you’re brushing your teeth. But I think that it would be easier to mandate how much water farmers are using for crops compared to how much water every person is using in their homes”), or offered design claims with major drawbacks (e.g., “I think that the best way to provide water is to collect rainfall or provide a way to turn salt water into clean water. However, changing salt water from the ocean to clean water would negatively impact the environment in the ocean. I don’t believe that I have a better solution right now”).

On the post-assessment, students provided significantly more integrated refutations, employing both implicit and explicit counterarguments. Implicit counterarguments (13 out of 32, 40.625%) were approximately evenly split between being in favor of (7/13, 53.85%) and opposed to (6/13, 46.15%) further restrictions on agricultural irrigation. Implicit counterarguments referred to a premise stated in the question text, such as farmers relying on irrigation for consistent yields or aquifer depletion threatening future water supply. For example, the response below, arguing in favor of further restricting agricultural irrigation, employed the implicit counterargument of farmers relying on irrigation for consistent yields.

I think that there needs to be restrictions because if we continue to use water the way we are now the issue will only get worse and there will be even less use of irrigation in a few years. Other solutions to the loss of water from the aquifer could be to engineer ways to make farmers use water in a more efficient way. There are plenty of things engineers can do and while it may be expensive to start, the benefits outweigh the costs. I think this is a better solution than restrictions because it lets farmers keep doing what they are doing while making them do it more efficiently to stop using as much water. [post-assessment]

In this response, the student’s claim was that agricultural water use should be further restricted, and they provided reasons concerning the loss of water to support their claim. They also refuted the opposing claim that agricultural water use should not be further restricted by posing a design claim (i.e., addressing a counterargument by creating a solution that preserves the benefits of an alternative while reducing the negative consequences of a counterargument). In this case, the design claim is to engineer technology that uses water more efficiently, which will conserve water (preserving the benefit of the restriction alternative) while reducing the negative consequences of that alternative (the implicit counterargument being that farmers in Nebraska rely on irrigation for consistent yields, as evidenced by the statement “it lets farmers keep doing what they are doing”). They also began to consider the tradeoffs of the engineering alternative by stating “There are plenty of things engineers can do and while it may be expensive to start, the benefits outweigh the costs,” although they did not fully explain those costs and benefits.

Implicit counterarguments on the other side of the issue often involved preserving water for the future, such as:

I do not think we should further water restriction because we already have NRDs [Natural Resource Districts] doing their best to try and keep the water level where it is. By further restricting farmers, we could potentially take away a lot of their profit which they use to live and provide for others. Another solution to this problem could be to implement new water saving technology. This would allow farmers to use water efficiently and not waste anything. I think that water saving technology is better than more restrictions because there are times when farmers do need to take out quite a bit of water due to a drought or something of that sort. This water saving technology could determine how much water farmers need to use and when they need to use it. [post-assessment]

In this response, the student was opposed to further restrictions on agricultural water use, and they provided reasons concerning farmer profitability to support their claim. They refuted the opposing claim that agricultural water use should be further restricted by posing a design claim of implementing technology that uses water more efficiently, which will maintain farmer profitability (preserving the benefit of the “opposed to restrictions” alternative) while reducing the negative consequences of that alternative (the implicit counterargument being that groundwater has been depleted, which may take thousands of years to replenish naturally, as evidenced by the statements “we already have NRDs doing their best to try and keep the water level where it is” and “this would allow farmers to use water efficiently and not waste anything”).

The majority of students who provided integrated refutations also included explicit counterarguments (19 out of 32, 59.375%), which were about evenly split between being in favor of (10/19, 52.63%) and opposed to (9/19, 47.37%) further restrictions on agricultural irrigation. The response below argued in favor of further restricting agricultural irrigation.

We should restrict irrigation for agriculture in Nebraska because we need to be ready for whenever there is a time of need and when water is low. Our environment is currently struggling, and changes are being made, but if we can save as much water as possible than we will be prepared. Water is often wasted due to runoff and water vapor, and slowing those two things will make a significant impact. Besides restricting irrigation, Nebraskans could also work with engineers to figure out new ways to use water more effectively. This solution is better than restricting irrigation because than farmers will not have to be so deeply impacted financially. Farmers need water, and should not have to feel nervous about running out of it. Therefore, by engineering new ways to use water, farmers can use a good amount while also limiting the amount that is lost due to runoff or water vaporization. [post-assessment]

In this response, the student’s claim was that agricultural water use should be further restricted, and they provided reasons concerning the human and environmental impacts of water to support their claim. Like the previous student, they also proposed the design claim of engineering technology that uses water more efficiently, but they explicitly acknowledged the counterarguments that farmers need water and farmers could be financially impacted by restrictions on water. Their design claim reduces the negative consequences of that counterargument by proposing a solution that conserves water while still allowing farmers to use appropriate amounts of water for irrigation.

In arguments opposed to further restricting agricultural irrigation, explicit counterarguments often referenced the need to preserve water for the future, such as:

I do not believe we should restrict irrigation. We need irrigation to water our crops, which provide sustenance for the meat we eat. Along with that, farmers are therefore [financially] supplemented. I think, as for irrigation, we should begin by using more sustainable ways of irrigating crops. Instead of center pivot sprayers being high above the crops, we can use adjustable center pivots that can reach closer to the plants. When the sprayers are closer to the plants, the evaporation of water is lessened, therefore we can use less water when irrigating. I think this is a better solution than restricting irrigation. It’s better because it allows us to start using less water in a practical sense rather than restricting it. It helps ensure that we will still have water in the future. [post-assessment]

In this response, the student provided reasons regarding food security and farmer profitability to support their claim that agricultural irrigation should not be further restricted. Their design claim involved using adjustable center pivots (i.e., low energy precision application) to minimize water loss due to evaporation, explicitly acknowledging that water needs to be preserved for the future. This design claim reduces the negative consequences of water depletion by using precision application while maintaining farmer profitability and the ability to use appropriate amounts of water for crops. They also weighed the proposed solutions (restrictions on agricultural irrigation vs. adjustable center pivots) and determined that adjustable center pivots are better because they reduce water loss while avoiding restrictions. These responses reflect the variety of integrated refutations provided, including both explicit and implicit counterarguments on both sides of the water conservation issue.

5.3. Wind Energy

The wind energy argumentation task asked students if restrictions on wind farm placement should be reduced in Nebraska, then to provide reasons for their response, suggest other solutions to the issue, and evaluate if the alternative solutions were better or worse than restricting agricultural irrigation. The number of students in favor of, opposed to, or undecided regarding reducing restrictions on wind farm placement in Nebraska is shown in

Table 8. Student stances on the wind energy argumentation task in the pre- and post-assessment responding to “Given this issue, should Nebraska reduce restrictions on wind farm placement?” (full question text in Table 1).

	Pre-Assessment: In Favor of Reducing Restrictions	Pre-Assessment: Opposed to Reducing Restrictions	Pre-Assessment: Undecided	Post-Assessment Totals
Post-assessment: in favor of reducing restrictions	20	14	6	40
Post-assessment: opposed to reducing restrictions	10	15	7	32
Post-assessment: undecided	7	6	5	18
Pre-assessment totals	37	35	18

. A total of 41.11% of students (37/90) were in favor of reducing restrictions on wind farm placement in Nebraska on the pre-assessment; on the post-assessment, 20 of these students did not change their stance, while 10 changed their stance to being opposed to reducing restrictions on wind farm placement and 7 changed their stance to being undecided. Furthermore, 38.89% of students (35/90) were opposed to reducing restrictions on wind farm placement in Nebraska on the pre-assessment; on the post-assessment, 15 students maintained this stance, 14 students changed their stance to being in favor of reducing restrictions on wind farm placement, and 6 changed their stance to being undecided. Finally, 20% of students (18/90) were undecided on the pre-assessment; on the post-assessment, 6 students were in favor of reducing restrictions on wind farm placement in Nebraska, 7 students were opposed to reducing restrictions on wind farm placement, and 5 remained undecided. In total, 55.56% of students (50/90) changed their stance on the wind energy task between the pre- and post-assessment.

Given that the wind energy was not discussed in the course, many responses in favor of reducing restrictions on wind farm placement directly referenced information in the question text. For example, the response below refers to wind not having the negative health impacts of coal.

Wind energy is healthier for us and our environment. I think the pros outweigh the cons here. Global warming is a major issue today along with climate change. If reducing these restrictions helps with the global warming issue, I completely agree with this solution. I just think eliminating greenhouse gases in general is best for our environment. So if reducing restrictions on wind energy placement helps reduce greenhouse gas emissions I’m all for it. I can’t really think of any alternatives that might be better than this one. I think reducing restriction on wind energy is the best strategy to help with the climate change issue. It is environmentally friendly and does not emit any greenhouse gases that will contribute to the climate change crisis. [post-assessment]

Other responses in favor of reducing restrictions on wind farm placement referenced Nebraska having a great deal of unrealized wind energy potential, such as:

Given the amount of country area our State has, increasing wind farms seems like a no brainer to me. I would like to see the coal use in Nebraska greatly reduced with wind farms making up a greater percentage. Being a midwestern (windy) state, we have the opportunity to be higher than the 14th highest as far as wind farms go. [pre-assessment]

Students who were opposed to reducing restrictions on wind farm placement also referenced information from the question text, such as negative impacts on wildlife and noise concerns, both of which are illustrated in the response below.

I do not think we should reduce wind farm placements. One reason is because I know I would not want to be living somewhere with constant noise coming from these wind farms, and I know I would also be upset if my property value was decreased because of something I could not control. Another reason that I do not think restrictions should be reduced is because it could harm animals and endangered species. There are many animals that live in the open land of Nebraska including some endangered species and if restrictions were lessened, then wind farms could destroy the animal environments Another solution to issue relating to energy is solar power. I think that solar power could be a better solution because it would not produce loud noises so it would not bother people close to solar power panels. Solar power can also be used within neighborhoods and within the town by placing panels on top of houses and buildings. [post-assessment]

Other responses opposed to reducing restrictions on wind farm placement cited personal experience in the farming community as reasons for their opposition, such as:

I do not feel like Nebraska should reduce the restrictions on wind farm placement because a lot of farmers and ranchers do not want to have wind turbines around their property. As I live on a farm/ranch, I know this to be true. When it comes to wind farm placement, I think the wind farms could be placed in places that are remote and around less people. I think that it is a better solution because it still respects farmers and ranchers while still producing wind energy. [pre-assessment]

After scoring the responses as described above, the Related-Samples McNemar Change test was conducted to determine if the frequency of any argument components significantly changed between the pre- and post-assessments. None of the argumentation components showed significant change on the pre- and post-assessment wind energy tasks (

Table 9. Counts of each argumentation component and results of Related-Samples McNemar Change test on the pre- and post-assessment wind energy argumentation task (n = 90). No results were statistically significant.

	Pre-Assessment	Post-Assessment	Test Statistic	Significance
Claim	71	75	0.375	0.541
Reasons	67	68	0.000	1.000
Counterargument	26	26	0.000	1.000
Basic refutation	8	7	0.000	1.000
Integrated refutation	21	17	0.375	0.541

), with each of the argumentation components staying fairly consistent (±4) from the pre- to post-assessment.

On the wind energy task, integrated refutations on both the pre- and post-assessments tended to involve weighing refutations, in which students considered the tradeoffs of maintaining vs. reducing restrictions on wind farm placement. For example, the student below decided that restrictions should be reduced because “the positives outweigh the negatives.”

Power sourced from wind energy is more beneficial to the environment than burning fossil fuels, and the positives outweigh the negatives. Sustaining human life is the ultimate outcome, and I believe that outweighs negatives seen through noise and impacts on wildlife. [Other solutions would be] find other ways to reduce use of fossil fuels, and to find more effective ways to harvest wind energy. Ways that do not involve a lot of noise or danger to the environment. [These are] better, as it is a hypothetical involving a better wind turbine! [pre-assessment]

As seen in the response above, many integrated refutations that involved weighing tradeoffs ultimately landed on the side of reducing restrictions on wind farm placement on both the pre- and post-assessments:

We should reduce restrictions because even though it might annoy a percentage of the surrounding communities, the impact it will have on preserving and protecting the environment from further coal pollution will make it worth it. We could invest in solar power technology or invest in creating quieter/less disruptive wind farms. I don’t think they are better [solutions] because they don’t have guaranteed results like reducing restrictions on wind farm placement does. [post-assessment]

We also noticed during coding procedures that many of the responses to the wind energy task did not address the question of maintaining or reducing restrictions on wind farm placement, but instead proposed other forms of alternative energy. These were not coded as integrated refutations since they did not address the wind energy prompt, therefore were not addressing a counterargument related to wind energy. Several students made a case for solar energy, such as the responses below.

I would say that they don’t reduce it yet but start working on using solar power and once they can gain enough power using solar they can reduce [restrictions]. I would start using solar power it is way better for the environment and would cost less money in the long run. [Solar power is] better because it would ultimately generate more power and use less energy and money to do so. [pre-assessment]

My reasoning is simply because solar farms are better in basically every way to wind farms. It’s been a while since I’ve had to do all my research on this topic so I don’t know sources and statistics off the top of my head […] but I do remember that solar farms are more efficient, less spacious, quieter, and cheaper than wind farms so really, there’s just no reason to use wind farms when solar farms are available. [post-assessment]

Other students argued for focusing on nuclear power and/or hydroelectric energy as opposed to wind energy, as shown in the responses below.

Wind turbines are great, and they should be put where they can. However, this isn’t the solution to the green energy problem. Nuclear power is [the] solution, although the American public has been traumatized by nuclear disasters originating in time where the technology was much more outdated. Nuclear power gives off zero greenhouse gasses, and the amount of all nuclear waste generated by such reactions is miniscule compared to the pollution generated by a coal fired power plant. [Nuclear power is] better, because it generates less waste, requires less government interference, and would be cheaper in the long run. Nuclear power pays for itself, wind power requires government subsidies to make financially practical. [pre-assessment]

The damage of our own well-being brought about by coal should be reduced thus making wind farms and nuclear power the best alternatives. Nuclear power is clean and outputs more power than coal. The only issue is if there were a leak or anything along those lines that could prove non-ideal to the plant. [post-assessment]

There are many better solutions to the power problem in Nebraska. For example, nobody ever seems to talk about the advantages of nuclear power and hydroelectric power. There is enough water in the state to damn up to produce electricity, and nuclear power is a limitless source of energy. Wind turbines are ugly, annoying to listen to and don’t ever pay for themselves without government subsidies. Throwing our resources into nuclear power is the best option. It’s expensive to construct, but unlike wind power it pays for itself. Nuclear power also never runs out. [Nuclear power plants] are more cost effective and less intrusive on Nebraskans. [post-assessment]

6. Discussion

Assessing students’ arguments regarding plastic pollution, water conservation, and wind energy provides insight on how students construct socioscientific arguments, the extent to which they identify and refute counterarguments, and the type of refutations that they provide in various integrated STEM contexts. Importantly, this work demonstrates that students’ socioscientific arguments differ based on the SSI context.

Results from the plastic pollution task indicate that students showed significant growth in making claims (i.e., taking a clear stance by providing a yes or no response to the question) and providing reasons supporting their claims, with a substantial but non-significant increase in basic refutations and no substantial change in counterarguments or integrated refutations in their responses. Taken individually, this finding refutes our prediction that instruction emphasizing evidence-based decision-making fosters stronger socioscientific arguments such that students are more likely to identify and refute counterarguments after instruction, and supports other research demonstrating that students may not engage with opposing claims when they are constructing arguments as individuals [34]. However, results from the water conservation task indicate that students showed significant growth in making claims and providing integrated refutations in which they either provided a design claim or weighing refutation, with a substantial but non-significant increase in providing reasons for their claims. This discrepancy suggests that the development of socioscientific arguments may be dependent upon the SSI context. Below, we discuss these findings and consider implications for SSI-based integrated STEM teaching and research.

6.1. Socioscientific Argument Strength Varied among SSI Contexts

Within critical integrative argumentation (CIA), arguments are considered stronger when they identify a counterargument, and stronger still if they refute a counterargument through a basic and/or integrated refutation [55]. In this study, students’ arguments regarding water conservation included significantly more integrated refutations (i.e., design claims and weighing refutations) on the post-assessment, meaning that they showed more growth in the strength of their arguments on this task compared to the plastic pollution or wind energy tasks. Focusing specifically on the difference between their growth on the plastic pollution and water conservation tasks, we suggest that this may be related to students’ identities regarding these issues. Many students in the College of Agricultural Sciences and Natural Resources identify as coming from a rural area (52%), and two of the three most common majors (Animal Science, Agribusiness, and Agronomy) are directly related to production agriculture. Given that the water conservation SSI directly addressed agricultural water use, students may find this SSI more relevant to their identity and prior knowledge as opposed to the plastic pollution SSI. Indeed, empirical studies have shown that having a stake in the outcome of a situation, or “vested interest,” motivates information processing [67], so it is possible that students who identified as stakeholders in the water conservation issue more easily identified counterarguments to their claims and refuted such counterarguments. Another recent study found that participants with extremely high or low concern about an issue demonstrated more sophisticated argumentation skills [68], suggesting that students who have very strong emotions about an issue may construct stronger arguments about that issue. We found numerous instances of students making connections between their identities and experiences and the water conservation issue. For example, one student began their response referring to their experience in FFA (Future Farmers of America), stating “Given that I have grown up in rural Nebraska and I have been apart [sic] of FFA, I realize that agriculture is the main reason why our country can function.” Another student established their identity in terms of farming, beginning their response with “As a Nebraska farm kid, I think irrigation is very important and essential to good crop yields across the state.” Other students referenced their communities and families, such as “I come from a farming community and background and restricting irrigation any further would only hurt crop yields and potential,” and “My dad was a farmer and lots of summers we would have dry spells, so irrigation was critical to make an income in our family and for the crops to prosper.” Based on these trends, it is possible that students’ perspectives as stakeholders in this issue may have enabled them to identify and refute counterarguments more easily. It is well-established that personal relevance is critical to student engagement in SSI-based instruction (e.g., [57,69]), so the local context and personal relevance of the water conservation issue compared to the global context of the plastic pollution issue may have facilitated greater gains in argumentation in the water conservation issue.

Given the relevance of students’ identities to their arguments on the water conservation task, one may expect similar trends on the wind energy task, which is also locally relevant to Nebraska and impacts agricultural regions. However, we found no significant growth in argumentation skills between the pre- and post-assessments on the wind energy task, indicating that any gains in the context of plastic pollution and water conservation did not transfer to the novel SSI. This contradicts our prediction that these skills would transfer, which we find especially curious based on prior work suggesting that transfer is related to “the direct relevance of the issue to students’ vested interests” [12]. In their responses, students again invoked their identities with statements such as “This is a problem in my hometown right now as the city wants to build another turbine so it hits close to home. There are so many wind towers up now that they are taking away land from farmers and other people who value the land,” and “I say no [we should not reduce restrictions on wind farm placement] because I come from a background of preserving wildlife more but also we need to find better ways to power our state.” Students also expressed concerns about the impacts of coal, with statements including “When it comes to coal being used it does create toxins that we put into our bodies just by breathing, but it also hurts our wildlife even more,” and “The repercussions from coal for energy are much more detrimental than what is negatively caused by utilizing wind energy. As wind turbines are used, the consequences that they cause can be controlled on a much more attainable level than the damage in which coal burning causes.” These statements suggest that students may identify as stakeholders in this issue and/or find it to be personally relevant, although this did not translate to gains in argumentation on this issue. Since this SSI was not discussed in class, this discrepancy could be due to a lack of knowledge of different alternatives to address this issue. Further, many of the integrated refutations identified on the post-assessment water conservation task included design claims based upon alternative solutions discussed during the course, while integrated refutations regarding the wind energy issue tended to weigh consequences stated in the question prompt with very few design claims posed.

6.2. Use of CIA to Assess Socioscientific Argumentation

In this study, we approached argumentation through the lens of CIA to assess the utility of CIA as a framework for assessing learners’ socioscientific arguments, given that the SSI community lacks a clear definition of a high-quality socioscientific argument and framework for assessing such arguments. This work demonstrates the utility of CIA in assessing socioscientific arguments. By applying the CIA framework, we were able to holistically assess students’ arguments for the presence of claims, evidence, counterarguments, and various refutations, while simultaneously identifying specific argument components (e.g., claims, reasons, integrated refutations) in which students showed growth, details which may have been overlooked if we had assessed arguments solely for claims or justifications or given each individual argument a numerical score. Further, we identified that student growth in argument components varied between SSI contexts, which prompts further questions about the contexts in which SSI-based instruction can foster socioscientific argumentation skills. CIA’s distinction between types of refutations provided another layer of interpretation, as we were able to distinguish between basic refutations, design claims, and weighing refutations. This allowed us to note SSI contexts in which students posed more design claims (water conservation), weighing refutations (wind energy), and basic refutations (plastic pollution), as well as how students integrated counterarguments into their refutations (i.e., implicitly or explicitly). By clearly identifying specific types of refutations, researchers and educators alike may be better positioned to develop instruction that fosters learners’ skills in identifying and refuting counterarguments. CIA is also well-suited for dialogic argumentation, in which counterarguments and refutations more naturally arise [33,34], and should be explored as a framework for facilitating and assessing students’ arguments in such settings.

We posed in our theoretical framework that, given the inherent social and personal aspects of SSI, socioscientific arguments should include social factors, values, and personal priorities, and suggested that integrated refutations may allow students to incorporate such objectives into their arguments. This may occur through (1) posing a design claim that meets an objective that they prioritize while reducing the negative impact of a lower-prioritized objective (expressed through a counterargument) or (2) weighing the importance of various objectives, values, or priorities. Although we did not code for specific instances of students citing their priorities, objectives, or values, many integrated refutations on the water conservation task implied such prioritization through statements such as “Adding more restrictions on irrigation is very negative for Nebraska because the state’s economy depends heavily on farming” (placing economics as a high priority) and “Having cattle and mass farms is necessary for to feed our growing population, but we need to restrict the amount of water that farmers are using before the aquifer is fully drained” (prioritizing the longevity of the aquifer over unlimited use of water for food production). On the plastic pollution task, students’ integrated refutations often included statements of values (e.g., “While plastic does have its negatives, there can be many positives including economic value, personal freedom value [and] ease of use […] banning plastics outright would lose many of the possible freedoms or uses above”) and/or considered personal and public preferences (e.g., “I think that biodegradable is better because people do not want to completely get rid of the use of plastics. […] it would be able to be implemented because people are very innovative and want to do the best for the effects on the earth and human health”). Integrated refutations on the wind energy task often included prioritizing clean energy over aesthetic and comfort concerns (e.g., “I think that people should be able to sacrifice some comfort in order to look out for the environment”). Although several studies evaluating socioscientific arguments have considered values, moral principles, and multiple perspectives (e.g., [48,49,53]), none of these frameworks explicitly address how students prioritize different objectives, values, and perspectives in their arguments. Employing CIA as an analytical framework can allow researchers to identify design claims and weighing refutations, as we have done in this study, and subsequently take the next step of analyzing how students include prioritization and weighing of values-based objectives in their arguments.

Finally, using CIA to assess students’ arguments can foster progress toward STEM and science literacy, such that students gain experience evaluating and proposing solutions to complex problems and making well-reasoned decisions regarding SSI that span multiple STEM disciplines. CIA reflects the practice of scientific argumentation [31], in which scientific models, explanations of phenomena, and solutions to problems are critically evaluated, counterarguments are posed and refuted, and scientists (ideally) consider tradeoffs and ultimately reach consensus. By using CIA to describe a high-quality socioscientific argument and evaluate learners’ progress toward constructing such high-quality arguments, learners can be authentically engaged in the full scientific practice of argumentation within integrated STEM contexts.

6.3. Impact of Structured Decision-Making on Socioscientific Argumentation

While this study did not directly test the relationship between structured decision-making (SDM) and socioscientific argumentation, our results suggest tentative connections between the two. First, there were significant changes in students’ use of argument components between the pre- and post-assessment, which may be related to their repeated use of the SDM process during the course. The SDM process was used to guide students through the SSI modules of plastic pollution and water conservation, in which they identified potential solutions to the SSIs, set measurable objectives, evaluate evidence regarding the impact of each potential solution on the objectives, consider tradeoffs of the potential solutions, and choose the solution that they felt best addressed the objectives. This very closely aligns with CIA, which defines high-quality arguments as those that make a claim, provide evidence for the claim, identify counterarguments, and refute those counterarguments through basic refutations, design claims, and/or weighing refutations [40,55]. Therefore, students’ engagement in the SDM process may have prepared them to construct high-quality arguments as described above. Student responses alluded to this potential explanation, evident in statements such as:

After taking this class, I realize that is really hard to say if they are good or bad. It depends what the objective is that we want to accomplish in deciding whether one is good or bad. No one solution is probably significantly better than the other and the need would vary depending on location. Not all solutions would be viable based on the place it was used at. In fact, it might even be fair to say that using a combination of the ones listed would be the best option as every solution has a disadvantage and that tends to be covered by other alternatives. [plastic pollution, post-assessment]

Further, we noted that students’ argumentation skills demonstrated on the plastic pollution and water conservation tasks did not show evidence of transfer to the wind energy argumentation task, as students did not demonstrate significant growth in any argument components on the wind energy argumentation task. Since students engaged in SDM around the issues of plastic pollution and water conservation and showed gains in argumentation components, while wind energy was not addressed in the course, it is possible that engaging in the SDM process around wind energy would have shown similar gains. Again, student responses referenced their lack of knowledge on the topic in statements such as “I know very little about this alternative and, therefore, do not want to be too quick to judge,” and “I just need more research on how these types of energy sources work and the costs,” demonstrating students’ awareness that further knowledge of the topic and alternative solutions to the issue may have facilitated stronger arguments. Previous work in this specific course context has identified multiple factors influencing transfer, such as engaging students’ prior knowledge and experiences, mastering skills of interest through repeated engagement, addressing the complex nature of decision-making, the roles of scientific knowledge and values in SSI-based decision-making, and considering multiple perspectives [12]. Given that these elements continued to be present in this course, we expected that argumentation skills would transfer to the wind energy issue, particularly given its local relevance to Nebraska. However, other scholars have identified that transfer is more likely when there is explicit instruction on the skills being studied (i.e., argumentation in this context), and close connections between the familiar (plastic pollution and water conservation) and unfamiliar (wind energy) contexts [68]. We intentionally selected the wind energy issue as a topic that students may feel closely connected to, akin to the water conservation issue, but the lack of familiarity with the nuances of the issue coupled with the fact that CIA was not explicitly taught may have limited the possibility of transfer.

6.4. Limitations

The goal of this work was to determine how instruction emphasizing evidence-based decision-making affected students’ socioscientific arguments measured by the critical integrative argumentation framework. Across three different SSI, we found that at the conclusion of the course, students provided significantly more claims about the SSI that were discussed in class, significantly more reasons about the plastic pollution SSI, and significantly more integrated refutations about the water conservation SSI. However, these findings are limited to this particular course context and group of students. Further, we recognize the inherent complexity in students’ decision-making and argumentation, particularly that emotions, interests, and motivations influence how students think about scientific and socioscientific issues [70,71,72]. Argumentation itself has important elements that are both cognitive (e.g., knowledge and epistemological understanding) [73] and social and psychological [74]. In future work, assessing student characteristics such as perceptions of being a stakeholder, motivated reasoning, emotions regarding the SSI, and personal epistemologies may shed more light on students’ arguments, specifically concerning the different outcomes between the plastic pollution and water conservation tasks. Similarly, we did not collect student demographic data in this study, which limits our analysis of how demographic factors (e.g., race, ethnicity, culture, geographic communities) may impact students’ perspectives on SSI.

Further, the course did not explicitly teach scientific argumentation, so students’ argumentation skills would likely be strengthened by direct instruction on constructing scientific arguments and incorporating strategies such as critical questions [40]. Such critical questions may include “what is the evidence?” and “is any of the evidence less important than those on the other side?” Recent work found that these critical questions provided students with a beneficial structure for evaluating arguments and counter arguments [40]; thus, incorporating such critical questions into the SDM process may provide students with greater scaffolding to consider the tradeoffs among alternatives more deeply and extensively.

7. Conclusions

This study used critical integrative argumentation (CIA; [40,55,56]) as a framework for assessing students’ socioscientific arguments regarding specific solutions to SSI before and after engaging in a science literacy course centered on structured decision-making. We found that students’ argumentation skills varied by SSI context, with differing gains in high-quality arguments in the water conservation and plastic pollution issues; students showed significant gains in integrated refutations (i.e., taking counterarguments into account through design claims and weighing refutations) on the water conservation issue, and substantial but statistically non-significant gains in basic refutations (i.e., arguments for why counterarguments are flawed) on the plastic pollution issue. We suggest this may be due to the personal relevance and local context of the water conservation issue motivating students to identify and account for counterarguments in their responses to the water conservation issue, which is supported within the SSI literature (e.g., [57,69]). While we expected that argumentation skills would transfer from the SSI contexts discussed in the course (plastic pollution, water conservation) to a novel SSI (wind energy), this was not observed in our results. After further consideration, we believe that the fact that students were not (1) explicitly taught to construct arguments according to the CIA framework and (2) exposed to alternative solutions to the wind energy issue may have compromised their ability to make a strong claim on either side of the issue, support their claims with evidence, and identify and refute counterarguments.

Given the lack of a consistent framework for evaluating socioscientific arguments in the SSI community, we posed CIA as a potentially fruitful avenue of exploration for students to engage in authentic (socio)scientific argumentation and for researchers to assess such arguments throughout STEM disciplines and in courses emphasizing integrated STEM outcomes (e.g., STEM literacy). We found CIA useful as an analytical framework, specifically because it allowed us to distinguish between types of refutations, note relationships between SSI contexts and types of refutations (e.g., design claims were most common in the water conservation task, basic refutations showed the greatest increase on the plastic pollution task) in which students posed more design claims (water conservation), weighing refutations (wind energy), and basic refutations (plastic pollution), determine how students integrated counterarguments into their refutations (i.e., implicitly or explicitly), and tentatively identify how students may include values, priorities, and various perspectives within integrated refutations. We also identified areas of overlap between CIA and structured decision-making (SDM), suggesting that engaging in SDM may facilitate the development of high-quality argumentation skills across STEM disciplines.

In summary, based on the results from all three argumentation tasks, we conclude that: (1) students’ argumentation skills are dependent upon the context of the SSI and likely related to the perceived relevance of the SSI to students’ lives and identities; (2) argumentation skills do not necessarily transfer to novel SSIs, potentially due to a lack of exposure to SSI background knowledge and alternative solutions; (3) engaging in SDM may facilitate the development of high-quality socioscientific argumentation skills; and (4) CIA is a useful and underexplored argumentation framework in SSI education research. While this work begins to explore the relationship between SSI-based structured decision-making and argumentation, more research is needed to understand this relationship and connections to students’ overall STEM literacy and 21st century skills. In light of current social positions on science and the increasing frequency of science denial in the general public, we encourage researchers to consider how students’ identities and perspectives influence outcomes regarding STEM literacy and incorporate evidence-based strategies to support students’ identities and foster lifelong STEM learning.