Evaluating a Novel Instructional Sequence for Conceptual Change in Physics Using Interactive Simulations

Abstract

This study investigated the effectiveness of a novel inquiry-based instructional sequence using interactive simulations for supporting students’ development of conceptual understanding, inquiry process skills and confidence in learning. The study, conducted in Beijing, involved two teachers and 117 students in four classes. The teachers participated in professional learning and were supported in enacting one of two different instructional approaches the Interactive Simulations Instructional Approach (ISIA) (experimental group) or conventional instruction (control group). Each student group completed pre-tests and post-tests, and classroom observations were conducted to ensure that the implementation of the intervention was consistent. Our findings reveal that students in the ISIA group demonstrated significantly greater gains in conceptual understanding, inquiry process skills and confidence in learning than their peers in the conventional instruction group. Neither students’ sex nor their levels of academic achievement showed main effects on students’ achievement in any of the three outcome types (understanding, skill, confidence). This study demonstrates that the combination of interactive simulations and inquiry-based learning can enhance the development of students’ conceptual understanding, inquiry process skills and confidence in learning.

1. Introduction

Two of the authors [1] outlined a novel instructional sequence using computer-based interactive simulations to support students and teachers in the inquiry-based learning of physics concepts. The instructional approach, named ISIA (Interactive Simulations Instructional Approach) (In the earlier publication the sequence was named Instructional Sequence for Interactive Simulations (ISIS), however intervening political and military events in the Middle East have made that acronym undesirable, leading to the change to ISIA), involves students in the processes of conceptual change. It draws on the special affordances of interactive simulations, with the intention of enhancing students’ conceptual understanding and developing their confidence and skills in inquiry-based learning. The theoretical framework of this approach is based on both Posner, Strike, Hewson and Gertzog’s [2] conceptual change theory and Vygotsky’s [3] Zone of Proximal Development (ZPD). The present study evaluated the effectiveness of this approach.

1.1. Context of the Study

Ongoing curriculum reform is a critical topic in the context of science education in China. China has undertaken a series of reforms since the early 1950s. One major milestone in this process occurred in 1999 when the State Council issued the “Decision on Promoting Quality Education in an All-Round Way”. In 2001, the Ministry of Education [4] issued the outline of the basic curriculum reform (trial version) and curriculum standards of 18 compulsory subjects (trial versions). In 2003, the Ministry of Education launched the program of curriculum reform and national curriculum standards for senior secondary physics and 14 other subjects.

According to the national curriculum standards [4] for senior secondary physics, there are two compulsory modules and three elective topics taught across in three years of high school. Year 12 students are expected to obtain (1) knowledge and skill; (2) processes and methods; and (3) emotions, attitudes, and values. Bloom’s taxonomy [5] is the theoretical framework behind these three aspects, which are popularly called the “Three Dimensions”, to operationalize the objectives of the curriculum standards.

The striking dimension in physics in relation to inquiry-based learning is the second dimension—processes and methods—which requires students to develop abilities in scientific inquiry including “questioning, solving problems and treating data”, “application of physics principles and scientific methods”, and “independent learning and cooperative learning”. Compared with the previous physics curriculum standards, scientific inquiry is considered for the first time as one of the teaching objectives alongside the teaching of physics content. This practice reflects the intention to establish a balance between knowledge-oriented and practice-oriented education.

An examination-oriented tradition, however, has greatly influenced the instructional application of scientific inquiry in China. In many cases, the implementation of the new curriculum did not change conventional approaches to instruction. Teaching, learning and school activities are still oriented towards exam preparation. Guo, Xing, Xu, and Zheng [6] found that Chinese students were typically well prepared for the examinations but exhibited low levels of experimental inquiry process skills in the four provinces investigated.

Teachers aspire to promote scientific inquiry instruction, but most of them do not know how to conduct inquiry-based instruction in practice. Relevant issues such as inquiry processes, inquiry structure, and organization of required curriculum contents are not addressed in the new physics curriculum standards [4]. Researchers cannot help but doubt that the new physics curriculum standards will be implemented effectively in practice [7]. Lack of knowledge and experience with inquiry-based instruction on the part of teachers is thought to act as a barrier to teaching science in this way [8]. Despite their aspirations to inquiry teaching, teachers prefer using easily operationalized conventional teaching methods including teacher-centered lectures, rote memorization of discrete science facts, “cookbook” laboratory activities, and summative tests, even in the context of findings indicating that conventional science classroom approaches have less than the desired effectiveness for supporting students’ conceptual change [9,10] and can ironically serve to destroy students’ innate curiosity about the world of science [11].

1.2. Literature Review

Despite difficulties in implementing inquiry-based instruction, there is growing evidence that inquiry-based learning can provide a constructivist environment where students can engage in active processes of knowledge construction [2,12,13,14]. Inquiry-based learning, if well supported, can be more effective than conventional instruction in its potential to switch students’ allegiance from their alternative conceptions to the established scientific conceptions [15,16]. Studies have found that in simulation-scaffolded environments, inquiry-based learning provided students with more effective learning opportunities, and students outperformed those participating in conventional instruction [16,17,18,19,20,21]. In a review of 79 studies regarding the effects of computer simulations for secondary school students, Scalise et al. [22] found that simulations enhanced students’ academic outcomes in 39 studies. Findings from these studies demonstrated that computer simulation also supported students’ scientific process skills such as developing research questions, conducting experiments, treating simulation data, and presenting research conclusions.

Rutten, Van Joolingen, and Van der Veen [23] reviewed 51 studies related to the effects of computer simulations on learning from 2001 to 2010. They found that computer simulation can enhance conventional science classrooms in that simulations provide useful visualizations and “virtual laboratories”. The results of many of these studies revealed that specific features of computer simulations supported the development of students’ conceptual understanding and promoted students’ predictive ability. For example, Sokolowski and Rackley [24] praised the use of the University of Colorado’s PhET simulations (https://phet.colorado.edu/) in science education. They explained that utilizing simulations enhanced the teaching of limits and helped students to immerse themselves in the virtual model of the physical world and in the inquiry processes. Easy Java Simulation (EJS) showed evidence of enhancing students’ physics conceptual learning. Kepler’s System Model, Geostationary Satellite around Earth Model, and One Dimensional Gravitational Model have all been used in the physics inquiry classroom [18,25,26] with the intention of supporting concept learning more effectively than traditional pen-and-paper instructional approaches.

Fan and Geelan [27] reviewed 57 studies to explore how teachers use computer simulations to meet science learning goals. They found that computer simulations positively impacted students’ learning gains in motivation, conceptual understanding, science process skills and scientific argumentation. However, some studies showed that students using simulations did not outperform those using conventional instruction [28,29]. Students using simulations sometimes had extra trouble in learning. For example, in some cases, students’ attention was distracted by highly attractive representations [30], or students were confused by multiple visualizations [31,32]. Some studies showed that students who learned science concepts using interactive simulations continued to demonstrate inadequate conceptual understanding [33,34]. As such, to investigate the effects of inquiry-based learning with interactive simulations on students’ conceptual change, an inquiry-based learning environment that integrated interactive simulations into students’ inquiry process was implemented and evaluated in the context of Beijing, China.

2. Design of an Inquiry-Based Instructional Sequence Using Interactive Simulations

Posner et al. [2] suggested that “conceptual change” is the type of learning that occurs in situations where students already hold conceptions about the phenomena of interest, but these conceptions are not scientifically accurate. “Conceptual change”, according to these authors, occurs only when students are dissatisfied with their existing concept and are convinced that the new conception is intelligible, plausible, and fruitful.

A review of the field of conceptual change pedagogy and its effectiveness, and the wide variety of approaches and contexts that have drawn on this theoretical frame, is beyond the scope of the present paper.

Drawing on conceptual change theory [2,35,36,37] and on Vygotsky’s notion of scaffolding and the zone of proximal development (ZPD) in the practice of conceptual development [38,39], Geelan and Fan developed the interactive simulations instructional approach (ISIA) for scaffolding learning activities that support students’ conceptual understanding. The ISIA model is outlined in much more detail, and its theoretical and methodological underpinnings explored, in our earlier work [1].

The ISIA includes five steps (see Figure 1). These are:

(1)

elicitation and clarification,

(2)

prediction and implications,

(3)

testing predictions using interactive simulations,

(4)

elucidation and linking, and

(5)

metacognitive evaluation and further testing.

Specifically, the first step of the ISIA explores alternative conceptions in order to elicit the range of preconceptions, misconceptions and scientific conceptions present within the class. This process also helps to begin the process of creating dissatisfaction on the part of students with existing misconceptions as they see the range of other ways of understanding the phenomenon of interest held by their peers and teacher. This discussion stimulates students’ motivation to actively change their existing conceptions. The second step requires students to write down their predictions for the outcomes of the interactive simulation and outline how their predictions arise from the conception that they hold.

To ensure that the new conception is intelligible, plausible, and fruitful, the third step of the ISIA supports students to test their predictions using interactive simulations. In this testing step, often there will be two (or more) concepts “competing”: a common student misconception (or a few such frameworks) and an established scientific concept. Alternative conceptions are resilient. Thus, repeated experiences and exposures may mean that more than one cycle through steps 2–4 of the sequence may be required in a single teaching sequence.

In the elucidation and linking step (Step 4), students present their findings while the teacher supports the presenting students as they seek to explain using accurate scientific language. After the students finish their presentation, they are encouraged to support their peers in adopting the correct scientific conception to explain the phenomenon of interest in their daily lives.

In the final step, students evaluate another group’s ISIA learning worksheet. In doing so, students come to realize the strengths and limitations of their learning experience, internalize the processes of inquiry-based learning, and establish a metacognitive perspective on others’ learning experiences. This step also focuses on reinforcing students’ ideas, and thus constructs a deeper conceptual understanding or re-constructs their understanding of the topic being learned. The achievement of conceptual change does end with the current topic. The learning process will lead students onward to another connected topic in that “science learning is very much a connected whole, and both knowledge and skills from one topic are relevant to other topics” [1] (p. 265).

While the students are involved in the ISIA learning experience, scaffolds are essential to provide support to engage them in problem-solving activities. Vygotsky’s [3] Zone of Proximal Development (ZPD) theory suggests that learning exists in “the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance, or in collaboration with more capable peers” [3] (p. 86). This highlights the effects of scaffolds (i.e., adult guidance and capable peers) on learning as occurring in the zone between what students can already do alone and unaided and what they cannot do, even with help and scaffolding. Complementarily, Quintana et al. [38] describe scaffolds as a range of assistance that can support problem-solving and cognitive task-performing that cannot be completed by students on their own. The range of assistance could include knowledgeable teachers, capable peers, modes of language, writing, images, technology tools and other social tools. Studies have shown that scaffolds benefit both instructional teaching and students’ learning [40,41,42]. Ge et al. [43] found that scaffolds can “support students to activate schemata, organize and retrieve knowledge, monitor and evaluate, and reflect on their learning”. The effectiveness of different forms of scaffolding has been reviewed in some studies [22,44]. Mayer [45] pointed out that an inquiry approach without any scaffolding or assistance does not always lead to learning. Specifically, scaffolds in this study included the ISIA, students’ teachers, their lab partners, PhET simulations, scientific language, and student worksheets. These scaffolds are of benefit to students’ learning and can be gradually removed until students can learn on their own.

In summary, the current study is based on conceptual change theory [2,35,36,37], the ZPD and scaffolding theory in the practice of conceptual development [38,39]. The following perspectives, which contribute to the current study, are articulated to underpin the interactive simulations instructional approach (ISIA) for students’ conceptual development:

(1)

Learning is constructed, rather than received; but commonalities exist between individuals [2,13].

(2)

Personal and social planes are equally important to the process of learning [3].

(3)

Existing ideas greatly influence students’ subsequent learning [2,3].

(4)

Conceptual change may take considerable time and have frequent reversals, involving experiencing formalized instruction and informal daily activities [37].

(5)

Conceptual change may take place through exchanging, modifying, and enriching the understanding of others [36,46].

(6)

In the ZPD, scaffolds from knowledgeable figures, language, technology tools, cognitive tools and activities are influential in supporting conceptual change [3,38,39], and

(7)

Conceptual change may be achieved when it meets the students’ zone of proximal development (ZPD) and the four conditions of conceptual change.

3. Methods

3.1. Purpose and General Method of the Study

This study used a controlled comparison educational trial to gather evidence on the effectiveness of simulation-supported inquiry-based instruction for enhancing students’ conceptual understanding, inquiry process skills, and confidence in learning. More specifically, this study aimed to explore the following questions:

(1)

What effect does simulation-supported inquiry-based instruction have on enhancing learners’ conceptual understanding, inquiry process skills and confidence in learning (compared with conventional instruction)?

(2)

Do the effects of simulation-supported inquiry-based instruction differ between male and female students?

(3)

Do the effects of simulation-supported inquiry-based instruction on students at different levels of academic achievement differ?

Data analysis consisted of an examination of the performances of the students on the pre-test and post-test of conceptual understanding, inquiry process skills and confidence in learning. Students’ conceptual understanding, inquiry process skills and confidence in learning were the dependent variables while treatment (experimental versus control), sex (male, female) and academic level (high, medium, low) were the independent variables.

Data were analyzed based on 2-tailed t-tests (p ≤ 0.05 for significance) because directionality of any statistical differences was not known before the analysis. ANOVA and ANCOVA were used to further explore the research questions.

3.2. Participants

117 Grade 10 Chinese students and two physics teachers participated in the current study. The sample comprised four classes with a mean age of 16.51 (SD = 0.87). The four classes were randomly assigned (as whole pre-existing classes rather than as individuals) to either an experimental group (n = 55, two classes) or a control group (n = 62, two classes). The experimental group students used the ISIA as part of their regular physics lessons. The control group students used conventional instruction to learn the same topics as ISIA students. Each teacher taught both an experimental and a control class to minimize teacher-related differences in outcomes.

Table 1. Descriptive statistics and ANOVA on pre-test in four classes.

	Conceptual Understanding				Inquiry Skills				Confidence
Class	Mean	SD	F	p	Mean	SD	F	p	Mean	SD	F	p
A	45.27	12.19	0.72	0.54	61.65	15.09	0.95	0.419	36.31	7	1.04	0.379
B	47.75	11.99			59.55	13.92			37.03	7.12
C	43.47	11.47			59.03	14.75			34.6	9.46
D	47.13	13.83			64.69	14.86			37.91	6.54

shows that there were no significant differences in achievement between the four classes at pre-test, on any of the three dimensions tested (conceptual understanding, inquiry skills and confident).

The participating teachers attended two-day teacher training workshops about the inquiry-based instructional sequence using interactive simulations. Topics included

(1)

PhET simulation introduction;

(2)

Exploring and practicing with PhET simulations;

(3)

Discussion about how teachers could use the simulations to facilitate students’ learning;

(4)

Introducing physics conceptual understanding and conceptual change and discussing students’ alternative conceptions;

(5)

Introducing the Force Concept Inventory test that would be used to assess students’ conceptual understanding of force and motion;

(6)

Inquiry skills survey introduction and discussion;

(7)

Introduction to inquiry-based instruction,

(8)

Guidelines for conceptual change instructional approach;

(9)

Discussion of details of the lesson plan and student worksheets; and

(10)

A summary of the educational research schedule.

Before the first lesson, the conceptual knowledge test, inquiry process skills survey and confidence survey in combination were used as a pre-test for all students. After the eight (8) week intervention the post-test panel was administered using instruments that were parallel to those administered at pre-test. Students who were absent for either the pre-test or post-test, who did not finish their answers or whose test paper was not completed were excluded from the data analysis. All data collected were de-identified for confidentiality purposes, and research ethics clearance was granted by the University of Queensland Human Research Ethics Committee.

3.3. Comparability of the Experimental and Control Groups

One-way between-group ANOVAs of students’ pre-test scores on the scientific concept test, inquiry process skill survey and confidence survey were conducted to see whether the classes were comparable. Given that they were (see Table 1), two classes were randomly chosen to be the experimental groups, and the remaining two became the control groups. This study is quasi-experimental in that it used pre-existing school classes rather than random assignment of students to the experimental and control conditions.

An examination of the skewness and kurtosis of the data revealed they were normally distributed (z_skewness-conceptual understanding = 1.52, z_kurtosis-conceptual understanding = 0.01, z_skewness-inquiry process skills = 1.94, z_kurtosis-inquiry process skills = 0.69, z_skewness-confidence = 2.38, z_kurtosis-confidence = 0.55). Homogeneity of variance was assessed using Levene’s Test for Equality of Variances and the result was not significant (p-conceptual understanding = 0.47, p-inquiry process skills = 0.65, p-confidence = 0.11). Given the sample size, the normal distribution of the data and the non-significance of the homogeneity of variance, follow-up ANOVAs were used to analyze the data.

3.4. Learning Activities

Both experimental groups and control groups studied a module on “Newton’s Laws of Motion”, a compulsory topic in the new physics curriculum for senior secondary school. Students in the experimental group had a 60-min inquiry-based instruction lesson once per week for eight weeks while students in the control group had a teaching sequence using the teachers’ usual approach to teaching this topic—which we describe as “conventional” instruction—on the same topic once per week for eight weeks.

Before each lesson, all participating teachers were requested to complete an ISIA lesson plan form. The teachers needed to integrate each topic and content into the ISIA. The lesson plan stated the five ISIA steps. It asked the teachers to fill in teaching objectives, student activities, and the teacher’s role. The participating teachers were requested to complete it after each lesson.

Table 2. Implementation of the Interactive Simulations Instructional Approach (ISIA).

Teaching Date: 16/05/2013		Teacher’s Name: Mr. Zhang (Z)
Teaching Steps (mins)	Teaching Objectives	Student Activities	Teacher Role
Step 1: Elicitation and clarification. (10–15)	To elicit students’ existing concepts and clarify the “target” scientific conception	Z’s introduction started with providing the lesson objectives and the roadmap of the content of the lesson. Z asked three questions related to Newton’s First Law and asked students to write down their initial ideas.	Facilitate student discussion to elicit their misconceptions.
Step 2: Prediction and implication. (15–20)	To outline the predictions and engage students in the implications of their prior conceptions on certain topics	Z described the situations again to prompt more discussions with students. Following that, he introduced the class sequence and interactive simulation that would be used in the current lesson.	Lead students to clarify their problem and propose predictions about the worksheet in discussion.
Step 3: Testing prediction through interactive simulations. (20–25)	To test predictions of competing conceptions using interactive simulations	Z led the class discussion on how to make a plan to test their hypothesis. Students used “Move and Force” simulation and scaffolding forms and questions from the student notes to explore their hypothesis. Most students appeared to be working collaboratively on their prediction problems, and some groups finished quickly and started to play games included with the simulation.	Facilitate students as they conduct their experiments through simulation, and solve the on-going problems supported by teacher, peers, and worksheet.
Step 4: Elucidation and linking. (35–40)	To clarify the findings and link results to the scientific conception through students’ presentation and teacher-student discussion.	Students gave their presentation. Z said, “Feel free to make the presentation in your style or using the worksheet. He also introduced four main aspects of the presentation. Four groups presented their exploration experiments with simulations. Z proposed several questions during or after each group’s presentation. Z cared about the questioning techniques and questioning time.	Guide students to clarify and link their findings through simulation, peers, and scientific discourse methods.
Step 5: Metacognitive evaluation and further testing. (40–50)	To evaluate the whole inquiry sequence to develop metacognitive inquiry thinking and deepened understanding of the scientific conception.	Z said to students, “I want you to mark your worksheet in your group and then invite one other group to re-mark your worksheet. The five criteria have been listed on the worksheet.”	Facilitate students’ completion of self-evaluation and other-evaluation in the worksheet.

shows an example of a completed ISIA lesson plan by a participating teacher regarding Newton’s Second Law. This table is a summarized and simplified example of the classroom observations that were completed by the researcher to ensure that teachers in the control classrooms taught the physics concepts as they usually would, using their “traditional” approaches to physics teaching, and that the teachers in the experimental classrooms accurately implemented the ISIA as intended. This is one of the measures taken to enhance the trustworthiness of the comparison undertaken in this study.

Students in the control group received conventional instruction, which used an experimental demonstration teaching method that is popular in Chinese classes. Teachers demonstrated experiments or “cookbook” experiments in front of the classroom. Meanwhile, students are requested to observe the teacher’s experiments. Before each experiment, the students are requested to read the lectures and textbooks. After demonstrating the experiments, the teacher provides relevant lectures with explanations to support students’ conceptual understanding. Courses ended with summative evaluations such as exercises, seatwork, and a quick quiz. During each lesson, classroom observations for teachers and students were conducted to ensure that the teachers in the experimental group correctly followed the inquiry-based instruction procedure and teachers in the control group taught using the “conventional” method.

3.5. PhET Interactive Simulations

This study was concerned with pedagogy and simulations that engage students in an inquiry process to help them acquire conceptual understanding and improve their inquiry skills. The simulations selected were provided by the Physics Education Technology (PhET) project. This project is a library of free online applications designed by the Interactive Simulations Project at Colorado University. There is a growing list of more than 90 physics simulations and the project has expanded to offer simulations for other subjects such as biology, chemistry, earth science and math. The central goal of PhET simulations is to support the implementation of inquiry learning. The design principles are based on research on how students learn [12]. PhET simulations have been used in a series of studies [47]. Chinese-translated versions of the physics simulations were used in the current study.

Studies have shown that PhET simulations can challenge, improve, correct, and reinforce conceptual understanding through self-driven exploration [48,49]. Students gained more knowledge when using the PhET simulation than when learning with real laboratory equipment [50]. Sokolowski et al. [24] also praised the application of the PhET simulations. They found that utilizing simulations enhanced the teaching of limits and helped students become immersed in the virtual model of the physical world and the inquiry processes. Therefore, the current study integrated PhET simulations into inquiry-based learning.

4. Data Sources

4.1. Test of Conceptual Understanding and Confidence Rank Survey

The Force Concept Inventory (FCI) is a common instrument for assessing understanding of force and motion developed by Hestenes et al. [51,52]. In this study, we modified the FCI into a two-tier test for measuring change in students’ misconceptions as a measure of concept development. In the original FCI, questions are multiple-choice questions, with one answer representing the “correct”, established scientific answer, and the remainder representing common student misconceptions. The adapted test included two additional elements: eliciting from students an explanation of why a particular option was chosen, and a chance for students to rank the confidence they felt about their chosen option.

The confidence rank was a Likert five-point survey appended to the conceptual understanding test. The following is an example of one item from the FCI, illustrating our additions:

Two metal balls are the same size, but one weighs twice as much as the other. The balls are dropped from the roof of a single-story building at the same instant. The time it takes the balls to reach the ground below will be:

• About half as long for the heavier ball as for the lighter one.
• About half as long for the lighter ball as for the heavier one.
• About the same for both balls.
• Considerably less for the heavier ball, but not necessarily half as long.
• Considerably less for the lighter ball, but not necessarily half as long.

Could you please explain why you choose this answer? You can use your physics knowledge or your own words to write down your understanding.

How sure are you of your answer to the question?

• Very sure;
• Sure;
• Neutral;
• Unsure;
• Very unsure.

4.2. Inquiry Process Skills Survey

Inquiry process skills were measured using a Likert five-point scale survey. A total of 13 items addressed four aspects of inquiry process skills. These four aspects were adapted from the study of White et al. [53], and included starting inquiry experiments, conducting inquiry experiments, cooperation, and communication during inquiry experiments, and evaluating inquiry experiments. An example of an item about the communication aspects of an inquiry experiment is presented below:

I understand the physical problems that I am exploring, but there are still some people who do not understand what I said or wrote.

5 = Strongly agree; 4 = Agree; 3 = No opinion; 2 = Disagree; 1 = Strongly disagree

4.3. Reliability and Validity of the Instruments

The Cronbach alpha coefficients of the conceptual knowledge test, confidence test and inquiry process skills survey were .81, .94 and .87 respectively, suggesting relatively high internal consistency for the scales with this sample. The internal consistency reliability of the inquiry process skills test was 0.88 for the pre-test (coefficient for the five sub-scores) and 0.86 for the post-test.

5. Results

5.1. Result 1: Conceptual Understanding

Students’ results on the test of conceptual understanding of physics were analyzed to address the following specific questions:

(1)

What effect does simulation-supported inquiry instruction have on enhancing learners’ conceptual understanding (compared with conventional instruction)?

(2)

Do the effects of simulation-supported inquiry instruction differ between male and female students in relation to conceptual understanding?

(3)

Do the effects of simulation-supported inquiry instruction differ on students at different levels of academic achievement in relation to conceptual understanding?

5.1.1. Conceptual Understanding Test

A one-way between-groups analysis of variance (ANOVA) was performed to explore the impact of instructional approaches on conceptual understanding, as measured by the conceptual understanding test at post-test. There was a statistically significant difference at the p < 0.05 level in scores for the two groups: F (1, 115) = 25.11, p = 0.000, η² = 0.181 which, according to generally accepted criteria [54], is considered a large effect size.

5.1.2. Sex

To assess the effectiveness of the two instructional approaches in promoting conceptual understanding for male and female students after treatment (post-test), a 2 × 2 between-groups analysis of covariance (ANCOVA) was used with students’ outcomes (pre-test) as a covariate. The results revealed that after adjusting for scores at pre-test, there was a significant interaction effect, F (1, 112) = 4.14, p = 0.044, with a small effect size (η² = 0.04). There was a statistically significant main effect for groups, F (1, 112) = 79.98, p = 0.000, η² = 0.42. The main effect for sex, F (1, 112) = 1.89, p = 0.17, did not reach statistical significance. The combination of the ISIA and interactive simulations supported the development of conceptual understanding on the part of students of both sexes.

5.1.3. Different Levels of Academic Achievement

Controlling for the effects of students’ conceptual understanding conducted before the commencement of the treatment (pre-test) as covariates, a 3 × 2 between-groups analysis of covariance (ANCOVA) was conducted to determine if there were significant differences between the groups at the three levels of conceptual understanding after the treatment (post-test). After adjusting for scores at pre-test, there was a significant interaction effect. F (2, 110) = 3.74, p = 0.027, with a moderate effect size (η² = 0.06). There was a statistically significant main effect for groups, F (1, 110) = 91.64, p = 0.000; and the effect size was large (η² = 0.45). The main effect for levels, F (2, 110) = 0.61, p = 0.55, did not reach statistical significance.

Table 3. Comparing means for conceptual understanding by sex and academic achievement level for pre-test and post-test.

		Experimental Group				Control Group
		n	M	SD	Difference	n	M	SD	Difference
Male	Pre-test	22	50.41	11.24	17.69	25	48.68	14.15	8.68
	Post-test		68.14	11.5			57.36	17.64
Female	Pre-test	33	44.03	12.04	22.21	37	44.09	10.78	9.8
	Post-test		66.24	11.1			53.89	10.39
Low	Pre-test	18	34.56	5.28	22.88	21	31.62	5.9	9.86
	Post-test		57.44	7.52			41.48	7.79
Middle	Pre-test	20	45.4	3.08	21.2	21	45.14	3.28	9.38
	Post-test		66.6	7.98			54.52	5.56
High	Pre-test	17	60.71	8.48	16.88	20	60	6.42	10.6
	Post-test		77.59	8.06			70.6	7.24

shows that the students from low, middle, and high levels in the experimental group obtained higher mean scores than their peers in the control group at post-test. The upshot of this is that the combination of the ISIA and interactive simulations supported the development of conceptual understanding on the part of students at all academic levels.

It is worth noting that “ceiling effects” may occur where students with lower pre-test scores have the scope to make larger gains at post-test than those already achieving higher scores at pre-test. Hake [55], reporting a summary of data gathered using the Force Concept Inventory (FCI), the same instrument used in this study, employed the concept of the normalized gain <g>, defined as the ratio of the actual average gain (%post–%pre) to the maximum possible average gain (100-%pre).

Table 4. Comparing normalized gain scores for conceptual understanding by academic achievement level for pre-test and post-test.

		Control Group
		n	M	<g>	n	M	<g>
Low	Pre-test	18	34.56	0.35	21	31.62	0.14
	Post-test		57.44			41.48
Middle	Pre-test	20	45.4	0.39	21	45.14	0.17
	Post-test		66.6			54.52
High	Pre-test	17	60.71	0.43	20	60.0	0.27
	Post-test		77.59			70.6

mirrors the lower section of Table 3 but reports normalized gain scores, <g>, for experimental and control groups at the three levels of academic achievement, rather than “raw” gain scores. It was not possible to re-calculate the ANCOVA for these scores however the differences between the experimental and control groups remain large while the differences within those groups between the gain scores at low, middle, and high levels of academic achievement remain relatively small.

5.2. Result 2: Inquiry Process Skills Survey

Three aspects of students’ inquiry process skills were analyzed:

(1)

What effect does simulation-supported inquiry instruction have on enhancing learners’ inquiry process skills (compared with conventional instruction)?

(2)

Do the effects of simulation-supported inquiry instruction differ between male and female students in relation to inquiry process skills?

(3)

Do the effects of simulation-supported inquiry instruction differ on students at different levels of academic achievement in relation to inquiry process skills?

5.2.1. Inquiry Process Skills Score

A one-way between-groups analysis of variance was performed to explore the impact of instructional approaches on inquiry process skills, as measured by the inquiry process skills test at post-test. There was a statistically significant difference at the p < 0.05 level in scores for the two groups: F (1, 115) = 71.36, p = 0.000, η² = 0.38, a large effect size.

5.2.2. Sex

We performed an ANCOVA using the pre-test scores as the covariant and found that there was no interaction effect, F (1, 112) = 0.14, p = 0.708. There was a statistically significant main effect for groups, F (1, 112) = 662.24, p = 0.000; and, the effect size was large (η² = 0.86). The main effect for sex, F (1, 112) = 1.58, p = 0.211, did not reach statistical significance. Examination of

Table 5. Comparing means for inquiry skills by sex and academic achievement level for pre-test and post-test.

		Experimental Group				Control Group
		n	M	SD	Difference	n	M	SD	Difference
Male	Pre-test	22	38.45	6.56	12.00	25	37.08	7.40	2.20
	Post-test		50.45	4.90			39.28	6.22
Female	Pre-test	33	35.51	7.15	11.94	37	35.78	8.74	2.11
	Post-test		47.45	6.13			37.89	7.65
Low	Pre-test	18	34.39	6.19	12.44	21	35.00	9.58	2.81
	Post-test		46.83	5.81			37.81	7.22
Middle	Pre-test	20	36.70	8.70	12.45	21	36.29	7.78	2.14
	Post-test		49.15	7.01			38.43	7.96
High	Pre-test	17	39.12	4.78	10.88	20	37.70	7.15	1.45
	Post-test		50.00	3.72			39.15	6.22

demonstrates that mean scores for male and female students in the experimental group improved more than for students in the control group at post-test. The mean difference indicated that the sex gap seemed to show no change from pre-test to post-test in either the experimental group or the control group.

5.2.3. Different Levels of Academic Achievement

The results of an ANCOVA revealed that, after adjusting for scores at pre-test, there was no interaction effect for different levels of academic achievement: F (2, 110) = 0.50, p = 0.61. There was a statistically significant main effect for groups, F (1, 110) = 685.19, p = 0.000; and, the effect size was large (η² = 0.86). The main effect for levels, F (2, 110) = 1.81, p = 0.17, did not reach statistical significance. Table 5 shows that students from low, middle, and high levels in the experimental group obtained higher mean scores than those in the control group at post-test.

5.3. Result 3: Confidence

Three aspects of students’ confidence in the answers they gave were analyzed.

(1)

What effect does simulation-supported inquiry instruction have on enhancing learners’ conceptual understanding (compared with conventional instruction)?

(2)

Do the effects of simulation-supported inquiry instruction differ between male and female students in relation to conceptual understanding?

(3)

Do the effects of simulation-supported inquiry instruction differ on students at different levels of academic achievement in relation to conceptual understanding?

5.3.1. Confidence Survey Score

A one-way between-groups analysis of variance was performed to explore the impact of instructional approaches on confidence in learning, as measured by the confidence survey at post-test. There was a statistically significant difference at the p < 0.05 level in scores for the two groups: F (1, 115) = 15.65, p = 0.000. The effect size, calculated using eta squared, was 0.12. An examination of the mean scores for each group showed that the mean for the experimental group (M = 75.65, SD = 8.88) was higher than the mean score for the control group (M = 67.60, SD = 12.57).

5.3.2. Sex

We conducted an ANCOVA using the pre-test scores as the covariant and found that there was a significant interaction effect, F (1, 112) = 14.10, p = 0.000, with a medium effect size (η² = 0.11). There was a statistically significant main effect for groups, F (1, 112) = 59.67, p = 0.000; and, the effect size was large (η² = 0.35). The main effect for sex, F (1, 112) = 0.44, p = 0.51, did not reach statistical significance. An examination of

Table 6. Comparing means for confidence by sex and academic achievement level for pre-test and post-test.

		Experimental Group				Control Group
		n	M	SD	Difference	n	M	SD	Difference
Male	Pre-test	22	69.54	11.27	9.14	25	66.88	15.69	5.88
	Post-test		78.68	9.03			72.76	11.67
Female	Pre-test	33	54.55	13.17	19.09	37	58.62	13.68	5.49
	Post-test		73.64	8.32			64.11	12.08
Low	Pre-test	18	52.06	14.84	20.55	21	54.62	16.69	6.67
	Post-test		72.61	8.84			61.29	13.80
Middle	Pre-test	20	63.75	11.88	13.60	21	62.62	12.22	5.33
	Post-test		77.35	8.69			67.95	11.39
High	Pre-test	17	65.76	13.21	11.12	20	68.95	12.53	4.90
	Post-test		76.88	8.84			73.85	9.26

shows that the male and female students in the experimental group obtained higher mean scores than their peers in the control group at post-test.

5.3.3. Different Levels of Academic Achievement

ANCOVA analysis indicated that after adjusting for scores at pre-test, there was a significant interaction effect, F (2, 110) = 4.43, p = 0.014, η² = 0.08. There was a statistically significant main effect for groups, F (1, 110) = 68.73, p = 0.000, η² = 0.39. The main effect for levels, F (2, 110) = 0.09, p = 0.91, did not reach statistical significance. An examination of Table 6 indicates that the students from three levels in the experimental group obtained higher mean scores than those in the control group at post-test.

6. Discussion

The aim of this study was to evaluate the relative effectiveness of two different instructional approaches (i.e., ISIA and conventional approach) on students’ learning and performance. The results confirm the positive effects of the interactive simulations instructional approach regarding students’ development of conceptual understanding, inquiry process skills and confidence in their own understanding.

6.1. ISIA Intervention Enhanced Students’ Conceptual Learning, Inquiry Process Skills and Confidence in Learning

This study found that an inquiry-based instructional sequence using interactive simulations enhanced students’ conceptual understanding of forces and motion. Teaching using ISIA provided different activities to support students’ acquisition and integration of physics concepts. This study also showed that conventional instruction improved students’ conceptual understanding but not as effectively as learning while using ISIA [10,13,56,57,58,59,60]. Students were requested to test their predictions using PhET interactive simulations after class discussion eliciting alternative conceptions directed students’ attention to their existing knowledge. It is plausible that multiple representations within the PhET simulations helped students understand concepts through the auditory-verbal channel and visual-pictorial channel, however this study did not generate evidence for the specific mechanism leading to the enhanced conceptual understanding on the part of students. The ISIA classrooms provided different types of verbal information (such as teacher’s questions and lectures and students’ discussions and explanations) to support students seeking to make sense of a specific concept. Research suggests that language can help students to understand and acquire the terms needed to describe relevant phenomena properly [61,62]. When students used PhET simulations to test their hypothesis, dynamic animations, images, charts, tables showing on the computer interface and printed words on textbooks helped to scaffold students’ understanding.

The findings reported here in relation to conceptual understanding are consistent with studies that show that representations of scientific concepts are effective in consolidating students’ learning [16,63,64]. In addition, in the ISIA learning setting, when students made presentations to explain their claims, in-depth conceptual understanding developed in the combination of verbal and visual materials, rather than isolated from one another [65].

To ensure that the established scientific concepts became intelligible, plausible, and fruitful [2], students needed to make presentations to defend their claims. In practice, the ISIA students were asked to present their claims based on data and explanation, to support arguments with claims, evidence, and reasons. The interviewed students reported that the use of scientific discourse was effective in promoting their understanding of forces and motion. Particularly, they benefitted from practicing the use of scientific language. Previous research has found that it is important for science learning to offer opportunities for students to practice scientific language [66,67,68]. China’s physics standards also explicitly call for the development of scientific language through learning core concepts [4]. ISIA provides one step specifically designed to help students understand and practice scientific language. This sequence could facilitate students “making the language of science their own” [69].

Students learning using the ISIA acquired skills in pursuing inquiry processes when they were involved in inquiry-based learning using PhET simulations. Teachers guided students to experience the five steps of the ISIA. Students’ worksheets also scaffolded students to complete investigation activities at each step. The last step asked students to evaluate their performance of each step and evaluate other groups’ performance based on the completed worksheets. These activities provided students with more chances to habituate to the processes of inquiry-based learning. This learning approach engaged students in more positive thinking and active learning [70].

Control group students, on the other hand, mainly observed their teachers doing experiments. Didactic lectures were used to transmit knowledge from teachers to students. Lessons observed during the eight weeks usually ended with exercises and a quick quiz. This strategy is prominent in the physics classroom because it offers easy organization and time-saving [71]. It is not surprising, therefore, that the ISIA students outperformed their peers in the control groups on conceptual understanding, inquiry process skills and confidence in learning.

6.2. ISIA Promisingly Addressed Sex Differences and Academic Levels Differences

This study found that overall, there was a statistically significant main effect for groups (i.e., experimental or control) on students’ conceptual understanding, inquiry process skills and confidence in learning. Sex did not reach statistical significance in students’ learning and performance. However, upon looking deeper, it was found that female ISIA students gained more in conceptual understanding (4.52 points) as shown in Table 4 and confidence in learning (9.96 points) as shown in Table 5 than did male students. Academic achievement levels did not reach statistical significance in students’ learning and performance.

In many countries, female students’ performance in physics learning appears to be lower than male students”, although this trend is changing over time. Studies show that sex-related differences closely related to concepts in the science curriculum, favoring boys [72,73]. One reason might be biological differences such as quantitative skills and spatial visualization [74]. Cultural beliefs could make girls less confident than boys because science is assumed to be included in male-type tasks [75] and women are stereotyped as dealing with humanities. In yet another study, one reason for female students underperforming in science could be lack of full support by their male peers [76]. The results from the current study did not explicitly address sex differences in achievement, but the sequence showed equal promise for the teaching of female and male students. The fourth step of ISIA provided opportunities for students to make presentations, which the female students enjoyed. Sixteen female students made presentations in this study, but only nine male students chose to present. Scientific language is different from daily language. This study found that language is also a powerful tool for physics learning. During presentations, students practiced how they could use scientific language to explain their claims. Teachers also provided additional elaborations when students’ explanations needed further clarification. There were no significant differences between the learning gains of male and female students in this study. The ISIA appears to support the learning of both sexes, while female students may find the opportunity to present more engaging. Further research including interviews with students would be required to test this possibility.

This study found that ISIA students from the middle level of academic achievement with a gain of 22.88 points from pre-test to post-test and at the low level with a gain of 21.20 points gained more in conceptual understanding when compared with their peers from the high achievement level with 16.88 points. The results were reversed in the control group. PhET interactive simulations first provided students multiple representations such as teacher explanation and peers discussion from the auditory-verbal channel as well as tables, images, and printed texts from visual-pictorial channel to enhance students’ understanding of each topic. When students retrieved information, if one representation is lost, the other is still available. Moreover, the multiple representations of the simulation interface can reduce potentially confounding language demands [77]. Studies also found that multiple representations of scientific concepts were effective in consolidating students’ learning [16,65,66]. Second, each topic was investigated through five steps of ISIA. Students—particularly those with low achievement—felt less pressure when they faced each step in sequence. Meanwhile, PhET simulations provided an interactive virtual lab. We observed that the ISIA students were more interested in completing investigative activities than those who learned through conventional instruction. However, conventional instruction approaches have been shown to have little effect on students’ misconceptions [10,59], and ironically can serve to destroy students’ innate curiosity about the world of science. Also, the ISIA classroom provided students with a non-competitive learning atmosphere. Lazarowitz and Hertz-Lazarowitz [78] found that non-competitive learning atmospheres support students who are struggling to understand. In this study, students in groups organized their learning pace, even if they were requested to finish their work within a certain period. We observed that most groups had extra time to review their materials and re-do the experiments. Lastly, using PhET simulations, teachers were afforded more time to observe students’ learning instead of focusing on students’ safety in laboratories. Two participating teachers were observed to engage in more interactive conversations with students in the experimental group compared to those in the control group. This provides a platform for teachers to diagnose their students’ problems and then provide further explanations. There were no significant differences between the learning gains at the low, middle, and high academic levels, suggesting that ISIA is capable of supporting learning for all students in a physics class, rather than supporting lower or higher achieving students differentially.

7. Conclusions

Based on these results, we can tentatively claim that the ISIA model contributes to student learning, including the development of physics concepts and inquiry process skills, as well as their confidence in their understanding. The ISIA sequence provided a platform to allow students to experience the inquiry process. The planned scaffolds such as the PhET simulations, teachers’ and peers’ help, and support from the worksheet and regulatory activities ensured that the implementation of each step of the inquiry instruction sequence was consistently applied. In summary, students learning with the ISIA model using interactive simulations learned more effectively than students using conventional instruction, with a large effect size. This finding was robust across both sexes and all academic achievement levels. Similarly, students gained confidence in their knowledge and developed skills in conducting scientific inquiry. These gains were also statistically significant with large effect sizes and were similarly not significantly different for male or female students. This study and its findings are in line with findings in other studies [16,23,79] that simulation-supported inquiry instruction had a positive effect on students’ knowledge construction concerning Newton’s Laws, when the inquiry instruction is conducted using a structured sequence and planned scaffolding. Further research is required to investigate how ISIA can be further improved in the context of China with larger student groups and in other areas of the country. Research with more students and in multiple international contexts is also required to better understand the educational affordances of combining inquiry-based pedagogy with computer-based interactive simulations.