2.2. Procedure
The course on magnetism and electrostatics consisted of three sessions and introduced the concepts of magnetic and electric fields as well as the gravitational field. Magnetic, electric, and gravitational forces are puzzling for students because of their property to act at a distance. The problem of the action at a distance is solved with the introduction of a force field. Thus, the aim of this unit was to show students how to use the concept of force fields as a mental tool in order to deal with the idea of action at a distance and to explain various magnetic and electric phenomena. Consequently, the unit often made use of field line depictions as supportive methods to enable the use of force fields as mental tools. The unit consisted of five different topics: (a) what is a field, (b) the direction of fields, (c) the strength of fields, (d) attraction and repulsion, and (e) the gravitational field, which was used as a transfer topic.
The instruction material made use of inquiry-based cognitively activating methods [
40]. Physics instruction still follows rather traditional procedures despite ample evidence that it often is inefficient for instance in lifting students’ prior conceptions. Cognitively activating instruction is a set of student-centered teaching methods focusing on conceptual understanding. It is intended to help take prior knowledge into account, change existing knowledge, and construct conceptual knowledge [
25,
40].
Whereas electricity, magnetism, and gravitation are conventionally treated in sequential order at school, in this unit the students were taught the topics in parallel using contrasting methods in order to emphasize the core concept of a force field, to facilitate transfer between the three topics, and integration of knowledge. Such contrasted comparisons are intended to help to differentiate superficially similar but differing concepts or phenomena by directly juxtaposing them and supporting the recognition of the similarities and differences between the concepts or phenomena. Whereas contrasted comparisons require more learning time and effort in the beginning, they help students eventually to carve out similarities and differences between the contrasted topics [
41]. The material used visual clues to delimit the electric, magnetic, and gravitational field and to make clear for every experiment which field is actually associated with it.
Besides various experiments on the visualization of magnetic and electric fields as well as on attraction and repulsion, the teaching material contained self-explanations, texts, and thought experiments, and aimed at enabling the students to learn how to interpret graphical representations of field lines. Self-explanations are found to be an effective method in enhancing students understanding by guiding students to consciously reflect upon the teaching material [
42,
43]. The teaching material also contained a glossary for unknown terms that the students wanted to look up.
All students were instructed by the same teacher—the first author—in a facility at his research institution. As the course had to take place outside of school hours and appointments had to be made individually with each child, the course was carried out in small groups of one to six children per session. Generally, children were invited to come on three subsequent weeks on the same day, but due to individual constraints, the time between sessions also varied, with a mean of 7.63 days (SD = 8.92 days) from timepoint one to two and a mean of 6.96 days (SD = 5.79 days) between timepoint two and three.
After every unit, students constructed a concept map using the software CmapTools [
44]. Here, the concept mapping was used primarily as an assessment tool for research purposes. That is, the concept maps were not part of the instructional unit
per se. Constructing these concept maps may have affected students’ learning, but the maps were not specifically used as supportive learning tools. That is, students were not instructed in which kind of map was a desirable one, given a skeletal structure, or asked to reflect upon the map.
The method of concept mapping has to be practiced carefully beforehand to make sure that everyone understands and applies the method correctly and to prevent any bias from difficulties with the method. Thus, we introduced concept mapping at the beginning of the first unit as an assessment tool to make knowledge visible. We explained, that in contrast to mind maps, there is not necessarily a single central concept or a certain hierarchy, but that concept maps reveal how several terms relate to each other. The students then practiced concept mapping with CmapTools, first by completing an incomplete concept map on the topic of sound and instruments to familiarize with the structure and the principle of justifying every link between concepts. Second, they also had to construct a concept map on their own on the topic of floating and sinking to get an idea how to structure terms in the concept map and how to use the software. The teacher also made the students aware that there is not a single solution, but that every concept map is correct in its own sense. The teacher also mentioned that the links in these concept maps can be drawn in any direction and the direction will not be relevant. At the end of the introduction, the teacher made the students aware that each student’s concept map looked different. The teacher then prompted a metacognitive question—what can concept maps assess differently compared to typical exam questions? The teacher also made the students aware that they can use concept mapping outside of the course for summarizing a topic.
After each of the three units, students constructed a concept map with 35 terms provided on the topic of Magnetism and Electrostatics. See
Figure 1 for an example of a concept map and the provided terms. On visual inspection, there are some unconnected nodes and the grammatical expression on the relations sometimes lack sophistication. The terms “Ferromagnetism” and “Electrostatics” have been arranged by the student below the category “I don’t know this yet”, meaning that they could not yet assign these two terms. Nevertheless, we regard this as a good concept map, stating the characteristics that a magnetic field is caused by electric current, an electric field by charges, and a gravitational field by mass. Whereas visual inspection is important to characterize the context of the nodes and to verify the interpretations, using network analysis helps to avoid subjective rating.
After each unit, students had 25 min to draw the concept map. In all cases, they had to construct a new map and could not access previous ones. They did not receive a root node or root question. Instead, the task was to depict how the different terms and depictions relate to each other. Altogether, the 30 students drew 87 concept maps (three students did not complete all three timepoints) which constitute the sample of concept maps that were analyzed in this study. Specific characteristics of the concept maps used in this study were that the students could link terms that they did not know with a specific node labelled “I don’t know this yet” or with a node labelled “this doesn’t fit”. Moreover, nine nodes consisted of visual depictions of field lines—magnets, electric charges, and cables that attract or repel each other, as well as the gravitational field of the earth. We used the field line depictions from the English Wikipedia [
45]. See
Table 1 for an overview of these pictorial nodes and their labels that are used throughout the text. All the pictures contained field line visualizations that work as mental tools. Normally, electric and magnetic fields are invisible and have to be uncovered with certain materials, such as iron powder, small compasses or nails, or ferrofluid. Using such depictions in concept maps enables the representation of complex information, which cannot be described in one or two terms, which is the usual element of a concept map. Depictions are one way to put in a lot of information and background context in one node.
We also classified each verbal node into one of six categories. Terms that were graspable, real-world objects we classified as “Objects”, with “Magnet”, “Compass”, “Cobalt”, “Iron”, “Nickel”, and “Metal” falling into this category. Terms that in contrast depicted mental tools that are not really observable but help to understand a physical observation—which the concept of the field characterizes—fell into the category “Models”, with “Magnetic Field”, “Electric Field”, “Geomagnetic Field”, “Gravitational Field”, “Field”, “Field Lines”, “Direction of Force”, “Dipole”, “Monopole”, “Magnetic North Pole”, and “Magnetic South Pole” falling into this category.
Terms that were related to the topic of magnetism or electrostatics but were not clear physical models related to the field nor graspable objects were categorized separately. Terms that described quantities were categorized as “Physical Quantities”—“Charge”, “Electric Current”, and “Mass” fell in this category. The observable behaviors of “Attraction” and “Repulsion” were classified as “Phenomena”, and “Magnetic” and “Magnetizable” were classified as “Attributes”. The terms “Magnetism”, “Ferromagnetism”, “Electrostatics”, and “Electromagnetism” were assumed to be terms of a higher hierarchy level and were categorized as “Topics”. Together with the depictions, we had all the nodes classified into seven categories (Depictions, Objects, Models, Physical Quantities, Phenomena, Attributes, and Topics) that we used to investigate conceptual differences between categories of terms.
To investigate misconceptions, one rater—the first author—rated every link of each concept map in terms of misconceptions. Being rather conservative in this rating, only connections that clearly expressed conceptions that differed from the current state of scientific knowledge were coded as misconceptions. For example, the connection from the Geomagnetic Field to the Gravitational Field together with some label stating “this is a”, “equal”, or “depiction” was coded as a misconception, as the geomagnetic field is not the same as the gravitational field. Another example would be the connection from Charge to Magnetic North Pole together with the label “the charge determines the direction of the pole”, which would be a misconception as magnetic poles do not have a charge.
In order to investigate expert knowledge, four experts constructed concept maps—the first author, the second author who holds a doctoral degree in physics, as well as one secondary school teacher from physics, and a chemist constructed concept maps independently from each other.
2.3. Statistical Analyses
A concept map can be regarded as a network consisting of nodes (also called vertices) and links (also called edges) between them, which enables the application of graph theory on concept maps. At first, we applied standard graph theoretic measures to get an overview of the properties of the concept maps. As the direction of links was explicitly mentioned to be irrelevant, we analyzed all maps as undirected graphs. To describe surface features of the concept maps, we counted the number of links and nodes at each timepoint, the density (which is the ratio of the number of existing links to the number of possible links), the mean distance (which is the average length of all shortest paths between nodes in the graph), and the diameter (which is the longest path between two nodes in the network). Not all children connected unused nodes to either the specific node labelled “I don’t know this yet” or “this doesn’t fit” but left them unconnected. We aggregated these three options to count the number of unknown nodes and analyzed them separately. Furthermore, we removed nodes that were added individually by some students, to ensure comparability of the concept maps across students.
Regarding the centrality of certain nodes, several measures exist, from counting the number of edges per node (degree centrality), over the average shortest path length to all other nodes (closeness centrality) to the number of paths that cross through a certain node (betweenness centrality). Each centrality measure answers a slightly different question. Using degree centrality (or other measures of radial centrality), we would suppose that only immediate neighbors of a node would be considered when this node was inserted in the concept map. Structure of the overall map would not play a role at all. Regarding the process of constructing a concept map, this assumption is probably too strict. Using closeness centrality, one would suppose that every shortest path of every node would be considered when this node was inserted in the concept map—an assumption that is also hardly justifiable. Yet, we assumed, that besides the directly adjacent neighbors also the paths that cross a certain node have been considered in the construction of the concept map and inform about the centrality of it. Thus, to combine the number of adjacent nodes as well as some structural properties of a node, we used betweenness centrality. Moreover, as a comparison measure, we applied PageRank centrality [
46] that takes the number of (incoming) edges but also the importance of adjacent nodes sending these edges into account. In contrast to the aforementioned degree and betweenness centrality indices, PageRank centrality weights each edge differently according to the importance the node it emerges from. As PageRank centrality was developed for directed graphs but we used undirected graphs, we split each undirected edge into two directed ones for calculating it.
To investigate the underlying structure or common links across all students, we constructed aggregated graphs from all student maps. We aggregated the maps across all timepoints, but also separately for each timepoint. This aggregation enables researchers to interpret common pathways and structural elements that describe core features of the concept maps at each timepoint. The most common structure can be visualized when rare edges are left out. Thus, to be able to interpret the maps, we pruned the aggregated concept maps until only the most common edges remained. To identify homogeneous subgroups of terms in the concept maps, which reveal major topics and have similar characteristics, we used clustering of the concept maps. We applied a clustering algorithm based on Optimal Community Structure [
47] to all student maps separately.
We used R for all analyses, with the following packages in alphabetical order: afex, CINNA, centiserve, comato, corrplot, dplyr, emmeans, ggplot2, igraph, lmerTest, magrittr, proxy, qgraph, rcartocolor, and readxl [
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58,
59,
60,
61,
62]. The R script and data is available on Open Science Framework (
https://osf.io/behft/). We used an alpha level of 0.05 for all statistical tests.