Transferring Knowledge in a Knowledge-in-Use Task—Investigating the Role of Knowledge Organization

Abstract

Knowledge-in-Use, i.e., the ability to apply what one has learned, is a major goal of education and involves the ability to transfer one’s knowledge. While some general principles of knowledge transfer have been revealed, the literature is full of inconclusive results and it remains hard to predict successful transfer. However, research into expertise suggests that how one organizes one’s knowledge is critical for successful transfer. Drawing on data from a larger study on the learning of energy, we employed network analysis to investigate how the organization of students’ knowledge about energy influenced their ability to transfer and what role achievement goal orientation may have played in this. We found that students that had more coherently organized knowledge networks were more successful in transfer. Furthermore, we also found a connection between mastery goal orientation and the organization of students’ knowledge networks. Our results extend the literature by providing evidence for a direct connection between the organization of students’ knowledge networks, their success in transfer, and their goal orientation and hint at the complexities in the relationship between mastery approach goal orientation and successful transfer beyond what is reported in the literature.

1. Introduction

A major goal of education is the ability to use what one has learned in new contexts. This ability to transfer one’s knowledge is considered especially important in the rapidly changing world we live in, where “schools have to prepare students for jobs that have not yet been created, technologies that have not yet been invented, and problems that we don’t yet know will arise” [1]. Similar emphasis on transfer can be seen in national science standards such as the US Next Generation Science Standards (NGSS) [2] with its emphasis on knowledge-in-use, i.e., the ability to integrate domain knowledge with scientific practices and thus apply one’s knowledge in numerous contexts, or the German science standards [3] that emphasize competence—an effectively analogous concept to knowledge-in-use. While the traditional primarily psychological study of transfer has revealed some of the fundamental cognitive mechanisms based on identical elements [4], analogy [5], or schemata [6], the literature is ripe with mixed results and predicting when and under what conditions transfer will be successful or not remains challenging—especially when it comes to discipline specific knowledge [7,8,9]. With respect to discipline specific knowledge, studies of expertise [10] and conceptual change [11]—the study of the development of conceptual knowledge—have also looked at transfer. However, those studies, rather than focusing on the process or mechanism of transfer itself, have found that the ability to successfully apply one’s knowledge across a wide range of contexts, i.e., transferring one’s knowledge, is a key characteristic of expertise and successful conceptual change. Furthermore, a key difference between experts and novices is that the knowledge networks of experts are organized around key ideas of the domain rather than surface features as in the case of novices [10,12,13]. Similarly, conceptual change can be considered as a reorganization of one’s knowledge networks [14]. Thus, the organization of learners’ knowledge networks evidently plays an important role in successful transfer. However, traditional research into transfer has not connected its findings to the organization of students’ knowledge networks and studies that have investigated how students organize their knowledge [15,16,17] have rarely connected their findings explicitly to transfer. In this paper, drawing on data from a study into the teaching and learning of energy in middle school, we will start to address this issue by investigating how the organization of students’ knowledge networks is related to their ability to transfer their knowledge in a knowledge-in-use task.

1.1. Knowledge-in-Use

Recent science standards such as the US NGSS, the German science standards, and the definition of literacy used by the Organisation for Economic Co-operation and Development (OECD) [18] emphasize that a central goal of science education is to enable students to apply what they have learned to make sense of phenomena in the natural and engineered world across an extensive range of contexts, i.e., to demonstrate knowledge-in-use [19]. To do so, students need to integrate central disciplinary ideas, such as energy, with scientific practices, such as constructing models. Those ideas and practices, however, have to be learned in some context in which students learning will be anchored [20]. Therefore, to demonstrate knowledge-in-use and apply what they have learned across contexts, students will need to generalize what they have learned from the specific contexts. In consequence, valid knowledge-in-use assessments require at least some degree of knowledge transfer [21]. This transfer may be characterized as horizontal, i.e., ideas are applied in new contexts but no new knowledge is required, vertical, i.e., within the same context but requiring new knowledge, or as combining horizontal and vertical aspects. In sum, knowledge-in-use emphasizes using one’s knowledge about scientific ideas and practices to make sense of phenomena, requiring different degrees of transfer. This description aligns well with the description of transfer as a process of sense making.

1.2. Transfer as Sense Making

Transfer as sense making [22] emphasizes that when students transfer knowledge, they do so to meet a certain goal, e.g., explaining a new phenomenon or solving a problem. To quantify those goals, in our study, we used goal-orientation theory [23] as our theoretical framework. According to goal orientation theory, students can adopt goals that revolve around either building competence and understanding (mastery-oriented goals) or goals focused on demonstrating skill and ability (performance-oriented goals). Having different goals will effect student motivation, self-efficacy, interest, effort, persistence and more [24,25,26]. In their transfer as sense making framework, Nokes-Malach and Mestre (2013) argued that the goals that students try to meet when they engage in a transfer task are critical to their success. For example, in a transfer tasks that requires explaining a new phenomenon, students that have a higher mastery approach goal orientation may invest more effort because their goal is a correct explanation. In contrast, students that have a performance goal orientation may not feel inclined to invest the same amount of effort as mastery-oriented students because they care less about having the right solution. This line of argument is supported by empirical findings that suggest that mastery goals are associated with a deeper processing of materials [27] and increased effort and complexity of used strategies when facing challenges [28]. This inclusion of motivational aspects sets the transfer as sense making framework apart from traditional theories of transfer that describe transfer as nothing but an application of existing knowledge in the form of rules or schemata (see e.g., [29] for an example) to new areas. Thus, transfer as sense-making provides a more holistic picture of the transfer process and recent research has successfully demonstrated the large effect that goal orientation can have on transfer [30]. Nevertheless, the prior knowledge that students bring to a transfer task plays a fundamental role in successful transfer [31]. This aspect has received only little attention in empirical work that follows the transfer as sense making perspective. This leaves open questions about the mechanism through which mastery goal orientation affects transfer. Nokes and Belenky [32] argued that there is a direct effect of mastery goals on transfer. However, as we pointed out, prior knowledge plays a critical role in transfer [31], and mastery goals have been linked to numerous effects, e.g., higher cognitive activation, that support learning [27]. This raises the question of whether the influence of mastery goals on successful transfer is mediated by prior knowledge. In this case, the positive effect of mastery goals reported in [30] could primarily be the outcome of students with higher master goal orientation also having greater prior knowledge, that is better organized prior knowledge.

1.3. The Role of Prior Knowledge for Transfer

Early studies of transfer tried to predict successful transfer by the ratio between the number of concepts required in the transfer problem and the number of those concepts held by the student [4]. Thus, knowledge was primarily characterized by the “size” of one’s prior relevant knowledge. Today, knowledge is typically characterized as a network-like structure [31] and this perspective is successfully employed in cognitive- and neuroscience [33,34]. From a network perspective, it is not so much the number of ideas that characterizes knowledge but how those ideas are connected and organized. Such a network-like view has also been adopted in accounts of transfer that are rooted in the conceptual change literature [35,36]. In these studies, the researchers, drawing on the Piagetian notion of accommodation [37], connect transfer to changes in students’ knowledge structures, that is, a reorganization of students’ knowledge networks. However, these arguments remain on a theoretical level, as students’ knowledge networks or the changes to them were not measured. The research into expertise, however, has empirically investigated the knowledge networks of experts and novices. A major finding has been that experts structure their knowledge networks differently than novices: experts organize their knowledge networks around key ideas in a domain whereas novices have rather unconnected knowledge networks [10,12]. Connecting this result with the finding that transfer is a key aspect of expertise [12,31]; a picture emerges where a connection between the structure of students’ knowledge networks and their ability to transfer their knowledge can be made, but lacking empirical evidence, that picture remains blurry. Unfortunately, research that has explicitly focused on the structure of students’ knowledge networks [13,15,16,17] has rarely connected its findings to transfer and thus, has not contributed to a clearer picture of the relationship between students’ knowledge structure and their ability to transfer.

1.4. Research Questions

While the transfer of learning is increasingly important in a rapidly changing world and expected by educational standards that emphasize knowledge-in-use, it remains challenging to predict successful transfer. Research emphasizes that the organization of students’ knowledge networks and motivational aspects such as goal orientation play a critical role. However, the relationship between the structure of knowledge networks, goal orientation, and successful transfer remains untested. To address this issue, we drew on data from a study that investigated the teaching and learning of energy and asked the following research questions:

RQ 1: How is the organization of students’ knowledge networks related to their ability to transfer their knowledge to a new context in a knowledge-in-use task?

RQ 2: What is the relationship between students’ goal orientation, the structure of their knowledge networks, and successful transfer?

2. Materials and Methods

2.1. Design and Sample

We drew on data from interviews, a transfer task, and a goal orientation measure that were part of a larger study that took place in the Midwestern US at the 7th grade level and investigated the learning of energy in middle school across two different curricular units. Both units were about 10 weeks long, followed project-based learning pedagogy [38], and emphasized knowledge-in-use. However, while one unit conceptualized energy in terms of different forms that can be transformed into each other, the other conceptualized it as a unitary quantity that can be transferred (see [39] for more details on the units).

Interviews and a written goal orientation survey were administered at the end of the units and a transfer task was administered a few weeks later. Drawing on a network-analytical approach, we used the interview data to construct knowledge networks for each student [40]. Linking these to the results from the transfer task allowed us to answer research question 1. Relating these results further with the students’ goal orientation allowed us to address research question 2.

In this study, we focused on a subsample that was interviewed, answered the goal orientation survey and took the transfer task. The subsample (N = 18) had a similar number of students from both units (8/10). While the subsample was small, it was representative of the sample as a whole (N = 394), as there was only a small difference in their average score and standard deviation on a test of energy understanding administered at the end of the units (

Table 1. Scores on energy understanding measured at the end of the unit for subsample used in this study and sample from larger project.

Group	Mean	Standard Deviation	Cohen’s d
Whole Sample	−0.03	0.62	0.08, 95% CI [−0.41; 0.56]
Subsample used in this study	0.02	0.44

).

To account for potential systematic differences between students from the two units, i.e., students from one unit generally scoring higher than the other, we standardized all measures within the units.

2.2. Measures

2.2.1. Interviews and Knowledge networks

Students were interviewed individually according to a semi-structured interview-about-instances interview protocol [41]. In this protocol, students were shown short (5–10 s) videos of a series of five phenomena and prompted to explain each one. After introductions and obtaining student consent, we showed students the video of the first phenomenon, and asked: “How can you use scientific ideas to explain why the [object in video] moves like it does?”. Note that students were not asked to use energy ideas to make sense of the phenomenon. After students’ initial response, non-instructional prompts were used to clarify ambiguous student statements and clarify students’ conceptions of ideas such as fields if necessary. Prompts used the language of the student, e.g., if a student said: “The ball loses energy.” The interviewer might ask “It loses energy, what do you mean? Can you describe this a little more?” Interviewers were explicitly instructed to avoid instructive prompts, e.g., provide ideas they did not come up with themselves, or leading prompts, e.g., prompts that focus students on certain aspects of a phenomenon. After prompting, students were shown the video of next the phenomenon.

All interviews addressed the identical five phenomena: a rolling golf ball stopping on sand, bread baking in a solar oven, an electric motor being turned on, spinning for some time, and being turned off, a cart oscillating between two springs and stopping, and a ball being released and falling out of the frame. The phenomena were selected to cover a broad range of phenomena to be rich in the sense that one could interpret and explain them on different levels of detail (e.g., in case of the oscillating cart one could just discuss the oscillation part or go on and also discuss why the cart eventually stops).

The interviews provide the basis for constructing the knowledge networks. To construct the knowledge networks, we first, using qualitative content analysis [42], identified the normative uses of science ideas in the explanation of each phenomenon. To derive the categories for this deductive analysis, we referred to the literature on energy learning progressions (e.g., [43]) and student conceptions (e.g., [44]) to identify relevant energy ideas (e.g., forms, transfer, transformation). For all other concepts we considered grade band appropriate conceptions, e.g., for the concept of “force” we considered the respective literature on student conceptions (e.g., [45]).

Table 2. Example part of category system used for deductive coding of ideas.

Category	Definition	Anchor Example	Borderline Cases
Energy forms	S talks about energy being manifest in a form of energy.	“When the ball is held in his hand it has gravitational potential energy”.	Incorrect usages of forms such as relating kinetic energy to height are coded as “Forms incorrect”.
Energy transfer	S talks about energy being transferred from one system (including fields and objects) to another.	“The fire is transferring energy up to the cup”.	Energy has to be transferred from one to another system. Energy only being transferred “from” or “to” something is not sufficient as coded as “Transfer incorrect”.
Energy Transformation	S talks about one form of energy being transformed into another.	“The gravitational energy is being converted to the kinetic energy”.	Energy has to be transformed from one form into another.
Gravity	S talks about gravity pulling something down.	“Because gravity is trying to force it down”.

shows an example of the category system. Interrater reliability was assessed with help of a second trained rater who scored randomly chosen interviews that represent 10% of the interview sample. We found 88.79% agreement between the raters. Next, we drew on a network analytical approach, detailed in [40], to construct the actual knowledge networks and to quantify them. The central assumption in constructing the knowledge networks is that ideas that co-occur in the explanatory account of one phenomenon are connected. As an example, consider that a student used the ideas “energy transformation”, “energy forms”, “speed in one phenomenon. We would consider these three ideas all to be connected with each other. The strength of the connection would be identical for all connections and would be one, since the ideas co-occurred in the explanation of one phenomenon. Now, we can aggregate the networks for all the phenomena that a student explained to construct knowledge networks for individual students. In these networks, connections between ideas become stronger if students consistently used them across the phenomena they explained. For example, consider that in a second phenomenon, the same student used the ideas “energy transformation”, “energy forms”, and “temperature”. Aggregating this and the previous network would result in a network where the strength of the connection between “energy transformation” and “energy forms” would be increased by one, because the two ideas would now have co-occurred in the explanations of two phenomena. The strength of all other connections would remain one, accordingly. Furthermore, “temperature” would be connected with “energy transformation” and “energy forms” since it co-occurred with those ideas in the same phenomenon. However, “temperature” would not be connected with “speed” since the two ideas have not co-occurred in the explanation of the same phenomenon.

Now, we use two networks measures to quantify the knowledge networks. The network measure coherence, which quantifies the overall connectedness of the networks, is high if students used a set of similar ideas consistently across phenomena and low if students used very different ideas in each phenomenon. Thus, coherence can be considered a measure of integrated knowledge [46] or span [14]. While coherence quantifies the overall structure of the knowledge networks, degree can be used to quantify how strongly individual ideas are connected in the knowledge network and thus provides a way to quantify how central a given idea is in the network. These measures go beyond simple frequency of terms or codes calculations sometimes used to quantify interview data, as they convey structural information about students’ knowledge networks. Furthermore, what sets these networks apart from concept maps [17], is that they are created on the basis of students engaging in the practice of constructing scientific explanations, i.e., they are based on performance data and thus, overcome a critical limitation of traditional concept mapping [47].

2.2.2. The Transfer Task

The transfer task was developed following evidence-centered design [48] with iterative refinements based on the feedback from expert teachers, think-aloud interviews, and extensive testing with students. The task, which took around 45 min, was centered around the question of “How do instant heat packs work?” and was structured as an inquiry activity. It did not require energy ideas beyond those addressed in the units but rather situated them in a new chemistry focused context of heat packs. Therefore, we can characterize it as a horizontal transfer task.

The task was administered in a scaffolded and a unscaffolded version. In both versions, students first received text that introduced the heat pack and saw a short demonstration of the heat pack. Students were then given the opportunity to engage with heat packs themselves and were asked to provide their initial ideas about how the heat pack works. Then, students in the unscaffolded version were asked to answer two questions about the heat pack (Figure 1) that required them to integrate energy ideas with different scientific practices, i.e., emphasize knowledge in use. Students in the scaffolded version were given a learning resource before being asked to answer the two questions. The learning resource provided some information about how the heat packs work but did not give away the answers to the questions.

The heat pack sciences questions were scored by two experienced scorers following a rubric that emphasized knowledge-in-use, as it considered to what extent students integrated the required scientific practice and their understanding of energy (see

Table 3. Rubric for heat pack science question 1.

Answer Component	Level of Performance
	0	1	2
Claim	Chris’ claim, or does not identify a “best” claim	Jordan’s claim	Harper’s claim (may refer to Jordan’s claim also being true)
Evidence (connecting to evidence observed in the phenomenon or presented in the reading)	Response does not connect to evidence observed in the phenomenon or provided in the learning resource, or simply restates the claim	Student cites evidence but, remains on the macroscopic level, e.g., the temperature of the heat pack solution increases during boiling	Evidence connects to particle rearrangement, e.g., particles in the crystal separate when the heat pack is boiled
Reasoning (connecting evidence to claim via energy ideas)	Refers to no correct energy ideas in support of their response, or simply restates the claim	Uses some correct energy ideas to link to evidence, e.g., energy is transferred to the solution if it heats up atoms speed up when something heats up	Connects changes in particle arrangement to energy transfer to/from the salt solution, e.g., energy is transferred to the solution as particles get farther apart Note: it is not necessary to connect to the idea of fields here.

for an example). Sum scores were calculated for each student.

2.2.3. Goal Orientation Measure

The mastery goal orientation measure was adopted from Vedder-Weiss and Fortus [49]. It consisted of five items with a five-point Likert-scale where students rated statements from “not true at all” (1) to “very true” (5). The statements in the five items reflect aspects of mastery goal orientation and thus, higher scores on the likert scale indicate stronger endorsement of mastery goal orientation. Figure 2 shows an example item for measuring the students’ mastery goal orientation.

In order to obtain students’ mastery orientation “score” as well as to validate the construct structure and transform the data into an interval scale for subsequent analysis, the polytomous Rasch analysis [50] was performed. The mastery goal orientation construct showed high reliability (α = 0.84), and all items fell within acceptable infit and outfit measures [51].

2.3. Analyses

2.3.1. Research Question 1—Linking Knowledge networks and Transfer

To address research question 1, we first investigated the relationship between the overall organization of students’ knowledge networks and their performance on the transfer task by regressing the networks’ measure of coherence on the transfer score. To identify the role of different energy ideas, we calculated a series of regression models to investigate the relationship between the degree of different energy ideas and the transfer score. In this analysis, we differentiated between the students from the two curricular approaches as in one case, energy forms and transformations were emphasized and energy transfer in the other case.

2.3.2. Research Question 2—The Interplay of Goal Orientation, the Structure of Students’ Knowledge networks, and Successful Transfer

To address research questions 2, we investigated the correlations between goal, orientation, the overall structure of students’ knowledge networks as measured by coherence and degree, and successful transfer. For degree, we used the degree of the energy ideas that proved to be the most important for successful transfer in research question 1.

3. Results

3.1. Research Question 1—Linking Knowledge networks and Transfer

Figure 3 provides an example of a low (0.21) and a high (0.57) coherence knowledge network. In the low coherence network, connections between ideas are rather weak (line thickness) and individual ideas are only connected to few other ideas (size of circles). “Heat” and “Temperature” even appear unconnected to the remainder of the network, indicating that when the student used these ideas, he did not use any of the other ideas. In contrast, the high coherence knowledge network shows that stronger connections and ideas are organized around a hub of energy forms, energy transformation, force, and speed ideas. Ideas about energy forms and transformations and forces are central ideas in physics and science [2] and speed is an important indicator of changes in energy and forces.

Figure 4 shows that students with more coherent knowledge networks score higher on the transfer task. The respective regression model supports this conclusion (${\beta }^2 = 0.54, p= 0.02, R^2= 0.29$).

Table 4. Regression model predicting transfer score. All variables standardized.

Energy Idea	Energy FormsTransformation Unit			Energy Transfer Unit
	β2	p	R2	β2	p	R2
Forms	0.61	0.06	0.37	0.14	0.75	0.02
Transformation	0.70	0.02	0.50	No student used Transformation ideas.
Transfer of energy	0.22	0.53	0.05	0.80	0.02	0.64

shows the series of regression models that show how the centrality of individual energy ideas in students’ knowledge networks is related to their success on the transfer task. For students from the unit that emphasized energy forms and transformation, the centrality of the energy transformation idea in their knowledge networks is strongly related to their success on their transfer task. For students from the unit that emphasized energy transfer, the centrality of the transfer of energy idea in their knowledge networks is strongly related to their success on their transfer task.

3.2. Research Question 2—The Interplay of Goal Orientation, the Structure of Students’ Knowledge networks, and Successful Transfer

Table 5. Correlations between transfer score, coherence, degree of energy transfer and energy transformation, and mastery.

	Transfer Score	Coherence	Degree	Mastery
Transfer score	-	-	-	-
Coherence	0.54 *	-	-	-
Degree	0.62 **	0.77 ***	-	-
Mastery	0.40 •	0.36	0.52 *	-

Note: • = p < 0.10; * = p < 0.05; ** = p < 0.01; *** = p < 0.001.

describes the relationships between mastery goal orientation, the network measures that describe students’ knowledge networks, and students’ score on the transfer task. We found medium sized correlations between students’ mastery goal orientation and the network measures that describe students’ knowledge networks as well as between students’ mastery goal orientation and their transfer score. This means that students with a higher mastery goal orientation do not only score higher on the transfer task but also tend to connect ideas about energy transfer or energy transformation more strongly in their knowledge networks.

4. Discussion

We found that students who had more coherently organized knowledge networks scored higher on a knowledge-in-use transfer task. Further, we were able to determine the importance of individual energy ideas for successful transfer. Interestingly, students with higher mastery goal orientation were more likely to organize their knowledge networks around those central ideas. In sum, the results show the potential of network analysis to aid our understanding of transfer as network analysis provides a more nuanced picture of students’ prior knowledge than, e.g., a concept inventory.

4.1. Prior Knowledge and Transfer—It Is All About the Connections

There is broad consensus that knowledge-in-use or the ability to successfully apply ones’ knowledge to make sense of the natural and engineered world is a major goal of education. However, making sense of the diverse phenomena that students encounter in their daily lives often requires knowledge transfer. Traditionally, transfer has been perceived as the challenge of mapping ones existing, that is, prior knowledge, on the new phenomenon [7,22]. In research question one, we also focused on the role of students’ prior knowledge for transfer. However, in contrast to prior studies, we drew on students’ knowledge networks to characterize their prior knowledge. Our finding that students with more coherent knowledge networks, i.e., knowledge networks that were more strongly connected and organized around a set of central ideas, performed better on the transfer task supports the rationale of modern science standards to focus on a small set of powerful ideas and emphasize connections between those [2,3,52]. Furthermore, it corroborates with findings from the research into expertise [10] and conceptual change [11].

The analysis of the role of individual ideas supports the argument that energy transformation and energy transfer (for students in the respective units) are what Lee and Liu [53] call linking ideas, i.e., especially important ideas that serve as hubs in students’ knowledge networks. While this role of energy transfer has been reported in knowledge-in-use assessments that require only little transfer [40], our findings extend this result to horizontal knowledge-in-use assessments and empirically demonstrate that also energy transformation can take this role in a learning environment that supports this.

More generally, demonstrating that students that have knowledge networks in which energy transfer is a central idea are more successful in regular knowledge-in-use assessments [40] and in a horizontal transfer task is an interesting finding because it supports the theoretical argument that energy transfer is a useful central idea within physics as well as across the sciences [53]. This draws on the larger question of whether ideas that are useful with a disciplinary frame are also useful when it comes to horizontal transfer. This question is important because, as Andreas Schleicher, the Director for the Directorate of Education and Skills at OECD, has argued, “schools have to prepare students for jobs that have not yet been created, technologies that have not yet been invented, and problems that we don’t yet know will arise” [1] and thus, instruction that emphasizes ideas that are helpful within a discipline as well as for transfer across disciplines is potentially preferable to instruction that emphasizes ideas which are only useful in one of these cases. The results in this study together with the findings reported in [40] provide first evidence that emphasizing energy transfer is useful within as well as across disciplines.

4.2. The Role of Goal Orientation for Transfer

Similarly to what Belenky and Nokes [30] found and as Nokes and Belenky [32] argued in the transfer as sensemaking model, we found mastery goal orientation to be related with successful transfer [30]. Extending the work of Belenky and Nokes [30], we also looked at the relationship between mastery goal orientation and students’ knowledge networks. We found that students with higher mastery goal orientation tended to structure their knowledge networks around central energy ideas such as energy transfer or energy transformation (as the network measure of degree shows) and also tended to have generally better organized knowledge networks (as measured by the network measure of coherence). These findings are in line with the results that suggest that mastery goal orientation leads to deeper processing of materials [27] and thus, to increased conceptual understanding which, in our study, is reflected in having better organized knowledge networks.

For the transfer as sensemaking framework, in general, our finding that mastery goal orientation is related to students’ transfer scores as well as the organization of their prior knowledge suggests that the role of mastery goal orientation may be more complex than the model of Nokes and Belenky [32] suggests: while our sample prohibits mediation analyses to further disentangle the role between mastery goal orientation, the structure of students prior knowledge, and successful transfer, the mere presence of the relation between mastery goal orientation and the structure of students’ knowledge networks implies that the role of mastery goals in transfer is more nuanced than just a direct effect during the sensemaking process as implied by Nokes and Belenky [32]. While Belenky and Nokes [54] have started to take a closer look at effects concerning students’ goal orientation in transfer tasks, our study shows how the fine grained analyses of students’ prior knowledge that network methods can deliver can provide a complimentary lens to explain individual differences in successful transfer.

4.3. How Network Analysis Can Aid the Understanding of Transfer

In research question 1, we were able to relate successful transfer with having knowledge networks that are organized around individual important ideas about energy. In research question 2, we found that students that endorse mastery goals more strongly tend to have knowledge networks that are organized around those important energy ideas. Given the emphasis that modern theories of transfer place on the coordination of ideas [9,11,55] it appears that this is exactly the grain size that is needed to make progress in the troubled field of transfer research [7,8,9]. Traditional assessments may give good estimates about how much a student knows or what ideas a student has mastered but tell us little about how a student knows. However, when successful transfer is not about how many ideas one has mastered but rather about how one can coordinate these ideas, it is not surprising that we still struggle to correctly predict successful transfer [7,8,9] when we use traditional measures of prior knowledge. The knowledge networks and respective measures used in this study address this gap as they provided us with information about how students coordinate ideas when making sense of phenomena. What we consider a critical distinction of the network approach from traditional concept mapping techniques is that the knowledge networks are constructed from student explanations, i.e., actual student performances in the domain of interest, and not constructed by the students’ themselves. If students construct networks themselves, the problem arises that this requires meta knowledge about the structure of the domain but not necessary competence in the domain. For example, a student may very well know that e.g., energy, forces, speed, and momentum are all connected ideas in physics, but not be able to coordinate these ideas in an actual explanation [47]. Since the knowledge networks in the approach used in this study were constructed from actual student performances, this problem disappears as evidenced by the large shares of variance in student performance in the transfer task that is explained by the network measures.

4.4. Limitations

The main limitation of our study is certainly the small sample size which is a consequence of the interview approach which is difficult to scale. Nevertheless, the results we have obtained are promising and warrant further research. In further research however, we suggest exploring the potentials of learning analytics and natural language processing methods to scale the approach. For example, automated analyses of student answers on worksheets and other artifacts may provide a data source that could be used to create knowledge networks.

Another limitation of this study is that we only looked at one horizontal task which limits the generalizability of our findings. Thus, future studies should look at a larger variety of tasks and also investigate vertical transfer tasks. However, the development of the transfer task in this study has been challenging, and others have faced challenges in the development of transfer tasks as well [21]. Thus, there seems to be a need for a set of well-functioning transfer tasks and design principles that can provide guidance in the development process of such tasks.

4.5. Using Network Analysis to Investigate the Structure of Students’ Knowledge networks—an Outlook

In the current approach to constructing and analyzing students’ knowledge networks, only students’ normative ideas are included. However, when it comes to developing conceptual understanding of major ideas such as energy, evolution, or chemical reactions, non-normative ideas play an important role [14]. Often collectively referred to as misconceptions, non-normative ideas have been a focus of discipline specific education research as they are believed to inhibit students from developing conceptual understanding (see e.g., [38,39]). However, as those ideas are often deeply rooted in students’ everyday experiences and cultural background, e.g., students thinking of energy as a kind of substance that flows [44], they have shown to be resistant to change [14,56,57]. In consequence, it has been proposed to try and build on these ideas [14,58], e.g., the idea that energy is like a substance can be a helpful metaphor for using energy ideas productively to make sense of a wide range of phenomena as long as one points out the limits of the metaphor as well [59,60]. However, whether one wants to avoid non-normative ideas or embraces them as resources, the interplay of normative and non-normative ideas should play an important role in the reasoning processes that students engage in when making sense of phenomena.

In principle, the interview data used in this study could also be coded for students’ non-normative ideas but when one would simply include them when constructing students’ knowledge networks following the current approach from [40], the measures would become hard to interpret. To address this issue, we propose that future research into students’ knowledge networks explores the potentials of multidimensional networks (also known as multilayer, multiplex or multilevel networks) that allow to describe multiple types of relations between the elements in a network and multiple types of elements in a network [61].

Finally, another way to extend the current network analytical approach would be to include affective data into the knowledge networks. Currently, relations between affective variables such as mastery goals in the present study and students’ knowledge networks are analysed after both have been measured and constructed independently. Integrating these measures as, e.g., in forma mentis networks [62], might reveal relations and subtleties that the current independent measurement and analysis of affective and cognitive variables hides.