# Learning Pathways and Students Performance: A Dynamic Complex System

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- R1.
- Does a strong relationship between OLR and learning performance emerge when analysis from a micro to a macro level is performed?
- R2.
- Can the equifinality property in an individual learning pathway be modelled to classify the learning performance?
- R3.
- Are the student’s learning pathways a complex system?

## 2. Related Work

## 3. Preliminaries on Learning Pathway Networks

#### 3.1. Construction of Learning Pathway Networks

_{1}and LP

_{2}be the adjacency matrix of two individual learning pathways. To combine them, the “or” matrix operation denoted by LP

_{1}|LP

_{2}is defined as 1 if LP

_{1}(i,j) ≠ 0 or LP

_{2}(i,j) ≠ 0; it is defined as 0, otherwise. This operation can be repeated for every student enrolled in the course to obtain a collective learning pathway network. Since the individual learning pathways follow different combinations in the sequence of learning resources, the collective learning pathways are complex networks, as shown in Figure 2. The direction of the arcs indicates the order in which the resources were reviewed.

#### 3.2. Extraction of the Relevant Nodes of Learning Pathway Networks

#### 3.3. Online Learning Rate

_{b}(l) is the minimum number of boxes to cover the network. For a detailed description of the box-covering algorithm, see [26]. The OLR can be computed on individual and collective networks. The OLR is a normalised measure that can be compared among different learning pathway networks where one means the fastest learning rate and zero is the slowest.

_{b}) equals the number of nodes (n = 35; note that node enumeration starts from 2). In other words, reviewing the material at the speed of sessions of size one will take 35 sessions to complete the course. Now the speed is five (l = Δ = 5)–which means sessions of size five (box size)—hence, the entire network is covered by a session (one box) that includes all the learning material (nodes). It means that N

_{b}is computed with boxes of diameter l = Δ; thus, N

_{b}= 1. The box-covering implementation in the Supplementary Material obtains the N

_{b}for l = [2, Δ−1]. When the student reviews the material in a linear pathway, the resulting plot of l vs. N

_{b}is similar to a straight line with a slope of −1/2.

## 4. Method

#### 4.1. Participants and Context

#### 4.2. Analysis of Emergence

_{b}. The Mann–Whitney U test was performed on OLR (computed in collective and individual networks) to determine if the students who failed the course obtained a lower OLR score than those who passed. Moreover, the OLR and final grade correlations were analysed using the partitioned collective network. Moreover, this analysis was performed using all individual networks. These analyses of individual and collective networks seek to answer research question one, as shown in Figure 5.

#### 4.3. Analysis of Equifinality

## 5. Results

#### 5.1. The Emergence of OLR and Learning Performance Correlation

_{a}= 5, N

_{f}= 5) = 0.000, z = −2.611, p = 0.008. This result shows that the OLR can differentiate between passing students and those with poor learning performance, as was found in previous research [8]. Moreover, a significant difference in the OLR computed on individual learning pathway networks was found between students who passed (Mdn = 0.669) and failed (Mdn = 0.591); U(N

_{a}= 373, N

_{f}= 51) = 4513.5, z = −1, p < 0.001, as shown in Figure 7.

^{2}) of the linear model of the OLR of the individual learning pathways networks and students’ final grades was 0.225, F(1,422) = 123.903, p < 0.001. The regression coefficient (β = 0.476, p < 0.001) indicates that the final grade increases as OLR. Meanwhile, the collective partitioned learning pathway networks were adj R

^{2}= 0.515, F(1,8) = 10.542, p = 0.012. Similarly, the regression coefficient is positive (β = 0.754, p = 0.012). These results suggest that a strong relationship emerges from individual learning pathways (effect size f

^{2}= 0.293) analysis to collective ones (effect size f

^{2}= 1.317) [29]. Furthermore, a complex topology emerges from the individual learning pathways when they are gathered to construct the collective ones, although the individual ones are sequential revisions with a few bifurcations, as shown in Figure 2 and Figure 8. The complex topology of collective learning pathways emerges from a non-additive process of the individual learning pathways, as the collective pathways are non-additive systems.

#### 5.2. Equifinality and Learning Performance

_{a}= 373, N

_{f}= 51) = 4943, z = −5.573, p < 0.001. The mean distances of the learning pathways of the students who passed (44) and failed the course (49) were far from the expected learning pathway designed by the faculty members and were higher for failed students. Moreover, the minimum distance for successful students was 31 and 30 for failed students. These values suggest that not even one student reviewed the learning material in the way that it was designed. Figure 9 shows the Levenshtein distance between sequences of students who (a) passed and (b) failed the course and the expected learning pathway. Intense blue in Figure 9a,b indicates that many deletions and insertions were performed to transform the current sequence into the expected one to review the course content. Moreover, Figure 9b shows many more sequences with intense blue than Figure 9a. Hence, students who failed the course did not browse the learning material as expected.

_{t}= 500, N

_{f}= 500) = 93,606, z = −6.957, p < 0.0001. The area under the receiver operating characteristic curve was higher for the fractal method (Mdn = 0.969) than for the topological sort (Mdn = 0.957); U(N

_{t}= 500, N

_{f}= 500) = 240,337.5, z = −2.170, p = 0.03. The Matthews correlation was analysed, and the Mann–Whitney U test showed that the fractal method (Mdn = 0.884) obtained a higher value than did the topological sort (Mdn = 0.857); U(N

_{t}= 500, N

_{f}= 500) = 212,427.5, z = −8.296, p < 0.0001, as shown in Figure 10. Thus, the equifinality presented in individual learning pathways is useful for classifying the students’ learning performance. Furthermore, the fractal node extraction approach outperformed the topological sort in classifying the learning performance.

^{®}was lower than the fractal method (f) (Mdn = 0.94); U(N

_{r}= 500, N

_{f}= 500) = 1209, z = −27.259, p < 0.0001. The area under the receiver operating characteristic curve was higher for the fractal method (Mdn = 0.969) than for the random forest (Mdn = 0.62); U(N

_{r}= 500, N

_{f}= 500) = 10, z = −27.372, p < 0.0001. Moreover, the Matthews correlation was analysed, and the Mann–Whitney U test showed that the fractal method (Mdn = 0.884) obtained a higher value than the random forest (Mdn = −0.025); U(N

_{t}= 500, N

_{f}= 500) = 0.0, z = −21.842, p < 0.0001, as shown in Figure 10. The low value of the Matthews correlation means that random forest incorrectly classified most positive and negative instances, and most of its positive and negative predictions were also incorrect [30,31]. These low results show that the random forest cannot learn how the sequences differentiate the students’ learning performance.

## 6. Conclusions

## Supplementary Materials

**a**) a graph. (

**b**) The result of the box number starting from node three for a box size one. (

**c**) The result of the box number for a box size two. (

**d**) The box assignment for the six nodes, varying the box size from one to five.

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Ladyman, J.; Lambert, J.; Wiesner, K. What is a complex system? Eur. J. Philos. Sci.
**2013**, 3, 33–67. [Google Scholar] [CrossRef] - Siegenfeld, A.F.; Bar-Yam, Y. An Introduction to Complex Systems Science and Its Applications. Complexity
**2020**, 2020, 6105872. [Google Scholar] [CrossRef] - Wilensky, U.; Jacobson, M.J. Complex Systems and the Learning Sciences. In The Cambridge Handbook of the Learning Sciences, 2nd ed.; Sawyer, R.K., Ed.; Cambridge Handbooks in Psychology; Cambridge University Press: Cambridge, UK, 2014; pp. 319–338. [Google Scholar]
- Quintana, R. Embracing complexity in social science research. Qual. Quant.
**2023**, 57, 15–38. [Google Scholar] [CrossRef] - Bar-Yam, Y. Dynamics of Complex Systems; Addison-Wesley: Boston, MA, USA, 1997. [Google Scholar]
- Hilpert, J.C.; Marchand, G.C. Complex Systems Research in Educational Psychology: Aligning Theory and Method. Educ. Psychol.
**2018**, 53, 185–202. [Google Scholar] [CrossRef] - Ramirez-Arellano, A.; Hernández-Simón, L.M.; Bory-Reyes, J. A box-covering Tsallis information dimension and non-extensive property of complex networks. Chaos Solitons Fractals
**2020**, 132, 109590. [Google Scholar] [CrossRef] - Ramirez-Arellano, A. Students learning pathways in higher blended education: An analysis of complex networks perspective. Comput. Educ.
**2019**, 141, 103634. [Google Scholar] [CrossRef] - Favela, L.H. Cognitive science as complexity science. WIREs Cogn. Sci.
**2020**, 11, e1525. [Google Scholar] [CrossRef] - Ramirez-Arellano, A.; Sigarreta Almira, J.M.; Bory-Reyes, J. Fractional Online Learning Rate: Influence of Psychological Factors on Learning Acquisition. Nonlinear Dyn. Psychol. Life Sci.
**2022**, 26, 289–313. [Google Scholar] - Pintrich, P.R. Multiple goals, multiple pathways: The role of goal orientation in learning and achievement. J. Educ. Psychol.
**2000**, 92, 544. [Google Scholar] [CrossRef] - Jacobson, M.J.; Levin, J.A.; Kapur, M. Education as a Complex System: Conceptual and Methodological Implications. Educ. Res.
**2019**, 48, 112–119. [Google Scholar] [CrossRef] - Linnenbrink, E.A.; Pintrich, P.R. Chapter 8—Multiple pathways to learning and achievement: The role of goal orientation in fostering adaptive motivation, affect, and cognition. In Intrinsic and Extrinsic Motivation; Sansone, C., Harackiewicz, J.M., Eds.; Academic Press: San Diego, CA, USA, 2000; pp. 195–227. [Google Scholar]
- Crosslin, M.; Breuer, K.; Milikić, N.; Dellinger, J.T. Understanding student learning pathways in traditional online history courses: Utilizing process mining analysis on clickstream data. J. Res. Innov. Teach. Learn.
**2021**, ahead-of-print. [Google Scholar] [CrossRef] - Crosslin, M.; Dellinger, J.T.; Joksimovic, S.; Kovanovic, V.; Gaševic, D. Customizable Modalities for Individualized Learning: Examining Patterns of Engagement in Dual-Layer MOOCs. Online Learn.
**2018**, 22, 19–38. [Google Scholar] [CrossRef] - Crosslin, M. Exploring self-regulated learning choices in a customisable learning pathway MOOC. Australas. J. Educ. Technol.
**2018**, 34. [Google Scholar] [CrossRef] - De Smet, C.; De Wever, B.; Schellens, T.; Valcke, M. Differential impact of learning path based versus conventional instruction in science education. Comput. Educ.
**2016**, 99, 53–67. [Google Scholar] [CrossRef] - Iatrellis, O.; Kameas, A.; Fitsilis, P. A novel integrated approach to the execution of personalized and self-evolving learning pathways. Educ. Inf. Technol.
**2019**, 24, 781–803. [Google Scholar] [CrossRef] - Iatrellis, O.; Kameas, A.; Fitsilis, P. EDUC8 ontology: Semantic modeling of multi-facet learning pathways. Educ. Inf. Technol.
**2019**, 24, 2371–2390. [Google Scholar] [CrossRef] - Wanichsan, D.; Panjaburee, P.; Chookaew, S. Enhancing knowledge integration from multiple experts to guiding personalized learning paths for testing and diagnostic systems. Comput. Educ. Artif. Intell.
**2021**, 2, 100013. [Google Scholar] [CrossRef] - Liu, H.; Li, X. Learning path combination recommendation based on the learning networks. Soft Comput.
**2020**, 24, 4427–4439. [Google Scholar] [CrossRef] - Ren, Q.; Peng, X.; Liu, X.; Zheng, Q.; He, T.; Hu, D.; Jiang, X.; Zhang, L. Network modelling and visualisation analysis of the undergraduate dental curriculum system in China. J. Comput. Commun.
**2021**, 9, 38–51. [Google Scholar] [CrossRef] - Gajewski, Ł.; Chołoniewski, J.; Hołyst, J. Key Courses of Academic Curriculum Uncovered by Data Mining of Students’ Grades. Acta Phys. Pol. A
**2016**, 129. [Google Scholar] [CrossRef] - Mai, T.T.; Bezbradica, M.; Crane, M. Learning behaviours data in programming education: Community analysis and outcome prediction with cleaned data. Future Gener. Comput. Syst.
**2022**, 127, 42–55. [Google Scholar] [CrossRef] - Ramirez-Arellano, A.; Bory-Reyes, J. The Resilience of Complex Network: An Approach for Relevant Nodes Extraction. Fractals
**2021**, 29, 2150009. [Google Scholar] [CrossRef] - Chaoming, S.; Lazaros, K.G.; Shlomo, H.; Hernán, A.M. How to calculate the fractal dimension of a complex network: The box covering algorithm. J. Stat. Mech. Theory Exp.
**2007**, 2007, P03006. [Google Scholar] - Hammersley, M.; Traianou, A. Ethics and Educational Research; British Educational Research Association: London, UK, 2012. [Google Scholar]
- Aho, A.V.; Corasick, M.J. Efficient string matching: An aid to bibliographic research. Comm. ACM
**1975**, 18, 333–340. [Google Scholar] [CrossRef] - Cohen, J. Statistical Power Analysis for the Behavioral Sciences; Academic Press: Cambridge, MA, USA, 1977. [Google Scholar]
- Chicco, D.; Tötsch, N.; Jurman, G. The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation. BioData Min.
**2021**, 14, 13. [Google Scholar] [CrossRef] [PubMed] - Chicco, D.; Jurman, G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genom.
**2020**, 21, 1–13. [Google Scholar] [CrossRef] [PubMed] [Green Version]

**Figure 2.**The collective learning pathway network from 42 students enrolled in Mathematics applied to Biological Science from January–July 2020 semester. a = activity, e = examination, l = lesson, r = reading.

**Figure 3.**Extracting relevant nodes by computing the boxes to cover the learning pathway network of the January–July 2020 semester. (

**a**) The minimum boxes to cover the network. (

**b**) Nodes with the highest betweenness in each box are deleted, which are nodes 36, 29, 26 and 32.

**Figure 4.**The relevant learning pathways are extracted by relevant node identification from the January–July 2020 semester. a = activity, e = examination, l = lesson, r = reading.

**Figure 7.**The significant difference in OLR by learning performance of individual learning pathways (

**a**) and collective ones (

**b**).

**Figure 8.**The student’s learning pathways, those of (

**a**,

**b**), obtained a final grade of seven and (

**c**,

**d**) of nine. A = activity, e = examination, l = lesson, r = reading.

**Figure 9.**Levenshtein distance between students who (

**a**) passed and (

**b**) failed the course and the expected learning pathway.

**Figure 10.**The accuracy of the Matthews Correlation Coefficient (MCC), and Area Under the Receiver Operating characteristic Curve (AUROC) for the classification of learning performance by the LSTM network. * Statistically different p > 0.05.

Semester | Partition | Short Name | Students Enrolled | Final Grade (Mean) |
---|---|---|---|---|

January–July 2020 | A | JJ2020A | 33 | 8.73 |

F | JJ2020F | 9 | 2.33 | |

August–December 2020 | A | AD2020A | 68 | 8.87 |

F | AD2020F | 7 | 2.29 | |

January–July 2021 | A | JJ2021A | 107 | 8.21 |

F | JJ2021F | 10 | 4.6 | |

August–December 2021 | A | AD2021A | 81 | 7.73 |

F | AD2021F | 20 | 3.05 | |

January–July 2020 | A | JJ2022A | 84 | 8.13 |

F | JJ2022F | 5 | 2.4 |

**Table 2.**The sequences extracted from learning pathways are split into several attributes to train and test the random forest. The learning pathways are shown in Figure 8. - means no value.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ortiz-Vilchis, P.; Ramirez-Arellano, A.
Learning Pathways and Students Performance: A Dynamic Complex System. *Entropy* **2023**, *25*, 291.
https://doi.org/10.3390/e25020291

**AMA Style**

Ortiz-Vilchis P, Ramirez-Arellano A.
Learning Pathways and Students Performance: A Dynamic Complex System. *Entropy*. 2023; 25(2):291.
https://doi.org/10.3390/e25020291

**Chicago/Turabian Style**

Ortiz-Vilchis, Pilar, and Aldo Ramirez-Arellano.
2023. "Learning Pathways and Students Performance: A Dynamic Complex System" *Entropy* 25, no. 2: 291.
https://doi.org/10.3390/e25020291