Enhancing Dropout Prediction in Distributed Educational Data Using Learning Pattern Awareness: A Federated Learning Approach

Abstract

Learning patterns are crucial for predicting student dropout in educational settings, providing insights into students’ behaviors and motivations. However, existing mainstream dropout prediction models have limitations in effectively mining these learning patterns and cannot mine these learning patterns in large-scale, distributed educational datasets. In this study, we analyze the representations of mainstream models and identify their inability to capture students’ distinct learning patterns and personalized variations across courses. Addressing these challenges, our study adopts a federated learning approach, tailoring the analysis to leverage distributed data while maintaining privacy and decentralization. We introduce the Federated Learning Pattern Aware Dropout Prediction Model (FLPADPM), which utilizes a one-dimensional convolutional neural network (CNN) and a bidirectional long short-term memory (LSTM) layer within a federated learning framework. This model is designed to effectively capture nuanced learning patterns and adapt to variations across diverse educational settings. To evaluate the performance of LPADPM, we conduct an empirical evaluation using the KDD Cup 2015 and XuetangX datasets. Our results demonstrate that LPADPM outperforms state-of-the-art models in accurately predicting student dropout behavior. Furthermore, we visualize the representations generated by LPADPM, which confirm its ability to effectively mine learning patterns in different courses. Our results showcase the model’s ability to capture and analyze learning patterns across various courses and institutions within a federated learning context.

1. Introduction

1.1. Motivation

The rapid advancement of Internet and Artificial Intelligence (AI) technologies has profoundly influenced various aspects of our lives, including smart city development [1,2,3,4], energy management [5,6], and financial services [7]. Particularly notable is its impact in the field of education, especially in the realm of Massive Open Online Courses (MOOCs). As an emerging form of online education, MOOCs are undergoing continuous transformation and innovation through the application of AI and Internet technologies.

The phenomenon of student dropout in MOOCs represents a critical challenge in online education, attracting extensive research and investigation. High dropout rates, prevalent in certain courses and programs, adversely affect student success and institutional reputations. To combat this, various dropout prediction models have been developed, employing machine learning algorithms and deep learning models [8,9,10,11]. These models aim to identify at-risk students early, enabling timely interventions to bolster retention rates. Computational approaches, including advanced algorithms and mathematical models, have been instrumental in enhancing the accuracy and efficiency of these predictions. Notably, the integration of parallel and distributed computing techniques, including federated learning approaches, has begun to transform dropout prediction in MOOCs. By leveraging distributed data while maintaining privacy and reducing computational burdens, federated learning offers a promising avenue for developing more robust, scalable, and efficient dropout prediction systems. Such advancements hold the potential not only to refine intervention strategies but also to significantly improve student engagement and retention outcomes in online educational platforms.

Extensive research has been conducted to unravel the factors influencing student dropout in various educational contexts, with a significant focus on Massive Open Online Courses (MOOCs). Studies such as Feng et al.’s analysis of XuetangX, China’s largest MOOC platform, shed light on distinct studying behaviors and their linkage to dropout motivations [12]. These investigations reveal correlations between dropout rates and various factors, including peer influence, engagement levels, course difficulty, workload, and content relevance [9,13,14]. Jin et al.’s definition of dropout as a sustained absence of learning behavior further underscores the need for sophisticated data analysis techniques in these large-scale educational datasets [14]. These insights form a crucial foundation for the development of advanced dropout prediction models, driving the need for more nuanced analytical approaches that can process complex data patterns and address data privacy considerations.

In addition to the factors mentioned earlier, the significance of learning patterns in predicting student dropout is paramount. Learning patterns, defined as a coherent set of learning activities that learners engage in over a specific time period [15], provide profound insights into student behavior and motivations. These patterns are critical in assessing the likelihood of students dropping out. Research indicates that students who share similar beliefs about learning and motivation tend to exhibit similar learning patterns [16]. However, the variability of these patterns across different courses, shaped by unique educational contexts and motivations [17,18], presents a complex challenge in accurately predicting dropout.

1.2. Significance

Addressing this challenge necessitates the adoption of advanced computational approaches. The application of particularly parallel and distributed processing techniques becomes crucial here. These techniques facilitate effective handling and analysis of diverse learning patterns, adapting to the dynamic nature of educational data. Moreover, integrating mathematical models and federated learning approaches into this framework not only enhances the accuracy of predictions but also aligns with the decentralized nature of MOOC data. By focusing on these unique learning patterns through sophisticated analytical models, our approach aims to refine dropout prediction methods in MOOCs, tailoring them to the intricacies of different courses and student behaviors.

Our study undertook a comprehensive analysis of existing dropout prediction models. Utilizing the t-SNE dimensionality reduction technique [19], we visualized the models’ representations to assess their effectiveness. This analysis uncovered two key limitations. First, we observed that mainstream models often fall short in adequately capturing the distinct learning patterns of students. Such inadequacy hinders their predictive accuracy, limiting their utility in identifying potential dropouts based on these patterns. Second, these models typically fail to represent personalized learning patterns that vary across different courses. This lack of customization in model representations leads to ineffective predictions of student dropout behavior in varied educational settings. These findings underscore the necessity for more sophisticated computational approaches. By leveraging such advanced methods, we aim to develop enhanced models that can more accurately capture and analyze the intricate and varied learning patterns of students, thereby significantly improving the precision of dropout predictions in MOOC environments.

1.3. Objectives

To effectively overcome the limitations of traditional dropout prediction models, we have developed the Federated Learning Pattern Aware Dropout Prediction Model (FLPADPM), an approach that maintains the original model architecture while integrating the principles of federated learning. In our federated learning setup, each client (e.g., different educational institutions or MOOC platforms) independently trains a local instance of FLPADPM on their own dataset. This local training process utilizes the one-dimensional convolutional neural network (CNN) and bidirectional long short-term memory (LSTM) layer, ensuring consistency in feature extraction and analysis of temporal relationships in student learning patterns.

During the local training phase, each client’s model learns to identify and understand the unique learning behaviors and patterns prevalent in their respective datasets. The course-specific learnable matrices, $\mathbf{K}$ and $\mathbf{V}$, within each local model, adapt to the distinct educational contexts of their data, offering personalized insights into student learning patterns. The attention mechanism, deployed at each client, tailors the dropout prediction to individual students by focusing on the most relevant learning patterns, as determined by the local data.

Once the local models complete their training, the key step in our federated learning approach comes into play: the aggregation of learned parameters on a central server. Here, the parameters from all clients are aggregated to update the global FLPADPM model. This aggregation process ensures that the global model benefits from the diverse and comprehensive learning insights gathered across all participating clients, enhancing its overall predictive accuracy.

1.4. Contributions

In this study, we present three key contributions to the field of dropout prediction in MOOCs. First, we identify and articulate the limitations of mainstream dropout prediction models, particularly their inability to effectively uncover and interpret the distinct learning patterns exhibited by students. We observe that these conventional models often overlook the personalized variations in learning patterns across different courses, leading to a shortfall in their predictive accuracy for student dropout behavior.

Second, in response to these challenges, we propose the Federated Learning Pattern Aware Dropout Prediction Model (FLPADPM), an innovative approach that integrates the original architecture of the Learning Pattern Aware Dropout Prediction Model (LPADPM) with the principles of federated learning. FLPADPM leverages a one-dimensional convolutional neural network (CNN) and a bidirectional long short-term memory (LSTM) layer to deeply analyze relationships, patterns, and the temporal dynamics of student data. Furthermore, we incorporate learnable matrices to model and understand the variations in learning patterns specific to different courses. This aspect of the model is crucial in capturing the unique educational contexts and thereby enhancing the accuracy of dropout predictions.

Third, through extensive empirical evaluation, we demonstrate that FLPADPM significantly outperforms existing state-of-the-art models in accurately predicting student dropout behavior. This superior performance is attributed to our model’s ability to effectively integrate and analyze personalized learning patterns within a federated learning framework. Our study underscores the importance of considering distinct learning patterns in dropout prediction and showcases the effectiveness of our novel model in capturing these patterns. By incorporating these nuanced insights into student behavior, FLPADPM enables the design of targeted interventions and support systems, ultimately aiming to bolster student engagement and improve retention rates in MOOC environments.

2. Related Work

2.1. Dropout Prediction Task

Several studies have been conducted to understand the factors influencing student dropout in Massive Open Online Courses (MOOCs). These studies have explored various aspects such as learning behaviors, course design, social interactions, and demographic factors. In this section, we review the relevant literature and discuss the findings of these studies.

One important factor that has been investigated is the relationship between learning behaviors and dropout rates. Wang et al. [20] analyzed MOOC data and found that users’ learning behaviors are relative for several consecutive days. They highlighted the need to consider factors that may affect dropout and incorporate them into dropout prediction models. Similarly, Wen et al. [21] studied the local correlation between learning behaviors and its impact on dropout prediction. They found that the time interval of the learning behavior has a significant influence on future user states.

Another aspect that has been explored is the influence of course design on dropout rates. Feng et al. [12] conducted a clustering analysis of users’ learning activity data and found that studying behavior can be grouped into distinct categories. They also observed a high correlation between dropout rates in similar courses and the strong influence of friends’ dropout behaviors on each other. In addition to learning behaviors and course design, demographic factors have also been studied. For example, Aldowah et al. [13] investigated the impact of academic skills, prior experience, and social support on dropout rates. They found that these factors play a significant role in predicting dropout behavior.

Furthermore, the use of machine learning algorithms [22] and deep learning models [23,24] has been explored for dropout prediction in MOOCs. Mrhar et al. [25] used a data mining method to analyze learners’ learning behavior and identify characteristics of learners who tend to drop out. They found that features such as number of threads viewed and number of forum posts can be used to predict learner attrition. Similarly, Liu et al. [26] proposed a learning behavior feature fused deep learning network model for dropout prediction, which considers temporal features among individual videos and incorporates general information such as student and course features. Other studies have focused on the analysis of clickstream data and the use of time series models for dropout prediction. Tang et al. [27] applied recurrent neural networks with long short-term memory cells to predict course dropout based on learners’ daily activities. They found that the time series model outperformed other contrast models in terms of prediction accuracy. Recently, advancements in deep learning for dropout prediction have been notable. Kumar et al. [28] introduced an Ensemble Deep Learning Network model that leverages ResNet-50 and Faster RCNN with an attention mechanism for improved dropout prediction accuracy. Additionally, Li et al. [29] explored the use of a deep learning model with an attention mechanism in a flipped classroom setting, aiming to enhance the prediction of students’ performance and the identification of critical learning stages.

While these studies have provided valuable insights into the predictors of dropout rates, there is a gap in the research when it comes to understanding the correlation between distinct learning patterns and dropout rates. This gap highlights the need for further investigation into the relationship between learning patterns and dropout rates in order to develop more accurate prediction models. In our study, we aim to address this gap by analyzing the impact of learning patterns on dropout rates in MOOCs.

2.2. Federated Learning

At present, federated learning is gaining increasing attention for its ability to train distributed models while preserving clients’ privacy. In the federated learning setting, multiple clients perform the machine learning tasks collaboratively under the coordination of the server without sharing their local data [30,31]. Many algorithms were proposed to learn a single global model which can perform well for all clients. For example, FedProx [32] utilized the regularization for global model to alleviate the distribution drift between local models and a global model; SCAFFOLD [33] aimed to correct local drift using control variates. To improve the global model learning with non-independent and identically distributed (non-IID) data, both Zhao et al. [34] and Yoon et al. [35] introduced a data sharing method and data augmentation technique among clients to mitigate statistical heterogeneity issues.

Differing from learning a single global model for all clients, personalized federated learning was proposed by designing a tailored model for each client under a traditional federated learning information communication setting. Per-FedAvg [36] and pFedMe [37] attempted to learn a shared meta-model for each client through a Model-Agnostic Meta-Learning Approach and Moreau Envelopes, separately. Moreover, Fedsteg [38] and Fedhealth [39] learn the shared knowledge among the related clients via transfer learning to improve the local model’s performance. Recently, various DNN-based personalized federated learning approaches were proposed for learning a common representation and personalized classifier by decoupling the neural network into the representation layers and decision layer [40,41]. FedPer [42] and FedRep [43] learn the entire neural network jointly over the local updating and only send the representation layers for aggregation; the difference between these two methods lies in the local updating. FedPer simultaneously updates both the representation layers and the decision layer, whereas FedRep alternately updates the two parts. FedPop [44] is a Bayesian federated learning framework which aims to learn a common deterministic representation layer and a personalized stochastic decision layer for capturing the uncertainty over the model parameters. Unfortunately, existing federated learning methods have not taken into consideration the application in dropout prediction.

3. Methodology

3.1. Problem Definition

The objective of this study is to predict student dropout rates in MOOCs within a federated learning framework, as shown in Figure 1. Formally, let C denote the set of courses, U the set of users, and I the set of institutions or MOOC platforms, each acting as a client in the federated learning system. The set of all enrollments is denoted as E, with each enrollment represented as a tuple $(u,c,i)$, indicating that user u has enrolled in course c offered by institution i. Our goal is to predict the dropout likelihood, denoted by a binary variable $y_{(u,c,i)}\in 0,1$, where 1 indicates dropout and 0 indicates continuation. The prediction is based on user activity data, represented by a matrix $X_{(u,c,i)}$ of dimensions $m\times n$, where m and n represent the number of days and types of learning activities, respectively. Each client (institution) trains a local model on its dataset, and these local models are aggregated to enhance the global prediction model.  

3.2. Student Activity Feature Extraction

Students often exhibit numerous behaviors within online education systems, where the temporal combination of different learning behaviors often constitutes their unique learning methods and styles. Therefore, to unearth these distinctive learning styles, it is essential to explore the complex coupling relationships between various learning behaviors, as well as the temporal dependencies between these behaviors, in order to discern each student’s unique learning pattern.

Inspired by existing models [45,46], we utilize a one-dimensional CNN to capture the relationships between different activity features and obtain the resulting feature matrix $S_{(u,c)}\in {\mathbb{R}}^{m\times N_c\times n}$, where m represents the number of historical active days, $N_c$ denotes the number of CNN channels, and n represents the number of student activity features.

The input to the CNN is the course identifier c and the activity information $X_{(u,c)}\in {\mathbb{R}}^{m\times n}$. The CNN is applied to learn the relationships between different activity features and generate the feature matrix $S_{(u,c)}$. The CNN operation is defined as follows:

$$S_{(u,c)}=\mathrm{CNN}\left(X_{(u,c)}\right),$$

(1)

where $\mathrm{CNN}(·)$ represents the one-dimensional Convolutional Neural Network.

Next, we utilize a Bi-LSTM layer to capture the temporal relationships between different days. The input to the Bi-LSTM layer is the feature matrix $S_{(u,c)}$, and the output is the hidden state matrix $O_{(u,c)}\in {\mathbb{R}}^{m\times 2N_h}$, where $N_h$ represents the number of hidden units in the Bi-LSTM layer. The Bi-LSTM capturing both forward and backward dependencies in student activity patterns. This dual-direction processing is critical for understanding the context of student interactions over time, providing a robust feature set for dropout prediction. The Bi-LSTM layer is defined as follows:

$$O_{(u,c)}=\text{Bi-LSTM}\left(S_{(u,c)}\right),$$

(2)

where $\text{Bi-LSTM}(·)$ represents the Bidirectional Long Short-Term Memory operation.

3.3. Modeling Distinct Learning Pattern

Due to factors such as class schedules and the nature of courses, different learning patterns often exist in different classes. Taking English and Mathematics classes as examples, varying class times may lead to different periodicities in learning English and Mathematics. Students often preview the material before class and review and complete assignments after class. Additionally, due to the distinct nature of English and Mathematics classes—the former being memory-oriented and the latter focusing on logical reasoning—the times of day students allocate for learning English and Mathematics tend to differ. Existing research indicates that memory tasks are more efficiently executed at night [47,48], while reasoning tasks are more effective during the day [49].

We aim to capture different learning patterns in different courses by generating two learnable matrices, ${\mathbf{K}}_c$ and ${\mathbf{V}}_c$, for each course. These matrices represent the keys and values of different learning patterns at different courses.

Firstly, we define the following variables: $\mathbf{K}\in {\mathbb{R}}^{\left|C\right|\times N_l\times 2m{N}_d}$, the learnable matrices representing the keys of different learning patterns for different courses. Here, $\left|C\right|$ denotes the number of courses, $N_l$ denotes the number of learning patterns, m denotes the number of historical days, and $N_d$ denotes the dimension of the output of Bi-LSTM. $V\in {\mathbb{R}}^{\left|C\right|\times N_l\times N_d}$ are the learnable matrices representing the values of different learning patterns at different courses.

Given the course enrollment c of user u, we can index the corresponding keys and values to obtain ${\mathbf{K}}_c$ and ${\mathbf{V}}_c$ for that course. Next, we calculate the attention weights between the student’s activity embedding $O_{(u,c)}$ and the keys ${\mathbf{K}}_c$ of different learning patterns using an attention mechanism. This allows us to measure the similarity between the student’s activities and the different learning patterns associated with each course. The attention weights are calculated as follows:

$$W_{(u,c)}^i=\frac{exp(K_c^i·O_{(u,c)})}{{\sum }_{k=1}^{N_l}exp(K_c^k·O_{(u,c)})},$$

(3)

where $i\in [1,2,\dots ,N_l]$ represents the index of the learning pattern and $N_l$ is the number of learning patterns. After obtaining the attention weights, we multiply them with the values $V_c$, then combine it with course embedding $e_c$ to obtain the student’s learning pattern $P_{(u,c)}$:

$$\begin{array}{cc}\hfill V_{(u,c)}^{sum}& =\sum _{i=1}^{N_l}{W}_{(u,c)}^i{V}_c^i,\hfill \\ \hfill P_{(u,c)}& =MLP(V_{(u,c)}^{sum}\oplus e_c),\hfill \end{array}$$

(4)

where ⊕ represents the concatenation operation.

3.4. Dropout Prediction

We concatenate the student’s activity, embedding $O_{(u,c)}$ and the personal learning pattern $P_{(u,c)}$, and pass them through a multi-layer perceptron (MLP) to obtain the predicted dropout probability ${\widehat{y}}_{(u,c)}$:

$$\begin{array}{cc}\hfill {\mathbf{e}}_{(u,c)}& =ReLu({\mathbf{W}}_e(P_{(u,c)}\oplus O_{(u,c)})+{\mathbf{b}}_e)\hfill \\ \hfill {\widehat{y}}_{(u,c)}& =\sigma \left(MLP(P_{(u,c)}\oplus O_{(u,c)})\right).\hfill \end{array}$$

(5)

Here, $\sigma$ denotes the sigmoid function and $ReLu$ is the rectified linear activation function.

3.5. Model Learning

In our learning setup, each participating institution or MOOC platform, denoted as a client, independently trains a local instance of the Learning Pattern Aware Dropout Prediction Model (LPADPM) on its dataset. The dataset for each client consists of user activity data and corresponding dropout indicators specific to the courses offered by that institution. We denote the model parameters from Student Activity Feature Extraction, Modeling Student Learning Pattern, and Dropout Prediction Module as ${\mathit{\theta }}_S$, ${\mathit{\theta }}_M$ and ${\mathit{\theta }}_D$; C represents the participating clients in each communication round.

The local training involves feeding the user activity data into a one-dimensional convolutional neural network (CNN) and a bidirectional Bi-LSTM layer to capture the relationships and temporal patterns in student activities. It should be emphasized that the CNN and Bi-STM layer operates locally on each client’s dataset. Training occurs in parallel across institutions, with each client contributing to the construction of a shared global model without direct data exchange, thus preserving data sovereignty. Furthermore, each client employs learnable matrices ${\mathbf{K}}_c$ and ${\mathbf{V}}_c$ to model the distinct learning patterns of courses offered by them. An attention mechanism is used to determine the relevance of these patterns to individual students’ activities. The loss function in local training at i-th client (institution) is shown as follows:

$$l_l=-\sum _{(u,c)\in {\left|X\right|}_i}{y}_{(u,c,i)}log\left({\widehat{y}}_{(u,c,i)}\right)$$

(6)

Our federated learning framework operates by distributing the model training process across multiple clients. Each client, representing a participating institution or MOOC platform, independently trains a local instance of the Bi-LSTM model on its own data. Once the local models complete their training, client i only send the ${\mathit{\theta }}_{i,D}$ to the server for model averaging while keeping the ${\mathit{\theta }}_{i,S}$ and ${\mathit{\theta }}_{i,M}$ locally. This process involves aggregating the weights and biases of the local models from participating clients to update the global model. The updated global model, which now reflects a broader understanding of student learning patterns across various institutions, is then distributed back to the clients. Algorithm 1 shows the entire federated training process.

Algorithm 1 FLPADPM: Federated Learning Pattern Aware Dropout Prediction Model

Input: T communication rounds, R local updating rounds, C random subset of all clients,  and initialization: ${\mathit{\theta }}_{i,S,0}^t$, ${\mathit{\theta }}_{i,M,0}^t$, ${\mathit{\theta }}_{i,D,0}^t$ Server executes:  for  $t=0,1,\cdots ,T-1$  do   Sample ${\mathcal{C}}^t$ clients with size C uniformly at random   for each client $i\in {\mathcal{C}}^t$ in parallel do    ${\mathit{\theta }}_{i,S}^{t+1},{\mathit{\theta }}_{i,M}^{t+1},{\mathit{\theta }}_{i,D}^{t+1}\leftarrow$ Client Update($i,{\mathit{\theta }}_{i,S}^t,{\mathit{\theta }}_{i,M}^t,{\mathit{\theta }}_D^t$)   end for   ${\mathit{\theta }}_D^{t+1}=\frac{1}{C}{\sum }_{i\in {\mathcal{C}}^t}{\mathit{\theta }}_{i,D}^{t+1}$  end for Client Update($i,{\mathit{\theta }}_{i,S}^t,{\mathit{\theta }}_{i,M}^t,{\mathit{\theta }}_D^t$):  ${\mathit{\theta }}_{i,S,0}^t\leftarrow {\mathit{\theta }}_{i,S}^t$, ${\mathit{\theta }}_{i,M,0}^t\leftarrow {\mathit{\theta }}_{i,M}^t$, ${\mathit{\theta }}_{i,D,0}^t\leftarrow {\mathit{\theta }}_D^t$  for  $r=0,1,\cdots ,R-1$do   Update ${\theta }_{i,S}^t$, ${\theta }_{i,M}^t$, ${\theta }_{i,D}^t$ with Stochastic Gradient Descent using Equation (6)  end for  Return  ${\mathit{\theta }}_{i,S}^{t+1},{\mathit{\theta }}_{i,M}^{t+1},{\mathit{\theta }}_{i,D}^{t+1}$

4. Experiment

4.1. Datasets Selection

We selected two widely used datasets, namely the KDD Cup 2015 dataset and the XuetangX dataset http://moocdata.cn/data/user-activity (accessed on 15 December 2023), to study the factors influencing student dropout in MOOCs.The combination of these datasets allows us to analyze dropout patterns from different angles and validate the generalizability of our findings.

The KDD Cup 2015 dataset is a comprehensive dataset in the field of educational data mining. It was released as part of the KDD Cup 2015 competition, which aimed to predict student dropout in MOOCs. The dataset contains a large amount of information about student activities, including log data, forum posts, and course enrollment data. It covers a diverse range of courses from different domains and includes data from multiple MOOC platforms.

The XuetangX dataset is another valuable resource for studying student dropout rate in MOOCs. XuetangX is one of the largest MOOC platforms in China, hosting a wide variety of courses. The dataset provides detailed information about student interactions, including video watching, forum participation, and assignment submissions. It also includes demographic information and course-related features.

The experimental setting follows the KDD Cup 2015 and XueTangX with only 30 days of historical data. Objectively, there are trend changes in the dataset, but due to the short time limit of the existing dataset and the small time change, it is difficult to analyze the change, so the experimental results will not be affected by the peak or change of the data value trend.

4.2. Baseline Models

We analysis several baseline models that have been widely used in dropout rate prediction, which are as following:

• CAL [50] is a fusion deep dropout prediction model that utilizes a CNN to obtain local feature representations and a bidirectional LSTM network to obtain a time-series feature vector representation.
• CLSA [51] uses a CNN to extract local features and a LSTM to obtain a time-series incorporated vector representation. The model incorporates a static attention mechanism to obtain an attention weight on each dimension.
• CNN [21] considers the local correlation of learning behaviors and proposes a new feature matrix and a CNN model for dropout prediction.
• DNN [12] is a three-layer deep neural network.
• CNNLS [52] combines CNN and LSTM to automatically extract features from MOOC raw data.
• CLMS [53] is a deep neural network that combines CNN, LSTM, and SVM. It incorporates an effective feature extraction strategy and considers the impact of sequential relationships and class imbalance on dropout prediction.
• CRM [54] automatically extracts features from raw MOOC data using a combination of CNN and RNN.
• CSEG [55] is a hybrid deep neural network, which utilize a combination of a CNN, Squeeze-and-Excitation Networks (SE-Net), and GRU to predict the probabilistic of student dropout.
• EDLN [28] uses ResNet-50 to extract local features, uses Faster RCNN to obtain vector representation, and then applies a static attention mechanism to the vector to obtain the attention weight of each dimension.

These baseline models have been widely used in the field of dropout rate prediction in MOOCs and provide a benchmark for evaluating the performance and representation of the proposed model in this paper.

4.3. Experiment Setting

We implement local model data by adopting a specific federated learning algorithm, FedAvg. Each client periodically performs stochastic gradient descent on their local data batch and computes the model updates. These updates are then securely transmitted to the central server where they are aggregated to refine the global model parameters. For training our models within the federated framework, we adopted a dynamic learning rate approach. Initially set based on empirical studies, the learning rate is adjusted using a decay function to balance convergence speed and model stability across federated rounds. Additionally, regarding hyperparameter optimization, including learning rate selection, was conducted a grid search strategy. We evaluated various learning rates for their impact on convergence and predictive performance, selecting the rate that minimized cross-validation loss while ensuring consistent learning across clients.

We evaluated the performance of our proposed Federated Learning Pattern Aware Dropout Prediction Model (FLPADPM) compared to several baseline models with a FedAvg framework [30] on the KDD Cup 2015 and XuetangX datasets. FedAvg is a popular federated learning framework. We used three evaluation metrics, namely area under the curve (AUC), root mean square error (RMSE), and mean absolute error (MAE), to assess the performance of the models. Each experiment is conducted with five random seeds, and the results are averaged for analysis. The experimental results presented in

Table 1. The results of the KDDCUP15 dataset.

MODEL	AUC	RMSE	MAE
CAL + FedAvg	$0.8656\pm 0.0412$	$0.4440\pm 0.0049$	$0.3800\pm 0.0065$
CLSA + FedAvg	$0.8551\pm 0.0204$	$0.4233\pm 0.0010$	$0.3595\pm 0.0016$
LSTM + FedAvg	$0.8621\pm 0.0013$	$0.4133\pm 0.0002$	$0.3463\pm 0.0021$
CLMS + FedAvg	$0.8899\pm 0.0005$	$0.4303\pm 0.0026$	$0.4022\pm 0.0055$
1D-CNN + FedAvg	$0.8921\pm 0.0014$	$0.4033\pm 0.0010$	$0.3321\pm 0.0023$
CNN + FedAvg	$0.8842\pm 0.0438$	$0.4046\pm 0.0011$	$0.3449\pm 0.0021$
DNN + FedAvg	$0.8866\pm 0.0158$	$0.4163\pm 0.0011$	$0.3667\pm 0.0036$
CRM + FedAvg	$0.8794\pm 0.0018$	$0.4285\pm 0.0022$	$0.3674\pm 0.0042$
CNNLS + FedAvg	$0.8364\pm 0.0933$	$0.4915\pm 0.0052$	$0.4473\pm 0.0060$
CSEG + FedAvg	$0.9004\pm 0.0001$	$0.3856\pm 0.0001$	$0.3053\pm 0.0011$
EDLN + FedAvg	$0.9012\pm 0.0018$	$0.3743\pm 0.00023$	$0.2927\pm 0.0012$
FLPADPM	0.9101 ± 0.0001	0.3658 ± 0.0001	0.2784 ± 0.0029

and

Table 2. The results of the XueTangX dataset.

MODEL	AUC	RMSE	MAE
CAL + FedAvg	$0.9589\pm 0.0065$	$0.3359\pm 0.0808$	$0.2742\pm 0.0972$
CLSA + FedAvg	$0.9474\pm 0.0117$	$0.3554\pm 0.0358$	$0.3018\pm 0.0579$
LSTM + FedAvg	$0.9511\pm 0.0015$	$0.3487\pm 0.00421$	$0.3007\pm 0.0213$
CLMS + FedAvg	$0.9679\pm 0.0004$	$0.3160\pm 0.0043$	$0.2351\pm 0.0047$
1D-CNN + FedAvg	$0.9773\pm 0.0143$	$0.3030\pm 0.0203$	$0.2386\pm 0.0321$
CNN + FedAvg	$0.9718\pm 0.0155$	$0.3031\pm 0.0120$	$0.2394\pm 0.0331$
DNN + FedAvg	$0.9666\pm 0.0188$	$0.4393\pm 0.0152$	$0.4356\pm 0.0170$
CRM + FedAvg	$0.9525\pm 0.0002$	$0.3054\pm 0.0009$	$0.2319\pm 0.0021$
CNNLS + FedAvg	$0.9577\pm 0.0291$	$0.3018\pm 0.0484$	$0.2409\pm 0.0444$
CSEG + FedAvg	$0.9718\pm 0.0001$	$0.3186\pm 0.0046$	$0.2515\pm 0.0082$
EDLN + FedAvg	$0.9698\pm 0.0016$	$0.3198\pm 0.0043$	$0.2517\pm 0.0063$
FLPADPM	0.9770 ± 0.0051	0.2713 ± 0.0253	0.2084 ± 0.0427

demonstrate the superior performance of FLPADPM compared to existing prediction methods on both the KDD Cup 2015 and XuetangX datasets.

On the KDD Cup 2015 dataset, the FLPADPM achieves an impressive AUC of 0.9101 ± 0.0001, indicating its ability to accurately distinguish between dropout and non-dropout instances. The FLPADPM also achieves a low RMSE of $0.3658\pm 0.0001$ and a low MAE of $0.2784\pm 0.0029$, indicating its high precision in predicting dropout rates. Similarly, on the XuetangX dataset, the FLPADPM achieves an outstanding AUC of $0.9770\pm 0.0051$, indicating its strong discriminatory power. The FLPADPM also achieves a low RMSE of $0.2713\pm 0.0253$ and a low MAE of $0.2084\pm 0.0427$, further confirming its accuracy in predicting dropouts.

The superior performance of the FLPADPM can be attributed to its incorporation of learning patterns into the dropout prediction model. By modeling and extracting distinct learning patterns across students and different courses, the FLPADPM gains a comprehensive understanding of students’ learning behaviors and patterns. This enables the model to make more accurate predictions.

For each dataset, the conversations are divided into three sets: the training set, the test set, and the validation set. The training set consists of the first 60% of the conversations, the test set consists of the last 20%, and the remaining 20% is allocated to the validation set. This division ensures that the model is trained on a sufficient amount of data while also having separate sets for testing and validation.

To optimize the hyperparameters of the baseline models, a search is conducted for weight decay and learning rate. The weight decay is searched within the range of $1\times {10}^{-5}$ to $1\times {10}^{-1}$, and the learning rate is searched within the range of 0.0001 to 0.01. Additionally, the optimal batch size is searched within the range of 16 to 64, while keeping other hyperparameters unchanged. This hyperparameter search helps to find the best combination of parameters for each model.

4.4. The Representation Analysis of Mainstream Dropout Prediction Model

In this section, we perform a visualization of the distributed representations of the baseline models using the training dataset after hyperparameter optimization. These distributed representations serve as inputs to the classifiers. The representations are grouped based on the course ID or category ID of the courses. We employ the t-SNE dimensionality reduction technique to plot the distributed representations of all users for each course or category. The t-SNE function is used with the following hyperparameter settings: perplexity is set to 60, the number of iterations is set to 5000, the learning rate is set to 100, and the random state is set to 42. By reducing the distributed representations to a two-dimensional plane, we can observe the grouping of representations based on the course ID or category ID. Different colors are used to represent the distributed representations belonging to different courses.

As shown in Figure 2 and Figure 3, we present the analysis results of the mainstream dropout prediction models in terms of their ability to capture distinct learning patterns and variations across different courses. The analysis is based on the representation generated by these models on the KDD Cup 2015 and XuetangX datasets.

Firstly, we observe that all models exhibit a distinct cluster at the edge of the figure in both datasets. This suggests that the models are able to identify certain extreme cases of dropout, where students exhibit clear signs of disengagement from the learning process. However, at the center of the figure, all models show a large cluster, indicating that they struggle to differentiate between different learning patterns within this region. Interestingly, the size of the center cluster in the XueTangX dataset is larger than that in the KDD Cup 2015 dataset. This implies that the mainstream dropout prediction models perform even worse in capturing distinct learning patterns in the XueTangX dataset compared to the KDD Cup 2015 dataset. To further analyze the performance of the models, we examine the size of the center cluster in the KDD Cup 2015 dataset. We find that the models can be ranked in terms of the size of their center cluster from largest to smallest as follows: CAL ≥ CLSA > DNN > CSEG > CLMS ≥ CNN ≥ CNNLS ≥ CRM. This indicates that the CAL model has the largest center cluster, while the CNNLS model has the smallest. These findings suggest that the mainstream dropout prediction models fail to effectively mine the distinct learning patterns at the center of the figure, resulting in the mixing of different learning patterns within this region. This limitation hinders the models’ ability to accurately predict dropout for students who exhibit more subtle signs of disengagement.

Furthermore, we discover that the representations of all courses are mixed together by all models in both the KDD Cup 2015 and XuetangX datasets. This finding contradicts our prior knowledge that learning patterns can vary across different courses [15,17,18]. The inability of the mainstream dropout prediction models to capture the variation of learning patterns across different courses suggests a significant limitation in their ability to provide accurate predictions for students in diverse educational contexts.

The analysis results demonstrate that the mainstream dropout prediction models are unable to effectively capture distinct learning patterns and variations across different courses. This limitation compromises their performance in accurately predicting dropout for students who exhibit subtle signs of disengagement.

4.5. Visualization of FLPADPM’s Representation

We analyze and compare the representation of students’ activity in the proposed model with the baseline model. This analysis helps to evaluate the effectiveness of the proposed model in capturing distinct learning patterns across students and different courses. We can combine the information from Figure 4 and Figure 5 to draw several conclusions.

Firstly, when comparing the representation of students’ activity between the proposed model and the baseline model, it is evident that our model’s representation of students’ activity is superior. This can be observed in Figure 4a,b, where distinct clusters can be identified in the center of the figure for both the KDDCup and XueTangX datasets. This indicates that the activity feature extraction module in our model has a strong ability to mine and capture distinct learning patterns across students.

However, it is important to note that the embedding of students’ activity at different courses still tends to mix together. This suggests that solely relying on the activity feature extraction module is not sufficient. In order to effectively model the variation of students’ learning patterns across different courses, additional modules or mechanisms need to be incorporated.

As shown in Figure 5a,b, the representation of our model successfully distinguishes learning patterns in different courses. Additionally, it is able to identify different learning patterns within a single course. This demonstrates the effectiveness of the learning pattern modeling module in our model, as it enables the model to capture and represent the variations in students’ learning patterns across different courses.

The visualization of the representation in our experiment provides strong evidence to support the claim that our proposed model outperforms the baseline model in capturing and representing distinct learning patterns across students and different courses. The combination of the activity feature extraction module and the learning pattern modeling module in our model enables a more comprehensive and accurate understanding of students’ learning patterns, leading to improved performance in dropout rate prediction.

5. Conclusions

In this study, we addressed the critical limitations inherent in existing mainstream dropout prediction models regarding their effectiveness in mining student learning patterns. Through a thorough analysis of the models’ representations, we identified two major shortcomings: their failure to capture distinct learning patterns and their lack of consideration for personalized variations across different courses. To address these gaps, we proposed the Learning Pattern Aware Dropout Prediction Model (LPADPM), which innovatively incorporates a one-dimensional convolutional neural network (CNN) and a bidirectional long short-term memory (LSTM) layer. This combination allows for an in-depth capture of relationships, patterns, and the temporal dynamics inherent in student data.

Furthermore, we introduced course-specific learnable matrices in LPADPM, enabling the model to adapt to variations in learning patterns across various courses. This addition significantly enhanced the model’s ability to tailor its predictions to the unique educational contexts of each course.

A key innovation in our approach was the integration of federated learning into the LPADPM framework. By adopting this decentralized learning structure, we enabled the model to train across multiple datasets from different institutions while preserving data privacy and autonomy. This federated learning approach not only improved the model’s predictive accuracy by leveraging diverse and comprehensive data but also maintained the confidentiality and security of individual datasets. At the same time, the application of federated learning in our study is critical for understanding the potential and challenges of privacy-preserving predictive analytics in educational settings. By analyzing the aggregation of local model updates, we can draw conclusions about the learning patterns without compromising student privacy or data security.

Our empirical evaluations using the KDD Cup 2015 and XuetangX datasets demonstrated that the FLPADPM outperforms state-of-the-art (SOTA) models with traditional federated learning framework in accurately predicting student dropout behavior. The visualizations of the representations generated by our model provided further evidence of its superior capability in effectively mining and interpreting learning patterns in various courses.

The successful implementation of LPADPM, enhanced with federated learning, marks a significant advancement in dropout prediction methodologies. It paves the way for more personalized, privacy-aware, and efficient predictive models in the realm of online education, particularly in MOOCs. This study not only contributes to the academic field of educational data mining but also offers practical insights for educators and platform administrators aiming to improve student retention and success in MOOCs.

6. Further Discussion

In this study, our deployment of the Federated Learning Pattern Aware Dropout Prediction Model (FLPADPM) has demonstrated notable advancements in predictive accuracy for student dropout rates within MOOCs. Moreover, the introduction of course-specific learnable matrices has innovatively enabled the customization of the model to the varied pedagogical methodologies across institutions. This bespoke approach sets a precedent for adaptive learning analytics, providing a pathway for educators to develop targeted interventions based on predictive insights.

However, the research encompasses certain limitations that must be acknowledged. The heterogeneity of data across different MOOC platforms presents a challenge to the federated learning approach, potentially impacting the model’s ability to generalize across diverse educational contexts. Additionally, the variability in student engagement within the datasets may introduce biases that could skew the predictive outcomes. These limitations highlight the need for extensive validation of the FLPADPM across a broader spectrum of educational settings and a more diverse student demographic to confirm its efficacy and scalability. It is also imperative to consider the computational overheads introduced by the federated learning process and the possible need for more efficient communication protocols to handle the increased data exchange.

Future research should focus on enhancing the FLPADPM by exploring alternative federated learning algorithms that can further optimize privacy and computational efficiency. Additionally, extending the model’s application to encompass a wider range of predictive tasks, such as course success and student engagement levels, could provide a more holistic tool for educational administrators. The potential for FLPADPM to inform real-time interventions and support personalized learning trajectories offers a promising horizon for the realm of online education, marking a step towards more responsive and student-centered learning environments.