Data Mining of Students’ Consumption Behaviour Pattern Based on Self-Attention Graph Neural Network

Abstract

Performance prediction is of significant importance. Previous mining of behaviour data was limited to machine learning models. Corresponding research has not made good use of the information of spatial location changes over time, in addition to discriminative students’ behavioural patterns and tendentious behaviour. Thus, we establish students’ behaviour networks, combine temporal and spatial information to mine behavioural patterns of academic performance discrimination, and predict student’s performance. Firstly, we put forward some principles to build graphs with a topological structure based on consumption data; secondly, we propose an improved self-attention mechanism model; thirdly, we perform classification tasks related to academic performance, and determine discriminative learning and life behaviour sequence patterns. Results showed that the accuracy of the two-category classification reached 84.86% and that of the three-category classification reached 79.43%. In addition, students with good academic performance were observed to study in the classroom or library after dinner and lunch. Apart from returning to the dormitory in the evening, they tended to stay focused in the library and other learning venues during the day. Lastly, different nodes have different contributions to the prediction, thereby providing an approach for feature selection. Our research findings provide a method to grasp students’ campus traces.

1. Introduction

Methods to improve education quality by mining off-line education and on-line learning platform data [1] has led to the development of educational data mining (EDM) [2]. Among the several problems in the field of EDM, predicting students’ scholastic performance is a key issue [3,4], and various statistical methods [5,6] and tools [7] have been developed to perform this task. However, these methods could not reflect the learning conditions of students over a specific period, and do not facilitate the discovery of new knowledge patterns from the data set for the development of new and accurate models. Advancements in machine learning has led to the emergence of powerful data visualization methods and a variety of models, such as clustering, classification, and prediction, including algorithms that can dynamically process data streams [8], and other algorithms such as the support vector machine have been used to detect students who may fail in courses as an early warning [9]. Research on machine learning using behavioural data for performance prediction has attracted a significant amount of attention, such as grade prediction using online behaviour [10,11,12], gaming behaviour [13], consumption behaviour [14] and travel behaviour [15]. To conduct on-line behaviour mining more deeply, artificial neural networks have been used for log data mining and desirable results have been achieved [16]. However, with regard to knowledge tracking, such as the prediction of student’s test questions, artificial neural networks have not yielded satisfactory result, and recurrent neural networks (RNNs) have had to be introduced [17]. Further, when dealing with long sequences, RNNs found that gradient explosion and gradient disappearance were prone to happen [18]. Therefore, to overcome such shortcomings, long short term memory networks (LSTM) were introduced [19]. LSTM can well solve the prediction problem with temporal data, while it has defects in processing spatial data. Further, the same behaviour will lead to ambiguity in the behavioural purpose if the difference in spatial location is ignored. For example, the behavioural meanings of fetching boiled water in the teaching building, in the dormitory, and in the bathroom are obviously different. We can, respectively, speculate that the purposes of corresponding behaviours are to study, to play games in the dormitory, and to take a bath. The lack of spatial location information will affect the prediction accuracy of models and the analysis of discriminative behavioural patterns. If the information in the spatial position was considered to mine behavioural characteristics contained in consumption data, the graph topological structure formed by behaviour cannot be well mined, and the above methods lost the ability to have good performance for such data with certain special structures. Naturally mining the data with graph structure for performance prediction requires the introduction of new tools. At present, graph neural networks (GNNs) have undergone rapid developments. Further, such networks have been extensively used [20,21,22] and are suitable for dealing with graph structure data. We observed that the behaviour of students can be demonstrated in the form of a graph and particular behavioural pattern and tendency can be observed, which inspired us to use graph-related tools to mine hidden information in behavioural patterns using students’ spatial location changes over time.

In this study, we aimed to utilize the information regarding spatial position changes over time reflected by consumption data collected from a school to extract behavioural pattern features and construct graph structures for mining discriminative behavioural characteristics and behavioural trends related to academic performance, as well as for performance prediction. Firstly, we proposed two guidelines for constructing the graph structures by extracting features from the consumption behaviour data. Secondly, we proposed an improved self-attention mechanism model based on previous graph self-attention mechanism; thirdly, we made use of graphs composed of behaviour characteristics to perform classification and determined discriminative learning and life behaviour sequence patterns.

The remainder of this article is as follows: In Section 2, we present some additional current research on consumption behaviour and discuss the possibility of using a GNN for graph mining. In Section 3, we explain the data we used in our study as well as the processing methods; we also describe the method of graph construction and the improved model. In Section 4, we report the results of the experiment and analyse the results obtained. In Section 5, we propose some open issues, summarize the paper and also present some shortcomings of our research.

2. Related Work

Continuously adapting to rapid development and educational innovation is highly crucial and has led to the emergence and application of EDM in various fields. Scholars investigated the three aspects of student performance, teaching equality, and policy making, and found student performance had the greatest significance [23,24]. Related research had also focused on performance prediction and the discussion of methods [25] and models [26], including data [27], and  models and methods were considerably improved for different purposes.

From the perspective of students, performance prediction [28,29], early warning of failure in subjects [30], sentiment analysis [31] and course recommendation [32,33] are key issues; besides, the trajectory of school behaviour is rich in information of learning habits of students to mine, and analyses of behavioural patterns based on spatial location change have been widely carried out for performance prediction. The trajectory of students’ behaviour at school helps us understand the various characteristics of learning status and lifestyle [34]. Dalvi [35] focused on students’ green and low-carbon behaviour, and Islam [36] studied electronic product consumption; in both these studies, only the lifestyles of students were investigated, and  the impact of such consumption bahaviour on academic performance was not studied. Mei [37] used campus behaviour data to predict academic performance. However, discriminative learning and life patterns that can help distinguish students with different learning levels was unclear. Further, time and location information has not yet been considered in most studies. Li [14] focused on behaviours that could reflect the regularity of students’ lives; however, behavioural patterns are not discriminative enough by nature. In study [38], based on the behaviour records of undergraduates’ smart cards, the authors studied the impact of students’ diligence and the regularity of their daily life on grades. Due to a lack of spatial information, the authors had to regard different behaviours as the same behaviour and the prediction results were affected. The main reason for this is the lack of consideration about spatial information. Thus, consideration of behavioural patterns with spatial location information is necessary.

Advanced statistical methods were extensively applied in EDM during its early stage, such as the t-test [39], for the prediction of academic performance [40,41]. Statistical methods were suitable for small sample data; otherwise, it was necessary to put forward intelligent algorithms. Machine learning algorithms were utilized for predicting academic performance [42] to warn students who might fail in certain courses [43] and predict the graduation rates [44]. Regarding machine learning algorithms that did not perform well on time series data, the improved recurrent neural network (RNN) based on artificial neural networks showed a good mining effect [45], and appeared to be ineffective for dealing with non-European data, such as those with a graph structure. There was also a lack of in-depth mining on student behaviour tendency reflected by the spatial location information. Due to the particularity of graph structure, it was also difficult to use general models to process and analyse. We therefore studied behavioural characteristics from the perspectives of spatial topological structure and time dimension, and we introduced a new and powerful tool for dealing with non-European structure data.

As a powerful tool for processing non-European structure data, graph neural networks have undergone rapid development and been wide applied, such as the knowledge graph [46], natural language processing [47], graph-based text representation [48] and graph embedding techniques [49]. In particular, some scholars have proposed the graph attention mechanism to improve the performance of node classification [50]. Some scholars have recommended graphs for recommendation systems [51], such as the music recommendation system in mobile networks [52,53] because of graphs’ powerful information representation abilities and wide applications. Specifically, Zhang [54] used bipartite graph to perform context-sensitive web service discovery. Notably, community detection is also a key task [55,56]. Consumption behaviour at school has structure and characteristics similar to those of social networks and graph. A topology structure must therefore be introduced to distinguish behavioural patterns and find students’ behaviour tendency. Inspired by the node classification method, we improved the present self-attention GNN to mine consumption behaviour data from both time and spatial aspects.

3. Methods

3.1. Data Description

As the behaviour of students is often the same regardless of the semester, we only collected behaviour data over a single month. The activities of first-year students are relatively messy due to their curiosity and are difficult to analyse, while the third-year and fourth-year students need to engage in some social work outside the school, which leads to very short time at school and consumption behaviours are lacking and inconvenient to analyse. Therefore, we considered the consumption data of all second-year students at Beijing Institute of Technology, which is characterized by science and engineering majors and has a male to female ratio of 2.2 to 1. Students use campus cards for on-campus consumption, and the corresponding data are transmitted and stored in the school’s campus card consumption system later, which is relatively convenient to access. Thus, from December 2020 to January 2021, we collected 752,725 pieces of consumption data generated during May 2020 and 3640 pieces of final exam scores of the course of data structure from two data systems, including campus card consumption system and educational administration system. The collected consumption data are detailed in

Table 1. Collected raw data.

ID Number	Consumption Money (RMB Cent)	Time	Action
36984	200	2018/5/2 9:18	Breakfast
36984	400	2018/5/2 11:47	Lunch
36984	1500	2018/5/2 16:52	Dinner
17347	300	2018/5/31 11:24	Lunch
17347	250	2018/5/31 11:25	Lunch
17347	38	2018/5/31 11:26	Lunch
17347	150	2018/5/31 11:27	Lunch
10075	180	2018/5/20 11:16	Lunch
10075	200	2018/5/20 11:39	Lunch
10075	1500	2018/5/20 17:50	Dinner
10075	300	2018/5/21 07:56	Breakfast
10075	50	2018/5/21 07:56	Breakfast

, and a more detailed explanation regarding the “action” is presented in

Table 2. Specific explanation of different consumption behaviours.

Behaviour	Action Explanation
Dinner	Consumption after 4:00 p.m. in the cafeteria.
Lunch	Consumption between 10:00 a.m. and 4:00 p.m. in the cafeteria
Breakfast	Consumption before 10:00 a.m. in the cafeteria.
Supermarket	Consumption in the supermarket
Library	Consumption in the library
Dormitory bathroom	Consumption in the dormitory bathroom
Dormitory boiled water	Consumption on the dormitory water
Gym	Consumption in the school gym
School bus	Consumption caused by taking the school bus between campuses
Management office	Consumption in the school management office
Classroom boiled water	Consumption of water available in the classroom

.

3.2. Data Preprocessing

Based on the above data, we performed certain preprocessing measurements, using the following pseudo-code of Algorithm 1.

Algorithm 1 Data pre-processing

Input: Raw data file Output: Processed data files for different students 1: Group data by $ID$ 2: while Student $ID$ equals some $ID$ do 3:   if Two adjacent rows of data are exactly the same then 4:    Delete a row 5:   end if 6:   if The consumption behaviour is in the gym, management office or school bus then 7:    Delete the row 8:   end if 9:   if Two adjacent behaviours are the same then 10:    if Time difference is less than 5 fimnutes then 11:     Delete the second row 12:    end if 13:   end if 14: end while 15: if More than 15 consumption data for some student then 16:   Generate a new data file for the student 17: else 18:   Do not consider the student’s behaviour data 19: end if

In collected data, “management office” refers to students’ short-term campus card recharging behaviour; “school bus” refers to the behaviour of commuting between campuses, and the uncertainty of destination campus makes it impossible to analyse behavioural purpose; as for “gym”, the overall time and purpose of staying in the gym based on this record cannot be inferred. Hence, we deleted related behaviours in the preprocessing algorithm. After the data preprocessing, we got the data of 3616 students and next considered extracting features from the following three aspects:

• Indicators of regularity
• Amount of consumption
• Behaviour sequence pattern

After obtaining the features, we performed the chi-square test, f-test and other feature selection methods, and finally determined thirty relevant features related to the grades.

Table 3. Extracted features based on students’ data and reality.

Type	Feature Name	Feature Explanation	Intermediate	Node ID
Indicators of regularity	Study-actual	Actual number of visits to library or classroom	No	0
	Paid-days	Number of days with consumption records	No	2
	Getting-up-num	Number of wake-ups	No	3
Amount of consumption	total-meals	Total number of meals	Yes	26
	Total-supermarket-num	Total number of times to go shopping	Yes	7
	Total-month-money	Total monthly cost	Yes	8
Behaviour sequence pattern	Dinner-dormitory-num	Number of times in the dormitory after dinner	No	13
	Lunch-study-num	Number of times to study after lunch	No	15
	Dinner-study-num	Number of times to study after dinner	No	16
	Dinner-study-dorm-bathroom-num	Number of times to study after dinner and go back to the dorm	No	21

presents some of the information related to the features that were extracted.

We noted that the features related to consumption money were intermediate, which meant that excessive consumption and low consumption were both abnormal phenomena. Thus, this type of feature was transformed to the maximum feature using Formula (1):

$$x_i=1-\frac{\left|x_i-x_{\mathrm{best}}\right|}{max\left\{\left|x_i-x_{\mathrm{best}}\right|\right\}}$$

(1)

The same method was applied to the other similar intermediate indicators, as presented in Table 3.

3.3. Graph Construction

After the extraction of the behavioural sequence features, the data lost the natural graph structure. As such, construction of suitable graphs from these features became a challenge. To reflect the continuity and trend of students’ behaviours, we constructed graphs based on the characteristics of students’ behaviours at school and daily experience and regarded the features as nodes.

3.3.1. p-Clique

For a graph (V, E), V refers to a set of elements called vertices and E is a multiset of unordered pairs (u, v) whose elements are called edges. Two vertices are said to be adjacent if there exists an edge between them. A clique in a graph refers to a set of pairwise adjacent vertices, and if the number of vertices involved is p, it is called a p-clique. The behavioural patterns of students in their daily school life are often consistent. While some patterns may not be related to one another obviously, the extracted behaviour sequence features will be interrelated. This interrelationship between the features is an important indicator that can be used to distinguish between students with different learning levels. Based on this, we arranged part of the extracted behaviour sequence features into a p-clique in the constructed graph.

3.3.2. Other Criterion

Two nodes are said to be connected and interrelated if there exists an edge between them. Hence, for two nodes to have a relationship, we considered that the following two criteria must be fulfilled:

• Necessary connection: This condition means that the two nodes are interrelated, for example, the edge between the total-month-money and total-lunch-money. The cost of lunch must be a part of the total monthly cost, and there exists a connection between the two features.
• Unnecessary connection: This condition means that if the values of the two potentially related features are non-zero at the same time, there exists an edge between the two features. For example, if getting-up and breakfast-study are both non-zero at the same time, then there is a connection between the two features, which means that the student may tend to get up early for breakfast.

Based on the above-mentioned criterion, we constructed graphs for each student, creating a total of 3616 graphs for graph-level classification, using 700 graphs as the test set. Below, we present two graphs that were constructed according to the above-mentioned method (Figure 1).

3.4. Model Description

We proposed an improved self-attention GNN based on previous research. The corresponding Algorithm 2 pseudo-code of improved self-attention model is as follows.

Algorithm 2 Graph classification based on improved self-attention GNN

Input: Adjacency matrix of constructed matrix $W_{adj}$, node feature vector V, graph indicators P and degree matrix D Output: The prediction labels on the test set 1: Initialize: initialize weight matrix W and bias $\epsilon$ 2: Normalize adjacency matrix by $L=D^{-\frac{1}{2}}(W_{adj}+I)D^{-\frac{1}{2}}$, in which D refers to the degree matrix 3: Build a graph convolutional layer to obtain the attention scores by calculating $Z=GCN(L,V,W,\epsilon )$ 4: Perform convolution twice, use activation function ReLu to process the scores, and concatenate the scores S 5: Perform self-attention pooling by function pooling ($W_{adj}$, S, P) and update the graph structure and adjacency matrix $W_{adj}$ according to the mask 6: Perform maximum pooling and average pooling, and concatenate the results 7: Predict the labels after the three fully connected layers

In the standard self-attention mechanism, given a group of nodes $\left(x_1,x_2,\cdots ,x_k\right)$ and weight matrices $W_Q,W_K,W_V$ that represent different linear transformations of features, the attention coefficients are computed to reflect the pair-wise importance of the nodes, as shown in Equation (2):

$$e_{ij}={\left(W_Q^T{x}_i\right)}^T\left(W_K^T{x}_j\right),\forall 1\le i,j\le k$$

(2)

Then $e_{ij}$ is normalized by all possible values of j using the Softmax function as shown in Equation (3):

$${\alpha }_{ij}=\frac{exp\left(e_{ij}\right)}{{\sum }_{1\le l\le k}exp\left(e_{il}\right)}$$

(3)

Finally, a weighted sum of transformed features is calculated as shown in Equation (4):

$${\overrightarrow{d}}_i=tanh\left(\sum _{1\le j\le k}{\alpha }_{ij}{W}_V^T{x}_j\right)$$

(4)

Additionally, a new node embedding vector set can be obtained when using a multi-head graph attention layer. In graph attention neural networks, the node used for the attention mechanism generally only aggregates the information of the first-order neighbours to update the information. In the improved model, due to the characteristics of the extracted features, and to utilize the neighbouring information of node $v_i$ better, we applied two graph convolution layers. In the convolution layers, we calculated the node embedding using a linear transformer W and used the activation function ReLu to calculate the raw attention score between pair-wise nodes using Equations (5) and (6):

$$Z=GCN(L,V,W,\epsilon )$$

(5)

$$e_{ij}=ReLU\left({\overrightarrow{a}}^T\left(z_i^{\left(l\right)}\parallel z_j^{\left(l\right)}\right)\right)$$

(6)

where $\overrightarrow{a}$ is a weight vector for learning and ‖ refers to concatenation. Next, we applied the SoftMax function to normalize $e_{ij}$, and a weighted sum based on attention on the features of all the neighbour nodes, as shown in Equation (7):

$$h_i^{(l+1)}=ReLu\left(\sum _{j\in \mathcal{N}\left(i\right)}{\alpha }_{ij}^{\left(l\right)}{z}_j^{\left(l\right)}\right)$$

(7)

$\mathcal{N}\left(i\right)$ refers to all the neighbours of node i. We also used the pooling method to handle redundant information and reduce the amount of calculations. The extracted features denote the occurrence frequency of consumption behaviour sequence patterns and the ReLu function shows the best performance on our dataset.

4. Result and Analysis

We compared our improved self-attention GNN to some other graph neural network variants on two classification methods, including some machine learning models. We will now discuss the difference in the loss reduction of the variant GCNs and the improved self-attention GNN mainly. Due to the uneven distribution of the labels, the results are weighted as demonstrated below.

4.1. Experiment Result

Since the proportion of students with a score of less than 60 was very small, we set a higher passing line. We used the score of 70 to divide the students’ scores into two categories defined as $\{1,2\}$ and conducted a training task. The student group whose scores are all greater than or equal to 70 is labeled 1, and the rest of the group whose scores are less than 70 is labeled 2. To construct a large graph and speed up calculations, we first batched all the training graphs, and then trained the self-attention GNN with 300 epochs, as shown in Figure 2. Compared with the other GNN variants trained using the same number of epochs, the loss of our improved model varied sharply during the training process.

Table 4. The two-category classification results on the test set.

Model	Accuracy	Precision	Recall	F1-Score
Improved self-attention graph neural network	84.86%	94.84%	84.86%	91.81%
Topology adaptive graph convolutional network (TAGConv GNN) [57]	68.57%	62.60%	68.57%	64.88%
GraphSAGE convolutional network (SAGEConv GNN) [58]	65.76%	65.29%	70.29%	67.04%
Simplified graph convolutional network (SGConv GNN) [59]	70.43%	66.19%	70.43%	67.71%
Logistic regression [60]	61.14%	81.39%	61.14%	66.64%
KNN [61]	84.00%	76.57%	84.00%	78.44%
Decision tree [62]	76.00%	77.00%	76.00%	77.00%

lists the performance of different models.

As can be seen from Figure 2, the cross-entropy loss reduction of the improved self-attention GNN was very fast, while it could quickly reach stability. While the descent process of the SAGEConv GNN exhibited fluctuations, the cross-entropy loss reduction was much smoother than our improved self-attention model. The remaining two graph neural network variants were both highly stable. This revealed the high sensitivity of our improved GNN model to variations in the data.

It was difficult to analyse the behavioural patterns of students who were outstanding in the two-category classification. Therefore, we further divided the students whose scores were greater than 70 into two categories and conducted a three-category classification task. Then, we used the scores of 70 and 85 to divide the students’ scores into three categories defined as $\{1,2,3\}$ for multi-category classification. Specifically, students with a score of less than 70 are considered as failing, students with a score of 70 or above and not exceeding 85 are considered as good and students with a score of 85 or above are considered excellent, corresponding to label 1, label 2 and label 3, respectively. We then performed 300 epochs. We observed that the process of loss reduction gradually stabilized, as shown in Figure 3. The performances of different models are listed in

Table 5. The three-category classification results on the test set.

Model	Accuracy	Precision	Recall	F1-Score
Improved self-attention graph neural network	79.43%	97.20%	79.43%	87.28%
Topology adaptive graph convolutional network (TAGConv GNN) [57]	64.71%	62.44%	64.71%	62.99%
GraphSAGE convolutional network (SAGEConv GNN) [58]	66.06%	65.91%	65.86%	65.43%
Simplified graph convolutional network (SGConv GNN) [59]	67.00%	63.51%	67.00%	64.56%
Logistic regression [60]	48.29%	72.80%	48.29%	53.49%
KNN [61]	79.14%	68.93%	79.14%	68.00%
Decision tree [62]	68.05%	67.16%	68.09%	68.17%

.

The improved self-attention GNN showed a high degree of data sensitivity from Figure 3; after reaching stability, the cross-entropy loss of our model still showed some fluctuations. After training SAGEConv GNN for 300 epochs, the cross-entropy loss kept on declining and fluctuating from the declining process of the cross entropy loss. Therefore, our improved self-attention GNN was better than SAGEConv GNN in this regard, which implied that the two models may have high data requirements and data sensitivity.

Based on the idea of hypothesis testing, we constructed an indicator based on the discriminative behavioural patterns to reflect the differences in behavioural patterns using Formula (8):

$$\mathrm{ratio}=\frac{{\sum }_{i=1}^n{x}_i/n}{{\sum }_{j=1}^m{y}_j/m}$$

(8)

In the two-category classification, x represents the number of occurrences of some behavioural patterns of outstanding students, while y represents the number of occurrences of corresponding behavioural pattern of lagging students; in the three-category classifications, three ratio indicators were used regarding three score categories, respectively. The numerator and denominator were divided by the corresponding number of people to make sure the number of people did not have an impact. Taking the two-category classification as an example, the related results are presented in

Table 6. Ratio of occurrences of different discriminative behavioural patterns in the two category classification tasks.

Behavioural Pattern Sequence	Average Times of Outstanding Students	Average Times of Lagging Students	Ratio	Node ID
Study-actual	5.3516	2.2573	2.3708	0
Paid-days	29.5463	27.5556	1.0722	2
Getting-up-num	10.7108	4.9357	2.1701	3
Dinner-dormitory-num	8.3119	6.0819	1.3667	13
Breakfast-study-num	1.2968	0.4269	3.0377	14
Lunch-study-num	1.0605	0.3275	3.2383	15
Dinner-study-num	0.7278	0.2982	2.4402	16
Classroom-lunch-or-dinner-num	1.8204	0.8655	2.1033	18
Breakfast-study-lunch-num	0.7675	0.2690	2.8530	19
Dinner-study-dorm-bathroom-num	0.2287	0.0526	4.4359	21
Total-meals	56.3516	42.7252	1.3189	26

.

4.2. Result Analysis

From the perspective of cross entropy loss decreasing process in Figure 2 and Figure 3, compared with other models, the improved self-attention model had higher cross-entropy loss in the initial stage of training, while with certain fluctuations, it could decrease and converge rapidly in both classification tasks. The initial cross-entropy loss and training processes of the other three graph neural network model variants were similar in two-category classification, which may be related to the fact that these models treated the nodes indiscriminately during each iteration. However, the loss of SAGEConv GNN maintained the downward trend in three-category classification. Judging from the model’s prediction accuracy and other performance, we could speculate that although the loss declined, the model might have been over-fitting, which led to poor performance on the test set. In addition, the training process of both our improved model and SAGEConv GNN demonstrated larger fluctuations, which revealed that they might be more sensitive to label distribution.

From the perspective of prediction accuracy in Table 4 and Table 5, the improved self-attention GNN performed better than any other model in both the classification tasks and achieved accuracy of 84.86% in two-category classification and 79.43% in three-category classification, respectively. Moreover, precision could reach 94.84% and 97.20%, and F1-score was able to achieve 91.81% and 87.28%, respectively. Indicators, such as precision, recall score and F1-score, revealed that improving the self-attention model could improve the prediction performance and converge to stability rapidly. Judging from several indicators, three variants performed similarly but worse than our improved model, in which the accuracy reached 68.57%, 65.76%, 70.43% and 64.71%, 66.06%, 67% in the two tasks, respectively. They focused on improving graph convolution methods. Compared with machine learning algorithms, the overall performance of the graph neural network variants was found to be superior, demonstrating the graph’s strong and in-depth ability to mine mutual influences between local nodes. As comparison, the machine learning models equalized the input features, ignoring their potential mutual influence, which resulted in their relatively unsatisfactory performance in some extent. The KNN classification performance was also relatively good, in which accuracy and recall rate were the same, reaching 84% and 79.14% in the two tasks, respectively. However, the KNN could not predict the behavioural trend and identify discriminative behavioural patterns. Its performance was dependent on the training set and the way the distance between data points was defined, which limited its applicability. The decision tree can give judgments through internal nodes and determine classification results based on leaf nodes; however, the depth and the number of leaf nodes of the decision tree for good predictive effect had certain uncertainty and it could not predict students’ behavioural tendency similar to the KNN. It was more suitable for ensemble learning to improve the accuracy, while in this research accuracy only achieved 76% and 68.05% in two tasks and bore a resemblance to the remaining three indicators. As for logistic regression, this method could be regarded as generalized linear regression, while a more complex non-linear relationship cannot be expressed, not to mention mining local mutual influence. The above shortcomings contributed to such a situation: Accuracy equaled to recall rate in both tasks by 61.14% and 48.29%, precisions were 81.39% and 72.8% and F1-scores were 66.64% and 53.49%. By paying more attention to the local structure information using the GNN, better local mutual information mining could be performed, except improving the prediction accuracy of our method. The improved self-attention GNN could further identify discriminative behavioural patterns and predict the behavioural trends of students from the node scores and the existence of edges, which provided an insight into the behavioural patterns of outstanding students.

Next we discussed the prediction of students’ behavioural trends. Because of the use of the self-attention mechanism, different feature nodes contributed to the performance prediction differently. We utilize different colour shades to represent the scores of the different nodes as shown in Figure 4. The darker the colour, the higher the score, demonstrating the relationship between the different behavioural patterns and their contribution to predicting the academic performance of the student. The first figure (Figure 4a) represents one of the graph trainings resulting from the two-category classification task, followed by three-category classifications. In two different classification tasks, different nodes showed different contributions for grade prediction.

Among them, the paid-days (node 2) had the highest score, as can be seen from Figure 4. The results showed that daily life habits of the student were relatively consistent and regular. In addition, we noticed that the behavioural pattern of studying after dinner and then returning to the dormitory (node 25), and the number of times to study after lunch (node 15) influenced the prediction greatly. In addition, we noticed that whether the student at least ate lunch and dinner during the day impacted on their academic performance; this behavioural pattern was also a reflection of the regularity in their daily life. Additionally, taking the graph shown in Figure 4a as an example and considering the behavioural tendency, it was most likely that the student would study and then return to the dormitory after finishing dinner according to single node 21. Among the neighbours of node 4 (number of times eating both breakfast and lunch or both lunch and dinner), the influence of node 6 (number of dinners) was relatively large, which showed that if the student tended to enjoy dinner, the student was most likely to eat all three meals. It is therefore possible to create a similar analysis for other students as well.

Taking discriminative behavioural patterns into consideration, in Table 6, except for following behavioural patterns, such as the number of days with consumption records, returning to the dormitory after dinner, and the total number of meals over a month, the ratios of the remaining indicators exceeded 2. On average, the number of occurrences of these behavioural patterns of outstanding students exceeded twice that of lagging students. Further, we speculated that some corresponding behavioural patterns of some top students occurred more frequently. The visualization results were consistent with the above-mentioned results, thereby proving the correctness and rationality of our conclusions. Therefore, we believe that these characteristics could be used as a reference to distinguish students with regard to their learning levels.

5. Open Issues and Conclusions

5.1. Open Issues

In fact, judging from the results, GNN still needs to be improved in EDM. Below we list some open issues for further research:

• How to use nodes and edges to reflect the relationship between features is still an open question.
• The learning data is various, such as the data generated during MOOC online course learning, which inspires us to construct graphs to present more kinds of data for information mining.
• As for node score based on self-attention mechanism, the features represented by different nodes can be ranked by importance. Thus, how to combine traditional feature selection methods and GNN to determine useful features is also worth exploring.
• Through the comparison of scores, we can judge what the student’s next behaviour pattern is most likely to be. However, based on the score values obtained from training, new quantification methods are needed.
• How to build a behavioural network of all students, and integrate more information including curriculum arrangements, to conduct more diversified analysis, including skipping classes and social interactions, is encouraged.

5.2. Conclusions

In this study, we focused on mining on-campus consumption data to identify discriminative behavioural patterns based on spatial location change, analyse behaviour trends, and predict students’ academic performance by constructing behaviour networks. Firstly, we preprocessed collected consumption data to extract features that could reflect students’ living habits and their learning status. Secondly, we attempted to construct campus behaviour networks from reality and artificially, including introducing p-clique. Thirdly, combining with the pooling method, an improved self-attention GNN was utilized for training and prediction, and good prediction performance on the test set was achieved. For discriminative behavioural patterns, the habit of getting up early and behavioural patterns of continuous learning until meal time (lunch or dinner) in classroom and learning after three meals were discriminatory for distinguishing students with different learning levels, since the defined ratios of these behavioural pattern features are greater than 2 and we held certain beliefs that these characteristics are discriminatory in the sense of average. These new knowledge discoveries of behavioural patterns were consistent with the visualization results, conformed to the actual situation, and had certain reference meaning. For the behavioural trend analysis, judging from the behavioural pattern represented by single node 21, the student was more likely to continue studying after dinner than to return to the dormitory; from the perspective of node interaction and the existence of edges (e.g., node 4 and its neighbour nodes), three meals of students who often eat dinner were also relatively regular.

However, some limitations need noting regarding the present study. An arguable weakness is that all the graphs we consider are still not large samples. When we consider multi-category classification, the current method may fall into the shortcomings of few training samples for each category. In the meantime, few-shot learning may contribute to solving the above problem. Another weakness is that we discard some information, such as excluding behaviour related to gym and school busses. In addition, application limitation exists in our model and it is hard to apply in other universities.

As for future research directions, we were considering building a bigger behavioual network involving all students at school by their consumption record and some useful and possible video material to carry out analyses for different purposes, such as detecting absenteeism.