Visual Geometry Group-SwishNet-Based Asymmetric Facial Emotion Recognition for Multi-Face Engagement Detection in Online Learning Environments

Abstract

In the contemporary global educational environment, the automatic assessment of students’ online engagement has garnered widespread attention. A substantial number of studies have demonstrated that facial expressions are a crucial indicator for measuring engagement. However, due to the asymmetry inherent in facial expressions and the varying degrees of deviation of students’ faces from a camera, significant challenges have been posed to accurate emotion recognition in the online learning environment. To address these challenges, this work proposes a novel VGG-SwishNet model, which is based on the VGG-16 model and aims to enhance the recognition ability of asymmetric facial expressions, thereby improving the reliability of student engagement assessment in online education. The Swish activation function is introduced into the model due to its smoothness and self-gating mechanism. Its smoothness aids in stabilizing gradient updates during backpropagation and facilitates better handling of minor variations in input data. This enables the model to more effectively capture subtle differences and asymmetric variations in facial expressions. Additionally, the self-gating mechanism allows the function to automatically adjust its degree of nonlinearity. This helps the model to learn more effective asymmetric feature representations and mitigates the vanishing gradient problem to some extent. Subsequently, this model was applied to the assessment of engagement and provided a visualization of the results. In terms of performance, the proposed method achieved high recognition accuracy on the JAFFE, KDEF, and CK+ datasets. Specifically, under 80–20% and 10-fold cross-validation (CV) scenarios, the recognition accuracy exceeded 95%. According to the obtained results, the proposed approach demonstrates higher accuracy and robust stability.

1. Introduction

Student engagement within online learning is an issue that teachers and parents have been paying close attention to [1,2]. Students who actively engage in learning activities are more likely to gain profound learning experiences and better academic achievements [3,4,5,6]. However, due to the separation of time and space between teachers and students during the online learning process, it is rather difficult for teachers to promptly perceive changes in students’ engagement. Meanwhile, students are highly susceptible to the interference of the external learning environment during online learning, which poses a challenge to their ability to maintain attention for a long period. As an important external manifestation of engagement, facial expressions can intuitively reflect students’ learning status and interest [7]. Through the precise identification and analysis of these facial cues, it is possible to deduce the engagement levels of students.

Human facial emotional expressions are achieved through the synergistic actions of complex facial action units (AUs). For instance, AU1 represents the raising of eyebrows, AU6 represents the constriction of eyes, and AU12 represents the stretching of the corners of the mouth. Combinations of these AUs constitute a rich variety of facial expressions. However, when a person only raises one side of the mouth when expressing happiness or only lifts one eyebrow when expressing surprise, such asymmetry is a natural characteristic of emotional expression. Therefore, recognizing these asymmetric expressions is of great significance for understanding students’ genuine emotional states. Moreover, as students participate in online learning for relatively long periods, they will inevitably occasionally deviate from the field of view of the camera, revealing the side or partial contour of their faces. Such deviations can also lead to asymmetry in the facial presentation within the images, further intensifying the complexity of facial emotion recognition (FER) and posing a tremendous challenge to the precise identification of facial emotions.

In recent years, deep learning technologies have made significant progress in tasks such as feature extraction, classification, and recognition. In the field of computer vision, deep learning has been widely applied to tasks like object recognition, face detection, and emotion detection. With the rapid development of artificial intelligence (AI) and the widespread application of big data, the role of facial expression recognition in the teaching and learning process has become increasingly prominent, attracting the attention of more and more researchers [8,9]. Deep learning-based facial expression recognition technology enables “end-to-end” learning, directly extracting features from input images, which reduces human intervention and lowers the complexity of feature selection and extraction. Relying on this technology, teachers can more accurately understand the learning emotions of each student, adjust teaching strategies in a timely manner, and enhance student engagement and learning outcomes [10,11]. Therefore, developing an automatic facial expression recognition system for students is of great significance for improving the quality of education and the learning experience.

Meanwhile, the You Only Look Once (YOLO) series of models, an efficient real-time object detection framework, has been widely applied in fields such as autonomous driving and surveillance. Introduced by Redmon et al. [12] in 2016, YOLO treats object detection as a single regression problem, directly predicting bounding boxes and class probabilities from images. This design enables YOLO to perform exceptionally well in both training and inference, allowing for the rapid identification of multiple objects within an image. However, YOLO still faces challenges in detecting small objects, dealing with complex backgrounds, and handling occlusions [13]. For instance, poorly designed anchor boxes may lead to missed detections of small objects, noise in complex backgrounds can interfere with object detection, and partially occluded objects may be misdetected or missed due to reliance on anchor boxes and non-maximum suppression (NMS). Moreover, to maintain real-time performance, YOLO may sacrifice adaptability to new data distributions, thereby reducing detection accuracy and robustness [14].

Based on these considerations, a novel VGG-SwishNet deep learning model is proposed, which can automatically detect seven types of facial emotions using the facial images in the dataset, and it is particularly proficient in identifying asymmetric emotions caused by facial expressions themselves and postures. The proposed model adopts the pre-trained visual geometry group 16 (VGG-16) architecture, combines it with transfer learning (TL) techniques, and introduces the Swish activation function and batch normalization (BN) layers into the model. TL strategies not only leverage the robust feature extraction capabilities of the VGG-16 model but also empower it to learn meaningful feature representations from new datasets. BN layers reduce internal covariate shifts, ensuring the stability of data distribution, which facilitates effective gradient propagation in subsequent layers. Furthermore, it accelerates the model’s training process, enhancing the convergence speed and overall stability of the model [15].

The Swish activation function was proposed in 2017 and has drawn widespread attention in the field of deep learning [16]. Compared to traditional activation functions, it allows for smoother nonlinear mapping and can automatically adjust its degree of nonlinearity through a self-gating mechanism, thereby helping to mitigate the vanishing gradient problem. When dealing with the asymmetry caused by facial postures, its nonlinear characteristics can assist the model in more effectively extracting complex asymmetric features under different postures, such as the changes in facial contours and facial features when the face is in profile. Regarding the asymmetry of facial expressions themselves, the smoothness of the Swish activation function enables the model to more accurately capture subtle differences in expressions, such as a raised corner of the mouth or a drooping eyelid. Therefore, the Swish activation function has a significant advantage in recognizing asymmetric facial expressions. In addition, dropout layers with a rate of 0.25 are incorporated to prevent model overfitting. Finally, the performance of the proposed model was evaluated on benchmark datasets and compared with the engagement detection and real-time performance of five different versions of the YOLOv9 model (YOLOv9t, YOLOv9s, YOLOv9m, YOLOv9c, and YOLOv9e) and the ViT-Base/16 model in online learning environments.

The general contributions of the paper can be outlined as follows:

(i)

A novel and effective VGG-SwishNet model is proposed, which introduces the Swish activation function and BN layers based on the original VGG-16 model to improve the recognition capabilities for asymmetric facial emotions.

(ii)

The proposed VGG-SwishNet model is applied to multi-face engagement detection in online learning environments, with visualization results provided. Additionally, the accuracy of the proposed model in engagement detection is validated through qualitative research methods involving semi-structured interviews.

The rest of this paper is organized as follows: Section 2 presents a brief review of existing FER and engagement detection methods. Section 3 provides a preliminary overview of the convolutional neural network (CNN), the VGG-16 model, TL, and Swish activation function; explains the architecture for engagement detection based on FER as well as the proposed VGG-SwishNet model; and describes the benchmark datasets and performance metrics used in the experiments. Section 4 offers a detailed explanation of image preprocessing and experimental setup and provides a thorough discussion and comparative analysis of performance evaluations on benchmark datasets, as well as the results and related issues concerning online engagement detection. Section 5 concludes the paper with a discussion of potential directions for future research.

2. Related Works

2.1. Facial Expression Recognition

Facial expression recognition has experienced remarkable advancements over the past few decades, with significant strides in incorporating machine learning and deep learning methodologies. Early facial expression recognition research mainly relies on traditional machine learning algorithms, such as support vector machines (SVMs), k-nearest neighbors (KNNs), random forests (RFs), and neural networks (NNs), often in conjunction with a suite of feature extraction techniques to bolster recognition accuracy. Common feature extraction methods include the Gabor filter, Haar wavelet, local binary pattern (LBP), local ternary pattern (LTP), histogram of oriented gradient (HOG), linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), principal component analysis (PCA), etc. For instance, Lee et al. [17] introduced an algorithm predicated on regularized discriminant analysis (RDA) for FER, harnessing the combined strengths of LDA and QDA. Holder and Tapamo [18] enhanced the traditional gradient local ternary pattern (GLTP) with the incorporation of a more precise Scharr gradient operator and PCA for dimensionality reduction, subsequently validating their approach using an SVM on the extended Cohn-Kanade (CK+) and JAFFE datasets. Bellamkonda and Gopalan [19] proposed a facial expression recognition model based on SVM, which uses Gabor wavelet and LBP for facial feature extraction and PCA for dimensionality reduction. Eng et al. [20] demonstrated the efficacy of HOG for feature extraction from facial images, employing an SVM classifier for expression categorization, and substantiated their method’s validity on the JAFFE and KDEF datasets.

With the breakthrough of deep learning in image processing, methods based on deep learning have begun to dominate facial expression recognition. Mollahosseini et al. [21] discussed the application of deep neural networks in facial expression recognition and proposed a deep learning framework based on GoogLeNet and AlexNet to recognize facial expressions. The model was verified on seven different datasets. Jain et al. [22] adopted a deep neural network model that integrates CNNs for feature extraction with recurrent neural networks (RNNs) for emotion classification. Sari et al. [23] devised a lightweight CNN architecture for facial expression recognition and tested it on the CK+, JAFFE, and KDEF datasets. The rapid development of deep belief networks (DBNs) has also attracted considerable research interest. Liu et al. [24] constructed an innovative boosted deep belief network (BDBN) framework for recognizing facial expressions, which amalgamates the capabilities of strong and weak classifiers to enhance emotion recognition. Zhao et al. [25] capitalized on the unsupervised feature learning attributes of DBNs and the strengths of multi-layer perceptrons (MLPs), achieving a deep learning-based approach to facial expression recognition that exhibited commendable performance on the JAFFE and CK datasets. Roy et al. [26] introduced ResEmoteNet, a novel deep learning architecture for FER that integrates CNNs, Squeeze-and-Excitation (SE) blocks, and residual networks. The inclusion of an SE block focuses on the significant features of the human face, enhancing feature representation and suppressing less relevant ones, thereby reducing loss and improving overall model performance.

In recent years, the introduction of the YOLO model has brought new opportunities for the development of facial expression recognition technology. Bharathi et al. [27] proposed a novel approach utilizing the YOLO algorithm for the real-time detection of individuals in video frames, followed by a shallow CNN model for facial expression classification. The model achieved a notable accuracy of 95.57% on the FER-2013 dataset, demonstrating its effectiveness in recognizing seven distinct facial expressions. Vanamoju et al. [28] proposed a deep learning classifier based on YOLO for FER. This study leveraged the single-shot detection capability of the YOLOv8 algorithm, using a single neural network to predict bounding boxes and emotion class probabilities, achieving a test accuracy of 72.56% on the FER-2013 dataset. Parambil et al. [29] conducted a comprehensive comparative study of the YOLO model series, evaluating the performance of YOLOv5 to YOLOv9 in emotion recognition tasks. Based on the AffectNet dataset, the study compared the strengths and weaknesses of different YOLO models in terms of accuracy, inference time, model size, and generalization ability. It was found that YOLOv9e achieved the best accuracy, while YOLOv8n struck a good balance between speed and accuracy. Additionally, Hasan et al. [30] proposed a structural model combining YOLO face detection and CNN to predict human facial emotions. This model detected faces using YOLO and extracted features with CNN to classify seven basic emotions. The experimental accuracy reached 94% on the FER-2013 dataset.

2.2. Engagement Detection

In engagement detection, Hasnine et al. [31] crafted an educational application that computes engagement levels and categorizes student engagement as “highly engaged”, “engaged”, and “disengaged”. Ayouni et al. [32] predicted student engagement levels based on students’ activities recorded in Blackboard reports and compared three different machine learning predictive models. Sharma et al. [33] used the Haar cascade algorithm for face detection and eye region extraction. A CNN is used to identify whether students face the camera and classify their attention status as “distracted” or “focused”. If a student’s attention is defined as “focused”, the facial emotion of the student is subsequently recognized, and the student’s engagement is classified as “very involved”, “nominally involved”, or “not involved”. Shen et al. [7] designed a CNN model based on domain adaptation for facial expression recognition to assess the learning engagement of online learners in the MOOC scenario, and the engagement of learners was classified as “great”, “not bad”, and “not so well”.

3. Methodology

3.1. Preliminary

CNNs represent a class of deep learning architectures that have been widely applied in areas such as image recognition [34,35]. They automatically extract features, reducing the need for manual feature engineering, and enhance model generalization through structures such as convolutional, pooling, and fully connected layers. Owing to their hierarchical architecture, CNNs can efficiently capture a rich representation of features ranging from low-level edges and textures to high-level semantic information such as object parts and shapes.

VGG-16 [36] is a paradigmatic deep convolutional neural network (DCNN) architecture developed by the Visual Geometry Group of Oxford University. It extracts rich features by stacking multiple convolutional and pooling layers. Additionally, the model demonstrates excellent generalization ability and performs admirably on the ImageNet dataset [37], making its pre-trained model a powerful feature extractor applicable to various image recognition tasks. Moreover, the VGG-16 model is straightforward to implement and expand, with its relatively simple structure facilitating easy implementation and modification, allowing for convenient fine-tuning to better adapt to facial expression recognition tasks.

TL is a commonly used machine learning technique that allows models trained on other tasks to be used as a starting point [38]. By fine-tuning the existing model, the knowledge acquired from one task can be transferred to another distinct yet related task, thereby adapting to the new learning task. It is particularly effective when data resources are limited or training costs are high [39].

The Swish activation function is a self-gated activation function that can automatically adjust its degree of nonlinearity. This function approaches linearity when the input value is large, allowing for gradient flow, and smoothly transitions near zero to facilitate stable gradient propagation. It also gradually saturates for small input values, thereby alleviating the vanishing gradient problem [40]. Additionally, the non-monotonicity and smoothness of the Swish activation function help stabilize gradient updates during backpropagation, promote faster convergence to optimal solutions, and enhance the model’s generalization ability to new data.

3.2. Proposed Architecture for Facial Emotion Recognition and Engagement Detection

An architecture for assessing the level of student engagement through recognizing their facial emotions is proposed, as depicted in Figure 1. The arrows indicate the direction of the workflow, and the red rectangles are used to highlight the facial regions. The FER model is based on a pre-trained VGG-16 network, which is further fine-tuned for our purposes. Following the accurate identification of students’ facial emotions using the proposed model, the level of student engagement can be automatically assessed by integrating the emotion weights (EW) associated with each emotion.

3.2.1. Facial Emotion Recognition

In this stage, the VGG-16 model initially introduced in Section 3.1 was deployed and fine-tuned. Fine-tuning is a commonly used TL technique. The principal advantage of this method is that it cannot only avoid greatly changing the beneficial feature representation within the pre-trained model but also make the model adjust moderately on the new dataset to meet the needs of new tasks. The VGG-16 model after fine-tuning is shown in Figure 2.

As shown in Figure 2, the proposed model consists of a total of 46 layers. In the original VGG-16 model, the rectified linear unit (ReLU) activation function is used uniformly as the activation function. However, numerous studies have indicated that the ReLU function can lead to the problem of “dying neurons”, which in turn affects the model’s expressiveness [41,42]. This is because the derivative of ReLU is zero when the input is less than zero. Once a neuron’s output becomes zero, its weights cannot be updated in subsequent training, causing the neuron to permanently output zero. In contrast, the Swish activation function possesses smooth properties and excellent gradient propagation characteristics, enabling more stable weight updates during training. Its non-monotonic nature allows it to better capture complex relationships among input features, thereby enabling the model to learn richer feature representations and enhance its ability to fit complex data patterns [43]. Additionally, the self-gating mechanism of the Swish activation function enables the model to adaptively adjust the activation level based on the magnitude and importance of the input, thus better handling different types of features [40]. Studies have also demonstrated that, in many challenging datasets, especially in deeper models, the Swish activation function outperforms the ReLU activation function [16]. Therefore, in the proposed model, all ReLU activation functions are replaced with Swish activation functions. The calculation formula of the Swish activation function is shown in Formulas (1) and (2).

$$f\left(x\right)=x·\sigma \left(\beta x\right)$$

(1)

$$\sigma \left(x\right)={\displaystyle \frac{1}{1+e^{-x}}}$$

(2)

where $\sigma \left(x\right)$ denotes the Sigmoid function, and $\beta$ is a hyperparameter used to control the nonlinearity of the Swish activation function.

Subsequently, a BN layer was added after each max-pooling layer in the improved VGG-16 architecture. The max-pooling layer is primarily used to reduce the spatial dimensions of the feature maps while retaining important and salient information. Introducing a BN layer after the pooling layer allows for the normalization of the features after pooling, thereby stabilizing the data distribution. This is because the pooling layer may alter the numerical distribution of the feature maps, while the BN layer, by normalizing the mean and variance, effectively reduces the problem of internal covariate shift, making the learning process of subsequent layers more stable and efficient [15,44]. The calculation formulas for the BN layer are shown in Equations (3)–(6).

$${\mu }_B={\displaystyle \frac{1}{m}}\sum _{i=1}^m{x}_i$$

(3)

$${\sigma }_B^2={\displaystyle \frac{1}{m}}\sum _{i=1}^m{(x_i-{\mu }_B)}^2$$

(4)

$$\widehat{x_i}={\displaystyle \frac{x_i-{\mu }_B}{\sqrt{{\sigma }_B^2+ϵ}}}$$

(5)

$$y_i=\gamma {\hat{x}}_i+\beta$$

(6)

where $m$ denotes the batch size, and $x_i$ represents an individual sample within the batch. ${\mu }_B$ and ${\sigma }_B^2$ are the mean and variance of the batch, respectively. The parameters $\gamma$ and $\beta$ are trainable, serving as the scaling and translation factors, respectively.

Finally, the last dense layer was replaced with a novel one for classifying facial images into one of seven emotion categories: angry, disgusted, fearful, happy, neutral, sad, and surprised. Additionally, two dropout layers were set with a dropout rate of 0.25 to prevent overfitting. The output layer utilizes a softmax function and cross-entropy loss function for multi-class classification, as shown in Formulas (7) and (8).

$$\sigma (z{)}_j={\displaystyle \frac{e^{z_j}}{\sum _{k=1}^K{e}^{z_k}}}$$

(7)

$$L=-{\sum }_{j=1}^K{y}_{j}log\left({\sigma \left(z\right)}_j\right)$$

(8)

where $K$ represents the total number of different categories, and $z_j$ is the $j^{th}$ element in the original output vector. $\sigma (z{)}_j$ is the prediction probability of the $j^{th}$ category after Softmax function transformation. $y_j$ is the $j^{th}$ element in the real label vector. $L$ is the loss value.

3.2.2. Engagement Detection

In the engagement detection stage, the Haar feature-based cascade classifier within OpenCV [45] was utilized to identify and extract facial regions. Then, the method proposed in Section 3.2.1 was used to recognize the emotion of the extracted face and output the dominant emotion and its emotional probability (DEP) corresponding to each face. Finally, the concentration index (CI) was obtained by multiplying the dominant emotion probability by the corresponding EW, and the formula is shown in Equation (9).

CI = DEP × EW

(9)

The EW is a scalar ranging from 0 to 1, mainly used to quantify the intensity of facial emotional expressions at a particular instant. In this study, the EW, defined by Sharma et al. [33], is shown in

Table 1. Emotion weights.

Emotion	Neutral	Happy	Surprised	Sad	Fearful	Angry	Disgusted
Emotion weight	0.9	0.6	0.6	0.3	0.3	0.25	0.2

. According to the obtained CI, a student’s level of engagement (highly engaged, nominally engaged, and not engaged) can be evaluated according to

Table 2. Engagement detection from concentration index.

Engagement Type	Concentration Index (CI)
Highly engaged	>50%
Nominally engaged	20–50%
Not engaged	<20%

.

3.3. Facial Expression Datasets

Three benchmark facial expression datasets were used to evaluate the proposed approach: the Japanese female facial expression dataset, the Karolinska directed emotional faces dataset, and the Extended Cohn-Kanade dataset. A brief description of the three datasets used is given below.

3.3.1. Japanese Female Facial Expression Dataset (JAFFE)

The JAFFE [46] dataset comprises a total of 213 grayscale images of 10 Japanese females, with each image measuring 256 × 256 pixels. The dataset includes six basic facial expressions: angry, disgusted, fearful, happy, sad, surprised, and neutral. Each expression exhibits subtle asymmetry when presented on the face, such as differences in the upward movement range of the corners of the mouth and the movements of the eye muscles. These slight variations provide abundant samples for the study of asymmetric facial expressions. Moreover, this dataset can also be used to evaluate the performance and generalization capabilities of models when dealing with a limited amount of data. Some of the images are shown in Figure 3.

3.3.2. Karolinska Directed Emotional Faces Dataset (KDEF)

The KDEF [47] is a dataset that consists of 4900 human facial expression images, with each image having a resolution of 562 × 762 pixels. This group of pictures contains facial expression images of 70 individuals (35 males and 35 females). Each individual shows six basic expressions (angry, disgusted, afraid, happy, sad, surprised) and neutral expressions. These expressions are captured from five distinct perspectives: full left profile, left half-profile, front view, right half-profile, and full right profile. The multi-angle images of the KDEF dataset facilitate the model’s ability to understand and recognize facial expressions from various perspectives, which is particularly beneficial for identifying asymmetric expressions that may occur naturally. Part of the image is shown in Figure 4.

3.3.3. Extended Cohn-Kanade Dataset (CK+)

The CK+ [48] dataset is an extension of the original CK dataset, comprising 593 video sequences from 123 subjects. The participants’ ages range from 18 to 50 years, and 69% of the participants are female. In terms of ethnicity, 81% are Euro-American, 13% are Afro-American, and 6% belong to other groups. This diverse demographic composition enables the models trained on this dataset to be more effectively generalized across different populations, thereby enhancing their universality and robustness. Among the 593 video sequences, 309 sequences are annotated with one of the six basic emotions: angry, disgusted, fearful, happy, sad, and surprised. Each video sequence captures the dynamic transition from a neutral facial expression to a peak emotional expression. For the purposes of this study, the first frame of each of the 309 sequences was selected to represent the neutral expression, while the final three frames were chosen to represent the target emotion. This approach resulted in a dataset comprising 1236 images, each with a resolution of 640 × 490 pixels. Some of the images are shown in Figure 5.

3.4. Performance Metrics

Four widely utilized performance metrics, namely accuracy, precision, recall, and the F1-score, were adopted to thoroughly assess the proposed approach. These well-established metrics offer insightful evaluations of classification model performance. Accuracy indicates the proportion of correctly predicted instances out of the total. Precision measures the accuracy of positive predictions among all predicted positives. Recall, or sensitivity, reflects the model’s ability to identify all actual positives. The F1-score provides a balanced view by combining precision and recall into a single metric. The formulas used for calculating these metrics are provided as Equations (10), (11), (12), and (13), respectively.

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(10)

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}}$$

(11)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(12)

$$\mathrm{F}1-\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=2\times {\displaystyle \frac{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}}$$

(13)

4. Experimental Results and Discussion

4.1. Image Preprocessing

In the preprocessing steps, the image size of each dataset was resized to 224 × 224 pixels. To ensure the integrity of information during resizing and to address variations in illumination and head pose, OpenCV’s Haar feature-based cascade classifier was employed for face detection in the original images, and facial region was cropped to reduce the interference of background information. To handle head pose variations, the 68-point facial landmark detector from the dlib library was used to detect key facial features and calculate the angles of the eyes, nose, and mouth. The face was then aligned to a standard position through affine transformation, thereby enhancing the model’s robustness to pose variations. To mitigate the impact of different illumination conditions, histogram equalization was applied to adjust the brightness and contrast of the images, which helps to reduce the influence of lighting variations on model performance. After alignment and normalization, the images were resized to 224 × 224 pixels using bicubic interpolation. This method ensures image quality and preserves more facial detail information while resizing. Following these preprocessing steps, the images were fed into the proposed VGG-SwishNet model to extract reliable features for classifying the images into seven different emotion categories.

4.2. Experimental Setup

The experiment was conducted on a 64-bit Microsoft Windows 10 PC with MATLAB R2024b software. The CPU model is 13th Gen Intel (R) Core (TM) i9-13900k (Intel Corporation, Santa Clara, CA, USA), the clock frequency is 3.00 GHz, the graphics card is NVIDIA GeForce RTX 3090 (NVIDIA Corporation, Santa Clara, CA, USA), and the RAM is 128 GB. In this study, experiments were conducted on the JAFFE, KDEF, and CK+ datasets utilizing two distinct splitting modes: (i) in one mode, 80% of the images served as the training set, and 20% were designated as the test set; (ii) in the other mode, the 10-fold CV case, the entire dataset was easily divided into ten parts. During each CV process, one part was used for testing, and the remaining nine parts were combined for training models. Each time a different part was selected as the test set, this process was repeated ten times to ensure that each sample can be used as a test. The training hyperparameters for the network model are detailed in

Table 3. Training hyperparameters.

Parameter	Value
Optimizer	adam
MaxEpochs	30
InitialLearnRate	0.0001
LearnRateSchedule	piecewise
LearnRateDropPeriod	10
LearnRateDropFactor	0.5
MiniBatchSize	16
ValidationFrequency	30

.

4.3. Experimental Analysis and Comparison

4.3.1. Performance Evaluation on Datasets

The performance of the proposed VGG-SwishNet model was assessed through evaluation on the JAFFE, KDEF, and CK+ datasets. The results show that under the scenarios of an 80–20% split and 10-fold CV, the test accuracy of our approach exceeds 95% on all three datasets, demonstrating excellent performance. An experiment was also designed to generate confusion matrices for the method across the three datasets. The confusion matrix was utilized to present the classification outcomes of a model across various datasets. Illustrating the correspondence between the actual and predicted classes enabled us to intuitively grasp the model’s classification accuracy, types of errors, and their distribution. Figure 6 illustrates the confusion matrix of the proposed model on the JAFFE dataset. From the results, it is evident that the model achieved a perfect classification accuracy of 100% for “Angry”, “Disgusted”, “Fearful”, “Happy”, “Neutral”, and “Surprised” expressions, indicating that all samples of these emotions were correctly classified. The “Sad” expression has a 17% chance of being misclassified as “Disgusted”. This is due to the subtle similarities in facial features or expression intensity between “Disgusted” and “Sad” expressions, leading to a small number of ambiguous facial expression images in the JAFFE dataset. The confusion matrix of the proposed method on the KDEF dataset is shown in Figure 7. Overall, the classification accuracies of the seven emotions are relatively high. The model performs exceptionally well for emotions such as “Happy”, “Neutral”, and “Surprised”. Figure 8 displays the confusion matrix of the proposed approach on the CK+ dataset. It can be clearly seen from the figure that the model achieved the best results on this dataset, with no images being misclassified.

To thoroughly assess the overall performance of our approach, the precision, recall, and F1-score for each emotion type were calculated on the JAFFE, KDEF, and CK+ datasets. Subsequently, the macro-averaging method was employed to mitigate the influence of class imbalance within the datasets on the model’s performance. The macro-averaging method is a common performance evaluation approach in multi-class classification problems, which measures the overall performance of the model by averaging the precision, recall, and F1-score of each type.

Table 4. Precision, recall, and F1-score of proposed model on each emotion type for JAFFE, KDEF, and CK+ datasets.

	JAFFE			KDEF			CK+
Emotion Type	Precision	Recall	F1-Score	Precision	Recall	F1-Score	Precision	Recall	F1-Score
Angry	1.00	1.00	1.00	0.95	0.98	0.96	1.00	1.00	1.00
Disgusted	1.00	0.86	0.92	0.94	0.95	0.95	1.00	1.00	1.00
Fearful	1.00	1.00	1.00	0.89	0.93	0.91	1.00	1.00	1.00
Happy	1.00	1.00	1.00	1.00	0.97	0.99	1.00	1.00	1.00
Neutral	1.00	1.00	1.00	1.00	0.97	0.98	1.00	1.00	1.00
Sad	0.83	1.00	0.91	0.91	0.93	0.92	1.00	1.00	1.00
Surprised	1.00	1.00	1.00	0.98	0.94	0.96	1.00	1.00	1.00
Macro Average	0.97	0.98	0.97	0.95	0.95	0.95	1.00	1.00	1.00

presents the macro-averaged results of the proposed model on the JAFFE, KDEF, and CK+ datasets. On the JAFFE dataset, the model achieved a macro-averaged precision of 97%, a macro-averaged recall of 98%, and a macro-averaged F1-score of 97%, demonstrating the effectiveness of the proposed method for facial expression recognition on small-scale pose datasets. The KDEF dataset includes images captured from five different angles: full left profile, half left profile, front view, half right profile, and full right profile, which are used to illustrate the model’s ability to classify emotions from images captured at various angles. The experimental results obtained using the KDEF dataset indicate that a macro-averaged precision of 95%, a macro-averaged recall of 95%, and a macro-averaged F1-score of 95% were achieved. Therefore, our proposed method can effectively recognize emotions from facial images taken from different angles. Additionally, the proposed method achieved a macro-averaged precision of 100%, a macro-averaged recall of 100%, and a macro-averaged F1-score of 100% on the CK+ dataset, further demonstrating the superior generalization capability and strong classification performance of our approach.

4.3.2. The Impact of the Activation Function on the Model’s Accuracy

This study evaluated the impact of various activation functions by replacing ReLU in the VGG-16 model with six alternatives (Swish, Softplus, ELU, LeakyReLU, ClippedReLU, and GeLU) and testing on the JAFFE, KDEF, and CK+ datasets. The validation losses are shown in Figure 9, Figure 10 and Figure 11, respectively. From these three figures, it is evident that the ClippedReLU function has the highest validation loss on the JAFFE and KDEF datasets, while the Softplus function has the highest validation loss on the CK+ dataset. Additionally, the test accuracy of the seven activation functions on the JAFFE, KDEF, and CK+ datasets was recorded, as depicted in

Table 5. The test accuracy of the proposed model on the JAFFE, KDEF, and CK+ datasets using seven activation functions.

	Test Accuracy
Activation Function	JAFFE	KDEF	CK+
ReLU	88.37%	93.27%	97.98%
Swish	97.67%	95.20%	100%
Softplus	95.35%	94.69%	97.20%
ELU	93.02%	95.41%	98.79%
LeakyReLU	90.70%	94.29%	99.19%
ClippedReLU	81.40%	92.24%	98.38%
GeLU	86.10%	92.45%	99.60%

. From Table 5, it can be observed that the Swish activation function achieved higher recognition accuracy rates on the JAFFE, KDEF, and CK+ datasets compared to ReLU and its variants LeakyReLU and ClippedReLU. This was primarily due to the smoothness of the Swish activation function, which, unlike ReLU, LeakyReLU, and ClippedReLU, has a continuous derivative across all intervals. Furthermore, the Swish activation function achieved a recognition rate of 97.67% on the JAFFE dataset and 100% on the CK+ dataset, surpassing other activation functions and the original ReLU activation function. Although on the KDEF dataset, the best recognition rate of the Swish function was only 95.20%, ranking second among all activation functions with the ELU activation function in first place, and the recognition accuracy of the ELU function on the JAFFE and CK+ datasets was significantly lower than that of Swish. Therefore, the proposed model with the Swish activation function can reliably recognize various facial emotions in most cases.

4.3.3. Comparison of Performance with Advanced Models

As a cutting-edge representative in the field of object detection, the YOLOv9 model possesses significant value for research and application purposes [49]. In recent years, Transformer-based models have also garnered widespread attention. Among them, the Vision Transformer Base with 16 × 16 Patches (ViT-Base/16) model, which uses patches with a size of 16 × 16, is capable of capturing more detailed facial image information and has demonstrated notable advantages in the FER task. Furthermore, it is a widely recognized and frequently used benchmark model. By comparing our method with different variants of the YOLOv9 model as well as the ViT-Base/16 model, the performance of our approach relative to advanced methods can be clearly and objectively assessed. In the study, all models underwent rigorous testing on the JAFFE, KDEF, and CK+ datasets, with identical hyperparameter configurations, as depicted in Table 3. Two distinct data partitioning strategies were employed: one involved allocating 80% of the dataset for training and 20% for testing, while the other utilized 10-fold cross-validation (CV) to comprehensively evaluate model performance. The experimental outcomes are detailed in

Table 6. A comparison of the accuracies achieved with advanced models on the JAFFE, KDEF, and CK+ datasets in the 80–20% case.

	80–20% Case
Model	JAFFE	KDEF	CK+
YOLOv9t	62.50%	68.16%	73.68%
YOLOv9s	65.62%	72.67%	74.09%
YOLOv9m	81.25%	83.27%	80.57%
YOLOv9c	78.12%	75.31%	79.35%
YOLOv9e	68.75%	79.39%	77.33%
ViT-Base/16	82.87%	79.91%	95.35%
Proposed model	97.67%	95.20%	100%

and

Table 7. A comparison of the accuracies achieved with advanced models on the JAFFE, KDEF, and CK+ datasets with 10-fold CV.

	10-Fold CV
Model	JAFFE	KDEF	CK+
YOLOv9t	56.25% ± 3.14%	72.65% ± 0.36%	88.11% ± 0.41%
YOLOv9s	59.38% ± 4.15%	77.65% ± 0.37%	90.42% ± 0.29%
YOLOv9m	68.22% ± 1.34%	87.86% ± 0.28%	94.58% ± 0.32%
YOLOv9c	71.88% ± 2.25%	87.45% ± 0.35%	95.55% ± 0.31%
YOLOv9e	68.75% ± 1.76%	88.47% ± 0.44%	92.45% ± 0.26%
ViT-Base/16	80.65% ± 3.26%	92.46% ± 0.24%	97.67% ± 0.28%
Proposed model	95.84% ± 1.45%	95.35% ± 0.34%	99.60% ± 0.21%

. The observation results indicate that, under both data partitioning strategies, the proposed model achieved the highest accuracy on all three datasets, followed by the ViT-Base/16 model. The YOLOv9t model had the lowest recognition rates across the datasets, followed by YOLOv9s. This is largely because their lightweight structures may compromise certain feature expression capabilities. The YOLOv9m, YOLOv9c, and YOLOv9e models, which have relatively complex architectures, demonstrate certain levels of accuracy across the three datasets but still show a gap compared to our approach. Overall, the proposed method shows good generalization ability and stability in the FER task.

4.3.4. Comparison of Performance with Prior Studies

In this study, the performance of the proposed method was compared with other state-of-the-art methods on the JAFFE, KDEF, and CK+ datasets.

Table 8. A comparison between the proposed method and existing methods on the JAFFE, KDEF, and CK+ datasets.

	Total of Samples				Test Accuracy
Method	JAFFE	KDEF	CK+	Data Splitting Method	JAFFE	KDEF	CK+
VGG-16 + TL [52]	213	–	4000	50–50%	93.7%	–	94.8%
VGG-16+ LogicMax [53]	213	–	1479	10-Fold cv	94.86%	–	98.62%
Improved CNN [23]	213	490	981	80–20%	86.24%	82.38%	88.23%
Efficient-SwishNet [43]	213	4900	1200	80–20%	95.02%	85.5%	100%
	–	980	–	80–20%	–	88.3%	–
WGFD + SVM [54]	213	490	–	LOSO	66.20%	83.47%	–
	–	–	1296	10-Fold cv	–	–	90.59%
Custom CNN [55]	213	–	654	10-Fold cv	91.58%	–	92.27%
Region Group Adaptive Attention Model [56]	213	4900	981	80–20%	95.20%	93.47%	99.59%
Appearance Fusion + SVM [57]	207	980	–	10-Fold cv	97.16%	90.12%	–
Improved Residual Network [58]	–	980	981	80–20%	–	93.38%	96.37%
LBP-CNN [59]	213	–	331	80–20%	75.0%	–	79.5%
LTP-CNN [59]	213	–	331	80–20%	77.3%	–	89.2%
CLBP-CNN [59]	213	–	331	80–20%	81.2%	–	91%
Fusion CNN [50]	213	4900	981	80–20%	93.07%	90.30%	98.22%
Fusion CNN-SVM [60]	213	–	981	80–20%	89.23%	–	96.19%
CCNN-SVM [51]	213	4900	974	80–20%	98.4%	90.43%	99.3%
	213	4900	974	5-Fold cv	96.62%	87.18%	99.3%
Our Proposed Method	213	4900	1236	80–20%	97.67%	95.20%	100%
	213	4900	1236	10-Fold cv	95.84%	95.35%	99.60%

lists the methods, the total number of samples in the three datasets, the data splitting methods, and the test accuracies. As shown in the tables, when the JAFFE dataset consisted of 213 images and the KDEF dataset contained 4900 images, under the 80–20% data split scenario, the Fusion-CNN method proposed by Jabbooree et al. [50], which integrates geometric features extracted using a β-skeleton undirected graph and ellipse parameters with appearance features from a 2D-CNN, achieved accuracies of 93.07% and 90.30% on these two datasets, respectively. In contrast, the CCNN-SVM model proposed by Rashad et al. attained accuracies of 98.4% and 90.43% on the same datasets [51]. Although the proposed method achieved an accuracy of 97.67% on the JAFFE dataset, it demonstrated superior performance on the KDEF dataset, outperforming the above two methods with a higher accuracy rate. On the CK+ dataset, the proposed model achieved a test accuracy of 100% using the 80–20% data split and 99.60% using 10-fold cross-validation (CV), showing strong competitiveness compared to other methods. It is evident that the proposed approach has significant advantages in the FER task compared to prior studies.

4.4. Result Visualization of Student Engagement Detection and Comparative Analysis

4.4.1. Experimental Setup

An approximately 4 min long teaching video was prepared for students to watch in advance. The students’ facial expressions were subsequently recorded during online learning. After the video recording was completed, the proposed model was employed to recognize and classify facial expressions from each frame of the video, with a frame rate of 10 frames per second. The operating environment was identical to that described in Section 4.2. A total of nine second-year graduate students majoring in educational technology participated in this assessment, including three males and six females.

4.4.2. Experimental Results

The proposed approach was tested on the recorded video, and the detection results are depicted in Figure 12. It can be clearly observed from the figure that all faces were recognized and marked with red rectangular boxes. Above the red rectangular box, the dominant emotion of the current face and its corresponding probability are indicated by white text on a red background. Below the box, the current engagement level is also marked with white text on a red background. The red rectangular boxes were added using a Haar feature-based cascade classifier for face detection, which visually highlights the regions of interest in the current facial expression recognition task. The detection results show that all students were “Highly engaged” during the process of watching the video in this experiment.

4.4.3. Semi-Structured Interviews

To further validate the effectiveness of the detection results, in-depth semi-structured interviews were conducted with the nine students involved in the experiment. This method is typically qualitative and is often used as an exploratory tool across various research fields. Based on the students’ responses, their levels of engagement in the online learning session were analyzed. The interview results indicated that eight students reported being “Highly engaged” while watching the video, and one student reported being “Nominally engaged”. In contrast, our proposed method detected that all nine students were “Highly engaged”. This discrepancy may be due to the unnatural behavior exhibited by some students during video viewing, which could have interfered with detection. Additionally, variations in camera resolution and lens quality can also affect the model’s detection. For example, low-quality cameras may produce images with insufficient detail, making it difficult for the model to accurately detect facial expressions. Moreover, different background noise levels, lighting conditions, and changes in network conditions can also impact the robustness and reliability of the model when applied in online learning scenarios. Therefore, despite minor differences between the interview and detection results, the proposed method still effectively detected students’ engagement levels. Overall, the proposed method performs well in assessing students’ engagement in online learning environments.

4.4.4. Comparative Analysis of Advanced Models

Figure 13 presents the engagement distribution of the proposed model, different versions of the YOLOv9 model, and the ViT-Base/16 model among the nine participants in the form of a two-dimensional bar chart. Each model’s performance is represented by bars of three different colors corresponding to the three levels of engagement: “Highly engaged”, “Nominally engaged”, and “Not engaged”. It is evident from the figure that the proposed method detected high engagement in all nine participants, which is largely consistent with the interview results. This indicates that the model can effectively capture facial expressions in online learning environments and accurately calculate the level of engagement. Moreover, our method exhibits high consistency in performance across different participants, demonstrating good generalization capabilities. In contrast, other models such as YOLOv9t and YOLOv9s performed relatively poorly on some participants, primarily due to their limited ability to capture subtle features when handling facial expression recognition tasks. Additionally, the detection results of the YOLOv9e model show that seven participants were classified as “Not Engaged”, which significantly deviates from the actual interview outcomes. This may be due to the model’s high complexity, which led it to learn specific noise and details. As a result, the model’s accuracy is relatively low in practical applications, especially in online learning environments. Therefore, the analysis results of the two-dimensional bar chart support the effectiveness and reliability of the proposed approach in detecting student engagement in online learning, providing a more effective tool for assessing student learning states.

4.4.5. Real-Time Testing and Comparison of Advanced Models

In this study, the real-time performance of different versions of the YOLOv9 model (YOLOv9t, YOLOv9s, YOLOv9m, YOLOv9c, and YOLOv9e), ViT-Base/16, and our method was evaluated in the online engagement detection task. Each version of the YOLOv9 model was optimized for different application scenarios and requirements in its design. The ViT-Base/16 model offers a good trade-off between accuracy and computational efficiency.

Table 9. Performance metrics of advanced models for engagement detection.

Model	Total Processing Time(s)	FLOPs (G)	Parameters (M)
YOLOv9t	175.20	7.7	2.1
YOLOv9s	267.63	26.4	7.3
YOLOv9m	274.84	76.3	20.1
YOLOv9c	296.24	102.1	25.5
YOLOv9e	665.64	189.0	58.1
ViT-Base/16	219.73	17.2	86.4
Proposed model	178.59	15.48	138.3

presents a comparison of the seven models in terms of the total processing time, floating-point operations per second (FLOPs), and the number of parameters. It can be seen that the YOLOv9t model has the shortest total processing time (175.20 s), followed by the proposed model, which has a total processing time of 178.59 s. In contrast, YOLOv9e has a significantly longer total processing time of 665.64 s. Additionally, the proposed method has FLOPs of 15.48G, second only to YOLOv9t, which enables faster inference and makes it highly advantageous for tasks requiring high real-time performance. In terms of the number of parameters, YOLOv9t and YOLOv9s, as lightweight models, both have parameters under 10M, allowing for rapid deployment in resource-constrained scenarios. Although our approach has a significantly larger number of parameters compared to the other models, it demonstrates a clear advantage in processing speed. This is attributed to the introduction of the BN layers, which further enhances computational efficiency. Therefore, in scenarios with high real-time requirements, such as online learning monitoring and real-time video analysis, the proposed approach, with its excellent processing speed and computational efficiency, can quickly capture students’ facial expressions and behavioral features, showing significant potential for practical applications.

5. Conclusions

Comprehending the extent of student engagement in online learning environments is crucial for improving teachers’ online teaching strategies and enhancing students’ online learning experiences. This paper proposed a novel VGG-SwishNet model, which combines the VGG-16 architecture with the Swish activation function for the task of facial expression recognition. This integration fully leverages the powerful feature extraction capabilities of VGG-16 and the smoothness and self-gating mechanism of the Swish activation function, significantly enhancing the model’s performance. The incorporation of the Swish activation function aids in alleviating the vanishing gradient problem, thereby enabling the model to more effectively capture subtle and asymmetric features within facial expressions. This aspect has received less attention in previous studies but is a crucial issue in facial expression recognition. The proposed model was evaluated on the JAFFE, KDEF, and CK+ datasets, with both 80–20% splitting and 10-fold CV being employed. The results indicate that under both validation methods, the model achieved the best test accuracy of over 95% on all three datasets, demonstrating excellent robustness and competitiveness. Building on the accurate identification of each emotion using the proposed method, student engagement levels were assessed by integrating the corresponding weights of each emotion and categorizing them into three tiers: “Highly engaged”, “Nominally engaged”, and “Not engaged”.

The proposed approach was tested on nine students in an online learning environment, combining the results with semi-structured interviews, and the effectiveness of the VGG-SwishNet model in recognizing emotions and detecting engagement levels online was confirmed. Moreover, the VGG-SwishNet model is not only applicable to facial expression recognition tasks but can also be extended to other multi-class problems that require processing asymmetric features, such as identifying asymmetric tumor characteristics in the field of medical image recognition. In future research, we will also continue to explore more lightweight versions of the model. Specifically, we plan to utilize well-known lightweight network architectures, such as MobileNet and EfficientNet. By integrating these lightweight architectures into the proposed model, we aim to reduce the number of model parameters and computational complexity while further improving its scalability in online learning environments. At the same time, we are also aware of some limitations in this study. For example, the datasets used in the experiment (JAFFE, KDEF, and CK+) have a relatively small number of samples and are strictly controlled, which may not fully reflect the complexity of real-world online learning environments. Therefore, based on the model proposed in this study, we plan to establish a facial expression recognition system in a real online learning environment. During this process, we will collect real-world data to establish a facial expression dataset for online learning, thereby further enhancing the accuracy of student facial expression recognition in online learning environments.

In this study, the primary focus has been on developing a novel VGG-SwishNet model that can automatically and accurately detect students’ facial expressions during online learning. To validate the effectiveness of our proposed method, its performance was first comprehensively evaluated. Subsequently, the model was applied to engagement detection, and its practical effectiveness was further validated through semi-structured interviews. Moreover, comparisons with different variants of the YOLOv9 model and the ViT-Base/16 model have also confirmed that the proposed method has significant advantages in terms of engagement detection accuracy and computational efficiency in the context of online learning. However, it is also deeply recognized that this is merely a preliminary exploration. In future research, we plan to develop a more comprehensive engagement assessment system. Building on the accurate recognition of facial expressions, we will incorporate additional indicators of engagement, such as head movements and eye contact, to provide a more holistic and accurate evaluation of students’ engagement in online learning. This will be an important direction for our future work. Moreover, gaining a deep understanding of how different students’ engagement varies with different learning content is also crucial as it holds significant importance for promoting personalized learning. Therefore, we will also explore the complex relationship between learning content and changes in student engagement and assess the impact of different learning conditions (such as different subjects, teaching methods, or levels of interaction) on student engagement. This will help teachers more effectively improve their teaching strategies and enhance students’ engagement in online learning.