MoCvT-SAR: A Lightweight MobileNetV4-CvT Hybrid Network for Student Activity Recognition in Smart Classrooms

Abstract

Student activity recognition (SAR) is becoming increasingly important in smart classroom environments, where automatic visual analysis can be used to analyze the activities of students. However, most of the current state-of-the-art deep learning (DL) models for recognizing student activities suffer from limitations like high computational complexity, limited interpretability, and lack of concern about efficient deployment. In this work, we develop an explainable and efficient SAR system using the MobileNetV4-CvT model. A systematic evaluation was first conducted using five representative lightweight models, namely MobileNetV4-Conv-S, RepViT-M0.9, SHViT, iFormer-S, and Lightweight CvT, under a unified experimental protocol. Experiments were performed on six student-centered activity classes derived from the EduNet dataset, resulting in 26,841 images divided at the video level into training, validation, and test sets to prevent frame-level information leakage. Guided by the baseline analysis and an extensive ablation study, the final hybrid model combines the efficient local feature extraction capability of MobileNetV4 with a single CvT attention block for contextual feature refinement. The proposed model achieved a test accuracy of 95.20%, Macro Precision of 97.12%, Macro Recall of 91.37%, Macro F1-score of 92.62%, and Weighted F1-score of 94.57%, while containing only 2.66 million parameters and requiring 30.92 MB of storage. Its prediction reliability was further supported by an Expected Calibration Error (ECE) of 3.19%. Moreover, Gradient-weighted Class Activation Mapping (Grad-CAM), SHapley Additive exPlanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME), and Integrated Gradients (IG) were used to provide qualitative illustrations of the visual cues associated with correct and incorrect predictions. The findings show that the proposed model offers a good trade-off between accuracy, efficiency, calibration, and explainability and is thus a promising approach to developing intelligent classroom monitoring systems.

1. Introduction

Student activity recognition (SAR) has emerged as an important research direction in smart classroom environments [1,2], where automated visual analysis can provide valuable information about learners’ participation, attention, interaction, and engagement [3] during educational activities [4,5]. Reliable recognition of classroom actions can support educators by providing objective and continuous information about behavioral patterns that are difficult to monitor manually, particularly in large and dynamic learning environments. Activities such as hand raising, reading, writing, talking, sleeping, and interacting with classroom objects represent meaningful indicators of classroom participation and learning behavior. With the increasing adoption of intelligent cameras, edge computing devices, and artificial intelligence (AI)-assisted educational technologies, there is a growing demand for vision-based recognition systems that combine high predictive performance, computational efficiency, and practical deployability.

Deep learning (DL) has significantly advanced visual activity recognition by enabling models to learn discriminative feature representations directly from image data [6,7]. Convolutional Neural Networks (CNNs) are particularly effective at extracting local and hierarchical spatial information, including edges, textures, body posture, hand position, facial orientation, and object-related cues. More recently, Vision Transformers (ViTs) and attention-based architectures have demonstrated strong capabilities in modeling long-range spatial dependencies and global contextual relationships. These abilities are highly pertinent to classroom settings, where the differentiation between visually similar activities can be contingent not only upon specific body parts but also on the spatial arrangement of people, things, and classroom-related items.

Although highly effective, typical deep Neural Network architectures and those based on Transformer models might be computationally expensive, consume large amounts of memory, and cause delay during inference process. This would hinder their application on low-resource platforms for implementing smart classrooms for real-time activities. Thus, much more emphasis has been placed on lightweight vision architecture approaches that seek to achieve the same level of recognition efficiency by cutting down on parameters, computations, storage, and inference. Hence, efficient vision architectures like lightweight CNNs and attention-based models could serve as good alternatives for smart classroom applications. Nevertheless, it is still an open question regarding their effectiveness in feature extraction [8,9,10,11].

The lightweight convolutional and attention-based networks have complementary properties. While CNNs offer powerful local feature extraction with low computational cost, the self-attention modules can model the dependence between distant regions of an image. For the classroom scene, local feature representations could encode patterns corresponding to hand gestures, face direction, books, smartphones, and body posture, while global modeling of context could take into account relations between those objects and the whole scene. Thus, combining the convolutional processing with attention mechanisms could offer a more complete representation for the SAR task. However, more complex models do not necessarily perform better. The effective hybrid model should offer the global attention with low computational cost and maintain strong capabilities of local representation by CNNs [2,12,13].

Motivated by these considerations, this study evaluates the following modern lightweight CNN and attention network architectures for SAR classification tasks using the EduNet dataset: MobileNetV4-Conv-S, RepViT-M0.9, SHViT, iFormer-S, and lightweight Convolutional Vision Transformer (CvT). Each of the selected architectures implements a specific approach for achieving efficient visual representation learning through approaches such as pure convolutional computation, channel-wise convolution architecture, single-head self-attention, efficient self-attention methods, and convolution with multi-head self-attention.

Based on the results of this comparative study, a lightweight hybrid architecture consisting of MobileNetV4 combined with attention based on CvT is proposed. MobileNetV4 is used as a lightweight convolutional backbone network that provides both local and hierarchical visual features, while the attention component based on CvT is included for capturing larger-scale spatial relationships via convolution-based self-attention. The proposed hybrid architecture of MobileNetV4 and CvT attention aims to preserve lightweight convolutional processing advantages and provide attention-based contextual modeling capabilities. In addition, an ablation study is carried out to analyze the impact of the following architectural and training parameters: input image size, number of attention blocks, attention-head configuration, attention position, and regularization parameters.

Besides performance, interpretation is another crucial factor to consider in the case of education-based AI tools. Typically, deep learning models are used in AI systems, which are complex in nature and their predictions are not easy to interpret. In the process of classroom surveillance, lack of transparency makes it difficult to find out whether the prediction made by the model is based on the behavioral aspects or background data. Explainable Artificial Intelligence (XAI) offers some analysis tools for analyzing the visual data that leads to the decision making of the model.

In order to analyze decision-making behavior of the suggested model, four XAI approaches have been added to the framework. These approaches include Gradient-weighted Class Activation Mapping (Grad-CAM), SHapley Additive exPlanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME), and Integrated Gradients (IG). These approaches can shed light on the role played by certain image regions in individual predictions. The use of all these approaches at the same time will help detect semantically meaningful patterns, such as body posture, hand posture, facial orientation, and classroom objects. Interpretability of the model is analyzed not only for correct predictions but also for incorrect ones.

The main contributions of this study are summarized as follows:

• A systematic comparative evaluation of contemporary lightweight CNN and attention-based architectures for SAR, including MobileNetV4-Conv-S, RepViT-M0.9, SHViT, iFormer-S, and a lightweight CvT model under a unified experimental protocol.
• The development of a lightweight MobileNetV4–CvT hybrid architecture that combines efficient convolutional feature extraction with convolution-enhanced self-attention to capture complementary local and global visual representations.
• A comprehensive ablation study examining the effects of important architectural and training configurations, including input image resolution, attention depth, attention-head configuration, attention placement, and regularization settings.
• An efficiency-oriented evaluation that considers classification performance together with model complexity indicators, including parameter count, model size, computational cost, and inference-related characteristics.
• The integration of Grad-CAM, SHAP, LIME, and IGs to provide complementary interpretations of the proposed model’s predictions and to identify the visual evidence contributing to correctly classified and misclassified activities.
• Empirical study on the trade-off between lightweight feature extraction using convolutions and contextual modeling using attention mechanism, offering some insights into the feasibility of using hybrid CNN–Transformer architectures for smart classrooms.

The remainder of this paper is organized as follows. Section 2 reviews previous research on vision-based SAR, lightweight CNN architectures, efficient attention-based models, hybrid CNN–Transformer approaches, and explainable AI. Section 3 presents the proposed methodology, including EduNet data preparation, the evaluated lightweight architectures, the proposed MobileNetV4–CvT model, training procedures, evaluation metrics, and the XAI framework. Section 4 presents and discusses the experimental results, including baseline comparisons, ablation studies, class-wise performance, computational efficiency, and explainability analysis. Section 5 discusses the limitations of the study and outlines directions for future research. Finally, Section 6 concludes the paper by summarizing the principal findings and contributions.

2. Related Work

The present section provides an overview of recently published literature concerning SAR that involves the use of light-weighted deep learning models in classroom settings. This body of literature includes small CNNs, YOLO-based detectors, light-weighted spatiotemporal models, and attention-enabled models that were developed to detect student behaviors, taking into account the need to simplify both the computational and memory requirements of these models.

2.1. Lightweight Deep Learning for Student Activity Recognition

Lightweight DL has become increasingly important for SAR, particularly in smart classrooms requiring efficient and real-time processing. Recent approaches focus on compact CNNs and lightweight attention-based architectures to balance recognition accuracy with computational efficiency.

Among YOLO-based approaches, Jia and He [12] considered the problem of small targets, occlusions, and complex background of the classroom in student behavior detection by developing an SBD method using YOLOv5 framework. In this paper, they introduced the Coordinate Attention (CoAttention) module inside the YOLOv5 backbone to enhance the feature representation, where the obtained feature maps were used as inputs for OpenPose to detect student pose. The system performance was analyzed on a real-world classroom dataset, which included several behaviors of students in different visual conditions. From the experiments, the maximum Mean Average Precision (mAP) value was found to be 82.1% which is higher than Faster R-CNN and YOLOv5 by 5.2 and 4.6 percentage points, respectively. Sheng et al. [14] addressed issues related to both accuracy and real-time performance in classroom settings. In particular, they developed a more accurate YOLOv8s detector using a Multi-Scale Large Kernel Convolution Module (MLKCM) and a Progressive Feature Optimization Module (PFOM). MLKCM is responsible for extracting information in multiple spatial scales while PFOM works on progressive feature optimization and maintains local and global information. The proposed algorithm was tested on SCB-Dataset3-S and SCB-Dataset3-U datasets. The dataset SCB-Dataset3-S includes 5015 images corresponding to hand-raising, reading, and writing activities. The results show mAP@0.5 of 76.5% and 95.0% on the two datasets, respectively, using 22.3 million parameters.

An enhanced YOLOv11-based framework has been designed by Wang et al. [15] that addresses challenges related to small object detection, occlusions, background complexities, and visual similarity among various classroom behaviors. It has a CSP-PMSA module that facilitates multi-scale contextual feature extraction, a Scale-Aware Head that enhances small object detection, and Multi-Head Self-Attention that helps in modeling global dependencies and visually similar behaviors. This model was tested using the STBD-08 dataset which consists of 8844 images and 267,888 annotations of behaviors belonging to eight categories of student activities. The proposed method yielded mAP@50 = 91.6% and mAP@50–95 = 75.7%, which is better than the baseline YOLOv11 with 2.7 and 2.6 percentage points improvement, respectively. Guo et al. [16] developed SCB-YOLO, which is a lightweight YOLOv11n-based network aiming at enhancing the detection of students’ behaviors without increasing computational complexity. This network consists of a GEIT module that helps extract pose contour information and edge information related to behaviors like hand-raising and writing along with a MANet_Star module for multi-scale feature fusion. The experiments on SCB-Dataset3-S, including 5015 images from the classrooms and three types of behavior (hand-raising, reading, and writing), led to a mAP@0.5 of 71.8% for SCB-YOLO, surpassing the YOLOv11n baseline by 2.6%. This network includes 4.3 million parameters and 11.4 Giga Floating-Point Operations (GFLOPs).

In addition to the YOLO based object detection techniques, Wasim et al. [17] proposed the recognition of academic activities based on sequences through surveillance videos on campus. The focus was on jointly capturing both spatial appearance and temporal dynamics with limited computation and memory overheads. The proposed Campus Activities Dataset (CAD) consists of four types of academic activities, i.e., writing on board, cleaning of board, paper distribution, and paper collection, each having 100 instances. Each activity instance was represented as a sequence of 10 RGB frames resized to $224\times 224$ pixels. The proposed 3D-CNN was compared with a ConvLSTM2D-based sequence-learning model and achieved the highest classification accuracy of 95%, while requiring 13.3 GFLOPs and approximately 18,464 KB of memory. The findings indicate that direct spatiotemporal learning through 3D convolutions can provide an effective and computationally efficient solution for video-based academic activity recognition.

Feng et al. [18] followed a lightweight CNN–attention direction by proposing IMRMB-Net for student behavior recognition under occlusion, small-target conditions, and environmental interference. The framework introduces an Inverted Mix Residual Mobile Block (IMRMB), which combines the Inverted Residual Mobile Block with Mixed Local Channel Attention to capture contextual, spatial, and channel information with limited computational overhead. DySample has been added to the neck structure to enable dynamic upsampling and enhanced small object detection, whereas a Focaler-ShapeIoU loss has been used to overcome challenging samples, class imbalance, and localization reliability problems. The model has been mainly tested using the UK_Dataset that consists of 8754 images of eight classroom behaviors, along with tests using subsets of SCB_Dataset. The proposed IMRMB-Net managed to get an mAP@50 of 93.3%, an mAP@50:95 of 78.7%, a speed of 60.37 Frames Per Second (FPS), and a computational complexity of 23.8 GFLOPs.

A summary of thelightweight DL studies for SAR is presented in Table 1, highlighting the adopted approaches, model architectures, datasets, reported performance, and principal contributions.

Table 1. Summary of lightweight deep learning approaches for student behavior and activity recognition.

Approach	Ref.	Year	Method/ Architecture	Dataset	Performance	Key Contributions
YOLO-based detection	[12]	2024	YOLOv5 with Coordinate Attention and OpenPose	Custom classroom dataset	mAP: 82.1%	Combines object detection, attention, and pose information to improve recognition under occlusion.
	[14]	2025	Improved YOLOv8s with MLKCM and PFOM	SCB-Dataset3-S and SCB-Dataset3-U	mAP@0.5: 76.5% (S); 95.0% (U)	Improves multi-scale feature extraction and local–global information aggregation.
	[15]	2025	Enhanced YOLOv11 with multi-scale features and MHSA	STBD-08	mAP@50: 91.6%; mAP@50–95: 75.7%	Enhances contextual modeling, small-object detection, and discrimination of similar behaviors.
	[16]	2026	SCB-YOLO with GEIT and MANet_Star	SCB-Dataset3-S	mAP@0.5: 71.8%; 4.3 M parameters; 11.4 GFLOPs	Combines edge-information enhancement with efficient multi-scale feature fusion.
3D-CNN-based sequence learning	[17]	2025	Lightweight 3D-CNN for spatiotemporal activity recognition	Campus Activities Dataset (CAD)	95% accuracy; 13.3 GFLOPs; 18,464 KB	Learns spatial and temporal activity information using a lightweight sequence-learning architecture.
CNN–attention based	[18]	2025	IMRMB-Net with lightweight attention and dynamic sampling	UK_Dataset and SCB_Dataset	mAP@50: 93.3%; mAP@50:95: 78.7%; 60.37 FPS	Targets occlusion and small-object recognition while reducing computational requirements.

2.2. Explainable Artificial Intelligence for Student Activity Recognition

Explainable Artificial Intelligence (XAI) has become increasingly important in machine learning-based educational systems because high-performing models often provide limited insight into how their predictions are generated. XAI techniques improve model transparency by identifying influential features and providing global or instance-level explanations that can support educators and learners in understanding model decisions.

For student performance prediction, Adnan et al. [19] developed an XAI-based framework for predicting and interpreting students’ performance at different stages of a course in a virtual learning environment. The study used 32,593 student records containing demographic, clickstream, and assessment information and evaluated several traditional and ensemble learning algorithms, including LR, SGD, NB, KNN, DT, SVC, RF, Bagging, Extra Trees, Gradient Boosting, AdaBoost, and Voting classifiers. Models were evaluated at 20%, 40%, 60%, 80%, and 100% of the course duration, RF achieved the highest accuracy of 91.96%, and was subsequently interpreted using SHAP and LIME to provide global and local explanations of student performance. The study demonstrated that explainable prediction can support early identification of student performance patterns, with the RF model providing the strongest predictive results. Johora et al. [20] similarly addressed undergraduate academic performance prediction by developing an explainable stacking ensemble model using data from 872 students from multiple institutions in Bangladesh. RF and Gradient Boosting have been used as base learners whereas SVC was used as meta-classifier. SHAP and LIME methods have been incorporated to give global/local explanations for the predictions and determine the important academics/behavioural features. The stacking ensemble obtained an accuracy score of 86.38%, a precision score of 86.80%, recall score of 86.37%, and F1 score of 86.49%.

In the case of predicting adaptability, Nnadi et al. [21] studied the factors that play a role in the capacity of students to adjust to various learning environments through explainable machine learning (ML). For the purpose of the study, the researchers used a dataset consisting of 1205 student entries and performed comparative analysis of various classifiers such as RF, SVM, XGBoost, Logistic Regression (LR), KNN, Gradient Boosting, CatBoost, and Neural Networks. In order to increase the interpretability, SHAP, LIME, Anchors, ALE, and Counterfactual were used together, which facilitated analysis of both global and local feature influences. RF had the greatest classification accuracy of 91%, while the next two methods were Neural Networks and CatBoost with accuracy of 90%.

For cognitive modelling, Niu et al. [22] examined the various factors affecting the cognition of the students and conducted comparisons of the interpretation results generated using different XAI approaches. A total of 1086 freshmen from university were used for data collection and personal features, family information, education experience, and relationship with teachers were some of the information considered in the study. Lasso, Random Forest, XGBoost, SVM, and Neural Networks were considered first followed by use of feature importance, Morris sensitivity, SHAP and LIME to investigate the influence of various factors in the study. After considering the important factors using PSM, the final RF model recorded MSE of 0.4953, RMSE of 0.7038, MAE of 0.554 and $R^2$ of 0.2642.

Mustapha [23] created an XAI-based Intelligent Tutoring System called Entity-Relationship Diagram-PRO (ERD-PRO) aimed at helping the students learn Entity-Relationship Diagram design in an intelligent way. It consists of reasoning and explanation tools and provides personal scenario-based assistance in finding entities, attributes, relationships, attribute types, and business rules. It was tested on the students of various educational establishments based on pre- and post-development surveys. The outcome is that 90% of users demonstrated their satisfaction with the system while 65% said that explanation facilities of the system successfully help in understanding difficult parts. In addition, it was found that 80% of users became more confident in ERD design thanks to the system.

For dropout prediction, Nti and Ramanayake [24] proposed an explainable ML framework for predicting student dropout and supporting personalized interventions in online education. The study used a publicly available dataset containing 10,000 student records with demographic, academic, engagement, and learning-preference features. RF and XGBoost were trained for dropout classification, while SHAP was applied to provide global and instance-level explanations of the factors influencing individual dropout predictions. XGBoost achieved the best overall performance with an accuracy of 80.0%, precision of 0.790, recall of 0.816, F1-score of 0.785, and AUC of 0.800, slightly outperforming RF, which achieved 78.8% accuracy. The explainability component was further used to translate individual risk predictions into information that could support targeted educational interventions.

For activity recognition, Bian et al. [25] introduced Explainable Multimodal Classroom Activity Recognition (EMCAR), extending conventional classroom activity recognition by jointly producing activity predictions and human-interpretable explanations. The authors constructed the EM-CARD dataset containing 588 classroom video clips and integrated visual, acoustic, and textual information using MLLMs and LLMs. A human–LLM collaborative pipeline was employed for multimodal clue extraction, explanation generation, and open-vocabulary activity label construction. Different unimodal and multimodal configurations were evaluated using precision, recall, F-score, and explanation-overlap metrics. The best multimodal configuration, using Gemma2 with visual, acoustic, and subtitle information, achieved a precision of 46.44%, recall of 39.58%, F-score of 41.58%, and overlap score of 8.84, demonstrating the potential of multimodal reasoning for explainable classroom activity recognition.

A concise summary of the reviewed XAI-based educational studies is presented in Table 2, highlighting their main tasks, employed AI/ML/DL methods, explainability techniques, datasets, and reported performance.

Table 2. Summary of recent studies applying Explainable AI (XAI) techniques in educational environments.

Task	Ref.	Year	AI/ML/DL Methods	XAI Techniques	Dataset	Performance
Performance Prediction	[19]	2022	LR, SGD, NB, KNN, DT, SVC, RF, Bagging, Extra Trees, Gradient Boosting, AdaBoost, and Voting	SHAP and LIME for global and local interpretation	VLE dataset, 32,593 records	RF accuracy: 91.96%
	[20]	2025	Stacking ensemble using RF and Gradient Boosting as base learners and SVC as meta-classifier	SHAP and LIME for global and local explanations	Bangladesh undergraduate dataset, 872 students	Accuracy: 86.38%
Adaptability Prediction	[21]	2024	Random Forest, SVM, XGBoost, LR, KNN, and other classifiers	SHAP, LIME, Anchors, ALE, and Counterfactual explanations	Student adaptability dataset, 1205 samples	RF accuracy: 91%
Cognitive Modelling	[22]	2025	Lasso, Random Forest, XGBoost, SVM, and Neural Networks	Feature Importance, Morris Sensitivity, SHAP, LIME, and PSM	University freshmen dataset, 1086 students	RMSE: 0.7038; $R^2$: 0.2642
Intelligent Tutoring	[23]	2025	ERD-PRO mixed-initiative Intelligent Tutoring System	Scenario-based reasoning and explanation modules	Students from different educational institutions	Student satisfaction: 90%
Dropout Prediction	[24]	2026	Random Forest and XGBoost	SHAP for global and instance-level explanations	Online learning dataset, 10,000 records	XGBoost accuracy: 80.0%
Activity Recognition	[25]	2026	MLLMs and LLMs using visual, acoustic, and textual information	Multimodal explanation generation and open-vocabulary explainable recognition	EM-CARD, 588 classroom video clips	Best F-score: 41.58%

Note: AI = Artificial Intelligence; ML = Machine Learning; DL = Deep Learning; XAI = Explainable Artificial Intelligence; LR = Logistic Regression; SGD = Stochastic Gradient Descent; NB = Naive Bayes; KNN = K-Nearest Neighbors; DT = Decision Tree; SVC = Support Vector Classifier; SVM = Support Vector Machine; RF = Random Forest; XGBoost = eXtreme Gradient Boosting; SHAP = SHapley Additive exPlanations; LIME = Local Interpretable Model-Agnostic Explanations; ALE = Accumulated Local Effects; PSM = Propensity Score Matching; VLE = Virtual Learning Environment; MSE = Mean Squared Error; ERD-PRO = Entity-Relationship Diagram-PRO; MLLM = Multimodal Large Language Model; LLM = Large Language Model; EM-CARD = Explainable Multimodal Classroom Activity Recognition Dataset.

2.3. Research Gap and Motivation

Despite the growing adoption of DL for SAR, most existing approaches primarily focus on improving recognition accuracy and real-time detection performance, while comparatively limited attention has been devoted to understanding and interpreting the decisions produced by these models. This limitation is particularly important in educational environments, where transparent predictions can help educators understand the visual evidence associated with recognized student activities and assess the reliability of automated classroom analysis.

Furthermore, XAI has been increasingly explored in educational applications, particularly for student performance prediction, adaptability assessment, cognitive modelling, intelligent tutoring, and dropout prediction. However, explainability remains less explored in vision-based SAR, especially for lightweight DL architectures intended for resource-constrained and real-time deployment. Existing studies also rarely investigate recognition performance, computational efficiency, and model interpretability within a unified framework.

To address these gaps, this study investigates several lightweight CNN and Transformer-based architectures, including MobileNetV4, RepViT, SHViT, iFormer, and CvT, under a common experimental setting. Based on this comparative analysis, a lightweight MobileNetV4–CvT hybrid architecture is developed to combine efficient local feature extraction with attention-based global representation learning. In addition, Grad-CAM, SHAP, LIME, and Integrated Gradients are employed to explain the visual evidence influencing model predictions, including both correctly classified and misclassified samples. The proposed framework therefore aims to jointly address recognition performance, computational efficiency, and interpretability, providing a transparent and deployment-oriented solution for SAR in smart classroom environments.

3. Proposed Methodology

The method used to develop and evaluate the suggested lightweight and interpretable SAR system is described in this section. The method outlines the EduNet data preparation process, the evaluated lightweight baseline models, the proposed MobileNetV4-CvT hybrid architecture, the XAI approaches, and the training and testing methods.

3.1. Overall Framework

The general methodology used in this research work is geared towards the investigation of lightweight and interpretable DL model of SAR in intelligent classrooms. As shown in Figure 1, the first step in this methodology is to prepare six classes of activities conducted by the students using the EduNet dataset that is Arguing, Eating_in_classroom, HandRaise, Reading_Book, Sitting_on_Desk, and Writting_on_Textbook. Video frames are extracted and divided into training, validation, and test sets using a video-level partitioning approach, where frames from the same video are ensured to be in the same set. The images are then resized, normalized, augmented in training, and encoded according to their respective labels of activity. Various lightweight DL architectures, such as MobileNetV4-Conv-S, RepViT-M0.9, SHViT, iFormer-S, and Lightweight CvT, are then examined under a unified experiment setting.

Figure 1. Overview of the proposed SAR framework, including EduNet-based data preparation, evaluation of lightweight baseline models, development and ablation of the MobileNetV4–CvT hybrid architecture, final performance assessment, and XAI analysis using Grad-CAM, LIME, IGs and SHAP.

Considering the complementary features found in the baseline models, MobileNetV4-Conv-S and CvT are further combined in a hybrid lightweight architecture. The former remains the main convolutional feature extractor, whereas the latter is used as an attention block added to the network at the deep feature stage in order to incorporate context into the representation learned. The main architectural decisions made for the hybrid model are studied via the ablation study in terms of the attention insertion point, number of CvT blocks used, the head structure of the attention block, and input image resolution. Once the final configuration is chosen, the proposed model is tested by using such metrics as accuracy, precision, recall, F1 score, confusion matrix, prediction confidence, model size, and number of parameters. Finally, the model is interpreted through the usage of Grad-CAM, LIME, IGs, and SHAP in order to check if the model utilizes meaningful regions related to student posture, gestures, objects, and class interactions.

3.2. EduNet Dataset and Image Preparation

This subsection explains the methodology followed for preparing the image dataset obtained from EduNet for use in this research. This includes the student activities that have been chosen and the video splits that have been made for training, validation, and testing purposes.

3.2.1. Dataset Description

The experiments were conducted using data derived from the EduNet classroom activity dataset [26]. EduNet was originally introduced as a video-based benchmark for classroom activity recognition and contains 7851 manually annotated video clips distributed across 20 teacher- and student-centered activity categories. The videos were collected from publicly available YouTube material and recordings captured in real classroom environments, thereby providing considerable variability in terms of viewpoint, illumination, background clutter, student appearance, and classroom setting. The original videos are of resolution $1280\times 720$ pixels, frame rate of 30 fps, and last roughly between 3.25 to 12.7 s in duration with a total video time of around 12 h.

In the present study, the task was reformulated as an image-based SAR problem using six selected student-centered categories from EduNet. The retained classes were Arguing, Eating_in_classroom, HandRaise, Reading_Book, Sitting_on_Desk, and Writting_on_Textbook. The original EduNet dataset contains 20 activity classes, including 11 student-related classes and 9 instructor-related classes. Since the present study focuses specifically on student activity recognition, only student-related categories were considered. Among the 11 student classes, six were selected because their corresponding scenes generally contain only one or two students, with the main subject located relatively close to the camera and therefore sufficiently visible for image-based activity recognition. The remaining student classes more frequently involve multiple individuals, larger subject-to-camera distances, or more complex interaction patterns, which would require explicit person localization or object-detection mechanisms and potentially a different multi-person recognition architecture. These activities were selected because they represent visually meaningful classroom behaviors with different body postures, hand configurations, and interactions with surrounding objects. The resulting image collection therefore focuses specifically on recognizing student activities from individual visual observations rather than modelling the complete temporal information available in the original video clips.

The EduNet dataset is not publicly distributed and was obtained directly from the original authors upon request for research purposes. No additional data involving human participants were collected in this study. The images were used only for activity recognition, and no attempt was made to identify individual students.

3.2.2. Training, Validation, and Test Sets

The preprocessed image dataset contains 26,841 images that are further divided among six activity categories. In particular, three separate datasets are used, which contain 21,574 images for training, 2264 images for validation, and 3003 images for testing (image distribution by each category can be found in Table 3). Sampling was done by extracting the frames at an interval of 30 frames. Six activity categories included a total of 312 videos as the source, comprising 14 for Arguing, 51 for Eating_in_classroom, 38 for HandRaise, 87 for Reading_Book, 71 for Sitting_on_Des, and 51 for Writting_on_Textbook.

Table 3. Distribution of images and source videos across the training, validation, and test sets.

Activity Class	Train	Validation	Test	Total
Arguing	1207	112	278	1597
Eating_in_classroom	5632	502	728	6862
HandRaise	2196	197	350	2743
Reading_Book	5652	725	782	7159
Sitting_on_Desk	3200	406	333	3939
Writting_on_Textbook	3687	322	532	4541
Source videos	250	35	27	312
Total images	21,574	2264	3003	26,841

The source videos were divided into 250 training videos, 35 validation videos, and 27 test videos. This resulted in an average of approximately 86 extracted images per source video. Notably, partitioning has been done not at the level of individual frames but at the level of entire videos. Namely, all frames obtained from one source video belong to one of three datasets, namely, training, validation, and testing. Thus, there are no frames from one video in multiple datasets. This helps avoid information leakage since highly similar consecutive frames from one video would otherwise be used both for training and evaluation of the network, which makes the latter more effective.

Every image belongs to one of the six classes, whose coding is as follows: 0 for Arguing, 1 for Eating_in_classroom, 2 for HandRaise, 3 for Reading_Book, 4 for Sitting_on_Desk, and 5 for Writting_on_Textbook. The video-level separation allows having different data for fitting, selecting, and evaluating the model.

Although no source video is shared across the three subsets, the same individual may appear in different subsets through distinct videos and different activity instances; therefore, the adopted protocol ensures video-level independence but not strict subject-level independence. All evaluated architectures were trained from scratch without pretrained weights, and the same training settings and hyperparameters were applied to all baseline models to ensure a fair comparison. In addition, the baseline architectures and the proposed MobileNetV4–CvT model were each evaluated using three independent random seeds, with the corresponding mean and standard deviation reported to assess performance stability across runs.

3.2.3. Image Preprocessing and Data Augmentation

All the images were first scaled to a fixed size of $192\times 192$ pixels before feeding them into the network to ensure an adequate input size with respect to the chosen lightweight network architectures. The pixel intensity values were scaled to the $[0,1]$ interval to ensure numerical consistency and aid in network optimization.

A certain level of data augmentation was performed on the training images to improve robustness and reduce overfitting. The augmentation process was kept modest enough so that it would not have a significant impact on the semantics of the classroom actions. Each training image could be randomly flipped horizontally, rotated by a small angle of 0.02 radians, randomly zoomed by around $\pm 4\%$ along each dimension of the image, and had its contrast value changed by a small factor of 0.05. Reflection padding was used while performing the rotation and zooming operations. The augmentation process can be represented as

$$\tilde{\mathbf{I}}=\mathcal{A}(\mathbf{I}),$$

(1)

where $\mathbf{I}$ denotes the original training image, $\tilde{\mathbf{I}}$ is its augmented counterpart, and $\mathcal{A}(·)$ represents the stochastic composition of horizontal flipping, rotation, zooming, and contrast adjustment.

Vertical flipping was deliberately excluded because it would generate physically unrealistic classroom configurations and could distort posture-related visual cues that are important for distinguishing student activities. Likewise, only relatively small geometric transformations were used, since strong rotations or zoom operations could remove discriminative information such as raised hands, books, desks, or eating-related objects. Data augmentation was active only during training, while validation and test images were evaluated without random augmentation to ensure a consistent and unbiased comparison of the investigated architectures.

3.3. Baseline Lightweight Architectures

To establish a consistent benchmark for the SAR task, representative lightweight architectures were selected to capture complementary feature-learning mechanisms. MobileNetV4-Conv-S and RepViT-M0.9 were included as efficient CNN-based models, since convolutional operations are particularly effective at learning local spatial patterns, textures, edges, and body-related visual features. In parallel, SHViT, iFormer-S, and Lightweight CvT were selected as attention-based or hybrid architectures because they can additionally model broader contextual relationships and long-range dependencies between image regions. Evaluating these models under the same experimental setting therefore provides a balanced comparison between local convolutional feature extraction and attention-based global representation learning, while also assessing predictive performance and computational complexity for lightweight SAR.

3.3.1. MobileNetV4-Conv-S

MobileNetV4 is a family of efficient CNNs developed for mobile and resource-constrained vision applications [27]. The architecture extends the MobileNet design philosophy by introducing the Universal Inverted Bottleneck (UIB), which provides a flexible formulation capable of representing different efficient convolutional blocks within the same framework. MobileNetV4 also introduces improved stage organization and block selection to achieve a better trade-off between recognition performance, latency, and computational complexity.

The UIB can be generally represented as

$$\mathbf{Y}={\mathcal{P}}_{\mathrm{proj}}\left[{\mathcal{D}}_{k_2}\left(\varphi \left({\mathcal{P}}_{\exp }\left({\mathcal{D}}_{k_1}(\mathbf{X})\right)\right)\right)\right],$$

(2)

where $\mathbf{X}$ is the input feature map, ${\mathcal{D}}_{k_1}$ and ${\mathcal{D}}_{k_2}$ denote optional depthwise convolutions, ${\mathcal{P}}_{\exp }$ and ${\mathcal{P}}_{\mathrm{proj}}$ represent channel expansion and projection operations, respectively, and $\varphi (·)$ denotes the nonlinear activation function.

In this study, the MobileNetV4-Conv-S variant is adopted. Conv-S is the small purely convolutional configuration of MobileNetV4 and does not incorporate Transformer self-attention. It combines Fused Inverted Bottleneck and UIB blocks to progressively extract spatial and semantic features while maintaining a relatively small computational footprint. The original classification layer is replaced with a task-specific output layer corresponding to the student activity classes considered in this work. MobileNetV4-Conv-S is used as one of the principal lightweight CNN baselines and also serves as the convolutional backbone of the proposed hybrid architecture.

3.3.2. RepViT-M0.9

RepViT is a lightweight convolutional architecture designed by revisiting efficient CNNs from the architectural perspective of ViTs [28]. Instead of incorporating self-attention, RepViT preserves a fully convolutional structure while separating spatial feature extraction and channel interaction into token-mixing and channel-mixing operations. This organization follows the general design philosophy of modern Transformer-like architectures while retaining the efficiency of convolutions.

A distinctive characteristic of RepViT is structural re-parameterization. During training, several computational branches can be used to improve representation learning, whereas these branches are merged into a single equivalent convolution during inference. This transformation can be summarized as

$${\mathbf{W}}_{\mathrm{rep}}={\mathbf{W}}_{3\times 3}+\mathrm{Pad}({\mathbf{W}}_{1\times 1})+{\mathbf{W}}_{\mathrm{id}},$$

(3)

where ${\mathbf{W}}_{\mathrm{rep}}$ is the equivalent inference kernel and the remaining terms represent the convolutional and identity branches used during training.

In this study, the RepViT-M0.9 configuration is employed as a lightweight CNN baseline. The M0.9 variant provides a compact model size while preserving the main RepViT design principles, including efficient spatial token mixing, channel mixing, structural re-parameterization, and selective channel recalibration. It is included to investigate whether a modern pure CNN architecture can provide competitive SAR performance without explicit self-attention.

3.3.3. SHViT

SHViT is a lightweight ViT architecture developed to reduce the computational redundancy of conventional multi-head self-attention [29]. Its central idea is that processing all feature channels using multiple attention heads is not always necessary for efficient visual recognition. SHViT therefore introduces SHSA, in which only a subset of feature channels is processed by self-attention, while the remaining channels bypass the attention operation.

Given an input representation $\mathbf{X}$, the channels are divided into attention and residual components:

$$[{\mathbf{X}}_{\mathrm{att}},{\mathbf{X}}_{\mathrm{res}}]=\mathrm{Split}(\mathbf{X}),$$

(4)

and self-attention is applied only to ${\mathbf{X}}_{\mathrm{att}}$:

$$\mathrm{SHSA}({\mathbf{X}}_{\mathrm{att}})=\mathrm{Softmax}\left({\displaystyle \frac{\mathbf{Q}{\mathbf{K}}^T}{\sqrt{d}}}\right)\mathbf{V}.$$

(5)

This is followed by concatenation of the attention enhanced representation with the bypassed feature channels. This approach reduces the computation required for global attention while still being able to capture long-range dependencies.

In the present study, SHViT will be examined as a lightweight attention-based method for SAR. The local convolutional operations in conjunction with partial channel global attention will make SHViT especially applicable to classroom activity recognition since local body postures and global spatial relations can contribute to student activity discrimination.

3.3.4. iFormer-S

The Inception Transformer, referred to as iFormer, is a hybrid vision architecture designed to jointly model high-frequency local information and low-frequency global information [30]. The model is motivated by the observation that convolutional operations are effective for extracting local details, whereas self-attention is particularly suitable for modelling global dependencies. To exploit these complementary properties, iFormer introduces an Inception Token Mixer that processes different feature groups using parallel branches.

Its general operation can be represented as

$$\mathbf{Y}=\mathrm{Fuse}\left[{\mathcal{H}}_{pool}({\mathbf{X}}_{h1}),{\mathcal{H}}_{conv}({\mathbf{X}}_{h2}),{\mathcal{L}}_{attn}({\mathbf{X}}_l)\right],$$

(6)

where ${\mathcal{H}}_{pool}$ and ${\mathcal{H}}_{conv}$ model high-frequency information through pooling and convolution, while ${\mathcal{L}}_{attn}$ captures low-frequency global dependencies using self-attention.

Another important characteristic of iFormer is its frequency-ramp strategy. Early layers allocate a larger proportion of channels to high-frequency processing, while deeper stages progressively assign more channels to low-frequency global modelling.

In this study, the iFormer-S variant is employed. The small configuration maintains the frequency-aware hybrid design while reducing the overall network complexity. It is selected to assess the effectiveness of combining local convolutional feature extraction and global attention for SAR within a relatively lightweight architecture.

3.3.5. Lightweight Convolutional Vision Transformer CvT

The CvT  [31] is a hierarchical Transformer architecture that incorporates convolutional operations into the tokenization and attention processes. Unlike the original ViT, which directly converts fixed image patches into tokens, CvT employs convolutional token embedding to preserve local spatial information and progressively construct hierarchical visual representations.

For an input feature map $\mathbf{X}$, convolutional token embedding can be expressed as

$${\mathbf{X}}_t=\mathrm{Flatten}\left({\mathrm{Conv}}_{k\times k}(\mathbf{X})\right).$$

(7)

CvT also replaces the conventional linear generation of queries, keys, and values with convolutional projections:

$$\mathbf{Q}={\mathcal{C}}_q({\mathbf{X}}_t),\mathbf{K}={\mathcal{C}}_k({\mathbf{X}}_t),\mathbf{V}={\mathcal{C}}_v({\mathbf{X}}_t),$$

(8)

where ${\mathcal{C}}_q$, ${\mathcal{C}}_k$, and ${\mathcal{C}}_v$ denote convolution-based projection operators. These operations introduce local spatial inductive biases before global self-attention is performed.

In this study, a Lightweight CvT architecture is developed rather than using the original larger CvT configurations. The proposed lightweight version preserves the essential CvT principles, namely convolutional token embedding and convolutional attention projections, while reducing the network depth, channel dimensions, and attention complexity. This adaptation produces a substantially smaller model suitable for SAR and also provides the attention mechanism subsequently integrated with MobileNetV4 in the proposed hybrid network.

For reproducibility, the detailed stage-wise configuration of the Lightweight CvT baseline is provided in Table 4. The network contains three hierarchical CvT stages with embedding dimensions of 48, 72, and 120, corresponding depths of 1, 2, and 3 blocks, and 1, 3, and 4 attention heads, respectively. Each attention block uses convolutional Q/K/V projections with a $3\times 3$ depthwise kernel, a query stride of 1, a key/value stride of 2, and an MLP expansion ratio of 2.0.

Table 4. Detailed stage-wise configuration of the Lightweight CvT baseline.

Stage	Output Size	Channels	Depth	Heads	Embedding Kernel/ Stride	KV Stride	MLP Ratio	Dropout
Stage 1	$40\times 40\times 48$	48	1	1	$7\times 7$ / 4	2	2.0	0.02
Stage 2	$20\times 20\times 72$	72	2	3	$3\times 3$ / 2	2	2.0	0.03
Stage 3	$10\times 10\times 120$	120	3	4	$3\times 3$ / 2	2	2.0	0.04

Note: The input resolution is $190\times 190\times 3$. Each CvT block uses a $3\times 3$ depthwise convolution for the Q/K/V projections, with query stride 1 and key/value stride 2. The classifier consists of layer normalization, global average pooling, dropout with a rate of 0.25, and a six-class softmax layer.

3.4. Proposed MobileNetV4–CvT Hybrid Architecture

The proposed architecture combines the computational efficiency of MobileNetV4-Conv-S with the global representation capability of convolutional self-attention derived from CvT (see Figure 2). Rather than replacing the convolutional backbone with a Transformer, the proposed design preserves MobileNetV4 as the principal feature extractor and introduces a single CvT-based attention block at a deep, low-resolution stage of the network. This strategy allows local spatial patterns to be progressively learned by efficient convolutional operations before global relationships between the resulting high-level features are modelled through self-attention. According to the selected configuration, input images are resized to $192\times 192$ pixels, and one CvT block with one attention head and a $3\times 3$ convolutional projection kernel is inserted at the penultimate 128-channel feature stage. The attention width ratio is set to $1.0$, meaning that the complete 128-dimensional feature representation participates in the attention operation. After attention refinement, the remaining MobileNetV4 blocks continue processing the enhanced representation before classification.

Figure 2. Architecture of the proposed MobileNetV4–CvT hybrid model. The $192\times 192\times 3$ input image is first processed by data augmentation and rescaling, followed by the early and middle MobileNetV4-Conv-S stages for hierarchical feature extraction. A single CvT attention block with one attention head and $3\times 3$ convolutional Q, K, and V projections is inserted at the penultimate $6\times 6\times 128$ feature stage. The refined representation is subsequently processed by the remaining MobileNetV4 blocks and the classification head, consisting of a $1\times 1$ convolution with 960 channels, global average pooling, a $1\times 1$ convolution with 1280 channels, dropout, a six-class $1\times 1$ classifier, flattening, and softmax prediction. Panels (a–c) detail the MobileNetV4 building blocks, CvT attention module, and classification head, respectively.

3.4.1. MobileNetV4 Feature Extraction Backbone

The feature extraction component follows a lightweight MobileNetV4-Conv-S design based on standard convolutions, Fused Inverted Bottleneck blocks, and Universal Inverted Bottleneck blocks. Given an input image $\mathbf{I}\in {\mathbb{R}}^{192\times 192\times 3}$, data augmentation is first applied during training, followed by pixel-value rescaling to the $[0,1]$ interval. The convolutional backbone then progressively decreases the spatial resolution while increasing the number of feature channels, enabling the extraction of hierarchical representations ranging from low-level appearance patterns to high-level semantic features.

The network begins with a $3\times 3$ convolutional stem with 32 channels and a stride of 2. Two Fused Inverted Bottleneck blocks subsequently produce feature representations with 32 and 64 channels. The deeper portion of the backbone is composed of UIB blocks that combine $1\times 1$ expansion and projection operations with optional $3\times 3$ or $5\times 5$ depthwise convolutions. A UIB transformation can be summarized as

$${\mathbf{F}}_{out}=\mathcal{P}\left[{\mathcal{D}}_m\left(\varphi \left(\mathcal{E}\left({\mathcal{D}}_s({\mathbf{F}}_{in})\right)\right)\right)\right],$$

(9)

where ${\mathcal{D}}_s$ and ${\mathcal{D}}_m$ denote the optional starting and middle depthwise convolutions, respectively, $\mathcal{E}$ represents the $1\times 1$ expansion operation, $\mathcal{P}$ denotes the $1\times 1$ projection, and $\varphi (·)$ is the nonlinear activation function. Residual connections are employed when the input and output dimensions are compatible.

After the initial convolutional stages, the feature dimensionality increases from 64 to 96 channels and subsequently to 128 channels. For the $192\times 192$ input resolution, successive stride-2 operations reduce the feature map to

$${\mathbf{F}}_p\in {\mathbb{R}}^{6\times 6\times 128}.$$

(10)

The selected CvT module is inserted at this point, after the first two 128-channel UIB blocks and before the remaining late-stage MobileNetV4 blocks. This penultimate placement is computationally advantageous because self-attention is applied to only $6\times 6=36$ spatial positions rather than to the considerably larger feature maps occurring in earlier stages.

3.4.2. CvT-Based Convolutional Self-Attention Module

The proposed hybrid introduces a single CvT-inspired block to enhance the deep MobileNetV4 representation with global contextual information. In contrast to conventional Transformer attention, where queries, keys, and values are produced directly using linear projections, the implemented CvT block first applies independent depthwise convolutions to the normalized feature representation. This introduces local spatial information into the attention formulation before global relationships are evaluated.

Let the input of the attention module be ${\mathbf{F}}_p\in {\mathbb{R}}^{6\times 6\times 128}$. Layer normalization is first applied, after which the query, key, and value representations are generated using separate $3\times 3$ depthwise convolutions, batch normalization, and learned projections:

$$\mathbf{Q}={\mathcal{P}}_Q\left[\mathrm{BN}\left({\mathrm{DWConv}}_{3\times 3}({\mathbf{F}}_p)\right)\right],$$

(11)

with the same procedure independently applied to obtain $\mathbf{K}$ and $\mathbf{V}$. The two-dimensional features are subsequently flattened into $N=36$ tokens with an embedding dimension of $C=128$.

The selected module employs one attention head. Consequently, the complete 128-dimensional feature representation is assigned to the single attention head, giving

$${\displaystyle d_h=\frac{128}{1}=128.}$$

(12)

For the single attention head, scaled dot-product attention is calculated as

$${\displaystyle \mathbf{A}=\mathrm{Softmax}\left(\frac{\mathbf{Q}{\mathbf{K}}^T}{\sqrt{128}}\right)\mathbf{V}.}$$

(13)

The resulting single-head attention output is projected back to the 128-dimensional feature space, reshaped to the original $6\times 6$ spatial arrangement, and added to the input through a residual connection. A second layer-normalization operation is followed by a feed-forward network with an expansion ratio of 2.0. Thus, the hidden representation contains 256 units and is processed using GELU activation:

$${\mathbf{F}}_{cvt}=\mathbf{Z}+{\mathcal{W}}_2\left[\mathrm{GELU}\left({\mathcal{W}}_1\left(\mathrm{LN}(\mathbf{Z})\right)\right)\right],$$

(14)

where $\mathbf{Z}$ denotes the attention-enhanced residual representation, ${\mathcal{W}}_1$ expands the feature dimension from 128 to 256, and ${\mathcal{W}}_2$ projects it back to 128. A dropout rate of 0.02 is applied within both the attention and feed-forward components. Hence, the CvT module combines local inductive bias from convolutional projections with global dependency modelling from single-head self-attention while introducing only one attention block into the lightweight backbone.

3.4.3. Feature Integration and Classification Head

Feature integration in the proposed architecture is performed sequentially rather than through concatenation of two independent CNN and Transformer branches. The CvT block directly refines the $6\times 6\times 128$ MobileNetV4 feature tensor, after which the attention-enhanced features are returned to the remaining MobileNetV4 stage. Four additional UIB blocks are subsequently applied while maintaining the spatial resolution and 128-channel dimensionality. The first two late blocks employ $5\times 5$ middle depthwise convolutions, whereas the final two use $3\times 3$ depthwise convolutions. This arrangement allows the globally refined representation produced by the CvT module to undergo additional convolutional processing before classification.

The resulting feature tensor is first projected from 128 to 960 channels using a $1\times 1$ convolution followed by batch normalization and ReLU activation. Global average pooling then aggregates the $6\times 6$ spatial representation into a global feature descriptor. A subsequent $1\times 1$ convolution expands this representation to 1280 channels with ReLU activation. To reduce overfitting, dropout with a rate of 0.30 is applied before the final classification layer.

The classifier projects the resulting representation to the K student activity classes using a $1\times 1$ convolution. The resulting logits are flattened and transformed into class probabilities through the softmax function:

$$P(y=k\mid \mathbf{I})={\displaystyle \frac{\exp (z_k)}{{\sum }_{j=1}^K\exp (z_j)}},$$

(15)

where $z_k$ denotes the logit associated with the kth activity class and K is the total number of classes.

Accordingly, the complete information flow of the selected hybrid architecture can be summarized as

$$\begin{array}{cc}\hfill {\mathbf{I}}_{192\times 192\times 3}& \to \mathrm{MobileNetV}{4}_{\mathrm{early}/\mathrm{mid}}\to {\mathbf{F}}_{6\times 6\times 128}\to {\mathrm{CvT}}_{1\text{-}\mathrm{head}}\hfill \\ \hfill & \to \mathrm{MobileNetV}{4}_{\mathrm{late}}\to {\mathrm{Conv}}_{1\times 1}^{960}\to \mathrm{GAP}\to {\mathrm{Conv}}_{1\times 1}^{1280}\to \mathrm{Dropout}\to \mathrm{Softmax}.\hfill \end{array}$$

(16)

The proposed design therefore preserves MobileNetV4 as the dominant lightweight feature extractor while introducing global contextual modelling only at a late, computationally inexpensive stage. The subsequent MobileNetV4 blocks further refine the attention-enhanced representation, providing a tightly integrated CNN–attention architecture rather than a simple combination of two independent models.

3.5. Explainable Artificial Intelligence Framework

Explainable Artificial Intelligence (XAI) refers to a set of methods designed to make the predictions of complex ML and DL models more understandable to humans. XAI techniques aim to identify the input features or regions that contribute most strongly to a model decision, thereby improving transparency and interpretability [32].

3.5.1. Gradient-Weighted Class Activation Mapping (Grad-CAM)

Grad-CAM is a gradient-based XAI technique that provides visual explanations for CNN predictions by identifying the spatial regions that contribute most strongly to a target class. It uses the gradients of the class score with respect to the feature maps of a selected convolutional layer to compute the importance of each feature map [33].

For a target class c, the importance weight of feature map k is computed as

$${\alpha }_k^c={\displaystyle \frac{1}{Z}}\sum _i\sum _j{\displaystyle \frac{\partial y^c}{\partial A_{ij}^k}},$$

(17)

where $A_{ij}^k$ denotes the activation at spatial location $(i,j)$ of feature map k, $y^c$ is the score associated with class c, and Z represents the number of spatial locations in the feature map.

The Grad-CAM localization map is then obtained as

$$L_{\mathrm{Grad}\text{-}{\mathrm{CAM}}^c}=\mathrm{ReLU}\left(\sum _k{\alpha }_k^c{A}^k\right),$$

(18)

where the ReLU operation retains the regions that have a positive influence on the prediction of the target class. In the implementation, Grad-CAM was computed from the head_960_relu layer, corresponding to the ReLU activation following the $1\times 1$ convolutional projection to 960 channels. The predicted class was used as the target class for generating each Grad-CAM explanation.

3.5.2. Local Interpretable Model-Agnostic Explanations (LIME)

LIME is a model-agnostic XAI technique that explains an individual prediction by approximating the original complex model with a simple interpretable model in the local neighborhood of the sample being explained [34]. Its objective can be expressed as

$$\xi (\mathbf{x})=\arg \underset{g\in G}{\min }\mathcal{L}\left(f,g,{\pi }_{\mathbf{x}}\right)+\Omega (g),$$

(19)

where f is the original model, g is the interpretable surrogate model, ${\pi }_{\mathbf{x}}$ defines the locality around the input $\mathbf{x}$, $\mathcal{L}$ measures the approximation error, and $\Omega (g)$ controls the complexity of the surrogate model. In this study, image segmentation was performed using the SLIC superpixel algorithm with 60 segments, a compactness value of 10, and a Gaussian smoothing parameter of $\sigma =1$. Each LIME explanation was generated using 600 perturbed samples. The top predicted class was considered, and up to 10 positively contributing superpixels were retained for visualization.

3.5.3. Integrated Gradients (IG)

IGs [35]: An Attribution Method based on Gradients which quantifies the contribution of each feature in the input using the gradients of the model integrated from a baseline reference point ${\mathbf{x}}^{\prime }$ to the actual input $\mathbf{x}$. The attribution of feature i is defined as

$$I{G}_i(\mathbf{x})=(x_i-x_i^{\prime }){\int }_0^1{\displaystyle \frac{\partial F\left({\mathbf{x}}^{\prime }+\alpha (\mathbf{x}-{\mathbf{x}}^{\prime })\right)}{\partial x_i}}d\alpha ,$$

(20)

where $F(\mathbf{x})$ denotes the model output and $\alpha$ controls the interpolation between the baseline and the input. For the implementation, an all-zero image was used as the reference baseline. The integral was approximated using 64 integration steps, corresponding to 65 interpolation points including the two endpoints, with consecutive gradients averaged using the trapezoidal approximation. The attribution map was generated for the predicted class.

3.5.4. SHapley Additive exPlanations (SHAP)

SHAP is an attribution-based XAI method derived from cooperative game theory that quantifies the contribution of each input feature to a model prediction [36]. For image-based classification, SHAP estimates how individual pixels or image regions influence the predicted class by comparing the model output obtained with and without subsets of features. The contribution of feature i is expressed using its Shapley value:

$${\varphi }_i=\sum _{S\subseteq N\setminus \left\{i\right\}}{\displaystyle \frac{\left|S\right|!(|N|-|S|-1)!}{\left|N\right|!}}\left[f(S\cup \{i\})-f(S)\right],$$

(21)

where N represents the complete set of input features, S is a subset of features excluding feature i, and $f(·)$ denotes the model prediction function. Positive SHAP values indicate image regions that support the predicted class, whereas negative values indicate regions that reduce the corresponding class prediction. SHAP explanations were generated using the shap.Explainer interface with an image masker based on Telea inpainting (inpaint_telea). The explanation was computed for the predicted class using a maximum of 250 model evaluations and a batch size of 16.

3.6. Evaluation Metrics

The performance of the proposed model was assessed using several complementary multi-class classification metrics, as summarized in Table 5. Accuracy provides an overall measure of correct predictions, while precision and recall evaluate the reliability of predicted classes and the ability to correctly identify samples from each class. The F1-score combines precision and recall into a single balanced measure. Both macro-averaged and weighted-averaged versions were considered to account for class-wise performance and class-frequency differences. In addition, the confusion matrix was used to analyze class-specific prediction patterns and identify the activities that were most frequently confused.

Table 5. Summary of the evaluation metrics used in this study.

Metric	Description
Accuracy	Overall proportion of correctly classified samples among all predictions.
Macro Precision	Average precision computed independently for each class, giving equal importance to all classes.
Macro Recall	Average recall computed independently for each class, giving equal importance to all classes.
Macro F1-score	Average class-wise F1-score, providing a balanced assessment across all classes.
Weighted Precision	Average precision weighted by the number of samples belonging to each class.
Weighted Recall	Average recall weighted according to the support of each class.
Weighted F1-score	Average F1-score weighted by the number of samples in each class.
Confusion Matrix	Matrix comparing true and predicted class labels to visualize correct classifications and class-specific misclassifications.

3.7. Model Training Strategy

All models were trained under a consistent experimental protocol to ensure a fair comparison. The input image resolution was fixed at $192\times 192$ pixels, with a batch size of 32 and a maximum training duration of 40 epochs. Model optimization was performed using the AdamW optimizer with a peak learning rate of $5\times {10}^{-4}$ and a weight decay coefficient of 0.01. To stabilize the initial optimization stage, a five-epoch linear warm-up was employed, after which the learning rate was progressively reduced following a cosine decay schedule to a minimum value of $1\times {10}^{-6}$. Gradient clipping with a maximum norm of 1.0 was additionally applied to limit excessively large parameter updates.

Categorical cross-entropy was adopted as the classification loss, together with a label-smoothing factor of 0.10 to reduce overconfident predictions and improve model generalization. An $L_2$ regularization coefficient of $1\times {10}^{-5}$ was also applied to the convolutional layers. Training was intentionally conducted for the complete 40 epochs without early stopping. During optimization, model weights were monitored according to the validation loss, and the checkpoint achieving the lowest validation loss was retained. After training, these best-performing validation weights were restored and subsequently used for the final evaluation on the training, validation, and independent test sets.

3.8. Computational Efficiency Metrics

In addition to classification performance, computational efficiency was considered to evaluate the suitability of the investigated architectures for lightweight SAR applications. The primary efficiency indicators were the total number of model parameters, the number of trainable and non-trainable parameters, and the model storage size. Parameter count provides a direct indication of architectural complexity and memory requirements, while the serialized model size reflects the practical storage cost associated with model deployment.

For a model containing P parameters represented using 32-bit floating-point values, the approximate parameter memory requirement is computed as

$$S_{\mathrm{FP}32}={\displaystyle \frac{4P}{{1024}^2}},$$

(22)

where $S_{\mathrm{FP}32}$ is expressed in megabytes and each parameter requires four bytes of storage. In addition to this theoretical estimate, the actual size of the saved .keras model was measured to account for the complete serialized architecture and associated model information. These efficiency measures were considered together with classification performance to assess the trade-off between predictive capability and model complexity.

4. Experimental Results and Discussion

This section presents and discusses the experimental results obtained from the evaluated lightweight architectures and the proposed MobileNetV4–CvT hybrid model. The analysis covers the experimental setup, baseline model comparison, ablation studies, final classification performance, prediction confidence and calibration, computational complexity, and XAI-based interpretation. The results are examined jointly to assess the trade-off between recognition accuracy, model efficiency, generalization, and interpretability for SAR in smart classroom environments.

4.1. Experimental Setup

All experiments were conducted in the Kaggle Notebook environment using two NVIDIA Tesla T4 GPUs. The models were implemented in Python using the TensorFlow and Keras frameworks. To ensure a consistent and fair comparison, all baseline architectures and the proposed MobileNetV4–CvT model were trained under the same experimental conditions. Input images were resized to $192\times 192$ pixels and processed in batches of 32 samples. Training was performed for a maximum of 40 epochs using the AdamW optimizer with a peak learning rate of $5\times {10}^{-4}$ and a weight decay of $0.01$. A five-epoch learning-rate warm-up was applied, followed by cosine decay to a minimum learning rate of $1\times {10}^{-6}$. Gradient clipping with a maximum norm of 1.0 was also employed to improve optimization stability.

Categorical cross-entropy with label smoothing of 0.10 was used as the training objective, while an $L_2$ regularization coefficient of $1\times {10}^{-5}$ was applied to reduce excessive parameter growth. Data augmentation was performed only on the training set and included horizontal flipping, slight rotation, zooming, and contrast adjustment. No early stopping criterion was used, allowing all models to complete the predefined training schedule. Instead, model checkpoints were saved according to the lowest validation loss, and the best-performing weights were restored for final evaluation.

4.2. Performance of Lightweight Baseline Models

Lightweight DL architectures are particularly important for SAR in smart classroom environments, where computational efficiency, memory requirements, and potential deployment on resource-constrained devices must be considered alongside predictive performance.

For this reason, the present study evaluated several recent lightweight architectures representing different design strategies, including MobileNetV4-Conv-S, RepViT-M0.9, SHViT, iFormer-S, and a Lightweight CvT model. The objective was to establish a consistent baseline and identify the architectural characteristics that are most suitable for the proposed hybrid framework. To assess the stability of the evaluated architectures across different random initializations, each model was trained using three independent random seeds (42, 123, and 2026), and the corresponding mean and standard deviation are reported in Table 6.

Table 6. Performance of the evaluated lightweight architectures across three independent training runs using random seeds 42, 123, and 2026.

Model	Accuracy (%)		Macro F1 (%)
	Val.	Test
MobileNetV4-Conv-S	95.05 ± 0.00	95.18 ± 0.27	92.69 ± 0.41
RepViT-M0.9	92.67 ± 0.15	94.98 ± 0.18	92.18 ± 0.52
SHViT	94.76 ± 0.26	93.38 ± 1.77	90.40 ± 2.53
iFormer-S	94.91 ± 0.26	94.75 ± 0.49	92.26 ± 0.54
Lightweight CvT	94.94 ± 0.05	95.07 ± 0.20	92.14 ± 0.30

Across the three independent runs, MobileNetV4-Conv-S achieved a mean test accuracy of 95.18 ± 0.27%, while Lightweight CvT reached 95.07 ± 0.20%. RepViT-M0.9 and iFormer-S obtained mean test accuracies of 94.98 ± 0.18% and 94.75 ± 0.49%, respectively. SHViT showed greater variability across runs, with a test accuracy of 93.38 ± 1.77% and a Macro F1-score of 90.40 ± 2.53%. Overall, the relatively small standard deviations obtained by most architectures indicate stable performance across the three random initializations, while the differences between the highest-performing models remain relatively small.

For a more detailed understanding of the classification performance, Table 7 presents the training, validation, and test results of the best performing run across the three random seeds. As seen from the results, all of the tested baseline models demonstrated an almost perfect training accuracy in the range of 99.40–100.00%, whereas the differences are more visible in validation and test sets. MobileNetV4-Conv-S showed the highest test accuracy of 95.34%, while demonstrating the highest class-balanced metrics in the standalone baselines in terms of Macro Precision of 97.31%, Macro Recall of 91.61%, Macro F1-score of 92.93%, and Weighted F1-score of 94.76%. The second best test accuracy was achieved by Lightweight CvT (95.20%), with the third one by RepViT-M0.9 (95.17%), fourth one by iFormer-S (94.54%), and fifth one by SHViT (91.87%). Regarding validation accuracy, MobileNetV4-Conv-S and iFormer-S achieved the same 95.05%, followed by Lightweight CvT (94.96%), SHViT (94.66%), and RepViT-M0.9 (92.58%).

Table 7. Overall training, validation, and test performance of the evaluated lightweight architectures for the best-performing run among random seeds 42, 123, and 2026.

Model	Accuracy (%)			Loss			Test Classification Metrics (%)
	Train	Val.	Test	Train	Val.	Test	Macro P.	Macro R.	Macro F1	Weighted F1
MobileNetV4-Conv-S	100	95.05	95.34	0.4264	0.6053	0.5950	97.31	91.61	92.93	94.76
RepViT-M0.9	100	92.58	95.17	0.4399	0.7744	0.6594	97.08	91.50	92.76	94.59
SHViT	99.40	94.66	91.87	0.4382	0.5689	0.6264	89.13	87.39	87.86	91.31
iFormer-S	99.42	95.05	94.54	0.4515	0.5835	0.5889	96.24	90.69	91.91	93.97
Lightweight CvT	100	94.96	95.20	0.3404	0.5522	0.5418	96.31	91.41	92.27	94.65

The loss values are another dimension from which the generalization performance of the models is examined. The architecture lightweight CvT had the smallest loss values for training, validation, and testing, with values of 0.3404, 0.5522, and 0.5418, respectively. Another model that had quite stable validation and test losses was iFormer-S, with values of 0.5835 and 0.5889, respectively. SHViT produced a validation loss of 0.5689 and a test loss of 0.6264, whereas RepViT-M0.9 exhibited the highest validation and test losses, reaching 0.7744 and 0.6594, respectively. Overall, MobileNetV4-Conv-S provided the strongest standalone test classification performance for the selected run, whereas Lightweight CvT combined competitive test performance with the lowest loss values. These complementary characteristics motivated the integration of MobileNetV4 and CvT within the proposed hybrid architecture.

The prediction confidence and computational characteristics of the evaluated lightweight architectures are reported in Table 8. RepViT-M0.9 produced the highest overall mean confidence of 95.94%, while Lightweight CvT achieved the highest mean confidence on correct predictions at 96.06%. However, RepViT-M0.9 also maintained a very high confidence of 96.09% for incorrectly classified samples, indicating that some incorrect predictions were made with considerable certainty. MobileNetV4-Conv-S showed similar confidence values for overall, correct, and incorrect predictions, reaching 91.67%, 91.65%, and 91.93%, respectively. However, the SHViT and iFormer-S models had relatively low confidence levels of 66.95% and 72.80%, respectively, in their wrong predictions. The lightweight CvT model still had a high confidence level of 95.56% and a correct prediction confidence of 96.06%, but their confidence in wrong predictions dropped to 85.82%.

Table 8. Prediction confidence and computational characteristics of the evaluated lightweight architectures.

Model	Mean Confidence (%)			Median Latency (ms/Image)	Throughput (FPS, Batch = 1)	Peak GPU Memory (MiB)	Model Size (MB)	Parameters
	Overall	Correct	Wrong
MobileNetV4-Conv-S	91.67	91.65	91.93	5.142	172.46	124.78	29.29	2,521,958
RepViT-M0.9	95.94	95.93	96.09	13.221	75.27	256.75	55.94	4,803,558
SHViT	87.62	89.45	66.95	9.084	109.45	243.01	59.63	5,164,990
iFormer-S	87.62	88.48	72.80	8.572	115.43	351.46	83.35	7,233,870
Lightweight CvT	95.56	96.06	85.82	6.219	159.34	55.25	7.26	592,182

Note: Confidence is the maximum predicted class probability; model size refers to the saved .keras file.

Inference efficiency and model complexity were also found to differ in distinct ways. For instance, MobileNetV4-Conv-S had the minimum median inference latency of 5.142 ms/image and maximum throughput of 172.46 FPS when the batch size was 1. The lightweight CvT model was efficient in terms of inference latency with 6.219 ms/image latency and throughput of 159.34 FPS. In terms of memory consumption and architectural compactness, Lightweight CvT was the most efficient model, requiring only 55.25 MiB of peak GPU memory, 592,182 parameters, and 7.26 MB of storage. MobileNetV4-Conv-S contained 2,521,958 parameters and required 29.29 MB of storage. RepViT-M0.9 and SHViT contained 4,803,558 and 5,164,990 parameters, corresponding to 55.94 MB and 59.63 MB, respectively. iFormer-S was the largest evaluated architecture, containing 7,233,870 parameters, requiring 83.35 MB of storage, and exhibiting the highest peak GPU memory consumption of 351.46 MiB.

Taken together with the classification results reported in Table 6 and Table 7, MobileNetV4-Conv-S and Lightweight CvT emerged as particularly promising from complementary perspectives. MobileNetV4-Conv-S achieved the strongest standalone test classification performance in the selected best-performing run and the fastest measured inference among the baseline architectures, whereas Lightweight CvT provided the most compact architecture, the lowest peak GPU memory consumption, and competitive classification performance. This complementarity motivated the development of the proposed MobileNetV4–CvT hybrid model, with the objective of combining efficient convolutional feature extraction with lightweight attention-based contextual modelling.

4.3. Ablation Study

An ablation study was conducted to systematically examine the influence of the main architectural choices of the proposed MobileNetV4–CvT hybrid model. The analysis focused on the placement of the CvT attention module, the number of CvT blocks, the number of attention heads, and the input image resolution. These experiments were used to identify the configuration that provides the most favorable balance between validation performance and architectural simplicity before evaluating the final hybrid model. An ablation study was conducted to systematically examine the influence of the main architectural choices of the proposed MobileNetV4–CvT hybrid model. The analysis focused on the placement of the CvT attention module, the number of CvT blocks, the number of attention heads, and the input image resolution. These experiments were used to identify the configuration that provides the most favorable balance between validation performance and architectural simplicity before evaluating the final hybrid model. However, it is worth mentioning that the results of the ablation studies were achieved through different setups in order to see the effect of each parameter separately. This means that the performance measures obtained during the ablation cannot be directly compared to the test results of the final model.

4.3.1. Effect of Attention Placement

The placement of the CvT attention module has an important influence on the performance of the proposed hybrid architecture, since it determines the stage at which global contextual information is introduced into the MobileNetV4 feature extraction process. As illustrated in Figure 3, three placement strategies were investigated: Penultimate, Both, and Final. The Penultimate and Both configurations achieved the highest validation accuracy of 95.05% and the same Macro F1-score of 81.66%. However, the Penultimate configuration yielded a lower validation loss of 0.6187 compared with 0.6330 for the Both configuration, indicating more stable validation behavior. In contrast, positioning the CvT module only at the Final stage reduced the validation accuracy to 94.43% and the Macro F1-score to 81.10%, with a validation loss of 0.6194. Considering the three evaluation criteria jointly, the Penultimate placement provided the most favorable balance between classification performance and validation stability. Therefore, this configuration was selected for the final MobileNetV4–CvT hybrid model.

Figure 3. Performance of the proposed hybrid model based on the placement of the CvT attention module.

4.3.2. Effect of the Number of CvT Attention Blocks

The number of CvT attention blocks is an important architectural parameter, as increasing the number of blocks can enhance contextual feature modelling but also increases the complexity of the hybrid network. As shown in Figure 4, using one or two CvT blocks resulted in the same validation accuracy of 95.05% and Macro F1-score of 81.66%. The two-block configuration achieved the lowest validation loss of 0.6106, compared with 0.6129 for a single block. However, this small reduction in loss did not translate into an improvement in either validation accuracy or Macro F1-score. Increasing the number of blocks to three slightly reduced the validation accuracy to 95.01% and the Macro F1-score to 81.18%, while increasing the validation loss to 0.6180. Considering the comparable predictive performance of the one- and two-block configurations, together with the lower architectural complexity of the former, a single CvT attention block was selected for the final proposed hybrid model. This configuration provides an appropriate balance between contextual representation capability, classification performance, and computational efficiency.

Figure 4. Performance of the proposed hybrid model based on the number of CvT attention blocks.

4.3.3. Effect of Attention Head Configuration

The number of attention heads plays an important role in determining how the CvT module distributes and captures contextual information within the extracted feature representation. As illustrated in Figure 5, the single-head configuration achieved the strongest validation performance, with an accuracy of 95.05%, a Macro F1-score of 81.66%, and the lowest validation loss of 0.6104. Increasing the number of attention heads to two reduced the validation accuracy to 94.70% and the Macro F1-score to 81.34%, while increasing the validation loss to 0.6190. A further increase to four heads resulted in a more noticeable performance decrease, reaching 93.82% validation accuracy and 80.54% Macro F1, together with the highest validation loss of 0.6260. These results indicate that increasing the number of attention heads does not provide additional benefits for the considered feature representation. Based on the three validation criteria, the single-head configuration provides the most favorable performance and the lowest architectural complexity.

Figure 5. Performance of the proposed hybrid model based on the number of attention heads in the CvT module.

4.3.4. Effect of Input Image Resolution

Input image resolution is a key factor in the performance of the proposed hybrid model, as it directly affects the amount of spatial detail available for feature extraction while also influencing computational cost. As shown in Figure 6, the $192\times 192$ configuration clearly achieved the strongest validation results, reaching an accuracy of 98.45% and a Macro F1-score of 96.83%, together with the lowest validation loss of 0.5019. In comparison, both $160\times 160$ and $224\times 224$ produced a validation accuracy of 95.05% and a Macro F1-score of 81.66%. Their validation losses were also higher, at 0.6123 and 0.6103, respectively. These results indicate that increasing the resolution beyond $192\times 192$ did not provide further improvement, while the lower resolution was insufficient to preserve the most discriminative visual information. Therefore, an input resolution of $192\times 192$ was selected for the final proposed hybrid model, as it provided the best overall balance between classification performance and input complexity.

Figure 6. Performance of the proposed hybrid model based on the input image resolution.

4.4. Performance of the Proposed MobileNetV4–CvT Model

To assess the stability of the proposed MobileNetV4–CvT model across different random initializations, the hybrid architecture was trained using three independent random seeds under the same experimental protocol. Across these runs, the model achieved a mean validation accuracy of 95.05 ± 0.15%, a mean test accuracy of 95.20 ± 0.17%, and a Macro F1-score of 92.62 ± 0.40%. These results indicate consistent performance across the three runs. The detailed results corresponding to the best-performing seed are presented below.

4.4.1. Proposed Model Learning Performance and Test

The learning behavior of the proposed MobileNetV4–CvT hybrid model shows a rapid and stable convergence during training. In the first few epochs, the model quickly captures the most discriminative visual patterns, leading to a sharp increase in training accuracy and a simultaneous decrease in training loss. As training progresses, the optimization becomes more gradual, with the training accuracy approaching saturation and reaching 99.95%, while the corresponding training loss decreases to 0.4421. The validation performance follows the same overall trend, although with moderate fluctuations, and stabilizes at an accuracy of 95.05% and a loss of 0.6176. These training dynamics are illustrated in Figure 7, where the accuracy and loss curves show that the model converges without a sustained degradation in validation performance.

Figure 7. Training and loss curves of the proposed MobileNetV4–CvT hybrid model.

The difference between the final training and validation accuracy indicates a moderate generalization gap, but the validation curve remains relatively stable during the later epochs rather than continuously decreasing. This behavior suggests that the model learns highly discriminative representations while maintaining reasonable generalization to unseen samples. This observation is further supported by the test performance, where the model achieves 95.20% accuracy with a loss of 0.6088. The close agreement between validation and test results indicates that the learned representation remains consistent beyond the training data and does not exhibit severe overfitting.

The class-wise prediction behavior of the proposed MobileNetV4–CvT model reveals strong discrimination for most of the evaluated student activities. Among the 3003 test samples, 2859 are correctly classified, corresponding to the overall test accuracy of 95.20%. As illustrated by the confusion matrix in Figure 8, the model correctly recognizes all test samples belonging to Eating_in_classroom, HandRaise, Reading_Book, Sitting_on_Desk, and Writting_on_Textbook, with 728, 350, 782, 333, and 532 correct predictions, respectively. This result indicates that the learned representation effectively separates these activities despite the visual variability that may occur in classroom environments.

Figure 8. Confusion matrix of the proposed MobileNetV4–CvT hybrid model on the test set.

The main source of classification error is concentrated in the Arguing class. Out of 278 Arguing samples, 134 are correctly recognized, while 140 are misclassified as Eating_in_classroom and four as HandRaise. This confusion suggests that some Arguing scenes may share visual characteristics with other interaction-based activities, particularly similar body orientations, hand movements, seated configurations, or interactions between multiple students. In contrast, no samples from the remaining five classes are incorrectly assigned to Arguing. Overall, the highly diagonal structure of the confusion matrix confirms the strong class-wise performance of the proposed model, while also identifying Arguing as the principal challenging class and an important target for further improvement.

The Test performance of the proposed MobileNetV4–CvT model confirms its ability to achieve a strong balance between predictive effectiveness and computational efficiency. As summarized in Table 9, the model attains a test accuracy of 95.20% with a loss of 0.6088. In addition to the overall accuracy, the model achieves a Macro Precision of 97.12%, Macro Recall of 91.37%, and Macro F1-score of 92.62%, indicating robust performance across the evaluated activity classes. The corresponding weighted metrics remain similarly high, with a Weighted Precision of 95.96%, Weighted Recall of 95.20%, and Weighted F1-score of 94.57%.

Table 9. Training and test performance of the proposed MobileNetV4–CvT model.

Model	Training Performance		Test Performance		Test Classification Metrics (%)
	Acc. (%)	Loss	Acc. (%)	Loss	Macro P.	Macro R.	Macro F1	Weighted P.	Weighted R.	Weighted F1
MobileNetV4–CvT	99.95	0.4421	95.20	0.6088	97.12	91.37	92.62	95.96	95.20	94.57

Note: Acc. denotes accuracy, P. denotes precision, and R. denotes recall.

The difference between the macro and weighted metrics is mainly influenced by the lower recognition performance observed for the Arguing class, as also reflected in the confusion matrix analysis. Nevertheless, the high Macro Precision indicates that the model produces relatively few false-positive predictions across the six activity categories, while the Macro Recall and Macro F1-score show that a strong class-balanced performance is maintained despite the difficulty associated with the challenging class. From a computational perspective, the proposed architecture contains 2,659,430 trainable and non-trainable parameters in total and requires only 30.92 MB when stored in the .keras format. These characteristics indicate that the proposed hybrid architecture preserves a relatively compact model size while maintaining high recognition performance, supporting its suitability for resource-constrained smart classroom applications.

The computational characteristics of the proposed MobileNetV4–CvT model are summarized in Table 10. The model achieved a median inference latency of 5.601 ms per image and a throughput of 177.17 FPS at a batch size of 1. In addition, the model required 49.67 MiB of peak GPU memory, while containing 2,659,430 trainable parameters and occupying 30.92 MB of storage. These results indicate that the proposed architecture maintains a compact computational profile while supporting high-throughput inference.

Table 10. Computational characteristics of the proposed MobileNetV4–CvT model.

Model	Median Latency (ms/Image)	Throughput (FPS, Batch = 1)	Peak GPU Memory (MiB)	Parameters	Model Size (MB)
MobileNetV4–CvT	5.601	177.17	49.67	2,659,430	30.92

Note: Model size refers to the saved .keras file.

4.4.2. Prediction Confidence and Calibration Analysis

The reliability of the proposed MobileNetV4–CvT model was further examined by analyzing both prediction confidence and probability calibration. The model achieves a mean confidence of 92.11% across all test predictions, while correctly classified samples exhibit a slightly higher average confidence of 92.23%. In contrast, the mean confidence associated with incorrect predictions decreases to 89.81%, as illustrated in Figure 9a. This difference indicates that the model generally assigns stronger confidence to correct decisions, although the relatively high confidence observed for some erroneous predictions shows that confidence alone cannot always distinguish between reliable and incorrect classifications.

Figure 9. Prediction confidence and calibration analysis of the proposed MobileNetV4–CvT hybrid model: (a) mean confidence for all, correct, and incorrect predictions, and (b) reliability diagram comparing observed accuracy with mean prediction confidence.

The calibration behavior was subsequently evaluated using the reliability diagram shown in Figure 9b. The proposed model achieves an ECE of 0.0319, corresponding to an average calibration discrepancy of 3.19%. Predictions in the dominant high-confidence region remain relatively close to the ideal calibration line, indicating good agreement between predicted confidence and observed accuracy for most test samples. Larger deviations are visible in some lower and intermediate confidence intervals, where the observed accuracy differs considerably from the predicted confidence. These fluctuations should be interpreted cautiously, since such confidence intervals may contain substantially fewer samples than the high-confidence region.

Despite the model having a reasonable ECE of 3.19%, the mean confidence of 89.81% on incorrect predictions is still a significant drawback of the model, especially when used in practical applications, like smart classrooms, as such high confidence misclassifications may result in an erroneous understanding of student behavior. In addition, the fact that the label smoothing during training was set to 0.10 may affect the confidence of the predictions, yet no independent analysis of its effect has been conducted. Future work will therefore examine different label-smoothing settings and explore post-hoc calibration techniques, such as temperature scaling, along with confidence-based rejection mechanisms to improve prediction reliability and reduce the impact of overconfident classification errors in practical classroom environments.

4.5. Explainability Analysis of the Proposed Model

To complement the quantitative evaluation, an explainability analysis was conducted to examine the visual evidence underlying the predictions of the proposed MobileNetV4–CvT model. Four complementary XAI techniques, namely Grad-CAM, LIME, IGs, and SHAP, were applied to the same representative test samples, as illustrated in Figure 10. The selected examples include both correctly classified and misclassified images, allowing the analysis to reveal the regions supporting reliable decisions as well as the visual cues that may contribute to incorrect predictions.

Figure 10. Comparative explainability analysis of four representative test samples using multiple XAI methods. Each row contains the same four samples, beginning with the original images and followed by Grad-CAM, LIME, IGs, and SHAP visualizations.

It should be emphasized that the following XAI results are intended as qualitative illustrations of the model’s decision behavior rather than as a quantitative validation of explanation reliability, faithfulness, or stability.

Grad-CAM highlights the spatial regions that contribute to the predicted class. As shown in Figure 10, the correctly classified Arguing samples exhibit relatively diffuse activation patterns, with the strongest responses occurring mainly around the image borders and surrounding background rather than being consistently concentrated on the students. Some activation also overlaps with the student interaction regions, but the maps indicate that contextual scene information contributes substantially to the prediction. A similar trend is also seen among the misclassified samples where the marked areas continue to be distributed across background and contextual areas rather than being localized to behaviorally significant body parts. This implies that the algorithm depends partly on visual context and this might account for the confusion among visually similar classroom behaviors.

LIME offers a local explanation by finding image segments that significantly impact a specific prediction. Superpixels picked out by the algorithm often correspond to areas including the students’ faces, arms, upper body parts, and interaction zones. For the correctly recognized samples, these regions correspond well to the expected discriminative cues. In the misclassified examples, LIME additionally highlights surrounding and background regions, indicating that the classifier may partially rely on contextual patterns shared across visually similar classroom activities.

IGs offers a more fine-grained attribution by estimating the contribution of individual pixels to the model output. The corresponding maps emphasize detailed structures around the students’ arms, faces, bodies, and nearby objects. For correctly classified samples, the strongest attributions are concentrated around the interaction-related regions, whereas in the misclassified cases, noticeable attribution is also observed around the desk and lower parts of the scene. This indicates that the prediction may be influenced by a combination of human posture and surrounding contextual information.

SHAP provides an additional attribution perspective by estimating the contribution of different image regions to the final prediction. The resulting explanations generally emphasize the students’ heads, upper bodies, arms, and interaction zones. For correctly classified images, several of these visually important regions are also identified by the other XAI methods, although the degree of agreement varies across samples and methods. In contrast, the misclassified samples show more dispersed influential regions, including areas around the desk and background. Overall, the four XAI methods provide complementary qualitative views of the visual evidence associated with the model predictions. However, visual agreement among these methods should not be interpreted as proof of explanation faithfulness or stability. A dedicated quantitative evaluation of XAI reliability, for example, using perturbation-based faithfulness or stability measures, is therefore left for future work.

4.6. Discussion and Comparison with Baseline Models

The comparative evaluation provides a broader view of the trade-off between classification performance, model complexity, and prediction reliability across the investigated lightweight architectures. Among the standalone baselines, MobileNetV4-Conv-S achieved the strongest test performance, with an accuracy of 95.34%, a Macro F1-score of 92.93%, and a Weighted F1-score of 94.76%. The proposed MobileNetV4–CvT model achieved a slightly lower test accuracy of 95.20%, corresponding to a difference of 0.14 percentage points, together with a Macro F1-score of 92.62% and a Weighted F1-score of 94.57%. Therefore, the introduction of CvT-based attention did not improve the overall test accuracy beyond the original MobileNetV4-Conv-S backbone. However, the hybrid model remained highly competitive while incorporating explicit contextual modelling through self-attention and maintaining a lightweight architecture.

Compared with the other attention-based and hybrid baselines, the proposed model shows a favorable performance profile. Its test accuracy of 95.20% is identical to that of Lightweight CvT and higher than RepViT-M0.9, iFormer-S, and SHViT, which achieved 95.17%, 94.54%, and 91.87%, respectively. The performance difference corresponds to 0.03 percentage points over RepViT-M0.9, 0.66 percentage points over iFormer-S, and 3.33 percentage points over SHViT. A similar trend is observed for the Macro F1-score. The proposed model achieves 92.62%, compared with 91.91% for iFormer-S, 92.76% for RepViT-M0.9, 92.27% for Lightweight CvT, and 87.86% for SHViT. These results indicate that the proposed architecture provides competitive class-balanced recognition while using a considerably smaller architecture than several of the evaluated lightweight baseline models. Nevertheless, RepViT-M0.9 and MobileNetV4-Conv-S achieve slightly higher Macro F1-scores, indicating that the integration of CvT attention does not consistently improve class-balanced performance across all baseline architectures.

The computational comparison further highlights this trade-off. The proposed MobileNetV4–CvT model contains 2,659,430 parameters and requires 30.92 MB of storage. This represents only a modest increase relative to MobileNetV4-Conv-S, which contains 2,521,958 parameters and occupies 29.29 MB. In contrast, RepViT-M0.9, SHViT, and iFormer-S require approximately 4.80, 5.16, and 7.23 million parameters, respectively, with corresponding model sizes of 55.94, 59.63, and 83.35 MB. Consequently, the proposed hybrid achieves a higher test accuracy than iFormer-S while using approximately 63.24% fewer parameters and a substantially smaller model size. It also remains considerably more compact than RepViT-M0.9 and SHViT. Lightweight CvT remains the smallest architecture, with only 592,182 parameters and a size of 7.26 MB, although its substantially lower parameter count is accompanied by a slightly lower Macro F1-score, while its test accuracy remains identical to that of the proposed hybrid model. Furthermore, the proposed model achieves a throughput of 177.17 FPS with a median inference latency of 5.601 ms/image, demonstrating competitive inference efficiency under the evaluated GPU conditions.

The confidence analysis provides an additional perspective on the behavior of the evaluated models. Lightweight CvT and RepViT-M0.9 produce the highest mean confidence on correct predictions, at 96.06% and 95.93%, respectively. On the other hand, RepViT-M0.9 shows high mean confidence of 96.09% on incorrect predictions as well. The designed hybrid method has 92.23% of mean confidence on correct predictions and 89.81% of mean confidence on incorrect predictions. Even though the confidence of the mistakes is still quite high, the calibration evaluation produces an ECE of 3.19%. Therefore, it can be concluded that the predicted probabilities are well-calibrated according to the actual accuracy. Calibration results provide additional insight into the problem, compared to typical classification evaluation metrics. It proves that predictive confidence should be taken into account along with calibration results.

In terms of the practical implementation in a classroom environment, the relatively high mean confidence of 89.81% for misclassified data is a crucial drawback because highly confident wrong predictions will likely decrease the reliability of an automatic monitoring system. Thus, it is necessary to note that confidence scores cannot be analyzed in isolation and need additional verification procedures in case of uncertain predictions or predictions with serious implications in classroom environments. It is important to mention that label smoothing was applied during training, which is an approach that affects the distribution of probabilities in order to prevent overly certain class probabilities. Despite the fact that the low ECE shows good calibration at the dataset level, it is important to research the connection of label smoothing and confidence for misclassified data.

Class-wise analysis shows that the residual mistakes are very localized and do not spread evenly among six activities. The five classes such as Eating_in_classroom, HandRaise, Reading_Book, Sitting_on_Desk, and Writting_on_Textbook are perfectly recognized by the system on test set, while most of the mistakes come from the Arguing class. Specifically, 140 Arguing instances are classified as Eating_in_classroom and four as HandRaise. This fact implies that the problem lies in particular visual ambiguity of the activity, but not in the general inability of the architecture to discriminate the activities. The XAI analysis further supports the investigation of this issue by showing how body posture, interaction regions, and surrounding contextual information influence both correct and incorrect predictions.

Overall, the experimental findings show that the proposed MobileNetV4–CvT model should not be interpreted as universally outperforming every individual baseline. MobileNetV4-Conv-S remains the strongest standalone model in terms of test accuracy and Macro F1-score, while Lightweight CvT provides the lowest computational complexity. Instead, the proposed architecture offers a compromise between these two characteristics by preserving most of the predictive capability of MobileNetV4 while introducing lightweight attention-based contextual modelling with limited additional complexity. Compared with larger lightweight architectures such as RepViT-M0.9, SHViT, and iFormer-S, the proposed model maintains competitive or higher classification performance with substantially fewer parameters. Together with its calibration assessment and multi-method XAI analysis, these characteristics support the use of the proposed hybrid architecture as a compact and interpretable approach for SAR in smart classroom environments. However, more extensive statistical evaluation using additional independent runs, together with evaluation on embedded hardware, would further strengthen the assessment of the statistical robustness and practical deployment characteristics of the proposed architecture.

It is important to emphasize that the reported test accuracy of 95.20% reflects performance on the six selected student activity classes from the EduNet dataset and should not be interpreted as evidence of generalization to all classroom activities or environments. Furthermore, the original video-based recognition task was reformulated as single-frame image classification, which excludes temporal information that may be essential for distinguishing visually similar student behaviors. Although video-level partitioning was adopted to reduce information leakage between the training, validation, and test sets, the evaluation remains limited to a single dataset and a restricted set of activities. Consequently, the reported results demonstrate the effectiveness of the proposed model under the considered experimental conditions, while its generalization to more diverse classroom scenarios, additional activities, and video-based recognition tasks requires further investigation.

Image-based student behavior recognition using EduNet has previously been investigated through a multi-stage framework in which video frames were first extracted and then processed independently using Viola–Jones detection, template matching, silhouette and skeleton feature extraction, and a Genetic Algorithm classifier. This approach achieved an average accuracy of 85.7% on EduNet, but did not report model parameter count or storage size and was not explicitly designed as a lightweight end-to-end DL architecture [37]. In comparison, the proposed MobileNetV4–CvT model achieves a test accuracy of 95.20%, a Macro F1-score of 92.62%, and a Weighted F1-score of 94.57%, while requiring only 2.66 million parameters and 30.92 MB of storage. Its main advantage lies in combining efficient convolutional feature extraction with lightweight self-attention, enabling the model to learn both local visual patterns and broader contextual relationships directly from classroom images. The proposed framework also extends beyond classification performance by incorporating calibration analysis and complementary XAI methods, thereby providing a more compact, interpretable, and deployment-oriented solution for image-based SAR.

It should be noted that these numerical results are not directly comparable because the previous EduNet-based study employs a different subset of activity classes and a different recognition methodology. Moreover, the original EduNet study evaluated all 20 student- and teacher-centered classes using a video-based I3D–ResNet-50 model and reported an overall accuracy of 72.3% for the two-stream I3D configuration [26]. In contrast, the present study addresses image-based recognition of six selected student activity classes using individual sampled frames. Therefore, the reported differences should be interpreted in the context of the different class subsets, input modalities, and experimental protocols rather than as a direct performance comparison.

5. Limitations and Future Work

Although the proposed MobileNetV4–CvT framework achieves competitive recognition performance while maintaining relatively low computational complexity, several limitations should be considered when interpreting the results. These limitations also provide directions for extending the present work toward more robust and deployment-oriented SAR systems.

• Limited number of activity classes: The present study focuses on six student-centered activities selected from the EduNet dataset. Although these classes represent meaningful classroom behaviors, they do not cover the full diversity of activities that may occur in real educational environments, such as sleeping, talking, mobile-phone use, teacher interaction, and collaborative group activities. Future work will extend the classification task to additional EduNet categories and other classroom behaviors to examine whether the proposed MobileNetV4–CvT architecture maintains its performance as the number and visual similarity of classes increase.
• Frame-based activity recognition: The current approach performs recognition from individual video frames and therefore does not explicitly model temporal information between successive observations. This may limit the discrimination of activities that have similar static appearances but different motion patterns. Future experiments will incorporate short frame sequences and lightweight temporal modelling approaches, such as Temporal Shift Modules, LSTM-based layers, temporal attention, and efficient video Transformers, and compare their performance with the current frame-based model [9,38].
• Dataset-dependent evaluation: The experimental evaluation is based on a single classroom activity dataset. Although video-level partitioning was used to prevent information leakage, variations in camera viewpoint, illumination, classroom layout, student appearance, and recording conditions may influence model generalization. Future work will therefore include cross-dataset and external-dataset evaluation, in which the model is trained on EduNet and tested or fine-tuned on other classroom activity datasets to measure its robustness under domain changes [39].
• Class-specific recognition difficulty: The confusion matrix shows that Arguing remains the most challenging activity, with several samples being misclassified as Eating_in_classroom. This indicates that visually alike body postures, gestures, and interactions are challenging to classify based on appearance data. Future research directions would include exploring class-aware data augmentation [40], hard example mining [41], balanced sampling [42], focal loss [43] and the use of pose keypoints and/or person-interaction features [44].
• Limited deployment and quantitative explainability assessment: The proposed model has currently been evaluated in terms of parameter count, model storage size, and computational complexity, while its practical performance on embedded hardware has not yet been investigated. In particular, inference latency, throughput, memory consumption, and energy efficiency remain to be evaluated under real-device deployment conditions. Moreover, the current XAI analysis based on Grad-CAM, LIME, IGs, and SHAP is qualitative and is intended to illustrate the visual evidence associated with the model predictions rather than to quantitatively validate the faithfulness or stability of the explanations. Future work will therefore focus on deploying the proposed model on real edge devices, such as NVIDIA Jetson platforms [45], to evaluate latency, FPS, memory usage, and power consumption. In addition, quantitative XAI assessment will be conducted using faithfulness, deletion/insertion, stability, and localization-based metrics to evaluate the consistency and reliability of the generated explanations [46,47].

6. Conclusions

This study investigated lightweight and explainable deep learning for student activity recognition (SAR) in smart classroom environments. Several recent efficient architectures, including MobileNetV4-Conv-S, RepViT-M0.9, SHViT, iFormer-S, and Lightweight CvT, were first evaluated under a unified experimental setting, after which a MobileNetV4–CvT hybrid model was developed to combine efficient convolutional feature extraction with lightweight contextual modelling.

The proposed hybrid model achieved a test accuracy of 95.20%, a Macro Precision of 97.12%, a Macro Recall of 91.37%, a Macro F1-score of 92.62%, and a Weighted F1-score of 94.57%. It contained 2.66 million parameters with a saved model size of 30.92 MB. Although MobileNetV4-Conv-S achieved the strongest standalone performance, with a test accuracy of 95.34%, the proposed MobileNetV4–CvT model remained highly competitive while incorporating lightweight contextual self-attention, calibration assessment, and complementary XAI-based interpretation. The main limitation of the current work is its frame-based evaluation on six activity classes from a single dataset, with the Arguing class remaining the most challenging category. In future research, attention will be paid to temporal modeling, more extensive evaluations involving multiple datasets, better disambiguation of activities, quantified XAI evaluation, and implementation on limited-resource edge devices or cameras.