FGD-Net: A Fine-Grained Gated Detail Network for Individual Student Behavior Detection in Classrooms

Abstract

Classroom behavior detection plays an important role in intelligent education by providing objective visual evidence for the analysis of learning engagement, classroom interaction assessment, and teaching evaluation. However, accurate behavior detection for individual students in real classroom environments remains challenging due to subtle behavior-related cues, visually similar action categories, occlusion, and complex background interference. To address these challenges, this paper proposes a Fine-grained Gated Detail Network (FGD-Net) for the fine-grained detection of individual student behaviors in classroom scenes. The proposed network improves behavior representation from three complementary aspects. Firstly, a Fine-grained Dynamic Recalibration Convolution Block (FDRC) is designed to enhance behavior-sensitive local regions, such as hands, arms, heads, and upper-body postures. Secondly, a Multi-path Gated Context Aggregation Block (MGCA) is introduced to aggregate complementary contextual information from multiple feature paths, thereby strengthening the semantic representation of visually similar classroom behaviors. Thirdly, a Shift-guided Detail Reconstruction Upsampling Block (SDRU) is developed to alleviate spatial detail loss during multi-scale feature fusion and improve the reconstruction of small-scale behavior-related cues. Extensive experiments are conducted on the SCB5 and SCB3 classroom behavior detection datasets. The experimental results show that FGD-Net achieves better detection performance than recent detectors on both datasets. These results demonstrate the potential of FGD-Net as an efficient visual perception model for scalable classroom video analytics and intelligent education applications.

1. Introduction

With the rapid development of smart education and visual learning analytics, classroom behavior detection has become an important research topic for understanding teaching and learning processes [1]. Student classroom behaviors, such as hand raising, reading, writing, and listening, provide intuitive visual cues for assessing learning engagement, classroom participation, and teacher–student interaction [2,3]. However, traditional classroom observation mainly relies on manual monitoring or post hoc video analysis [4], which is time-consuming, subjective, and difficult to use to provide timely feedback in large-scale classroom scenarios. To overcome these limitations, computer vision-based methods have been increasingly introduced to automatically perceive and analyze the learning-related states of students from classroom images or videos [5].

From a data-processing perspective, networked classroom cameras can generate continuous, high-volume, and unstructured visual streams. Transforming these visual data into structured behavior events is an important perceptual stage for scalable learning analytics and cognitive classroom systems. Such applications require detection models to achieve not only accurate behavior recognition but also compact computation and real-time processing capability so that classroom video streams can be analyzed efficiently under practical resource constraints.

Existing computer vision-based methods for classroom behavior analysis can be broadly grouped into three categories: video action recognition-based methods [6], pose estimation-based methods [7], and object detection-based methods [8,9]. Video action recognition methods typically use temporal cues from consecutive frames to model dynamic behavior patterns [10]. However, these methods usually require continuous video clips, dense temporal annotations, and relatively high computational costs, which limits their flexibility in real-time classroom applications [11]. Pose estimation-based methods represent student behaviors using human skeletal joints and provide interpretable structural information. However, their effectiveness strongly depends on accurate keypoint localization. In crowded classroom scenes, body parts are often occluded by desks, chairs, or nearby students, which makes pose estimation less reliable for fine-grained behavior detection [12]. In contrast, object detection-based methods provide a more direct and deployment-friendly solution by locating individual students and classifying the corresponding behavior of each student in classroom images or video frames.

Although object detection-based methods are more suitable for real-time classroom behavior analysis, they still face several task-specific challenges in complex classroom scenes. First, discriminative behavior-related regions, such as hands, arms, heads, and upper-body postures, usually occupy small spatial areas and produce subtle visual responses, making them vulnerable to occlusion by nearby students, desks, chairs, and cluttered backgrounds [13]. Second, visually similar categories, such as reading and writing, require multi-granularity semantic context, while insufficient contextual interaction may lead to ambiguous predictions [14]. Third, fine behavior-related details may be weakened during top-down multi-scale feature fusion, especially when low-resolution semantic features are combined with high-resolution localization features [15,16]. Therefore, an effective classroom behavior detector should simultaneously enhance local discriminative cues, aggregate behavior-related contextual semantics, and reconstruct fine spatial details during feature fusion.

To address these challenges, this paper proposes FGD-Net, an object detection-based network designed to improve fine-grained individual student behavior detection in complex classroom scenes. The present study focuses on the behavior categories provided by the selected classroom behavior detection benchmarks, while broader classroom behavior taxonomies are discussed in Section 2. FGD-Net improves the detection framework from three complementary perspectives: local discriminative region enhancement, multi-scale contextual semantic aggregation, and spatial detail reconstruction during feature fusion. The main contributions of this work are summarized as follows:

• We propose FGD-Net, an object detection-based model for detecting and classifying fine-grained behaviors of individual students in complex classroom environments. The proposed model improves the perception of subtle behavior cues and enhances detection robustness under background interference, occlusion, and scale variation.
• We introduce a Fine-grained Dynamic Recalibration Convolution Block (FDRC) to enhance behavior-sensitive local regions, such as hands, arms, heads, and upper-body postures. By dynamically recalibrating local feature responses, FDRC improves the representation of subtle behavior cues that are easily affected by classroom interference.
• We introduce a Multi-path Gated Context Aggregation Block (MGCA) to strengthen multi-granularity semantic interaction and contextual representation. This block helps distinguish visually similar classroom behaviors by aggregating complementary information from different feature paths.
• We introduce a Shift-guided Detail Reconstruction Upsampling Block (SDRU) to alleviate spatial detail loss during top-down feature fusion. SDRU improves the reconstruction of fine spatial details and enhances the detection of small-scale behavior-related regions.

2. Related Works

2.1. Classroom Behavior Analysis

Classroom behavior analysis aims to automatically perceive the learning states of students and classroom interactions from visual data, thereby providing objective support for teaching evaluation and classroom management. Existing studies have explored various visual cues and modeling strategies for this task. Trabelsi et al. [17] developed a real-time attention monitoring system based on YOLOv5 variants and reported an average accuracy of 76%. Shou et al. [18] modeled student actions from classroom time-series images using an improved AIA network, achieving 92% recognition accuracy and an mAP of 81.3% on their student behavior dataset. To address low-resolution visual observations, Zhang et al. [19] incorporated super-resolution into classroom behavior recognition and reported an accuracy of 91.0%, with ${\mathrm{mAP}}_{50}$ and ${\mathrm{mAP}}_{50:95}$ values of 91.5% and 70.0%, respectively. Jia and He [20] further combined object detection with pose estimation to exploit both appearance and structural cues, achieving an mAP of 82.1% in complex classroom environments. Beyond visual classroom environments, the automatic detection of affective and cognitive engagement from online learning discussions has also been investigated for learning achievement prediction [21].

Existing classroom behavior analysis research covers a substantially broader range of observable student activities than the three categories considered in the present experiments. In addition to hand raising, reading, and writing, previous studies have investigated behaviors such as listening, note-taking, standing, answering questions, discussing or talking, looking around, turning around, bowing the head, leaning on the desk, sleeping, yawning, eating or drinking, and using mobile phones or computers. These behaviors reflect different aspects of learning activity, classroom participation, body posture and orientation, peer interaction, and potentially off-task behavior. The specific behavior taxonomy varies across datasets and application scenarios, indicating that comprehensive classroom behavior analysis generally requires a richer set of behavior categories than those available in current small-scale detection benchmarks.

Although these methods have advanced intelligent classroom analysis, reliable behavior recognition in real classroom environments remains challenging due to dense student distributions, occlusion, low-resolution targets, and subtle differences among behavior categories.

2.2. Object Detection for Classroom Behavior Detection

Object detection-based methods provide a direct and deployment-friendly solution for classroom behavior analysis by simultaneously localizing students and classifying the corresponding behaviors. General object detectors can be broadly divided into two-stage and one-stage detectors. Two-stage detectors, represented by Faster R-CNN [22], first generate candidate regions and then perform classification and bounding-box regression. This design usually provides strong localization capability but introduces a relatively complex inference pipeline. In contrast, one-stage detectors, such as SSD [23] and YOLO [24], directly predict object categories and bounding boxes within a unified network, making them more suitable for real-time classroom monitoring scenarios.

Owing to their favorable balance between detection accuracy and inference efficiency, YOLO-based detectors have been widely used in classroom behavior detection. For example, Chen and Guan [25] improved YOLOv4 for teacher–student behavior recognition and reported an AP of 97.85% and an inference speed of 58 FPS on their student behavior dataset. Tang et al. [26] enhanced YOLOv5 using multi-scale feature fusion and attention mechanisms, obtaining an average accuracy of 89.8% and a recall of 90.4%, while maintaining an inference speed of 33 FPS. More recently, Cao et al. [27] proposed YOLO-AMM based on multi-dimensional feature optimization, improving ${\mathrm{mAP}}_{50}$ and ${\mathrm{mAP}}_{50:95}$ by 3.1 and 4.0 percentage points, respectively, over YOLOv8n, while achieving an inference speed of 169.1 FPS. Transformer-based detectors have also been explored for student behavior detection. Lin et al. [28] reported a precision of 85.3%, a recall of 84.1%, an ${\mathrm{mAP}}_{50}$ of 88.5%, and an ${\mathrm{mAP}}_{50:95}$ of 62.1% using an improved RT-DETR-based method on the FSCB dataset. These methods benefit from advanced feature representation and global dependency modeling, but differences in datasets, behavior categories, and evaluation protocols make direct numerical comparisons across studies difficult.

From an architectural perspective, these classroom-oriented detectors mainly improve general-purpose detection backbones through attention mechanisms, multi-scale feature fusion, or broader feature optimization. In contrast, FGD-Net is designed around three specific difficulties encountered in fine-grained classroom behavior detection. FDRC emphasizes weak behavior-sensitive local regions such as hands, arms, and upper-body postures; MGCA strengthens contextual interaction for distinguishing visually similar behaviors; and SDRU explicitly reconstructs spatial details that may be lost during top-down feature fusion. Therefore, rather than relying on a single generic enhancement strategy, FGD-Net organizes complementary local enhancement, contextual aggregation, and detail reconstruction within a unified YOLO-based detector. Since the reported results of previous classroom detectors were obtained on different datasets and evaluation protocols, their numerical values are not directly comparable; the effectiveness of FGD-Net relative to representative YOLO detectors is therefore evaluated under identical settings on SCB5 and SCB3 in Section 4.

Although existing detection-based methods have made promising progress, they still suffer from weak fine-grained behavior cues, visually similar behavior categories, and detail loss during feature fusion. These limitations motivate more targeted designs for local discriminative region enhancement, contextual semantic aggregation, and spatial detail reconstruction.

2.3. Feature Enhancement and Multi-Scale Feature Fusion

Effective feature representation is crucial for detecting fine-grained behaviors in complex classroom scenes. Attention mechanisms have been widely introduced into object detection to enhance informative feature responses. For example, SE [29] adaptively recalibrates channel-wise features, while CBAM [30] further combines channel and spatial attention to highlight discriminative regions. Recent feature enhancement methods further explore dynamic response recalibration and local detail modeling to strengthen weak discriminative regions under complex backgrounds [31,32,33]. Beyond local response enhancement, contextual semantic aggregation is also important for distinguishing visually similar behavior categories. Non-local networks [34] and global context modeling methods [35] capture long-range dependencies and improve semantic representation. Recent designs based on gated feature interaction and efficient contextual aggregation further enhance the modeling of complex semantic relationships [36]. For multi-scale detection, feature pyramid structures and their variants improve the interaction between high-level semantic features and low-level spatial details. FPN [15] constructs a top-down feature pyramid with lateral connections, while PANet [16] further strengthens bottom-up path aggregation. Other feature pyramid variants, such as EfficientDet [37], NAS-FPN [38], and AugFPN [39], improve multi-scale feature fusion from different perspectives. Beyond feature-level multi-scale fusion, adaptive input-space zooming has also been explored for small-object detection. Wang et al. [40] proposed an adaptive image zoom-in framework with bounding-box transformation for UAV object detection, demonstrating that explicitly enlarging small object regions can improve their feature representation with limited additional latency. Meanwhile, recent upsampling and feature reconstruction methods show that spatial detail recovery can be improved through content-aware, dynamic, or shift-guided feature refinement [41,42,43,44].

However, classroom behavior detection requires more targeted modeling of subtle local cues, behavior-related contextual semantics, and fine spatial details. Motivated by these observations, this work designs dedicated modules for local discriminative region enhancement, contextual semantic aggregation, and spatial detail reconstruction.

3. Methods

3.1. Overall Architecture

The overall architecture of the proposed Fine-grained Gated Detail Network (FGD-Net) is shown in Figure 1. FGD-Net follows a hierarchical object detection framework composed of a backbone, a neck, and three multi-scale detection heads. Given an input classroom image, the backbone extracts multi-level features. The Fine-grained Dynamic Recalibration Convolution Block (FDRC) is introduced in the shallow and middle stages to enhance behavior-sensitive local cues, while the Multi-path Gated Context Aggregation Block (MGCA) is employed in the deeper stage to strengthen contextual semantic representation. In the neck, the Shift-guided Detail Reconstruction Upsampling Block (SDRU) is used during top-down feature fusion to reconstruct fine spatial details and reduce the loss of small- and medium-scale behavior-related information.

Figure 1. Overall architecture of the proposed Fine-grained Gated Detail Network (FGD-Net).

The fused multi-scale features are subsequently fed into three detection heads corresponding to different target scales. Specifically, the high-resolution detection head is mainly responsible for small-scale targets and retains richer spatial details, making it particularly suitable for students located in the back rows or farther from the camera. The medium-resolution detection head focuses on medium-scale student instances, while the low-resolution detection head is mainly responsible for large-scale targets that are generally closer to the camera. The predictions from all three scales are jointly used to determine behavior categories and bounding-box locations. In this way, differences in student-to-camera distance are reflected as variations in target scale in the image plane and are handled through the multi-scale detection mechanism. Through the collaboration of FDRC, MGCA, and SDRU, FGD-Net improves local discriminative perception, contextual aggregation, and detail-preserving feature fusion for fine-grained classroom behavior detection.

3.2. Fine-Grained Dynamic Recalibration Convolution Block

In classroom behavior detection, discriminative cues are often localized in fine-grained regions, such as hands, arms, heads, and upper-body postures. These regions usually occupy only limited spatial areas and are easily affected by desks, chairs, nearby students, and cluttered classroom backgrounds. This makes conventional convolutional blocks less effective in highlighting behavior-related local responses. To address this problem, we propose a Fine-grained Dynamic Recalibration Convolution Block (FDRC), which enhances fine-grained behavior-sensitive regions through lightweight feature aggregation and dynamic channel-spatial recalibration.

As shown in Figure 2, FDRC adopts a split-and-aggregation structure. Given an input feature map $\mathbf{X}\in {\mathbb{R}}^{B\times C_{\mathrm{in}}\times H\times W}$, a convolution–batch normalization–SiLU (CBS) layer first projects the input into an intermediate feature ${\mathrm{CBS}}_{\mathrm{in}}\left(\mathbf{X}\right)\in {\mathbb{R}}^{B\times 2C\times H\times W}$. The projected feature is then equally split into two branches along the channel dimension:

$$[{\mathbf{X}}_1,{\mathbf{X}}_2]=\mathrm{Split}\left({\mathrm{CBS}}_{\mathrm{in}}\left(\mathbf{X}\right)\right),{\mathbf{X}}_1,{\mathbf{X}}_2\in {\mathbb{R}}^{B\times C\times H\times W}.$$

(1)

Figure 2. Structure of the proposed Fine-grained Dynamic Recalibration Convolution Block (FDRC).

The first branch is directly preserved to retain the original convolutional representation, whereas the second branch is fed into the proposed Dynamic Recalibration Attention Unit (DRAU) for feature enhancement. The output of FDRC is obtained by concatenating the preserved feature, the original second-branch feature, and the recalibrated feature, followed by a CBS fusion layer:

$$\mathbf{Y}={\mathrm{CBS}}_{\mathrm{out}}\left(\mathrm{Concat}\left({\mathbf{X}}_1,{\mathbf{X}}_2,\mathrm{DRAU}\left({\mathbf{X}}_2\right)\right)\right),\mathbf{Y}\in {\mathbb{R}}^{B\times C_{\mathrm{out}}\times H\times W}.$$

(2)

where $\mathbf{Y}$ denotes the output feature of FDRC. This design preserves the efficiency of convolutional feature extraction while introducing an additional dynamic enhancement path for behavior-related local cues.

Since ${\mathbf{X}}_1$, ${\mathbf{X}}_2$, and $\mathrm{DRAU}\left({\mathbf{X}}_2\right)$ have the same dimension ${\mathbb{R}}^{B\times C\times H\times W}$, their concatenation produces a feature of dimension ${\mathbb{R}}^{B\times 3C\times H\times W}$, which is subsequently fused by ${\mathrm{CBS}}_{\mathrm{out}}$.

DRAU is designed to adaptively recalibrate feature responses along both channel and spatial dimensions. As illustrated in Figure 2, each residual channel-spatial attention block in DRAU consists of two convolutional transformations, followed by channel attention (CA) and dynamic spatial attention (DSA). Different from the CBS layers used for feature projection and fusion in FDRC, the first transformation in DRAU adopts a Conv-BN-LeakyReLU structure, while the second transformation adopts a Conv-BN structure. Given the input feature $\mathbf{F}\in {\mathbb{R}}^{B\times C\times H\times W}$, DRAU can be formulated as

$$\mathrm{DRAU}\left(\mathbf{F}\right)=\mathbf{F}+{\mathrm{Conv}}_{\mathrm{f}}\left({\mathcal{B}}_N\circ \cdots \circ {\mathcal{B}}_1\left(\mathbf{F}\right)\right)$$

(3)

where ${\mathcal{B}}_i(·)$ denotes the i-th residual channel-spatial attention block, and ${\mathrm{Conv}}_{\mathrm{f}}$ is used for feature integration. In the implemented FGD-Net, the number of sequential residual channel-spatial attention blocks in DRAU is set to $N=2$. The DRAU preserves the feature dimensions, i.e., $\mathrm{DRAU}\left(\mathbf{F}\right)\in {\mathbb{R}}^{B\times C\times H\times W}$.

Inside each residual channel-spatial attention block, the input feature is first transformed by a Conv-BN-LeakyReLU layer and then by a Conv-BN layer:

$$\mathbf{H}={\mathrm{BN}}_2\left({\mathrm{Conv}}_2\left(\psi \left({\mathrm{BN}}_1\left({\mathrm{Conv}}_1\left(\mathbf{F}\right)\right)\right)\right)\right)$$

(4)

where $\psi (·)$ denotes the LeakyReLU activation function and $\mathbf{H}\in {\mathbb{R}}^{B\times C\times H\times W}$. Then, $\mathbf{H}$ is refined by CA and DSA, and the block output is obtained through residual learning:

$$\mathcal{B}\left(\mathbf{F}\right)=\mathbf{F}+\mathrm{DSA}\left(\mathrm{CA}\left(\mathbf{H}\right)\right)$$

(5)

The residual connection preserves the original representation and facilitates feature propagation.

The channel attention branch estimates channel-wise importance by combining global average pooling and global max pooling:

$${\mathbf{A}}_c=\sigma \left(\mathcal{M}\left(\mathrm{AvgPool}\right(\mathbf{H}\left)\right)+\mathcal{M}\left(\mathrm{MaxPool}\right(\mathbf{H}\left)\right)\right)$$

(6)

where $\mathcal{M}(·)$ is a lightweight transformation implemented by $1\times 1$ convolutions, $\sigma (·)$ denotes the Sigmoid function, and ${\mathbf{A}}_c\in {\mathbb{R}}^{B\times C\times 1\times 1}$ denotes the channel attention weights. The channel-refined feature is obtained by

$${\mathbf{H}}_c=\mathbf{H}\odot {\mathbf{A}}_c$$

(7)

Inspired by the input-adaptive convolution mechanism of Dynamic Convolution [45], the DSA branch further captures spatially varying behavior cues by generating an input-conditioned spatial kernel from the global context of the channel-refined feature ${\mathbf{H}}_c\in {\mathbb{R}}^{B\times C\times H\times W}$. Different from Dynamic Convolution, which adaptively aggregates multiple predefined convolution kernels, DSA directly generates a sample-specific spatial kernel from the global feature descriptor for spatial recalibration. Specifically, adaptive global average pooling first compresses each feature map into a channel-wise global descriptor of dimension ${\mathbb{R}}^{B\times C\times 1\times 1}$. The descriptor is subsequently processed by two $1\times 1$ convolutions with an intermediate ReLU activation. The first convolution preserves the channel dimension ($C\to C$), whereas the second maps the descriptor to $k^2$ kernel coefficients ($C\to k^2$). The coefficients are then reshaped into a sample-specific spatial kernel:

$$\mathbf{K}=\mathrm{Reshape}\left[{\mathrm{Conv}}_{1\times 1}^{C\to k^2}\left(\delta \left({\mathrm{Conv}}_{1\times 1}^{C\to C}\left(\mathrm{GAP}\left({\mathbf{H}}_c\right)\right)\right)\right)\right],\mathbf{K}\in {\mathbb{R}}^{B\times 1\times k\times k},$$

(8)

where $\delta (·)$ denotes the ReLU activation. In our implementation, $k=3$; therefore, each input sample independently generates one $3\times 3$ dynamic spatial kernel from its global context.

The spatial attention map is then produced by applying the generated kernel to the channel-averaged spatial descriptor:

$${\mathbf{A}}_s=\sigma \left(\mathrm{DynConv}\left({\mathrm{Mean}}_c\left({\mathbf{H}}_c\right),\mathbf{K}\right)\right)$$

(9)

The final spatially recalibrated feature is computed as

$${\mathbf{H}}_s={\mathbf{H}}_c\odot {\mathbf{A}}_s$$

(10)

Through the combination of CA and DSA, DRAU dynamically recalibrates channel and spatial responses, enabling FDRC to emphasize behavior-sensitive local regions while suppressing redundant background activations. Therefore, when embedded into the shallow and middle stages of the backbone, FDRC provides more discriminative fine-grained features for recognizing classroom behaviors with subtle local differences.

3.3. Multi-Path Gated Context Aggregation Block

In classroom behavior detection, different behaviors often exhibit highly similar visual appearances and require contextual reasoning beyond isolated local patterns. For example, reading, writing, listening, and hand raising may involve subtle differences in upper-body posture, arm position, head orientation, and interactions with desks or books. Therefore, a single-path convolutional structure may be insufficient to capture the diverse semantic cues needed to distinguish complex classroom behaviors. To address this problem, we propose a Multi-path Gated Context Aggregation Block (MGCA), which aggregates complementary features from multiple convolutional paths and introduces a Partial Gated Convolutional Mixer (PGCM) to enhance contextual representation while maintaining computational efficiency.

As shown in Figure 3, MGCA first projects the input feature into an intermediate space and then constructs multiple feature paths with different transformation characteristics. Given an input feature map $\mathbf{X}\in {\mathbb{R}}^{B\times C_{\mathrm{in}}\times H\times W}$, the initial projection is formulated as

$${\mathbf{X}}_m={\mathrm{CBS}}_{\mathrm{in}}\left(\mathbf{X}\right)$$

(11)

where ${\mathbf{X}}_m\in {\mathbb{R}}^{B\times 2C\times H\times W}$. Based on this intermediate feature, MGCA generates several complementary representations, including a channel projection branch, a lightweight depthwise convolution branch, two split branches, and an iterative gated mixing branch:

$${\mathbf{X}}_1={\mathrm{CBS}}_1\left({\mathbf{X}}_m\right),{\mathbf{X}}_2=\mathrm{DWBranch}\left({\mathbf{X}}_m\right)$$

(12)

$$[{\mathbf{X}}_3,{\mathbf{X}}_4]=\mathrm{Split}\left({\mathbf{X}}_m\right)$$

(13)

where $\mathrm{DWBranch}(·)$ denotes a lightweight branch composed of pointwise convolution, depthwise convolution, and pointwise projection. Starting from ${\mathbf{X}}_4$, the PGCM branch progressively refines contextual features:

$${\mathbf{X}}_{4+i}={\mathrm{PGCM}}_i\left({\mathbf{X}}_{3+i}\right),i=1,2,\dots ,N$$

(14)

In the implemented FGD-Net, each MGCA contains two sequential PGCM blocks, i.e., $N=2$. Finally, all generated features are concatenated and compressed by a CBS layer:

$$\mathbf{Y}={\mathrm{CBS}}_{\mathrm{out}}\left(\mathrm{Concat}\left({\mathbf{X}}_1,{\mathbf{X}}_2,{\mathbf{X}}_3,{\mathbf{X}}_4,{\mathbf{X}}_5,\dots ,{\mathbf{X}}_{4+N}\right)\right)$$

(15)

where $\mathbf{Y}$ denotes the output feature of MGCA. Through this multi-path aggregation design, MGCA combines direct channel information, efficient spatial responses, and progressively enhanced semantic representations, which is beneficial for modeling complex classroom behavior patterns.

Figure 3. Structure of the proposed Multi-path Gated Context Aggregation Block (MGCA).

The core component of MGCA is the proposed Partial Gated Convolutional Mixer (PGCM), which is designed to enhance local contextual interactions with limited computational cost. Given an input feature $\mathbf{F}$, PGCM first applies partial convolution (PConv) for efficient spatial mixing. Specifically, only a subset of channels is processed by spatial convolution, while the remaining channels are directly preserved:

$$\mathrm{PConv}\left(\mathbf{F}\right)=\mathrm{Concat}\left(\mathrm{Conv}\left({\mathbf{F}}_p\right),{\mathbf{F}}_r\right)$$

(16)

where ${\mathbf{F}}_p$ and ${\mathbf{F}}_r$ denote the processed and reserved channel groups, respectively. This design reduces redundant spatial computation while maintaining information flow across all channels.

After partial spatial mixing, PGCM adopts the Convolutional Gated Linear Unit (CGLU) introduced in TransNeXt [36] for gated feature modulation. The input feature is first projected and divided into two parts:

$$[\mathbf{U},\mathbf{V}]=\mathrm{Split}\left({\mathrm{Conv}}_{1\times 1}\left(\mathbf{F}\right)\right)$$

(17)

Then, the gating branch is enhanced by depthwise convolution and activated by GELU:

$$\mathbf{Z}=\mathrm{DWConv}\left(\mathbf{U}\right),\widehat{\mathbf{F}}={\mathrm{Conv}}_{1\times 1}\left(\mathrm{GELU}\left(\mathbf{Z}\right)\odot \mathbf{V}\right)$$

(18)

The output of PGCM is obtained through residual learning:

$$\mathrm{PGCM}\left(\mathbf{F}\right)=\mathbf{F}+\mathrm{CGLU}\left(\mathrm{PConv}\left(\mathbf{F}\right)\right)$$

(19)

In this design, PConv provides efficient local spatial mixing, while CGLU adaptively controls the propagation of informative semantic responses through gated modulation.

By integrating multi-path aggregation with partial gated mixing, MGCA improves the representation ability of high-level and fused features without introducing heavy computation. When applied to the deeper backbone, MGCA helps the detector capture richer contextual semantics for complex classroom behaviors, thereby alleviating the limitation of single-path feature representation.

3.4. Shift-Guided Detail Reconstruction Upsampling Block

In classroom behavior detection, multi-scale feature fusion is essential for recognizing behaviors with different spatial extents. However, during upsampling and aggregation in the neck, cues associated with small- and medium-scale behaviors may be weakened or spatially blurred. This issue is particularly harmful for subtle classroom actions, such as hand raising, writing, reading, and head turning, because their discriminative information is often located in compact local regions. To alleviate the loss of spatial details during feature fusion, we propose a Shift-guided Detail Reconstruction Upsampling Block (SDRU), which improves the spatial reconstruction ability of upsampled features through efficient up-convolution and four-directional shift-based channel mixing.

As shown in Figure 4, SDRU first upsamples the input feature map and then applies depthwise convolution to refine local spatial responses with low computational cost. Given an input feature map $\mathbf{X}\in {\mathbb{R}}^{B\times C\times H\times W}$, the upsampling stage can be formulated as

$${\mathbf{X}}_u=\mathrm{DWConv}\left(\mathrm{UP}\left(\mathbf{X}\right)\right)$$

(20)

where $\mathrm{UP}(·)$ denotes a $2\times$ upsampling operation and $\mathrm{DWConv}(·)$ denotes depthwise convolution. Compared with standard convolution, depthwise convolution performs spatial filtering independently for each channel, making the upsampling refinement more efficient.

Figure 4. Structure of the proposed Shift-guided Detail Reconstruction Upsampling Block (SDRU).

After obtaining the upsampled feature ${\mathbf{X}}_u$, SDRU introduces a Four-directional Shift Channel Mixer (FSCM) to enhance local neighborhood interaction. FSCM builds on the parameter-free spatial shift principle of Shift [46], but restricts the channel-wise shifts to four cardinal directions (left, right, up, and down) for lightweight spatial detail reconstruction. Specifically, the feature is first divided into four channel groups:

$$[{\mathbf{X}}_1,{\mathbf{X}}_2,{\mathbf{X}}_3,{\mathbf{X}}_4]={\mathrm{Split}}_c\left({\mathbf{X}}_u\right)$$

(21)

where each group contains one-quarter of the channels. These four groups are then shifted along different spatial directions:

$$\begin{array}{cc}\hfill {\tilde{\mathbf{X}}}_1& ={\mathrm{Shift}}_{\mathrm{left}}\left({\mathbf{X}}_1\right)\hfill \\ \hfill {\tilde{\mathbf{X}}}_2& ={\mathrm{Shift}}_{\mathrm{right}}\left({\mathbf{X}}_2\right)\hfill \\ \hfill {\tilde{\mathbf{X}}}_3& ={\mathrm{Shift}}_{\mathrm{up}}\left({\mathbf{X}}_3\right)\hfill \\ \hfill {\tilde{\mathbf{X}}}_4& ={\mathrm{Shift}}_{\mathrm{down}}\left({\mathbf{X}}_4\right)\hfill \end{array}$$

(22)

The shifted features are concatenated along the channel dimension to obtain the spatially mixed representation:

$${\mathbf{X}}_s=\mathrm{Concat}\left({\tilde{\mathbf{X}}}_1,{\tilde{\mathbf{X}}}_2,{\tilde{\mathbf{X}}}_3,{\tilde{\mathbf{X}}}_4\right)$$

(23)

This operation enables different channel groups to capture neighboring spatial positions from different directions, thereby improving local information exchange without introducing additional learnable parameters.

Finally, a convolutional layer is used to fuse the shifted channel groups and adjust the output representation:

$$\mathbf{Y}=\mathrm{Conv}\left(\mathrm{FSCM}\left({\mathbf{X}}_u\right)\right)$$

(24)

where $\mathrm{Conv}(·)$ denotes a $1\times 1$ convolution and $\mathbf{Y}\in {\mathbb{R}}^{B\times C\times 2H\times 2W}$ is the output of SDRU. Channel shuffle is also adopted before the shift operation to promote cross-channel information interaction.

4. Experiments

4.1. Datasets

Experiments are conducted on two student classroom behavior detection datasets, namely SCB5 and SCB3. Both datasets contain three common classroom behavior categories: hand raising, reading, and writing. These three categories represent only a subset of the broader range of classroom behaviors investigated in the literature. Their use in the present study is determined by the annotation scope of the SCB5 and SCB3 benchmarks and should not be interpreted as a complete taxonomy of student classroom behavior.

Representative cropped samples of the three behavior categories are shown in Figure 5. These enlarged examples mainly illustrate the category-specific visual characteristics of hand raising, reading, and writing. In the original classroom scenes, however, student instances exhibit substantial variations in target scale, occlusion, crowd density, and background complexity. Representative challenging cases are therefore additionally presented in Figure 6. In particular, distant students may occupy only a small spatial region, while hands, arms, heads, books, and upper-body postures can be partially occluded by desks, chairs, or neighboring students. These factors substantially increase the difficulty of fine-grained classroom behavior detection.

Figure 5. Cropped representative samples of the three classroom behavior categories, including hand raising, reading, and writing. The instances are enlarged for visual clarity.

Figure 6. Representative challenging cases in the SCB datasets, including small and distant targets, partial occlusion, crowded student distributions, and complex classroom background interference. Ground-truth boxes are shown to indicate the target behavior instances.

SCB5 was constructed from real classroom teaching scenarios [47]. The version used in this study contains three annotated behavior categories: hand raising, reading, and writing, with 5193 images for training and 1671 images for testing. Multiple students may appear simultaneously in the same classroom image, and the annotated behavior instances exhibit considerable variations in target scale, crowd density, occlusion, background complexity, and viewing direction. Therefore, SCB5 is used as the main benchmark for quantitative comparison, ablation studies, and visualization analysis in this paper.

SCB3 is an earlier student classroom behavior detection dataset containing the same three behavior categories—hand raising, reading, and writing [48]. It contains 4266 images, including 3418 training images and 848 test images. Compared with SCB5, SCB3 has a smaller data scale and is used as an additional benchmark to evaluate whether the proposed method remains effective under a different sample distribution. The detailed statistics of the two datasets are shown in Table 1.

Table 1. Statistics of the SCB datasets used in the experiments.

Dataset	Categories	Training Set	Test Set	Total
SCB3	3	3418	848	4266
SCB5	3	5193	1671	6864

The SCB datasets also contain variations in camera viewpoint. In some classroom images, students are observed from frontal or oblique directions, whereas in others they are viewed partially or predominantly from the rear. Such viewpoint changes can reduce the visibility of fine-grained behavior-related cues, particularly hand–object interactions and upper-body configurations associated with reading and writing. Rear and oblique views may also introduce additional self-occlusion and overlap with desks, chairs, or neighboring students, thereby increasing the difficulty of behavior detection.

In addition to the classroom behavior detection datasets, we further conduct supplementary experiments on the KITTI dataset [49]. The purpose of introducing KITTI is to examine whether the effectiveness of the proposed modules is specific to the visual distribution of the SCB datasets or can also be observed in a substantially different object detection setting. Although KITTI contains outdoor traffic scenes and different semantic categories, it shares several challenging visual factors with classroom detection, such as scale variation, partial occlusion, cluttered backgrounds, and the need to preserve discriminative local details. Therefore, KITTI provides an additional external benchmark for assessing the general applicability of the proposed modules under a different data distribution. The SCB5 and SCB3 datasets remain the primary benchmarks for evaluating classroom behavior detection, while KITTI is used only as a supplementary validation.

4.2. Implementation Details

All experiments are implemented using a PyTorch 2.2.2-based YOLO training framework and conducted on a single NVIDIA A10 GPU with 24 GB memory. The software environment includes Python 3.8.20, CUDA 11.8, and NVIDIA driver 520.61.05. To ensure a fair comparison, all models are trained under the same experimental settings unless otherwise specified.

During training, the input image size is set to $640\times 640$, which is a commonly adopted resolution in YOLO-based detectors and provides a practical trade-off between preserving behavior-related visual details and computational cost. The batch size is set to 32 according to the available memory of the single NVIDIA A10 24 GB GPU, while allowing stable mini-batch optimization. The maximum number of training epochs is set to 300 to provide sufficient training iterations for model convergence. Importantly, the same input resolution, batch size, and number of training epochs are used for all compared models to ensure a fair comparison.

The SGD optimizer is adopted with an initial learning rate of 0.01, a momentum of 0.937, and a weight decay of 0.0005, while the warm-up period is set to 3 epochs. These optimization parameters follow the standard configuration of the adopted YOLO training framework and are kept unchanged across all compared models, thereby avoiding model-specific hyperparameter tuning that could affect the fairness of the comparison. Mixed precision training is enabled to reduce GPU memory consumption and improve training efficiency without changing the experimental protocol. The random seed is fixed to 0, and deterministic training is enabled to improve reproducibility; the specific seed value itself has no special significance and is used only to ensure a consistent experimental initialization.

For data augmentation, the default augmentation strategy of the adopted training framework is used, including HSV augmentation, random translation, scaling, horizontal flipping, and Mosaic augmentation. Mosaic augmentation is kept enabled throughout training to increase scene and object-scale diversity, which is particularly useful for classroom images containing multiple students at different scales. Validation is performed after each training epoch. During evaluation, the IoU threshold for non-maximum suppression is fixed at 0.7, and the maximum number of detections per image is set to 300. These evaluation parameters follow the unified evaluation configuration and are kept identical for all models so that the reported performance differences are attributable to the model architectures rather than different post-processing settings. Other hyperparameters follow the default settings of the training framework. The main experimental settings are summarized in Table 2.

Table 2. Main experimental settings.

Setting	Value
GPU	NVIDIA A10 24 GB
Driver/CUDA	520.61.05/11.8
Python version	3.8.20
Input size	$640\times 640$
Epochs	300
Batch size	32
Optimizer	SGD
Initial learning rate	0.01
Momentum	0.937
Weight decay	0.0005
Warm-up epochs	3
AMP	Enabled
Random seed	0
Mosaic augmentation	Enabled throughout training
NMS IoU threshold	0.7
Maximum detections	300

4.3. Evaluation Metrics

To quantitatively evaluate the detection performance of different methods, we adopt several commonly used object detection metrics, including Precision, Recall, F1-score, ${\mathrm{mAP}}_{50}$, ${\mathrm{mAP}}_{75}$, and ${\mathrm{mAP}}_{50:95}$. In addition, GFLOPs, the number of parameters, model size, and FPS are used to evaluate the computational complexity and inference efficiency of each model.

Precision measures the proportion of correctly detected positive samples among all predicted positive samples, whereas Recall measures the proportion of correctly detected positive samples among all ground-truth positive samples. They are defined as follows:

$$\mathrm{P}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$

(25)

$$\mathrm{R}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

(26)

where $\mathrm{TP}$, $\mathrm{FP}$, and $\mathrm{FN}$ denote true positives, false positives, and false negatives, respectively.

F1-score is the harmonic mean of Precision and Recall, providing a balanced evaluation of detection accuracy. It is defined as

$$\mathrm{F}1={\displaystyle \frac{2\times \mathrm{P}\times \mathrm{R}}{\mathrm{P}+\mathrm{R}}}$$

(27)

Average Precision (AP) is calculated from the precision–recall curve of each category. The mean Average Precision (mAP) is obtained by averaging AP values over all categories:

$$\mathrm{AP}={\int }_0^{1}p\left(r\right)dr$$

(28)

$$\mathrm{mAP}={\displaystyle \frac{1}{C}}{\displaystyle \sum _{i=1}^C}{\mathrm{AP}}_i$$

(29)

where C denotes the number of categories and ${\mathrm{AP}}_i$ represents the AP value of the i-th category.

In this paper, ${\mathrm{mAP}}_{50}$ denotes the mAP value calculated at an IoU threshold of 0.5, whereas ${\mathrm{mAP}}_{75}$ denotes the mAP value calculated at an IoU threshold of 0.75. ${\mathrm{mAP}}_{50:95}$ denotes the average mAP over multiple IoU thresholds from 0.5 to 0.95 with a step size of 0.05, providing a more comprehensive evaluation of localization accuracy.

For the scale-wise evaluation, ${\mathrm{AP}}_S$, ${\mathrm{AP}}_M$, and ${\mathrm{AP}}_L$ denote the Average Precision for small, medium, and large objects, respectively, following the COCO evaluation protocol. Specifically, objects with bounding-box areas of $A<{32}^2$, ${32}^2\le A<{96}^2$, and $A\ge {96}^2$ ${\mathrm{pixels}}^2$ are categorized as small, medium, and large, respectively. Similarly, ${\mathrm{AR}}_S$, ${\mathrm{AR}}_M$, and ${\mathrm{AR}}_L$ denote the corresponding Average Recall values for the three object-scale groups. These scale categories are defined according to bounding-box area in the image plane rather than the physical distance between the object and the camera.

For efficiency evaluation, GFLOPs is used to measure computational cost, and the number of parameters reflects model complexity. Model size indicates the storage cost of the trained weights. FPS is used to measure inference speed. Specifically, ${\mathrm{FPS}}_{\mathrm{full}}$ denotes the speed of the full inference pipeline, including preprocessing, model inference, and post-processing, while ${\mathrm{FPS}}_{\mathrm{infer}}$ denotes the pure model inference speed. In the following comparison tables, Precision, Recall, F1-score, and mAP values are reported in percentage form.

4.4. Comparison with State-of-the-Art Methods

To comprehensively evaluate the effectiveness of the proposed method, we compare it with several recent YOLO-based detectors [50,51,52,53,54,55,56] and enhanced YOLO-family models [57,58] on two classroom behavior detection datasets, namely SCB5 and SCB3. SCB5 is used as the main benchmark because it contains more samples and more diverse classroom scenes, while SCB3 is further adopted to examine the generalization and stability of the proposed method across different data scales and scene distributions. For a fair comparison, all models are trained and evaluated under the same experimental settings. The comparison covers both detection accuracy and computational efficiency.

Table 3 presents the comparison results on the SCB5 dataset. The proposed method achieves the best performance across all accuracy-related metrics. Compared with the strongest baseline results, the proposed method improves Precision from 59.07% to 61.10%, Recall from 62.64% to 64.16%, and F1-score from 60.22% to 62.47%. In terms of average precision, the proposed method achieves 64.68% ${\mathrm{mAP}}_{50}$, 54.20% ${\mathrm{mAP}}_{75}$, and 46.46% ${\mathrm{mAP}}_{50:95}$, outperforming the best baseline results by 2.19, 1.57, and 0.73 percentage points, respectively. These results indicate that the proposed model can more effectively distinguish fine-grained classroom behaviors and improve the detection of behavior-related local regions.

Table 3. Comparison results on the SCB5 dataset.

Method	P (%)	R (%)	F1 (%)	${\mathbf{mAP}}_{50}$ (%)	${\mathbf{mAP}}_{75}$ (%)	${\mathbf{mAP}}_{50:95}$ (%)	GFLOPs	Params (M)	Size (MB)	${\mathbf{FPS}}_{\mathbf{full}}$	${\mathbf{FPS}}_{\mathbf{infer}}$
YOLOv8n	55.96	60.22	57.98	59.02	48.21	41.64	8.1	3.01	6.0	139.70	155.58
YOLOv9t	58.36	62.64	60.22	62.49	52.63	45.73	7.6	1.97	4.5	54.43	56.71
YOLOv10n	58.18	59.61	58.59	59.83	49.54	42.50	6.5	2.27	5.5	135.69	142.55
YOLOv11n	55.51	61.30	58.25	59.26	48.00	41.49	6.3	2.58	5.2	115.20	125.57
YOLOv12n	56.56	59.89	58.11	59.15	48.66	42.19	6.3	2.56	5.3	89.85	95.87
YOLOv13n	41.50	49.70	45.20	41.30	25.88	24.39	6.2	2.45	5.2	63.05	66.39
YOLOv26n	56.63	61.34	58.75	59.58	48.63	42.20	5.2	2.38	5.2	105.24	109.35
Hyper-YOLO	59.07	60.69	59.87	60.70	50.04	43.43	10.8	3.94	7.9	442.39	596.87
Gold-YOLO	53.31	59.82	55.51	55.98	44.16	38.28	12.1	5.61	12.0	104.15	116.16
Ours	61.10	64.16	62.47	64.68	54.20	46.46	8.1	3.18	6.4	95.42	100.99

Note: Bold values indicate the best results for each metric; higher values are better for P, R, F1, mAP, and FPS, whereas lower values are better for GFLOPs, Params, and model size.

In terms of model complexity, the proposed method contains 3.18 M parameters with a model size of 6.4 MB, indicating that the additional feature enhancement modules introduce only a moderate increase in model complexity. Although the proposed method is not optimized for maximum inference speed, it achieves 95.42 FPS for the full inference pipeline and 100.99 FPS for pure model inference on SCB5, which remains well within the real-time operating range.

More importantly, the accuracy–efficiency trade-off should be considered relative to the underlying lightweight detector. Compared with YOLOv11n, the proposed method improves ${\mathrm{mAP}}_{50}$, ${\mathrm{mAP}}_{75}$, and ${\mathrm{mAP}}_{50:95}$ by 5.42, 6.20, and 4.97 percentage points, respectively, on SCB5, while the full-pipeline inference speed decreases from 115.20 FPS to 95.42 FPS. A similar trend is observed on SCB3, where the corresponding improvements are 4.68, 5.71, and 4.97 percentage points, while the inference speed decreases from 116.74 FPS to 90.80 FPS. These results indicate that the proposed modules provide consistent improvements in localization and detection accuracy at the cost of a moderate reduction in inference speed, while retaining real-time processing capability.

It should also be noted that Hyper-YOLO achieves substantially higher inference speed than FGD-Net. This indicates that FGD-Net does not provide the best solution when maximum throughput is the primary requirement. However, FGD-Net consistently achieves higher detection accuracy across the two classroom datasets while still operating above 90 FPS in the full inference pipeline. Therefore, the proposed method is intended to provide an accuracy-oriented real-time solution for fine-grained classroom behavior detection rather than to maximize inference throughput alone.

Table 4 further reports the comparison results on the SCB3 dataset. Consistent with the results on SCB5, the proposed method achieves the best overall detection performance. Specifically, the proposed method obtains the highest Precision, F1-score, ${\mathrm{mAP}}_{50}$, ${\mathrm{mAP}}_{75}$, and ${\mathrm{mAP}}_{50:95}$. Compared with the strongest baseline results, the proposed method improves Precision from 64.44% to 64.98% and F1-score from 64.73% to 66.37%. In terms of average precision, compared with Hyper-YOLO, which achieves the strongest baseline performance, the proposed method improves ${\mathrm{mAP}}_{50}$ from 69.58% to 70.78%, ${\mathrm{mAP}}_{75}$ from 56.89% to 58.39%, and ${\mathrm{mAP}}_{50:95}$ from 49.82% to 50.52%. Although YOLOv9t achieves a slightly higher Recall, it has lower Precision, F1-score, and average precision. These results show that the proposed method provides a better overall accuracy balance under a different data scale and scene distribution.

Table 4. Comparison results on the SCB3 dataset.

Method	P (%)	R (%)	F1 (%)	${\mathbf{mAP}}_{50}$ (%)	${\mathbf{mAP}}_{75}$ (%)	${\mathbf{mAP}}_{50:95}$ (%)	GFLOPs	Params (M)	Size (MB)	${\mathbf{FPS}}_{\mathbf{full}}$	${\mathbf{FPS}}_{\mathbf{infer}}$
YOLOv8n	64.44	63.84	64.14	66.83	54.05	47.07	8.1	3.01	6.0	127.18	141.57
YOLOv9t	61.31	68.69	64.73	69.09	56.53	49.76	7.6	1.97	4.5	54.12	56.50
YOLOv10n	62.37	65.18	63.75	67.41	55.08	47.92	6.5	2.27	5.5	136.43	144.30
YOLOv11n	61.79	64.70	62.94	66.10	52.68	45.55	6.3	2.58	5.3	116.74	127.63
YOLOv12n	61.58	65.07	63.23	67.05	54.53	47.68	6.3	2.56	5.3	90.31	96.68
YOLOv13n	53.90	53.22	53.55	52.83	37.62	33.92	6.2	2.45	5.2	58.32	61.09
YOLOv26n	63.00	62.58	62.75	65.96	53.36	46.22	5.2	2.38	5.2	104.14	108.80
Hyper-YOLO	63.60	64.05	63.82	69.58	56.89	49.82	10.8	3.94	7.9	460.68	597.75
Gold-YOLO	55.77	59.34	57.35	58.45	43.45	38.58	12.1	5.61	12.0	102.14	113.15
Ours	64.98	67.83	66.37	70.78	58.39	50.52	8.1	3.18	6.4	90.80	96.37

Note: Bold values indicate the best results for each metric; higher values are better for P, R, F1, mAP, and FPS, whereas lower values are better for GFLOPs, Params, and model size.

The consistent improvements on SCB5 and SCB3 demonstrate the effectiveness and robustness of the proposed model. In particular, the results suggest that the proposed modules enhance behavior-sensitive feature representation and improve multi-scale feature fusion, which is beneficial for recognizing student classroom behaviors under different data scales and scene distributions.

To evaluate the sensitivity of the reported performance to random initialization, we further repeated the experiments for YOLOv9t and FGD-Net using three random seeds (0, 42, and 3402). YOLOv9t was selected because it is one of the strongest competing detectors on SCB5 and achieves performance closest to FGD-Net. As shown in Table 5, FGD-Net consistently outperforms YOLOv9t in ${\mathrm{mAP}}_{50}$, ${\mathrm{mAP}}_{75}$, and ${\mathrm{mAP}}_{50:95}$ under all three random seeds. Averaged over the three runs, FGD-Net achieves 64.33%, 53.86%, and 46.29% in ${\mathrm{mAP}}_{50}$, ${\mathrm{mAP}}_{75}$, and ${\mathrm{mAP}}_{50:95}$, respectively, compared with 62.93%, 53.01%, and 45.68% for YOLOv9t. The corresponding average improvements are 1.40, 0.85, and 0.61 percentage points. Although the margins over this strong baseline are moderate, the consistent improvements across different random initializations indicate that the reported gains are not dependent on a single favorable seed.

Table 5. Performance stability of YOLOv9t and FGD-Net under three different random seeds on SCB5. The last two rows report the mean ± standard deviation over the three runs.

Method	Seed	P (%)	R (%)	F1-Score (%)	${\mathbf{mAP}}_{50}$ (%)	${\mathbf{mAP}}_{75}$ (%)	${\mathbf{mAP}}_{50:95}$ (%)
YOLOv9t	0	58.36	62.64	60.22	62.49	52.63	45.73
FGD-Net	0	61.10	64.16	62.47	64.68	54.20	46.46
YOLOv9t	42	58.45	63.24	60.38	62.53	53.03	45.29
FGD-Net	42	61.18	63.30	62.12	63.65	53.26	45.82
YOLOv9t	3402	61.00	62.59	61.69	63.78	53.38	46.02
FGD-Net	3402	61.49	62.75	62.00	64.65	54.12	46.60
YOLOv9t	Mean ± Std.	$59.27\pm 1.50$	$62.82\pm 0.36$	$60.76\pm 0.81$	$62.93\pm 0.73$	$53.01\pm 0.38$	$45.68\pm 0.37$
FGD-Net	Mean ± Std.	$\mathbf{61}.\mathbf{26}\pm \mathbf{0}.\mathbf{21}$	$\mathbf{63}.\mathbf{40}\pm \mathbf{0}.\mathbf{71}$	$\mathbf{62}.\mathbf{20}\pm \mathbf{0}.\mathbf{24}$	$\mathbf{64}.\mathbf{33}\pm \mathbf{0}.\mathbf{59}$	$\mathbf{53}.\mathbf{86}\pm \mathbf{0}.\mathbf{52}$	$\mathbf{46}.\mathbf{29}\pm \mathbf{0}.\mathbf{42}$

Note: Bold values indicate the better mean performance between YOLOv9t and FGD-Net for the corresponding metrics.

Per-Class Performance Analysis

Table 6 further reports the class-wise detection performance of all compared methods on SCB5. The results reveal noticeable differences in recognition difficulty among the three classroom behavior categories. Across most methods, hand raising generally achieves relatively strong performance because its raised-arm configuration provides a more distinctive visual cue. In contrast, reading tends to exhibit lower Precision, F1-score, and AP values than hand raising and writing. For FGD-Net, reading achieves 54.69% Precision, 59.27% F1-score, and 42.63% ${\mathrm{mAP}}_{50:95}$, which are lower than the corresponding results for the other two categories. This suggests that reading remains the most challenging behavior among the three classes, potentially because its discriminative cues are relatively subtle and its upper-body posture and hand–object interactions can be visually similar to those of writing.

Table 6. Per-class detection performance of different methods on the SCB5 dataset.

Method	Behavior	P (%)	R (%)	F1 (%)	${\mathbf{mAP}}_{50}$ (%)	${\mathbf{mAP}}_{75}$ (%)	${\mathbf{mAP}}_{50:95}$ (%)
YOLOv8n	Hand raising	62.34	65.63	63.94	65.43	49.16	43.97
	Reading	50.87	65.63	54.47	53.23	43.33	37.23
	Writing	54.66	56.41	55.52	58.40	52.14	43.73
YOLOv9t	Hand raising	64.54	69.32	66.84	68.92	53.72	48.07
	Reading	51.07	63.36	56.55	56.56	47.98	40.97
	Writing	59.47	55.24	57.28	62.00	56.19	48.14
YOLOv10n	Hand raising	62.32	65.02	63.64	64.73	48.94	43.32
	Reading	51.31	62.04	56.17	55.48	46.13	39.37
	Writing	60.92	51.77	55.97	59.27	53.56	44.82
YOLOv11n	Hand raising	61.46	66.70	63.98	65.71	48.06	43.28
	Reading	50.83	58.69	54.48	52.88	43.22	36.96
	Writing	54.23	58.50	56.29	59.18	52.71	44.21
YOLOv12n	Hand raising	61.39	65.91	63.57	65.19	48.76	44.08
	Reading	50.91	58.46	54.42	53.66	44.63	38.11
	Writing	57.40	55.31	56.34	58.59	52.59	44.38
YOLOv13n	Hand raising	40.32	46.13	43.03	38.59	18.76	20.41
	Reading	40.59	52.42	45.75	40.60	26.39	24.26
	Writing	43.59	50.57	46.82	44.70	32.49	28.49
YOLOv26n	Hand raising	61.95	67.19	64.46	65.74	49.53	44.23
	Reading	50.55	61.99	55.68	54.25	43.96	37.78
	Writing	57.41	54.85	56.10	58.76	52.41	44.58
Hyper-YOLO	Hand raising	65.46	65.74	65.60	65.84	49.81	44.66
	Reading	51.87	59.19	55.29	54.85	44.87	38.68
	Writing	59.89	57.15	58.49	61.42	55.44	46.96
Gold-YOLO	Hand raising	56.37	69.67	62.32	64.06	45.65	41.42
	Reading	43.51	62.54	51.32	49.57	38.67	33.41
	Writing	60.04	47.25	52.89	54.32	48.14	40.02
Ours	Hand raising	66.87	69.63	68.22	69.12	53.70	47.08
	Reading	54.69	64.69	59.27	59.97	50.11	42.63
	Writing	61.74	58.18	59.91	64.95	58.78	49.67

Note: Bold values indicate the best result for each behavior category and metric among all compared methods.

To further determine which behavior categories benefit most from the proposed improvements, we compare FGD-Net with its YOLOv11n baseline at the class level. For hand raising, F1-score and ${\mathrm{mAP}}_{50:95}$ increase from 63.98% and 43.28% to 68.22% and 47.08%, corresponding to improvements of 4.24 and 3.80 percentage points, respectively. More pronounced improvements are observed for reading: Recall increases by 6.00 percentage points, while ${\mathrm{mAP}}_{50}$, ${\mathrm{mAP}}_{75}$, and ${\mathrm{mAP}}_{50:95}$ increase by 7.09, 6.89, and 5.67 percentage points, respectively. For writing, Precision increases by 7.51 percentage points, and ${\mathrm{mAP}}_{50}$, ${\mathrm{mAP}}_{75}$, and ${\mathrm{mAP}}_{50:95}$ improve by 5.77, 6.07, and 5.46 percentage points, respectively. Although the Recall for writing decreases marginally from 58.50% to 58.18%, its F1-score and all AP metrics are clearly improved.

These class-wise results indicate that the proposed improvements are particularly beneficial for reading and writing, for which behavior discrimination relies more strongly on localized posture and hand–object cues. At the same time, the comparatively lower performance of reading indicates that visually similar fine-grained classroom behaviors remain difficult to distinguish and should receive particular attention in future refinement of the model.

4.5. Scale-Wise Detection Analysis

To further investigate the influence of apparent target size, we report scale-wise AP and AR on SCB5 in Table 7. Following the standard COCO scale definition, objects with bounding-box areas smaller than ${32}^2$ ${\mathrm{pixels}}^2$, between ${32}^2$ and ${96}^2$ ${\mathrm{pixels}}^2$, and larger than or equal to ${96}^2$ ${\mathrm{pixels}}^2$ are categorized as small, medium, and large, respectively. Since the SCB dataset does not provide explicit annotations of the physical distance between each student and the camera, object scale in the image plane is used as an observable proxy for distance-related difficulty. Students located farther from the camera generally occupy fewer pixels and may also be more susceptible to partial occlusion by nearby students or classroom furniture. Therefore, the small-object results provide complementary quantitative evidence for evaluating detection performance under challenging distant-view conditions.

Table 7. Detection performance for objects of different scales on the SCB5 dataset. Following the COCO evaluation protocol, small, medium, and large objects are defined by bounding-box areas of $A<{32}^2$, ${32}^2\le A<{96}^2$, and $A\ge {96}^2$ ${\mathrm{pixels}}^2$, respectively.

Method	${\mathbf{AP}}_{\mathit{S}}$ (%)	${\mathbf{AP}}_{\mathit{M}}$ (%)	${\mathbf{AP}}_{\mathit{L}}$ (%)	${\mathbf{AR}}_{\mathit{S}}$ (%)	${\mathbf{AR}}_{\mathit{M}}$ (%)	${\mathbf{AR}}_{\mathit{L}}$ (%)
YOLOv8n	30.4	26.8	42.7	55.2	56.9	68.7
YOLOv9t	31.0	29.9	45.8	58.3	59.2	72.6
YOLOv10n	34.7	28.7	43.5	59.5	59.4	72.7
YOLOv11n	26.5	25.6	42.8	54.6	56.7	69.2
YOLOv12n	33.7	27.1	43.1	60.1	58.9	71.0
YOLOv13n	23.7	14.7	26.0	37.3	49.7	61.7
YOLOv26n	33.0	27.2	43.4	53.1	59.6	71.5
Hyper-YOLO	32.7	27.1	44.9	49.5	57.6	70.6
Gold-YOLO	30.8	25.4	39.3	49.8	58.8	69.1
Ours	36.5	31.7	46.6	60.6	61.5	73.6

Note: Bold values indicate the best results for the corresponding metrics among all compared methods.

As shown in Table 7, FGD-Net achieves the highest AP and AR across all three target-scale groups. Compared with the YOLOv11n baseline, ${\mathrm{AP}}_{\mathrm{S}}$, ${\mathrm{AP}}_{\mathrm{M}}$, and ${\mathrm{AP}}_{\mathrm{L}}$ increase from 26.50%, 25.60%, and 42.80% to 36.50%, 31.70%, and 46.60%, corresponding to improvements of 10.00, 6.10, and 3.80 percentage points, respectively. Similarly, ${\mathrm{AR}}_{\mathrm{S}}$, ${\mathrm{AR}}_{\mathrm{M}}$, and ${\mathrm{AR}}_{\mathrm{L}}$ improve by 6.00, 4.80, and 4.40 percentage points. The largest AP improvement is observed for small targets, indicating that the proposed multi-scale and detail-preserving design is particularly beneficial for student instances occupying relatively small image regions.

It should be noted that object scale is not equivalent to physical distance, and the current SCB annotations do not allow the effect of occlusion to be isolated independently. Therefore, the scale-wise results provide an indirect but quantitative analysis of distance-related difficulty rather than a direct measurement of performance degradation with physical distance.

4.6. Visualization Analysis

To provide a more intuitive understanding of the feature responses of different detectors, we employ Grad-CAM++ to generate activation heatmaps for several representative YOLO-based models. The input resolution is fixed at $640\times 640$, and the confidence threshold is set to 0.5. For the detection task, both classification- and bounding-box-related outputs are included in the backward target, and activation responses are extracted from selected intermediate feature layers. The resulting Grad-CAM++ maps are normalized for visualization, where warmer regions indicate stronger model responses and cooler regions indicate weaker responses. The visualization results are shown in Figure 7.

Figure 7. Heatmap visualization comparison of different YOLO-based detectors in classroom behavior detection scenes. Warmer regions indicate stronger model responses.

As shown in Figure 7, existing YOLO-based methods can generally focus on student regions, but their activation responses may be scattered or incomplete in complex classroom scenes. For example, YOLOv8 and YOLOv9 tend to generate discontinuous responses in dense student areas, and some behavior-related regions are not sufficiently highlighted. YOLOv10 and YOLOv26 produce stronger activations, but some responses spread to background regions, such as desks, chairs, and surrounding classroom areas, which may weaken the discrimination of fine-grained behaviors.

In contrast, the proposed method shows more concentrated and complete activation responses on students and behavior-related body regions. In particular, it better highlights key visual cues, such as heads, arms, hands, and upper-body postures, under crowded classroom conditions. This indicates that the proposed FDRC, SDRU, and MGCA modules enhance fine-grained local features, preserve spatial details, and aggregate contextual information. Therefore, the heatmap visualization further confirms that the proposed method obtains more discriminative feature representations for classroom behavior detection.

To quantitatively evaluate the spatial concentration of the activation responses, we introduce two target-localization measures. Let ${\mathbf{A}}_i\in {[0,1]}^{H\times W}$ denote the Grad-CAM++ activation map of the i-th image, independently normalized over its full spatial extent, and let ${\mathbf{M}}_i\in {\{0,1\}}^{H\times W}$ denote a binary target-region mask constructed from the union of all ground-truth behavior bounding boxes in that image. The Activation Localization Ratio (ALR) is defined as

$${\mathrm{ALR}}_i={\displaystyle \frac{{\sum }_{x,y}{\mathbf{A}}_i(x,y){\mathbf{M}}_i(x,y)}{{\sum }_{x,y}{\mathbf{A}}_i(x,y)+ϵ}},$$

(30)

where a higher ALR indicates that a larger proportion of the total activation response is concentrated within the annotated target regions. To further quantify the contrast between target and background responses, we define the mean target activation and mean background activation of the i-th image as

$$T_i=\mathrm{Mean}\left({\mathbf{A}}_i\mid {\mathbf{M}}_i=1\right),B_i=\mathrm{Mean}\left({\mathbf{A}}_i\mid {\mathbf{M}}_i=0\right),$$

(31)

respectively. Based on these quantities, the Target-to-Background Activation Ratio (TBR) is defined as

$${\mathrm{TBR}}_i={\displaystyle \frac{T_i}{B_i+ϵ}}.$$

(32)

A larger TBR indicates a greater contrast between the normalized activation within the annotated target regions and that of the surrounding background. For quantitative evaluation, each raw Grad-CAM++ map is independently normalized over its full spatial extent before metric calculation, without applying the bounding-box-based renormalization used only for visualization. ALR and TBR are computed independently for each validation image and are subsequently averaged over the entire SCB5 validation set. Specifically, the reported mean values are calculated as

$$\overline{\mathrm{ALR}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\mathrm{ALR}}_i,\overline{\mathrm{TBR}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\mathrm{TBR}}_i,$$

(33)

where N denotes the number of validation images. Likewise, the reported mean target and background activations are obtained by averaging $T_i$ and $B_i$, respectively, over all validation images. Since TBR is first computed for each individual image and then averaged, the reported mean TBR is not necessarily equal to the ratio between the dataset-level mean target and background activations. All reported statistics are calculated over the entire SCB5 validation set rather than only the representative images shown in Figure 7.

As shown in Table 8, FGD-Net achieves the highest mean ALR of 0.3271 and the highest mean TBR of 1.2975 among the compared detectors. In comparison with the YOLOv11n baseline, the mean ALR increases from 0.3036 to 0.3271, indicating that a larger proportion of the total Grad-CAM++ response is localized within the annotated target regions. Meanwhile, the mean TBR increases from 1.1280 to 1.2975, demonstrating a stronger contrast between the normalized activation in the target regions and that in the surrounding classroom background.

Table 8. Quantitative comparison of Grad-CAM++ activation localization on the SCB5 validation set. ALR and TBR are reported as mean ± standard deviation over individual validation images, while the target and background activation values denote their corresponding dataset-level means.

Method	ALR	TBR	Mean Target Activation	Mean Background Activation
YOLOv8n	$0.3184\pm 0.1973$	$1.1157\pm 0.3071$	0.3484	0.3283
YOLOv9t	$0.3162\pm 0.2014$	$1.2012\pm 0.3061$	0.3676	0.3137
YOLOv10n	$0.2581\pm 0.1940$	$1.0335\pm 0.2858$	0.3414	0.3343
YOLOv11n	$0.3036\pm 0.1969$	$1.1280\pm 0.2714$	0.3552	0.3223
YOLOv26n	$0.2987\pm 0.1903$	$1.0271\pm 0.2681$	0.3634	0.3579
Ours	0.3271 ± 0.2034	1.2975 ± 0.2406	0.3999	0.2951

Note: Bold values indicate the best mean results among all compared methods; higher values are better for ALR, TBR, and mean target activation, whereas lower values are better for mean background activation.

FGD-Net also produces the highest mean normalized activation within the target regions (0.3999) and the lowest mean normalized background activation (0.2951) among the compared models. This result is consistent with the qualitative observations in Figure 7: the proposed model allocates relatively stronger normalized responses to behavior-related target regions while suppressing activation in irrelevant background areas. Therefore, the improved spatial concentration observed in the heatmaps is supported by quantitative activation-localization statistics rather than visual inspection alone.

4.7. Supplementary Evaluation on the KITTI Dataset

To provide an external verification of the general applicability of the proposed modules, we further evaluate FGD-Net on KITTI under the same comparison protocol. In this experiment, KITTI is treated as an independent object detection benchmark, and the objective is to examine whether the architectural modifications that improve detection on SCB5 and SCB3 can still provide consistent benefits under a substantially different visual distribution.

The comparison results are reported in Table 9. The proposed method achieves the best overall detection performance in terms of Recall, F1-score, ${\mathrm{mAP}}_{50}$, ${\mathrm{mAP}}_{75}$, and ${\mathrm{mAP}}_{50:95}$. Specifically, FGD-Net obtains 88.32% ${\mathrm{mAP}}_{50}$ and 65.10% ${\mathrm{mAP}}_{50:95}$, outperforming the YOLOv11n baseline by 3.54 and 4.55 percentage points, respectively. Compared with other recent YOLO-based detectors, the proposed method also achieves the highest F1-score and average precision metrics, indicating that the proposed modules improve feature representation and localization quality under cross-scenario conditions. Although FGD-Net introduces a moderate increase in parameters and computational cost, it maintains real-time inference speed, achieving 99.74 FPS in the full inference pipeline and 105.18 FPS in pure model inference. These results provide supplementary evidence that the proposed architectural modifications remain effective under a substantially different object detection setting.

Table 9. Supplementary comparison results on the KITTI dataset.

Method	P (%)	R (%)	F1 (%)	${\mathbf{mAP}}_{50}$ (%)	${\mathbf{mAP}}_{75}$ (%)	${\mathbf{mAP}}_{50:95}$ (%)	GFLOPs	Params (M)	Size (MB)	${\mathbf{FPS}}_{\mathbf{full}}$	${\mathbf{FPS}}_{\mathbf{infer}}$
YOLOv8n	86.74	80.10	83.11	87.24	68.95	61.95	8.1	3.01	6.0	151.27	164.99
YOLOv9t	86.96	73.30	79.54	84.57	70.79	61.10	7.6	1.97	4.4	56.97	58.77
YOLOv10n	87.63	75.52	80.33	84.81	73.36	63.18	6.5	2.27	5.5	147.69	152.48
YOLOv11n	85.59	77.66	81.29	84.78	68.52	60.55	6.3	2.58	5.2	122.18	130.81
YOLOv12n	87.04	79.07	82.68	86.30	68.44	61.82	6.3	2.56	5.3	97.30	102.48
YOLOv13n	84.58	71.96	77.25	81.38	62.66	56.37	6.2	2.45	5.2	60.73	83.37
YOLOv26n	90.58	75.86	82.57	86.06	70.55	63.26	5.2	2.38	5.1	111.66	118.42
Ours	89.18	81.52	84.97	88.32	73.98	65.10	8.1	3.18	6.4	99.74	105.18

Note: Bold values indicate the best results for each metric; higher values are better for P, R, F1, mAP, and FPS, whereas lower values are better for GFLOPs, Params, and model size.

4.8. Ablation Study

To verify the individual and cumulative contributions of the proposed modules, we conduct controlled ablation experiments on the SCB5 dataset using YOLOv11n as the common baseline. The first row of Table 10 corresponds to the original YOLOv11n without any of the proposed modules, while FDRC, SDRU, and MGCA are progressively introduced in the subsequent configurations. The final row represents the complete FGD-Net. Therefore, the ablation study directly measures the performance changes relative to the YOLOv11n baseline under the same experimental settings. For consistency, the YOLOv11n baseline and the complete FGD-Net in Table 10 correspond to the YOLOv11n and Ours entries reported in Table 3, respectively.

Table 10. Ablation study of the proposed modules based on the YOLOv11n baseline on the SCB5 dataset.

FDRC	SDRU	MGCA	P (%)	R (%)	F1 (%)	${\mathbf{mAP}}_{50}$ (%)	${\mathbf{mAP}}_{75}$ (%)	${\mathbf{mAP}}_{50:95}$ (%)	GFLOPs	Params (M)	Size (MB)	${\mathbf{FPS}}_{\mathbf{full}}$	${\mathbf{FPS}}_{\mathbf{infer}}$
			55.51	61.30	58.25	59.26	48.00	41.49	6.3	2.58	5.2	115.20	125.57
✓			59.60	62.97	61.08	62.36	52.01	44.78	7.7	2.98	6.1	46.50	67.34
✓	✓		59.90	63.74	61.50	63.61	53.31	45.81	8.1	3.06	6.3	48.80	67.12
✓	✓	✓	61.10	64.16	62.47	64.68	54.20	46.46	8.1	3.18	6.4	95.42	100.99

Note: Bold values indicate the best results for the corresponding metrics; higher values are better for P, R, F1, mAP, and FPS, whereas lower values are better for GFLOPs, Params, and model size.

As shown in Table 10, introducing FDRC brings a clear improvement over the baseline. Specifically, Precision increases from 55.51% to 59.60%, Recall increases from 61.30% to 62.97%, and F1-score increases from 58.25% to 61.08%. Meanwhile, ${\mathrm{mAP}}_{50}$, ${\mathrm{mAP}}_{75}$, and ${\mathrm{mAP}}_{50:95}$ are improved by 3.10, 4.01, and 3.29 percentage points, respectively. These results indicate that FDRC can effectively enhance fine-grained behavior-related feature representation, which is beneficial for recognizing subtle classroom behaviors.

After further introducing SDRU, the detection performance is consistently improved. Compared with the model using only FDRC, the combination of FDRC and SDRU improves ${\mathrm{mAP}}_{50}$ from 62.36% to 63.61%, ${\mathrm{mAP}}_{75}$ from 52.01% to 53.31%, and ${\mathrm{mAP}}_{50:95}$ from 44.78% to 45.81%. This demonstrates that SDRU can better preserve and reconstruct spatial details during feature upsampling, thereby improving the detection of small-scale and posture-sensitive behavior regions.

When MGCA is added, the full model achieves the best overall performance. Compared with the baseline, the complete model improves Precision, Recall, and F1-score by 5.59, 2.86, and 4.22 percentage points, respectively. In terms of average precision, ${\mathrm{mAP}}_{50}$, ${\mathrm{mAP}}_{75}$, and ${\mathrm{mAP}}_{50:95}$ are improved by 5.42, 6.20, and 4.97 percentage points, respectively. These improvements show that MGCA further strengthens multi-scale contextual feature aggregation and enhances the discriminative ability of the model.

Although the complete model increases the computational cost from 6.3 GFLOPs to 8.1 GFLOPs and the number of parameters from 2.58M to 3.18M, it still maintains a compact model size of 6.4 MB and achieves 95.42 FPS in the full inference pipeline. Therefore, the proposed model obtains a favorable balance between detection accuracy and inference efficiency.

To further examine whether the performance improvement of SDRU can be attributed simply to the use of a more advanced upsampling operator, we compare the proposed SDRU with conventional Upsample, CARAFE [41], and DySample [42] under the same FGD-Net framework. As shown in Table 11, SDRU achieves the best overall detection performance, reaching 64.68% ${\mathrm{mAP}}_{50}$, 54.20% ${\mathrm{mAP}}_{75}$, and 46.46% ${\mathrm{mAP}}_{50:95}$. Compared with conventional Upsample, CARAFE, and DySample, SDRU improves ${\mathrm{mAP}}_{50:95}$ by 1.36, 1.08, and 1.49 percentage points, respectively.

Table 11. Comparison of different upsampling strategies within the FGD-Net framework on the SCB5 dataset.

Method	P (%)	R (%)	F1 (%)	${\mathbf{mAP}}_{50}$ (%)	${\mathbf{mAP}}_{75}$ (%)	${\mathbf{mAP}}_{50:95}$ (%)	GFLOPs	Params (M)	Size (MB)	${\mathbf{FPS}}_{\mathbf{full}}$	${\mathbf{FPS}}_{\mathbf{infer}}$
Upsample	60.58	62.62	61.40	62.82	52.90	45.10	7.7	3.09	6.3	93.64	98.66
CARAFE [41]	60.03	63.43	61.53	63.28	52.93	45.38	7.9	3.23	6.6	90.67	95.36
DySample [42]	58.30	63.39	60.54	62.52	52.21	44.97	7.7	3.10	6.3	92.37	97.12
SDRU	61.10	64.16	62.47	64.68	54.20	46.46	8.1	3.18	6.4	95.42	100.99

Note: Bold values indicate the best results for the corresponding metrics among all compared upsampling strategies.

In terms of computational efficiency, SDRU introduces only a moderate increase in computational cost, with 8.1 GFLOPs and 3.18 M parameters. Nevertheless, it achieves 95.42 FPS for the full inference pipeline and 100.99 FPS for pure model inference, which are higher than those obtained with CARAFE and DySample in the same framework. These results indicate that the advantage of SDRU does not arise solely from replacing the original upsampling operator, but from the combined effect of upsampling, local spatial interaction, and detail reconstruction.

To further verify the choice of the four-directional shift strategy in FSCM, we construct several directional-shift variants while keeping all other components unchanged. Specifically, X0 denotes the configuration without spatial shifting, which is equivalent to removing the shift operation in FSCM; X1V and X1H denote single-direction vertical and horizontal shifts, respectively; X2V and X2H denote bidirectional vertical (up and down) and horizontal (left and right) shifts, respectively; X2A denotes a two-axis single-direction configuration using down and right shifts; and X4 denotes the complete four-directional configuration using up, down, left, and right shifts, which corresponds to the FSCM adopted in the final FGD-Net.

As shown in Table 12, removing the spatial shift operation (X0) results in the lowest ${\mathrm{mAP}}_{50:95}$ of 44.46%, indicating that directional spatial interaction contributes positively to feature reconstruction. Introducing a single directional shift improves the performance to 45.75% for X1V and 46.18% for X1H. Bidirectional configurations also provide consistent improvements over X0, although their effectiveness varies with the selected directions.

Table 12. Ablation study of different directional shift configurations in FSCM on the SCB5 dataset.

Configuration	P (%)	R (%)	F1 (%)	${\mathbf{mAP}}_{50}$ (%)	${\mathbf{mAP}}_{75}$ (%)	${\mathbf{mAP}}_{50:95}$ (%)
X0	60.53	61.95	61.21	62.73	53.27	44.46
X1V	60.45	62.25	61.24	63.60	53.03	45.75
X1H	60.16	63.88	61.80	63.97	53.77	46.18
X2A	59.02	63.42	60.88	62.99	52.10	45.01
X2V	60.27	63.18	61.48	63.12	53.39	45.64
X2H	61.35	63.69	62.33	64.16	53.67	45.84
X4	61.10	64.16	62.47	64.68	54.20	46.46

Note: Bold values indicate the best results for the corresponding metrics among all shift configurations.

Among all configurations, the complete four-directional strategy X4 achieves the best overall detection performance, with 64.68% ${\mathrm{mAP}}_{50}$, 54.20% ${\mathrm{mAP}}_{75}$, and 46.46% ${\mathrm{mAP}}_{50:95}$. Compared with X0, X4 improves ${\mathrm{mAP}}_{50:95}$ by 2.00 percentage points. It also outperforms the best single-direction configuration X1H by 0.28 percentage points and the best two-direction configuration X2H by 0.62 percentage points in ${\mathrm{mAP}}_{50:95}$. These results support the use of complementary horizontal and vertical neighborhood information and indicate that the four-directional FSCM is more effective than restricted directional-shift alternatives for spatial detail reconstruction.

4.9. Failure and Robustness Analysis

Representative failure cases of FGD-Net are shown in Figure 8. Three typical failure modes can be observed. First, Figure 8a shows a high-confidence false-positive prediction, where a student who is not annotated as reading is incorrectly detected as the reading class. In this case, the relatively low viewing angle and the student’s upper-body posture produce visual characteristics similar to those of reading samples, which can cause an erroneous activation of the reading detector. This example indicates that posture ambiguity remains a potential source of false-positive predictions.

Figure 8. Representative failure cases of FGD-Net on the SCB5 dataset: (a) a high-confidence false-positive reading prediction caused by ambiguous posture and viewing angle; (b) a missed hand-raising instance in which the raised arm is largely occluded by a foreground student; and (c) a writing instance incorrectly classified as reading when the hand–pen interaction is partially occluded.

Occlusion constitutes another important source of detection error. In Figure 8b, the hand-raising student is located farther from the camera and occupies a relatively small image region, while most of the raised arm is occluded by a foreground student. As a result, the most distinctive cue for hand raising is only partially visible, leading to a false-negative detection. Figure 8c illustrates a different effect of occlusion. Although the student is writing, the right hand and pen interaction, which provides an important local cue for distinguishing writing from reading, is partially obscured. Consequently, the instance is incorrectly classified as reading. These examples suggest that occlusion can affect the detector in two different ways: severe loss of discriminative cues may result in missed detections, whereas partial occlusion of fine-grained hand–object information may increase confusion between visually similar behavior categories.

The missed detection in Figure 8b also illustrates the coupled influence of target scale and occlusion. Students located farther from the camera generally occupy fewer pixels, and their behavior-related regions can be more easily obscured by foreground students or classroom furniture. Nevertheless, the failure cases indicate that small targets combined with substantial occlusion remain challenging even for the proposed model.

Figure 9 compares the class-wise Precision–Recall curves of YOLOv11n and FGD-Net on the SCB5 dataset. FGD-Net shows a clear upward shift in the overall Precision–Recall characteristics, with the class-wise ${\mathrm{AP}}_{50}$ values increasing from 65.7%, 52.8%, and 59.1% to 69.1%, 59.9%, and 65.0% for hand raising, reading, and writing, respectively. The improvement is particularly evident for reading and writing, which are more difficult to distinguish because their discriminative cues are concentrated around relatively small hand–object and upper-body regions. Although reading remains the most challenging category, the improved curve indicates that FGD-Net achieves a more favorable Precision–Recall trade-off than the YOLOv11n baseline.

Figure 9. Class-wise Precision–Recall curves of YOLOv11n (left) and FGD-Net (right) on the SCB5 dataset.

The normalized confusion matrices in Figure 10 provide further insight into the remaining class-specific errors. Compared with YOLOv11n, FGD-Net improves the correct prediction ratios for hand raising, reading, and writing from approximately 0.65, 0.52, and 0.45 to 0.70, 0.63, and 0.48, respectively. The largest improvement is observed for reading, which is consistent with the per-class AP results.

Figure 10. Normalized confusion matrices of YOLOv11n (left) and FGD-Net (right) on the SCB5 dataset.

Nevertheless, a noticeable confusion between writing and reading remains. For true writing instances, a substantial proportion is still predicted as reading, reflecting the visual similarity between these two behaviors. Both classes can exhibit similar seated upper-body postures, while their distinction often depends on small local cues such as hand position, pen interaction, and book or notebook appearance. When these regions are partially occluded or occupy only a small number of pixels, the model may fail to preserve sufficient discriminative information for correct classification.

The background-related entries also indicate that missed detections remain an important source of error. Although FGD-Net reduces the proportion of true behavior instances assigned to the background category for all three classes, small, distant, or heavily occluded students may still be missed because their behavior-specific features are weak or incomplete.

4.10. Deployment Experiment and Practical Applications

To evaluate the deployment performance of FGD-Net in practical applications, we developed a classroom behavior detection system for real-world monitoring scenarios. As shown in Figure 11, FGD-Net was deployed on a platform equipped with an NVIDIA GeForce RTX 3060 GPU with 8 GB memory, enabling real-time behavior detection and visualization within a unified interface.

Figure 11. Classroom behavior detection system interface based on FGD-Net.

The current prototype is primarily intended to support the automatic observation and recording of observable classroom behaviors, including hand raising, reading, and writing. The detected behavior instances can provide basic information for summarizing the occurrence and distribution of these activities during a class. By further aggregating detection results over time, the system can potentially support classroom activity analysis, such as estimating the frequency of participation-related behaviors and examining temporal changes in observable student activities. Such information is intended to serve as auxiliary evidence for teachers rather than as a direct or automatic evaluation of students’ learning quality or teaching effectiveness.

At the current stage, the system focuses on behavior detection and visualization rather than higher-level pedagogical inference. More complex applications, such as longitudinal behavior analysis, individual activity profiling, and the integration of multiple behavioral cues for classroom learning analytics, require additional temporal modeling, richer behavior categories, and further pedagogical validation. Therefore, the present deployment experiment mainly demonstrates the feasibility of using FGD-Net as a visual perception component for future classroom behavior analysis systems.

Example Teacher-Facing Analytics Workflow

To further illustrate how the frame-level detection results of FGD-Net can be transformed into teacher-interpretable information, we consider a simple classroom analytics workflow. For each sampled video frame at time t, FGD-Net outputs the locations, behavior categories, and confidence scores of detected students. Let $n_c\left(t\right)$ denote the number of detected students exhibiting behavior c, where $c\in \{\mathrm{hand}\mathrm{raising},\mathrm{reading},\mathrm{writing}\}$, and let $n\left(t\right)={\sum }_c{n}_c\left(t\right)$ denote the total number of detected student behavior instances in that frame. For a predefined lesson interval W, such as one or five minutes, the classroom-level activity proportion of behavior c can be calculated as

$$A_c\left(W\right)={\displaystyle \frac{1}{\left|W\right|}}\sum _{t\in W}{\displaystyle \frac{n_c\left(t\right)}{max\left(n\right(t),1)}},$$

(34)

where $\left|W\right|$ denotes the number of sampled frames within the interval. This quantity represents the average proportion of detected students exhibiting a particular observable behavior during the selected lesson interval.

By calculating $A_c\left(W\right)$ over consecutive time windows, a lesson can be summarized as a temporal sequence of reading, writing, and hand-raising activities. These statistics can be presented to teachers using temporal curves, stacked activity charts, or lesson-level summaries. For example, a teacher could examine whether reading behavior becomes dominant during an assigned reading period, whether writing activity increases during an exercise session, or whether hand-raising activity becomes more frequent during question-and-answer interactions. In this way, the detector provides structured observational information that can assist teachers in reviewing how classroom activities evolve across different instructional stages.

At the individual level, longitudinal behavior statistics would additionally require maintaining student identities across video frames. When a multi-object tracking module is available, the proportion of observed time that student i spends exhibiting behavior c can be expressed as

$$R_{i,c}={\displaystyle \frac{T_{i,c}}{T_{i,\mathrm{obs}}}},$$

(35)

where $T_{i,c}$ denotes the cumulative tracked duration for which student i is classified as behavior c, and $T_{i,\mathrm{obs}}$ denotes the total duration for which the student is reliably observed. Since the current FGD-Net operates as a frame-level detector and does not explicitly perform student identity tracking or temporal behavior modeling, such individual-level duration statistics are considered a potential downstream extension rather than a validated output of the present system.

Importantly, the above activity proportions describe observable classroom behavior patterns rather than latent cognitive states. They should therefore not be interpreted directly as quantitative measures of engagement, concentration, understanding, or learning achievement. Instead, these behavioral summaries are intended to provide supplementary observational evidence that teachers can interpret together with the instructional context and other educational information.

5. Discussion

5.1. Interpretation of the Main Findings

The working hypothesis of this study is that fine-grained classroom behavior detection can benefit from jointly modeling behavior-sensitive local regions, contextual semantics, and spatial details during multi-scale feature fusion. The experimental results on SCB5 and SCB3 generally support this hypothesis. FGD-Net achieved the highest F1-score and average precision on both datasets, indicating that its improvements are not limited to a single classroom dataset or evaluation metric. On SCB5, the complete model improved ${\mathrm{mAP}}_{50}$, ${\mathrm{mAP}}_{75}$, and ${\mathrm{mAP}}_{50:95}$ by 5.42, 6.20, and 4.97 percentage points, respectively, over the YOLOv11n baseline. The relatively large improvement at the stricter IoU threshold of 0.75 suggests that FGD-Net improves not only behavior classification but also the localization of compact behavior-related regions.

The ablation results provide further evidence for the proposed hypothesis. Among the three modules, FDRC produced the largest initial improvement, increasing ${\mathrm{mAP}}_{50:95}$ from 41.49% to 44.78%. This result indicates that dynamically strengthening local regions is particularly important for classroom behavior detection, where discriminative information is frequently concentrated around hands, arms, heads, books, and upper-body postures. This observation is consistent with previous classroom behavior studies that identify small targets, occlusion, and weak posture differences as important sources of detection errors [14,25,26]. The heatmap results in Figure 7 also show that FGD-Net produces more concentrated activation responses around student body regions and fewer responses in irrelevant classroom backgrounds. Nevertheless, these activation maps should be regarded as qualitative supporting evidence rather than direct proof of the internal mechanism of FDRC.

After SDRU was introduced, all three mAP metrics were further improved. In particular, ${\mathrm{mAP}}_{75}$ increased from 52.01% to 53.31%, suggesting that spatial detail reconstruction contributes to more accurate localization. Conventional top-down feature pyramids may weaken or spatially blur small behavior-related cues during upsampling and feature aggregation [15,16]. The improvement obtained with SDRU indicates that shift-guided reconstruction can partially alleviate this problem while introducing only a small parameter increase. When MGCA was subsequently added, Precision increased from 59.90% to 61.10%, and the complete model achieved the best overall performance. This result suggests that complementary contextual paths help suppress background interference and reduce ambiguous predictions among visually similar behaviors.

The results are also related to, but different from, those of previous classroom behavior analysis methods. Earlier studies have employed temporal action modeling, skeleton-based representation, attention mechanisms, and combinations of object detection and pose estimation [7,18,20]. These approaches provide useful temporal or structural information but may require continuous video clips, reliable keypoint estimation, or additional annotations. In comparison, FGD-Net uses a single-frame object detection framework and improves task-specific feature representation without requiring pose or temporal labels. This gives the model a relatively simple and deployment-friendly inference pipeline. However, the present findings should be viewed as complementary to pose- and video-based methods rather than evidence that spatial appearance features can completely replace them. Temporal and skeletal information may still be beneficial when different behaviors have nearly identical appearances in an individual frame.

The consistent improvements on SCB5 and SCB3 demonstrate stability across two classroom benchmarks with different sample sizes and scene distributions. However, because both datasets contain the same three behavior categories and belong to the SCB benchmark family, these results mainly demonstrate within-task generalization. They do not fully establish generalization across schools, age groups, teaching activities, or cultural environments. In comparison, the KITTI experiment evaluates the proposed architecture under substantially different object categories and visual scenes. Relative to the YOLOv11n baseline, FGD-Net improved ${\mathrm{mAP}}_{50:95}$ from 60.55% to 65.10%. This result suggests that the proposed feature enhancement and reconstruction operations are not restricted to classroom images. Nevertheless, the KITTI experiment demonstrates the transferability of the network design rather than cross-domain generalization of classroom behavior understanding.

5.2. Implications for Cognitive Classroom Analytics

From the perspective of cognitive computing, FGD-Net can be regarded as a visual perception component that transforms classroom image or video streams into structured behavior events. Instead of storing and manually reviewing all classroom video frames, a detection system could extract the locations, categories, confidence scores, and temporal distributions of observable student behaviors. These structured results could subsequently be combined with learning analytics to describe classroom participation patterns, identify behavioral changes, or locate video intervals that require teacher review. Therefore, the contribution of FGD-Net is not limited to improving an object detection benchmark; it also provides a possible perception layer for data-driven intelligent classroom systems.

The computational characteristics of FGD-Net are relevant to this application scenario. The complete model contains 3.18M parameters, has a model size of 6.4 MB, and achieves 95.42 FPS in the full inference pipeline on the experimental platform. These results indicate that task-specific feature enhancement does not necessarily require a large-scale network. Compared with heavier feature-fusion models, the compact architecture may facilitate integration with edge–cloud classroom analytics, in which video frames are processed near the data source and only derived behavior events or aggregated statistics are transmitted to a server. Such a design could reduce communication and storage requirements when processing long-duration classroom videos.

The prototype shown in Figure 11 further demonstrates that FGD-Net can be integrated into a classroom behavior visualization interface. However, this prototype should currently be interpreted as a deployment feasibility demonstration rather than a complete validation in an operational classroom. The present experiment does not measure long-term system stability, end-to-end response latency, multi-camera throughput, or the effect of the generated information on teaching decisions. These factors need to be evaluated before the proposed system can be considered ready for large-scale practical use.

It is also important to distinguish observable classroom behavior from latent cognitive or educational states. Hand raising, reading, and writing provide useful visual evidence, but none of these behaviors is an unambiguous indicator of attention, engagement, understanding, or learning performance. For example, a student may remain cognitively engaged without performing one of the three detected behaviors, whereas writing may have different meanings under different teaching activities. Consequently, FGD-Net should be used to support, rather than replace, teacher interpretation. The detection results should preferably be aggregated over suitable time intervals and interpreted together with lesson context instead of being used as isolated evidence for ranking or penalizing individual students.

Classroom video may also contain identifiable and potentially sensitive student information. Practical applications therefore require appropriate data governance, including informed authorization, data minimization, secure storage, access control, and clearly defined retention periods. Model performance should also be examined across different classroom layouts and student groups to reduce the risk of uneven system reliability. Where possible, on-device inference and the retention of derived statistics rather than raw video may help reduce privacy exposure. These considerations are essential for translating visual behavior detection into a responsible cognitive classroom analytics system.

5.3. Limitations and Future Directions

Several limitations define the scope of the present findings. First, SCB5 and SCB3 contain only three behavior categories: hand raising, reading, and writing. Although these categories include fine-grained posture differences, they do not cover the wider range of interactions and learning-related behaviors found in real classrooms. In addition, both datasets are derived from the same benchmark family. The present experiments therefore cannot establish whether the model will maintain its performance under different camera viewpoints, classroom layouts, age groups, illumination conditions, teaching styles, and cultural contexts. Future studies should evaluate FGD-Net using independently collected and multi-institutional classroom datasets with more diverse behavior categories. The present study also does not explicitly analyze class-frequency imbalance among the three annotated behaviors. Since long-tailed distributions can lead to insufficient optimization signals for sample-scarce categories [59], future work should quantify the category distribution of classroom datasets and investigate appropriate class-balanced learning strategies when substantial imbalance is present.

Second, although the revised evaluation includes class-wise performance, confusion matrices, scale-wise analysis, and representative failure cases, the current SCB annotations do not support an independent quantitative stratification according to occlusion severity, student density, or physical camera distance. Future studies could therefore evaluate these factors using datasets with explicit annotations for occlusion level, crowd density, and viewing distance.

Third, FGD-Net processes individual images and does not explicitly model behavior duration, transitions, or student identity across frames. As a result, an ambiguous posture in a single frame may be incorrectly classified even when its temporal context would make the behavior clear. Future work could combine FGD-Net with multi-object tracking and lightweight temporal modeling to construct behavior trajectories from frame-level detections. Pose, gaze, scene context, or other modalities could also provide complementary information. However, the increased annotation and computational costs of such extensions should be carefully considered to preserve the real-time processing capability and moderate computational complexity of the current model.

Finally, inference speed is dependent on hardware, software implementation, batch size, and measurement protocol. The results obtained on the NVIDIA A10 platform and the RTX 3060 prototype do not guarantee equivalent performance on low-power edge devices or under multiple concurrent video streams. From an application perspective, future work will further extend the current detection prototype toward temporal behavior logging and classroom activity analysis by integrating behavior tracking, temporal modeling, and a broader range of classroom behavior categories. Future deployment studies should evaluate end-to-end latency, sustained throughput, GPU memory consumption, energy usage, and operational stability in real classroom environments. Model quantization, pruning, knowledge distillation, and adaptive frame sampling could be investigated to improve deployment efficiency. Domain adaptation, uncertainty estimation, privacy-preserving learning, and human-in-the-loop validation are also important directions for developing classroom behavior analytics systems that are generalizable, transparent, and responsible.

6. Conclusions

In this paper, we propose FGD-Net, a classroom behavior detection network designed for fine-grained student behavior analysis in complex classroom scenes. To address the challenges of subtle behavior cues, visually similar behavior categories, multi-scale posture variations, and background interference, FGD-Net introduces three dedicated modules: the Fine-grained Dynamic Recalibration Convolution Block (FDRC), the Multi-path Gated Context Aggregation Block (MGCA), and the Shift-guided Detail Reconstruction Upsampling Block (SDRU). Specifically, FDRC enhances behavior-sensitive local regions, MGCA strengthens multi-granularity contextual semantic aggregation, and SDRU improves spatial detail reconstruction during multi-scale feature fusion.

Extensive experiments on the SCB5 and SCB3 datasets demonstrate the effectiveness of FGD-Net. On the SCB5 dataset, FGD-Net achieves 64.68% ${\mathrm{mAP}}_{50}$ and 46.46% ${\mathrm{mAP}}_{50:95}$, outperforming the strongest baseline by 2.19 and 0.73 percentage points, respectively. On the SCB3 dataset, FGD-Net further obtains the best overall performance with 70.78% ${\mathrm{mAP}}_{50}$ and 50.52% ${\mathrm{mAP}}_{50:95}$, showing its robustness under different data scales and classroom scene distributions. It should be emphasized that the current study evaluates FGD-Net only for three classroom behavior categories, namely hand raising, reading, and writing, because these are the annotated behavior classes provided by the SCB5 and SCB3 datasets used in this work. Therefore, the present results should not be interpreted as demonstrating comprehensive recognition of all classroom behaviors. The supplementary KITTI experiment further indicates that the proposed architectural modifications remain effective under a substantially different object detection setting. Ablation studies verify the positive contribution of FDRC, MGCA, and SDRU. Extending FGD-Net to a broader classroom behavior taxonomy will require datasets with richer and sufficiently balanced annotations, adaptation and retraining of the classification head for an expanded label space, and further evaluation of inter-class confusion among visually similar behaviors. For behaviors such as peer interaction, listening, mobile-device use, or other activities that may depend strongly on temporal evolution and interaction context, additional temporal modeling, student tracking, and multi-frame information may also be required. Future work will therefore focus on extending the current three-class detection framework toward more comprehensive classroom behavior analysis.