Next Article in Journal
SPPs Structure Touch Sensing Method with Microstrip Transmission Line
Previous Article in Journal
Input-Adaptive Dynamic Neural Network for Efficient Object Detection Toward Resource-Constrained Deployment
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Graph Convolutional Network for Action Recognition in Occluded Skeleton Data

1
Reading Academy, Nanjing University of Information Science and Technology, No.219 Ningliu Road, Pukou District, Nanjing 210044, China
2
School of Mathematical, Physical and Computational Sciences, University of Reading, Whiteknights, P.O. Box 217, Reading RG6 6AH, UK
3
CICAEET, Nanjing University of Information Science and Technology, Nanjing 210044, China
*
Author to whom correspondence should be addressed.
Electronics 2026, 15(11), 2311; https://doi.org/10.3390/electronics15112311
Submission received: 4 April 2026 / Revised: 11 May 2026 / Accepted: 16 May 2026 / Published: 26 May 2026
(This article belongs to the Section Artificial Intelligence)

Abstract

Skeleton-based human action recognition has achieved significant progress, but local occlusions and missing joints in complex environments (e.g., occlusion and low-light conditions) still degrade recognition accuracy and stability. Existing GCN-based methods aggregate features uniformly across joints and lack mechanisms to suppress unreliable observations or recover structural semantics under large-area occlusion. To address this, we propose a Robust Occlusion-Compensated Graph Convolutional Network (ROC-GCN) with two complementary components: an adaptive dropout module that suppresses spatiotemporal noise via attention-guided Bernoulli sampling with dynamic spatial–temporal fusion, and an Occlusion Compensation Graph Convolution Module that compensates occluded features through Local–Global Body-Prior-Guided Attention together with feature-guided and multi-hop aggregation. To enable systematic evaluation, we further construct two complementary occlusion benchmarks on NTU RGB+D 60/120 covering spatial-random and spatiotemporal-continuous occlusion, and additionally validate the model on a real-world missing-joint subset. On standard NTU60/120 X-Sub, ROC-GCN improves Top-1 accuracy by +0.41% and +0.48% over the baseline, with the Top-1 standard deviation reduced from 0.61 → 0.17 and 0.47 → 0.10. On the occlusion benchmarks, Top-1 accuracy further improves by +0.98% and +0.73%, and consistent gains are also observed on the real-world missing-joint validation, confirming improved robustness and training stability.

1. Introduction

Human action recognition (HAR) has been widely applied in fields such as intelligent surveillance [1], elderly movement assessment [2], human–robot interaction [3], and sports rehabilitation [4] and has become an important research topic in computer vision and artificial intelligence [5]. Common input modalities include wearable sensor data, vision-based data, and their combinations. With the rapid development of deep learning, vision-based action recognition methods have achieved remarkable progress [6], particularly models that integrate Graph Neural Networks (GNNs) and Temporal Convolutional Networks (TCNs), which have shown rapid advancement in recent years [7].
Although GCN-based methods such as ST-GCN [8], 2s-AGCN [9], CTR-GCN [10], and GATCN [11] have achieved outstanding performance on standard benchmarks, their effectiveness rests on the implicit assumption that the input skeleton is complete and reliable. In practical deployment, this assumption frequently fails due to occlusion, low-light conditions, or sensor noise [12], exposing three limitations that are common to current GCN-based methods rather than specific to any individual model:
(1)
Existing GCN-based methods aggregate features uniformly across all joints and lack the ability to distinguish reliable joints from corrupted observations caused by occlusion, low-light conditions, or pose estimation failures. Consequently, noisy joints are propagated through the graph together with valid motion features, which significantly reduces representation robustness under incomplete skeleton observations.
(2)
Most existing methods also exhibit limited capability in recovering missing structural semantics under large-area occlusion scenarios. When semantically important joints (e.g., hands or head regions) and their neighboring joints are simultaneously missing, local graph aggregation alone becomes insufficient for reliable semantic inference. Without effective global structural reasoning and body-prior-guided compensation, the model struggles to reconstruct coherent motion representations from incomplete skeleton structures.
(3)
Mainstream skeleton datasets (e.g., NTU RGB+D 60 [13]) contain limited occlusion scenarios, hindering models from learning robust representations under incomplete observations and resulting in poor generalization in complex environments.
To address these limitations, we propose a Robust Occlusion-Compensated Graph Convolutional Network (ROC-GCN) based on BPG-GCN [14]. The main contributions include the following four aspects:
(1)
Instead of perturbing the graph stochastically, we propose an Adaptive Body-Prior-Guided Drop Module that jointly suppresses spatial and temporal noise via spatial dropout, temporal dropout, and a dynamic weight fusion mechanism. By adaptively balancing spatial and temporal dropout ratios according to the noise pattern of each input, the module enhances model robustness and efficiency under complex conditions.
(2)
To prevent occluded regions from being recovered through neighbors that may themselves be occluded, we develop a Local–Global Body-Prior-Guided Attention mechanism that uses body prior knowledge with a global body information channel, allowing visible motion information to compensate for occluded regions through anatomically related rather than merely adjacent joints.
(3)
To break the reliance on one-hop aggregation under severe occlusion, we introduce a feature-guided mechanism that incorporates structural topology into image features, together with a multi-hop mechanism that enlarges the receptive field and captures long-range joint dependencies, enabling more comprehensive feature modeling when local neighborhoods are corrupted.
(4)
To enable systematic evaluation of occlusion robustness, we construct two complementary occlusion datasets covering diverse occlusion patterns, providing a controllable protocol for assessing model behavior under different types of incomplete observations, while further validating the robustness of the proposed method on real-world missing-joint datasets.

2. Related Work

2.1. GCN-Based Skeleton Action Recognition

Skeleton-based action recognition methods model the human body as a graph structure where joints serve as nodes and bones as edges. Compared to approaches relying on RGB frames or optical flow, skeleton data is more compact and robust than RGB-based representations, particularly in scenarios involving occlusion, illumination variations, or complex backgrounds.
This inherent graph-structured property has inspired graph-based modeling approaches. Liu et al. [15] constructed spatiotemporal adjacency matrices to effectively capture spatiotemporal dependencies for human action recognition. Subsequently, adaptive adjacency matrices derived from physical body structure or semantic priors were proposed to enhance inter-joint dependency modeling and improve representational capacity for complex actions [16]. However, fixed graph structures still limit the flexibility of skeleton modeling [17], making it difficult to capture diverse action patterns, especially with limited training samples. To address this, Rong et al. [18] introduced DropEdge, which randomly removes edges during training to improve model generalization. Cheng et al. [19] developed DropGraph, which randomly discards nodes and their neighborhoods to simulate missing skeleton data, thereby enhancing model robustness to incomplete skeletons. Brau and Jiang [20] imposed constraints on the lengths of 3D body parts and utilized independently trained networks to learn prior distributions of 3D poses, further strengthening pose constraints. Hu et al. [21] employed a Multi-level Topological Channel Attention Network (MTCAN) to integrate coarse-to-fine topological features via channel attention and limb coordination modules inspired by body prior knowledge. Yin et al. [22] proposed a multi-branch attention graph convolution network that employs several transformation matrices to extract feature information contributing to occluded keypoints. Hu et al. [23] proposed an AWD-GCN that constructs dynamic adjacency matrices via attention mechanisms to model multi-partition relationships under diverse actions, accompanied by multi-scale and multi-level position attention for enhanced spatial feature discrimination. To better balance efficiency and performance, Wu et al. [24] proposed ASR-GCN with a gated Adaptive Spatial Information Reconstruction Unit for adaptive spatial feature processing, which achieves excellent performance on skeleton-based action recognition benchmarks with minimal parameter and computational overhead. Beyond accuracy, HPI-GCN [25] accelerates inference via re-parameterization and over-parameterization, while PCCTR-GCN [26] introduces a pose correction module together with channel-wise topology refinement to mitigate coordinate errors induced by viewpoint changes and occlusion.

2.2. Occluded Human Skeleton-Based Action Recognition

In practical scenarios, human skeleton sequences are often affected by factors such as perspective changes, human interactions, environmental occlusions, and detection errors. These factors can lead to varying degrees of joint loss or positional shifts, resulting in a significant decline in action recognition performance [27]. To enhance the robustness of skeleton-based action recognition models under occlusion conditions, researchers have proposed multiple approaches.
One group of methods enhances the model’s adaptability to incomplete skeletons through simulated occlusion data augmentation. PD-GCN [28] simulates occlusion scenarios to fully extract both local and global skeletal information, improving recognition accuracy for occluded skeletons. ASDA [29] and MSR-Net [30] employ adversarial occlusion generators to address the scarcity of occluded samples in real-world datasets. SkeletonMAE [31] occludes joints at both frame and joint levels, encoding only unoccluded joints while predicting masked ones. DAG [32] effectively simulates diverse occlusion scenarios through joint masking with instance-paste augmentation. Wu et al. [33] decoupled occlusion handling into a two-stage pipeline, first imputing missing keypoints via a spatiotemporal generative adversarial interpolation network (STGAIN), then performing action recognition through a frame-joint motion masked autoencoder (FJMAE). Chen et al. [34] introduced a self-supervised KNN-imputation framework (IosPSTL) that achieves state-of-the-art performance on the occluded versions of NTU-60 and NTU-120. Vernikos and Spyrou [35] reconstructed missing joints via a 3D generative adversarial network for occluded HAR.
Another group of methods enhances model stability at the graph structure level. Song et al. [36] proposed a multi-stream model based on GCN, aiming to learn rich features across all joints to reduce the impact of data degradation and improve the robustness of occluded data. Chen et al. [37] propose multi-view spatiotemporal occlusion and introduce the MGL module to fuse multi-view features. Several studies have attempted to directly infer missing keypoints. OPEC-NET [38] utilizes graph convolutional networks to infer occluded joints. STIP [39] addresses occlusion by enhancing keypoint semantic information. MSFGCN [40] constructs a multi-stream network where each branch handles distinct occlusion scenarios. For actions with limited range, it proposes extracting multi-scale motion features to distinguish actions with different motion ranges. Guo et al. [41] proposed an overcomplete Graph Convolutional Denoising Autoencoder (GCDAE) that serves as a preprocessing module for pretrained backbones, rectifying noisy joints while preserving intact details to improve robustness under various skeleton noise conditions.
However, existing methods still have certain limitations. On one hand, most current approaches model occlusion from a single perspective—such as simulating missing joints through data augmentation only, or enhancing feature aggregation capabilities solely at the graph level—making it difficult to simultaneously address both the noise introduced by occlusion and the loss of structural information. On the other hand, most approaches assume occlusion patterns exhibit consistency across spatial or temporal dimensions, lacking unified modeling capabilities for diverse occlusion mechanisms (e.g., random joint absence or continuous structural occlusion). This results in limited robustness and stability of models when handling complex occlusion scenarios.
Based on the above analysis, we propose an action recognition method for complex occlusion scenarios from two perspectives: ‘noise suppression’ and ‘structural compensation’. This method dynamically suppresses unreliable joints and anomalous frames across spatiotemporal dimensions through an adaptive drop mechanism. It combines Local–Global Body-Prior-Guided Attention with body prior knowledge to perform structural modeling and feature propagation in occluded regions, thereby achieving stable and consistent performance improvements across diverse occlusion patterns.

3. Method

To address issues such as missing keypoints, noisy point interference, and local structural damage caused by occlusion, we propose an action recognition method based on adaptive drop and occlusion compensation. This method consists of two core modules: (1) the Adaptive Body-Prior-Guided Drop (A-BPGD) module; (2) Occlusion Compensation Graph Convolutional Network (OC-GCN). The overall architecture is illustrated in Figure 1.
A-BPGD adaptively filters unreliable nodes and anomalous frames by learning spatiotemporal dimensional weights, thereby enhancing input skeleton stability at the data level. In contrast, OC-GCN integrates local–global prior attention with feature-guided and multi-hop mechanisms to compensate occluded regions at the feature level, fortifying inference robustness under severe occlusion.

3.1. Adaptive Body-Prior-Guided Drop Module

Random graph perturbation strategies such as DropEdge [18] and DropGraph [19] typically assume that all joints contribute equally during feature propagation and therefore apply stochastic dropping in a uniform manner. However, under occlusion conditions, corrupted observations are usually distributed unevenly across both spatial joints and temporal frames. Uniform random dropping may therefore remove informative motion cues while failing to effectively suppress unreliable observations. To address this issue, we propose an Adaptive Body-Prior-Guided Drop (A-BPGD) module, which leverages attention-guided Bernoulli sampling to suppress noise in both the spatial and temporal dimensions while preserving informative features. A dynamic weight fusion mechanism is introduced to adaptively balance the spatial and temporal dropout strategies. The architecture of the Drop S Block and Drop T Block is illustrated in Figure 2.

3.1.1. Drop S Block

The Drop S Module identifies and suppresses noisy joints or unreliable keypoints. Given an input skeleton sequence X R C × T × V , where C denotes the number of channels, T denotes the length of the time series, and V denotes the number of joints. The Drop S Block first extracts joint-level importance through dual-path feature compression. Specifically, max pooling and mean pooling operations are applied along the temporal and channel dimensions, respectively, to obtain compressed features X max R 1 × V and X avg R 1 × V . Subsequently, we use two non-shared multi-layer perceptrons to generate a spatial attention map F s R 1 × V . The overall attention process can be summarized as
F s = Attention ( X ) = σ MLP 1 ( Max ( X ) ) + MLP 2 ( Mean ( X ) )
where σ ( · ) denotes the Sigmoid function. After obtaining the spatial attention map, we employ an attention-guided Bernoulli sampling strategy to determine which joints should be dropped. Spatial attention maps characterize the feature reliability of each joint: joints with clear features and distinct motion patterns typically receive higher attention scores, while joints with degraded features due to occlusion or noise exhibit lower scores. Based on this characteristic, our approach treats the attention scores as probability parameters of a Bernoulli distribution and performs independent sampling for each joint node. Since joints with higher attention scores possess a greater probability of retention, this mechanism can adaptively protect critical joints while dropping unreliable noise joints with high probability. Then, we obtain the dropout matrix M 1 , multiply the adjacency matrix A p by the dropout matrix M 1 , and obtain the space mask matrix M s R 1 × V . By multiplying the spatial masking matrix element-wise with the input skeleton data, we get the skeleton data X 1 R C × T × V with spatial noise removed. This process can be represented as
M 1 = Bernoulli ( F s )
M s = M 1 × A p
X 1 = X M s
where × denotes matrix multiplication and ⊙ denotes element-wise multiplication. A R V × V [8] denotes the adjacency matrix, derived from body prior knowledge, which represents the connections between joints. A p denotes the p-th power of the adjacency matrix A, where the exponent p controls the neighborhood range to be dropped.

3.1.2. Drop T Block

The skeleton data X R C × T × V is fed into the temporal dropout module and then the temporal attention map F t R 1 × T is generated using the same method. After obtaining the temporal attention map, determine the frames to be dropped using a Bernoulli distribution based on attention, thus obtaining a dropout matrix M 2 . To ensure more thorough discarding, a one-dimensional max pooling operation (MaxPool1d) is applied to remove each selected frame and its adjacent frames, analogous to the spatial dropout operation. Then, we obtain a time discard mask matrix M t and multiply the input skeleton data X by the temporal mask matrix M t to obtain the skeleton data X 2 R C × T × V after temporal discard. The time dimension discard process can be represented as
F t = Attention ( X )
M 2 = Bernoulli ( F t )
M t = MaxPool 1 d ( M 2 )
X 2 = X M t
Here, ⊙ denotes element-wise multiplication.

3.1.3. Dynamic Weight Fusion Module

Spatial drop focuses on filtering out anomalous keypoints (e.g., noisy data), while temporal discard concentrates on removing abnormal frames (e.g., occlusion or motion-blurred frames). The importance of these two types of noise varies across different scenarios. Therefore, we design a dynamic weight fusion module to adaptively balance the contributions of the spatial and temporal drop. The skeleton data X 1 from the Drop S Block and the skeleton data X 2 from the temporal dropout modules are dynamically fused to obtain the final skeleton data X ˜ R C × T × V , combining both spatial and temporal dropout. This process can be expressed as follows:
X ˜ = α · X 1 + ( 1 α ) · X 2
where α denotes the fusion weight, which is generated by a lightweight attention mechanism. The process is described as follows:
α = σ FC Concat ( X 1 , X 2 )
FC denotes a fully connected layer that outputs a scalar value, which is normalized to the range [ 0 ,   1 ] through a Sigmoid function. The dynamic weight fusion mechanism adaptively adjusts the contribution of temporal and spatial dropout, thereby improving model flexibility and recognition accuracy.

3.2. Occlusion Compensation Graph Convolution Module

The Occlusion Compensation Graph Convolution Module is designed to address action recognition under occluded situations. When occlusion occurs, the randomly distributed visible joints are often insufficient for accurate action inference. However, the skeleton data embodies strong anatomical priors—for example, arm movements are coordinated with the torso, and symmetric limbs (e.g., left and right arms) often exhibit correlated motion patterns. Leveraging these priors, we partition the skeleton into five semantic body parts and integrate both global and local information [42]. The OC-GCN module adopts a Temporal Convolutional Network–Graph Convolutional Network (TCN–GCN) architecture and embeds a Local–Global Body-Prior-Guided Attention (LG-BPGA) mechanism, a feature-guided mechanism, and a multi-hop mechanism in the intermediate layers. After the denoised skeleton data X ˜ are processed by the TCN-GCN to extract the intermediate feature map Z, the LG-BPGA module performs occlusion compensation to generate the enhanced feature map Z and then it is further refined through feature-guided mechanism and a multi-hop mechanism. The overall architecture is illustrated in Figure 3.

3.2.1. Local–Global Body-Prior-Guided Attention Mechanism

Conventional attention mechanisms generally model joint relationships in a global and homogeneous manner while insufficiently considering the anatomical structure of the human body. Under occlusion conditions, however, correlations among joints are often highly dependent on local body-region semantics and structural connectivity. Relying solely on global attention may therefore weaken the ability to recover missing semantic information from structurally related joints. To address this issue, the proposed LG-BPGA module partitions the skeleton into five semantic regions (torso, left arm, right arm, left leg, right leg) based on body prior knowledge. It compensates for occlusion by integrating local attention with global features, as illustrated in Figure 4.
Local Feature Extraction:
Given the intermediate feature map Z R C × T × V , it is partitioned along the spatial dimension into five regions: the torso, left arm, right arm, left leg, and right leg, denoted as Z i R C × T × V i , i [ 1 ,   5 ] . V i denotes the set of joints in each segment, and follows the left–right symmetry rule. Then, each regional feature map (torso, left arm, right arm, left leg, and right leg) is first subjected to max pooling and average pooling, followed by a 2D convolution to compute channel-wise attention, producing local attention feature maps F i c R C × 1 × 1 , i [ 1 ,   5 ] . To reduce parameter overhead without compromising performance, the hidden layer size is set to R C / r × 1 × 1 , where r denotes the reduction rate. Finally, the local attention feature map F i c is multiplied element-wise with the corresponding body part feature map Z i , and the results are concatenated to derive the final output Z 6 . This process can be represented as
F i c = σ ( Conv 2 d [ MaxPool 2 d ( Z i ) ; AvgPool 2 d ( Z i ) ] )
Z i = Z i F i c
Z 6 = [ Z 1 ; Z 2 ; Z 3 ; Z 4 ; Z 5 ]
Here, σ ( · ) denotes the Sigmoid function; ⊙ denotes element-wise multiplication and [ ; ] denotes concatenation in spatial dimensions.
Global Feature Fusion:
After obtaining Z 6 , the global image feature map Z is fused with the feature map Z 6 to produce a feature map Z R C × T × V that contains both global and local information. This process can be represented as
Z = Concat ( Z 6 , Z )

3.2.2. Feature-Guided Mechanism

Standard graph convolution propagates information only between immediate neighbors and offers no mechanism to inject the skeletal topology directly into the feature space. Under occlusion, this leaves the model relying entirely on message passing for recovery. To address this, we propose the feature-guided mechanism. By leveraging the topological structure of skeletons, this mechanism propagates semantic information from visible joints to missing regions, thereby enhancing the completeness of the overall feature representation.
To model the spatial topological relationships among human joints and enable information propagation from visible to occluded regions, skeleton sequences are represented as graph structures. Specifically, skeleton data is represented as a graph G = ( V , E ) . V denotes the set of vertices, corresponding to joints in the human skeleton. V = { v i } , i [ 1 ,   5 ] , v i denotes the subset of joints corresponding to the torso, left arm, right arm, left leg, and right leg. E denotes the set of edges between nodes, describing the connection relationships between joints. The edges are constructed based on the physical connection between adjacent joints in the natural skeletal structure of the human body. The node features of graph G are represented by a feature matrix M R C × V , where C denotes the number of feature channels, and V represents the number of joints.
To recover obscured or missing node information, the feature map Z is projected onto a regularized structural grid constructed based on the skeleton topology. By exploiting the relative topological relationships among nodes within this grid, structured interpolation sampling is performed on the features of visible nodes, thereby generating an enhanced feature map Z .
The structural grid derived from skeleton topology enables feature interpolation to recover human structural semantics from visible regions when joints are missing. The resulting enhanced feature maps Z are then processed by graph convolutional neural networks, preserving local details from the original features while incorporating global topological relationships. This improves the model’s robustness to occlusion.

3.2.3. Multi-Hop Mechanism

When an occluded joint’s immediate neighbors are themselves occluded, one-hop aggregation no longer has a reliable source of information. The multi-hop mechanism therefore enlarges the receptive field along the skeletal topology with learnable per-hop weights, allowing the model to adaptively draw on more distant but visible joints when local context is corrupted. To capture distant dependencies and enhance structural inference capabilities under extensive occlusion conditions, we introduce a multi-hop mechanism. Starting with the initial node feature matrix M derived from enhanced feature maps Z , we update the node feature matrix M through the multi-hop mechanism. The update process can be summarized as follows:
M = σ k = 1 K w k · ( W ( 0 ) M + W ( 1 ) M A ^ k )
Here, σ ( · ) represents the Sigmoid function, w k is a learnable weight matrix used to adjust the importance of different hops for features. k denotes the number of jumps, K denotes the maximum number of hops. W ( 0 ) and W ( 1 ) are the weight matrices corresponding to the self and neighbor transformations, respectively. A ^ is the symmetrically normalized version without self-connections, A ^ k denotes k-hop neighborhood. The normalization process is as follows
A ^ = D ˜ 1 2 A ˜ D ˜ 1 2
where A ˜ = A + I , I is the identity matrix, and A is the adjacency matrix. D ˜ = diag ( A ˜ 1 v ) is a diagonal matrix, 1 v R V represents a vector of dimension V. When the adjacency matrix is not normalized, variations in node degrees may cause certain features to be overemphasized while others are suppressed. Normalization ensures balanced feature propagation across nodes and mitigates the risk of excessive smoothing.

4. Experiment

4.1. Dataset

4.1.1. Standard Dataset

NTU-RGB+D 60: We conducted experimental validation on the Cross-Subject (X-Sub) partition of the NTU RGB+D 60 [13] dataset. This dataset contains 60 action categories with a total of 56,880 samples, performed by 40 subjects. Cross-subject partitioning evenly divides the 40 participants into a training group (20 individuals) and a test group (20 individuals), ensuring no overlap between the two groups to evaluate the model’s cross-subject generalization capability.
NTU-RGB+D 120: We conducted experimental validation on the Cross-Subject (X-Sub) partition of the NTU RGB+D 120 [43] dataset. This dataset is an extended version of NTU RGB+D 60, comprising 120 action categories with a total of 114,480 samples completed by 106 subjects. The Cross-Subject partition evenly divides the 106 participants into a training group (53 subjects) and a testing group (53 subjects), with no overlap between the training and testing sets.

4.1.2. Occlusion Dataset

To systematically evaluate the robustness of models under occluded joints, we construct two skeleton occlusion datasets with distinct occlusion mechanisms based on the NTU RGB+D 60 and NTU RGB+D 120 datasets. The NTU60 dataset employs a spatial random occlusion strategy, while the NTU120 dataset utilizes a spatiotemporal continuous occlusion strategy. These aim to simulate two typical failure modes commonly encountered in real-world applications: joint-level noise and structural occlusion. This approach enables a comprehensive evaluation of the model’s adaptability and robustness in complex occlusion scenarios.
NTU60 Occlusion Dataset (Spatial Random Occlusion): The NTU60 occlusion dataset is constructed using a spatially random occlusion strategy. For each skeleton sequence, a fixed proportion of joints is randomly occluded, producing spatially discontinuous occlusion patterns. The occlusion rate is set to 50% (approximately 12 of 25 joints per sample). The dataset follows the Cross-Subject protocol of NTU RGB+D 60 and preserves the original train–test splits. During training, 10 occlusion variants are generated per sample using different random seeds to improve robustness to diverse spatial occlusion patterns. The final dataset contains 40,091 training samples and 16,487 testing samples (56,578 total) covering all 60 action classes. As shown in Figure 5, occlusions occur across different body regions and action categories, producing irregular and spatially discontinuous missing patterns in the skeletal structure.
NTU120 Occlusion Dataset (Spatiotemporal Continuous Occlusion): The NTU120 occlusion dataset is constructed using a spatiotemporal continuous occlusion strategy, which is adopted with an occlusion rate of 50% (approximately 12 of 25 joints per sample). The dataset follows the Cross-Subject protocol of NTU RGB+D 120 and preserves the original training and testing splits. During training, 10 occlusion variants are generated for each sample by initializing occlusion from different seed joints, enabling the model to learn robust representations under continuous occlusion over spatially varying regions. The final dataset contains 63,026 training samples and 50,919 testing samples, totaling 113,945 samples across all 120 action classes.
In the spatial dimension, occluded joints are not independently sampled. Instead, occlusion regions are generated via topology-constrained expansion along skeletal connections from randomly selected seed joints, producing spatially contiguous masked areas. Peripheral joints (e.g., hands, feet, and head) are preferentially selected as seeds, yielding semantically coherent limb-level occlusions that reflect realistic distal-body occlusion patterns. In the temporal dimension, the occlusion configuration is fixed across all frames within a sequence, enforcing temporal consistency. As shown in Figure 6, the set of occluded joints remains unchanged despite pose variations. This design simulates persistent real-world occlusion (e.g., static viewpoints or object interference) while avoiding stochastic masking.
The two occlusion strategies employed in the NTU60 and NTU120 occlusion datasets exhibit distinct complementary design objectives. Spatial random occlusion primarily characterizes joint-level noise and detection failures, challenging model robustness against high-frequency, non-structural disturbances, while spatiotemporal continuous occlusion focuses on simulating structural information loss caused by environmental factors, placing greater emphasis on the model’s ability to model skeletal topology and temporal consistency.
By introducing differentiated occlusion mechanisms under varying data scales and action complexities, we enable a more comprehensive analysis of model performance across diverse real-world occlusion scenarios.

4.2. Implementation Details

All experiments were conducted on an NVIDIA GeForce RTX 4080 SUPER GPU using the PyTorch (2.5.1) framework. The training process employed an SGD optimizer (with Nesterov momentum), an initial learning rate of 0.1, and learning rate decay at epochs 35 and 55 (decay rate 0.1), with a total of 65 training epochs. The batch size was set to 64, with a weight decay coefficient of 0.0004. A warm-up strategy was applied during the first 5 epochs to gradually increase the learning rate.
To ensure the reliability and stability of experimental results, each model was trained using five distinct random seeds. Models were separately trained on four different input modalities (Joint, Bone, Joint Motion, Bone Motion), with average results and standard deviations reported. The final results were obtained through weighted fusion, with fusion weights set to [ 0.6 , 0.6 , 0.4 , 0.4 ] .
In the adaptive dropout module, the hidden layer reduction ratio r for spatial and temporal attention is set to 4. The dynamic weight fusion module adaptively adjusts the spatiotemporal dropout ratio via a lightweight fully connected layer, with weight α normalized to the [ 0 ,   1 ] range through a Sigmoid function. Top-1 accuracy and Top-5 accuracy are adopted as primary evaluation metrics. Top-1 accuracy measures how well the model’s top-ranked category matches the true label, while Top-5 accuracy evaluates whether the true label appears among the model’s top five recognized categories.

4.3. Comparison with Other Methods

4.3.1. Comparison with Mainstream Methods

To comprehensively validate the effectiveness of the proposed method, we conducted a systematic comparison with mainstream skeleton-based action recognition models on NTU RGB+D 60 and NTU RGB+D 120. This comparison encompasses traditional GCN architectures, spatiotemporal Transformer structures, and recent models focused on occlusion robustness. The table lists the parameter scales of each method and their performance under the NTU60 X-Sub and NTU120 X-Sub settings.
The results shown in Table 1 demonstrate that classic GCN-based models (such as ST-GCN, 2s-AGCN, and SGN) exhibit robust performance in standard scenarios. Subsequent approaches like CTR-GCN, BlockGCN, and MTGCN further improved recognition accuracy through more refined topological modeling or lightweight strategies. Notably, BlockGCN achieved 93.1% accuracy on the NTU60 X-Sub dataset, demonstrating the crucial role of skeletal topology in spatiotemporal modeling. However, these models are predominantly optimized for unobstructed conditions and lack dedicated mechanisms to handle structural discontinuities and semantic information loss caused by occlusion. Consequently, they exhibit significant performance degradation when confronted with complex inputs such as occlusions or noise. To address the common occlusion challenge in real-world scenarios, RA-GCN and MSFGCN enhance feature recovery capabilities through multi-stream fusion and multi-scale geometric representations; Transformer-based architectures like SkeletonMAE and VQM-VAE explicitly reconstruct missing joint positions or semantic information through reconstruction-based frameworks, thereby improving adaptability to structurally incomplete inputs. Despite demonstrating positive progress in occlusion robustness, these methods remain constrained in practical application due to structural complexity, high parameter counts, and high computational costs. The proposed ROC-GCN demonstrates strong performance on both the NTU60 and NTU120 datasets. More importantly, ROC-GCN achieves this with a parameter size of only 2.51 M, significantly lower than most advanced occlusion-targeted methods. These results highlight the effectiveness of the proposed design in achieving an effective balance between model efficiency and robustness to occlusion.

4.3.2. Performance Comparison of Standard Datasets

To evaluate the performance and generalization capability of the proposed method, experiments were conducted on NTU RGB+D 60 and NTU RGB+D 120. Each dataset was trained with five random seeds, reporting the mean and standard deviation of Top-1 and Top-5 accuracy. Performance was analyzed for both modality fusion and individual modality (Joint, Bone, Joint Motion, and Bone Motion).
Table 2 compares ROC-GCN with the baseline on the NTU60 and NTU120 datasets, demonstrating consistent performance gains and improved training stability across both benchmarks.
Under the NTU60 X-Sub protocol, ROC-GCN achieves 89.77% Top-1 and 98.46% Top-5 accuracy, improving the baseline by 0.41 and 0.18 percentage points, respectively. In addition to accuracy gains, training stability is substantially enhanced, with Top-1 standard deviation reduced from 0.61% to 0.17% and Top-5 standard deviation from 0.28% to 0.02%, indicating markedly improved training consistency.
On NTU120 X-Sub, ROC-GCN similarly achieves superior performance, with Top-1 accuracy achieving 86.72%, gaining 0.48 percentage point improvement. Meanwhile, the Top-1 standard deviation decreases from 0.47% to 0.10%, demonstrating strong robustness under large-scale and diverse action distributions.
The consistent reduction in performance variance across both datasets further validates the effectiveness of the proposed adaptive dropout mechanism in suppressing perturbations caused by noisy joints and anomalous frames.
Table 3 presents the per-modality comparison between the baseline and our method on both datasets. On the NTU60 dataset, the proposed method consistently improves recognition accuracy across all modalities, outperforming the baseline in overall performance. The largest gain is observed in the Bone Motion modality (+0.71%), which can be attributed to the effective temporal noise suppression provided by the Adaptive Drop Module. By dynamically adjusting spatiotemporal drop ratios, the module enhances the robustness of motion feature extraction under noisy or occluded conditions.
Training stability is also significantly improved across all modalities. The standard deviation is reduced from 0.83% to 0.51% for Joint, 1.62% to 1.01% for Bone, 1.40% to 0.87% for Joint Motion, and 0.83% to 0.48% for Bone Motion, indicating more consistent optimization behavior.
On the NTU120 dataset, the Joint modality achieves the highest accuracy (83.05%) in both models, confirming that joint coordinates remain the primary semantic representation for action recognition. Our method further improves accuracy by 0.25% while reducing standard deviation by 18.8%, suggesting that the Local–Global Body-Prior-Guided Attention mechanism effectively focuses on key action regions (e.g., hands, legs) while reducing interference from background joints and extraneous movements through global semantic guidance.
Among motion-related modalities, Bone Motion (+1.23%) shows the largest performance gain, highlighting the model’s enhanced ability to compensate for missing structures under complex spatiotemporal dynamics. Meanwhile, the marked reduction in standard deviation (e.g., 27.3% in Joint Motion) indicates effective suppression of spatiotemporal noise accumulation, resulting in more stable training and more consistent prediction.
Overall, experimental results on standard datasets demonstrate that the proposed method consistently improves recognition accuracy and substantially enhances training stability under unobstructed conditions. The observed accuracy gains, together with the consistent reduction in standard deviation across both datasets, indicate improved robustness to noisy joints and anomalous inputs. This mitigates training perturbations arising from sensor noise and keypoint detection errors, further confirming the effectiveness and general applicability of the proposed method in common application scenarios.

4.3.3. Performance Comparison of Occlusion Datasets

To evaluate the accuracy and robustness of the model in recognizing actions under complex occlusion scenarios, we designed two occlusion datasets based on NTU60 and NTU120 for assessment. Table 4 shows the model’s performance on the NTU60/120 occlusion dataset in detail.
On the NTU60 occlusion dataset, our method achieves a Top-1 accuracy of 79.72%, improving the baseline by 0.98 percentage points, indicating that ROC-GCN effectively mitigates performance degradation under spatial random occlusion. Training stability is also improved, with Top-1 and Top-5 standard deviations reduced by 12.6% and 12.7%, respectively, demonstrating more consistent and reliable recognition across spatial random occlusion and levels of joint loss.
On the NTU120 occlusion dataset, our method achieves consistent performance improvements, with Top-1 and Top-5 accuracy increasing by 0.73 and 0.60 percentage points over the baseline, respectively. The model also exhibits markedly enhanced stability, as reflected by 26.7% and 65.6% reductions in Top-1 and Top-5 standard deviations. These results indicate that ROC-GCN effectively suppresses noise propagation under spatiotemporally continuous occlusion, maintaining stable optimization and inference despite structural information loss.
Figure 7 illustrates the comparative results across the four datasets.
As shown in Table 5, the proposed method demonstrates targeted performance improvements across different skeleton modalities and occlusion settings. Its behavior is closely related to the adopted occlusion strategy and the representational characteristics of the skeleton modality.
As further illustrated in Figure 8, the per-modality accuracy on the NTU60/120 occlusion datasets reflects these targeted improvements.On the NTU60 occlusion dataset, the proposed method achieves consistent performance gains across all four skeleton modalities, with the most pronounced improvement observed in the Joint modality. Specifically, Top-1 accuracy increases by 5.32 percentage points over the baseline, demonstrating the enhanced capability of the proposed architecture to model spatial dependencies under random joint absence conditions. When spatial occlusions disrupt local joint consistency, the model effectively exploits the global skeletal topology to integrate information from visible joints, thereby alleviating performance degradation caused by occlusion. Moreover, the standard deviation in the Joint modality is substantially reduced from 11.29 to 2.81, further confirming the improved training stability under Spatial Random Occlusion scenarios.
On the NTU120 occlusion dataset, the Joint Motion and Bone Motion modalities achieved improvements of 1.66 and 1.16 percentage points, respectively. These results demonstrate that under spatially continuous and temporally consistent occlusion conditions, the proposed Occlusion Compensation Graph Convolution Module and Local–Global Body-Prior-Guided Attention mechanism synergistically enhance the model’s ability to capture dynamic temporal action variations and relative joint motion relationships. Consequently, the model maintains stable recognition performance even in scenarios with missing structural information.
Overall, experimental results on the occlusion benchmarks comprehensively validate the effectiveness of the proposed method under complex input conditions. By dynamically suppressing unreliable joints across both spatial and temporal dimensions, the Adaptive Body-Prior-Guided Drop substantially reduces sensitivity to anomalous inputs, thereby mitigating feature degradation induced by occlusion and detection errors. Under spatiotemporal continuous occlusion, the coordinated interaction between the Local–Global Body-Prior-Guided Attention mechanism, the feature-guided mechanism, and multi-hop mechanism enables more complete exploitation of structural information from visible joints. This facilitates robust inference of action semantics in occluded regions and alleviates performance loss arising from structural context disruption. Furthermore, the two occlusion datasets constructed in this study provide a systematic and controllable evaluation framework. By incorporating complementary occlusion mechanisms—Spatial Random Occlusion and Spatiotemporal Continuous Occlusion—they enable comprehensive training and assessment across diverse occlusion patterns, effectively reducing evaluation bias caused by insufficient or uneven occlusion distributions in standard data.

4.3.4. Validation on Real-World Missing Joints

While the synthetic occlusion benchmarks enable controlled evaluation, real-world joint losses caused by sensors exhibit more complex and less predictable patterns. To assess whether the robustness observed under synthetic settings transfers to practical scenarios, we further evaluate ROC-GCN on a real missing-joint subset of NTU RGB+D 120 (X-Sub).
The subset is obtained by intersecting the X-Sub test split with the official samples_with_missing_skeletons.txt released with NTU RGB+D 120, which enumerates sequences containing missing or invalid joint coordinates in the raw Kinect output. Sequences with entirely absent joint data were discarded and only samples with partial joint loss were retained. No synthetic masking is applied. This yields 179 samples, for which we record the joint-visibility ratio j [ 0 ,   1 ] . Figure 9 illustrates a representative real-world missing-joint sample.
As shown in Table 6, ROC-GCN improves Top-1 accuracy by 3.36 percentage points on the full real-missing set, and the improvement scales with the severity of spatial loss, reaching + 18.18 , + 18.42 , and + 21.43 on the j < 0.95 , j < 0.80 , and j < 0.70 subsets, respectively, indicating that the proposed compensation mechanisms are selectively activated precisely when the visible-joint set is small. These results confirm that the robustness observed on the synthetic spatial benchmarks transfers to naturally occurring sensor-induced joint losses.

4.3.5. Per-Class Accuracy Analysis

In the class-wise evaluation on the NTU60 random occlusion dataset shown in Figure 10 and Figure 11, the proposed model achieves consistent improvements across multiple representative action categories. For fine-grained upper-limb interaction actions—such as A11: reading, A28: phone call, and A29: playing with phone/tablet—the method maintains high recognition accuracy even when key semantic joints (e.g., hands) are partially occluded. This robustness arises from global structural reasoning and neighborhood-aware dynamic compensation, which enable reliable inference under incomplete local observations. Furthermore, for action categories with distinct semantic representations in the head or torso regions—such as A44: headache, A45: chest pain, and A47: neck pain—the model yields markedly larger improvements over the baseline. Because discriminative cues are confined to specific body regions, global structural context alone is often insufficient under occlusion. The proposed Local–Global Body-Prior-Guided Attention mechanism more effectively reconstructs regional semantics from visible joints, improving recognition reliability. These results demonstrate clear advantages in handling actions governed by subtle upper-limb motion or localized semantic cues, while effectively mitigating representation loss caused by occlusion of critical joints.

4.3.6. Computational Efficiency Analysis

To provide a comprehensive evaluation of computational complexity and inference efficiency, we perform a comparison between the proposed ROC-GCN and the original BPG-GCN in terms of parameter count (Params), computational cost (FLOPs), and average inference speed. The results are presented in Table 7.
The improved model increases the parameter count from 1.82 M to 2.51 M and computational cost from 15.86 GFLOPs to 26.66 GFLOPs, primarily due to the multi-hop and feature-guidance mechanisms that enhance feature aggregation and spatial modeling. Despite this increase, inference time increases moderately from 17.67 ms to 24.45 ms, maintaining 40.89 FPS and meeting real-time requirements. These results indicate that the proposed model achieves stronger spatiotemporal representation and improved occlusion robustness while preserving high inference efficiency. To quantify the contribution of each component, ablation studies were performed on the NTU60 dataset under identical training and evaluation settings, with modules progressively introduced.

4.3.7. Ablation Experiment

Table 8 reports the ablation results of each proposed module, using BPG-GCN as the baseline and progressively incorporating the adaptive dropout module, the Local–Global Body-Prior-Guided Attention (LG-BPGA) mechanism, and the combined multi-hop and feature-guided design.
Introducing the adaptive dropout module improves Top-1 and Top-5 accuracy by 0.55 and 0.39 percentage points, respectively. This module suppresses spatially noisy joints and temporally abnormal frames, thereby improving the quality of input data. The dynamic weight fusion mechanism adaptively adjusts the spatiotemporal discard ratio across different scenarios, enabling the model to flexibly handle various types of noise interference. Building on this, integrating the LG-BPGA mechanism yields a further 0.33 percentage point gain in Top-1 accuracy, reaching 89.56%. By modeling local body semantics and incorporating global information, the model better preserves structural coherence and compensates for missing information under occlusion. Finally, adding the multi-hop and feature-guided mechanisms further increases Top-1 accuracy by 0.28 percentage points, achieving 89.84%. It is worth noting that introducing the multi-hop and feature-guided mechanisms leads to a 0.06% drop in Top-5 accuracy, which lies within the seed-to-seed fluctuation range observed in our experiments. Meanwhile, Top-1 accuracy continues to improve by 0.28%, indicating that long-range aggregation contributes discriminative information rather than amplifying noise. By enlarging the graph receptive field and guiding feature propagation along skeletal topology, this component enhances long-range dependency modeling and improves robustness to structural information loss.

5. Conclusions

This paper addresses the challenge of skeleton-based human action recognition in complex occlusion scenarios by proposing a robust recognition framework that integrates an adaptive dropout mechanism with an occlusion compensation strategy. The Adaptive Body-Prior-Guided Drop Module employs an attention-guided Bernoulli sampling strategy to dynamically suppress interference from unreliable joints and anomalous frames across spatiotemporal dimensions. The Occlusion Compensation Graph Convolution integrates Local–Global Body-Prior-Guided Attention, feature guidance, and multi-hop mechanisms to enhance the model’s ability to compensate for structural information loss and long-range dependencies.
Additionally, two complementary occlusion datasets for skeleton occlusion were constructed based on the NTU60/120 dataset, simulating joint-level perception failures and spatiotemporally continuous structural occlusions, respectively, with both datasets set to a 50% occlusion rate.
Extensive experiments demonstrate consistent performance gains and significantly reduced variance on both standard and occlusion benchmarks, indicating improved recognition stability and robustness. Moreover, the proposed model maintains a compact parameter size and high inference efficiency, achieving an effective balance between accuracy and computational cost, thereby demonstrating its practical applicability to real-world deployment scenarios for occluded action recognition.

Author Contributions

Conceptualization, S.J. and K.H.; methodology, S.J. and K.H.; software, S.J. and S.S.; validation, S.S., Y.C. and C.C.; formal analysis, S.J.; investigation, S.J., S.S., Y.C. and C.C.; data curation, S.J. and Y.C.; writing—original draft preparation, S.J.; writing—review and editing, K.H.; visualization, S.J. and C.C.; supervision, K.H.; project administration, K.H.; funding acquisition, K.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

The NTU RGB+D 60 and NTU RGB+D 120 datasets require an application to NTU for access, available at https://rose1.ntu.edu.sg/dataset/actionRecognition/ (accessed on 15 Jan 2026). The occlusion benchmarks constructed in this work and the source code will be made available from the corresponding author on reasonable request.

Acknowledgments

The authors would like to express heartfelt thanks to the reviewers and editors who submitted valuable revisions to this article. During the preparation of this manuscript, the authors used ChatGPT 5.4 for the purposes of language polishing and improving readability. The authors have reviewed and edited the output and take full responsibility for the content of this publication.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Wu, L.; Huang, C.; Zhao, S.; Li, J.; Zhao, J.; Cui, Z.; Yu, Z.; Xu, Y.; Zhang, M. Robust fall detection in video surveillance based on weakly supervised learning. Neural Netw. 2023, 163, 286–297. [Google Scholar] [CrossRef]
  2. Ying, Y.; Wang, H.; Liao, J.; Xing, Y.; Ma, L.; Zhou, H. Assessment of Movement Disorders in the Elderly Based on Skeletal Action Recognition. Electronics 2025, 14, 1437. [Google Scholar] [CrossRef]
  3. Dallel, M.; Havard, V.; Dupuis, Y.; Baudry, D. Digital twin of an industrial workstation: A novel method of an auto-labeled data generator using virtual reality for human action recognition in the context of human–robot collaboration. Eng. Appl. Artif. Intell. 2023, 118, 105655. [Google Scholar] [CrossRef]
  4. Gao, M.; Ju, B. Attention-enhanced gated recurrent unit for action recognition in tennis. PeerJ Comput. Sci. 2024, 10, e1804. [Google Scholar] [CrossRef] [PubMed]
  5. Ni, B.; Li, T.; Yang, X. Learning semantic-aligned action representation. IEEE Trans. Neural Netw. Learn. Syst. 2018, 29, 3715–3725. [Google Scholar] [CrossRef]
  6. Liu, M.; Meng, F.; Chen, C.; Wu, S. Novel motion patterns matter for practical skeleton-based action recognition. In Proceedings of the AAAI Conference on Artificial Intelligence; Association for the Advancement of Artificial Intelligence: Palo Alto, CA, USA, 2023; Volume 37, pp. 1701–1709. [Google Scholar] [CrossRef]
  7. Tu, Z.; Zhang, J.; Li, H.; Chen, Y.; Yuan, J. Joint-bone fusion graph convolutional network for semi-supervised skeleton action recognition. IEEE Trans. Multimed. 2023, 25, 1819–1831. [Google Scholar] [CrossRef]
  8. Yan, S.; Xiong, Y.; Lin, D. Spatial temporal graph convolutional networks for skeleton-based action recognition. In Proceedings of the AAAI Conference on Artificial Intelligence; Association for the Advancement of Artificial Intelligence: Palo Alto, CA, USA, 2018. [Google Scholar]
  9. Shi, L.; Zhang, Y.; Cheng, J.; Lu, H. Two-stream adaptive graph convolutional networks for skeleton-based action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); IEEE: Piscataway, NJ, USA, 2019; pp. 12018–12027. [Google Scholar] [CrossRef]
  10. Chen, Y.; Zhang, Z.; Yuan, C.; Li, B.; Deng, Y.; Hu, W. Channel-wise topology refinement graph convolution for skeleton-based action recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV); IEEE: Piscataway, NJ, USA, 2021; pp. 13359–13368. [Google Scholar]
  11. Zhou, S.B.; Chen, R.R.; Jiang, X.Q.; Pan, F. 2s-GATCN: Two-Stream Graph Attentional Convolutional Networks for Skeleton-Based Action Recognition. Electronics 2023, 12, 1711. [Google Scholar] [CrossRef]
  12. Hong, M.; Sun, B.; He, J.; Zhang, Y.; Yuan, L. Reducing hallucination in MLLMs for action recognition with adaptive masking and dual prompting. Neurocomputing 2026, 662, 131861. [Google Scholar] [CrossRef]
  13. Shahroudy, A.; Liu, J.; Ng, T.T.; Wang, G. NTU RGB+D: A large scale dataset for 3D human activity analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); IEEE: Piscataway, NJ, USA, 2016; pp. 1010–1019. [Google Scholar] [CrossRef]
  14. Hu, Q.; Liu, H.; Wang, H.Q.; Liu, M. Body prior guided graph convolutional neural network for skeleton-based action recognition. In Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); IEEE: Piscataway, NJ, USA, 2023; pp. 1–5. [Google Scholar] [CrossRef]
  15. Liu, Z.; Zhang, H.; Chen, Z.; Wang, Z.; Ouyang, W. Disentangling and unifying graph convolutions for skeleton-based action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); IEEE: Piscataway, NJ, USA, 2020; pp. 140–149. [Google Scholar] [CrossRef]
  16. Song, Y.F.; Zhang, Z.; Shan, C.; Wang, L. Constructing stronger and faster baselines for skeleton-based action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2023, 45, 1474–1488. [Google Scholar] [CrossRef]
  17. Wang, S.; Zhang, Y.; Zhao, M.; Qi, H.; Wang, K.; Wei, F.; Jiang, Y. Skeleton-based action recognition via temporal-channel aggregation. arXiv 2022, arXiv:2205.15936. [Google Scholar]
  18. Rong, Y.; Huang, W.; Xu, T.; Huang, J. DropEdge: Towards deep graph convolutional networks on node classification. In Proceedings of the International Conference on Learning Representations (ICLR), Addis Ababa, Ethiopia, 26–30 April 2020. [Google Scholar]
  19. Cheng, K.; Zhang, Y.; Cao, C.; Shi, L.; Cheng, J.; Lu, H. Decoupling GCN with DropGraph module for skeleton-based action recognition. In Proceedings of the European Conference on Computer Vision (ECCV); Springer: Cham, Switzerland, 2020; pp. 536–553. [Google Scholar]
  20. Brau, E.; Jiang, H. 3D human pose estimation via deep learning from 2D annotations. In Proceedings of the 4th International Conference on 3D Vision (3DV); IEEE: Piscataway, NJ, USA, 2016; pp. 582–591. [Google Scholar]
  21. Hu, K.; Shen, C.; Wang, T.; Shen, S.; Cai, C.; Huang, H.; Xia, M. Action recognition based on multi-level topological channel attention of human skeleton. Sensors 2023, 23, 9738. [Google Scholar] [CrossRef] [PubMed]
  22. Yin, Y.; Liu, M.; Zhu, Q.; Zhang, S.; Hussien, N.A.; Fan, Y. Multibranch attention graph convolutional networks for 3-D human pose estimation. IEEE Trans. Instrum. Meas. 2023, 72, 1–12. [Google Scholar] [CrossRef]
  23. Hu, K.; Jin, J.; Shen, C.; Xia, M.; Weng, L. Attentional weighting strategy-based dynamic GCN for skeleton-based action recognition. Multimed. Syst. 2023, 29, 1941–1954. [Google Scholar] [CrossRef]
  24. Wu, Y.; Xu, Z.; Huang, Y.; Liu, J.; Yu, Y. ASR-GCN: Adaptive spatial information reconstruction GCN for skeleton-based action recognition. Neural Netw. 2026, 197, 108508. [Google Scholar] [CrossRef] [PubMed]
  25. Wang, J.; Li, Z.; Liu, B.; Cai, H.; Saada, M.; Meng, Q. High-performance inference graph convolutional networks for skeleton-based action recognition. Neurocomputing 2025, 653, 131078. [Google Scholar] [CrossRef]
  26. Gao, Y.; Duan, X.; Dai, Q. Skeleton-Based Action Recognition Using Graph Convolutional Network with Pose Correction and Channel Topology Refinement. Comput. Mater. Contin. 2025, 83, 701–718. [Google Scholar] [CrossRef]
  27. Xia, R.; Liu, H.; Li, A.; Liu, X.; Zhang, Y.; Zhang, C.; Yang, B. Incomplete graph learning: A comprehensive survey. Neural Netw. 2025, 190, 107682. [Google Scholar] [CrossRef]
  28. Chen, Z.; Wang, H.; Gui, J. Occluded skeleton-based human action recognition with dual inhibition training. In Proceedings of the 31st ACM International Conference on Multimedia (MM); ACM: New York, NY, USA, 2023; pp. 2625–2634. [Google Scholar] [CrossRef]
  29. Bin, Y.; Cao, X.; Chen, X. Adversarial semantic data augmentation for human pose estimation. In Proceedings of the European Conference on Computer Vision (ECCV); Springer: Cham, Switzerland, 2020; pp. 606–622. [Google Scholar]
  30. Ke, L.; Chang, M.C.; Qi, H. Multi-scale structure-aware network for human pose estimation. In Proceedings of the European Conference on Computer Vision (ECCV); Springer: Cham, Switzerland, 2018; pp. 713–728. [Google Scholar]
  31. Wu, W.; Hua, Y.; Zheng, C.; Wu, S.; Chen, C.; Lu, A. SkeletonMAE: Spatial-temporal masked autoencoders for self-supervised skeleton action recognition. In Proceedings of the IEEE International Conference on Multimedia and Expo Workshops (ICMEW); IEEE: Piscataway, NJ, USA, 2023; pp. 224–229. [Google Scholar] [CrossRef]
  32. Xu, L.; Zhao, L.; Sun, X.; Wang, D.; Li, G.; Yan, K. A comprehensive framework for occluded human pose estimation. In Proceedings of the ICASSP 2024—2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); IEEE: Piscataway, NJ, USA, 2024; pp. 3405–3409. [Google Scholar] [CrossRef]
  33. Wu, S.; Lu, G.; Han, Z.; Chen, L. A robust two-stage framework for human skeleton action recognition with GAIN and masked autoencoder. Neurocomputing 2025, 623, 129433. [Google Scholar] [CrossRef]
  34. Chen, Y.; Peng, K.; Roitberg, A.; Schneider, D.; Zhang, J.; Zheng, J.; Chen, Y.; Liu, R.; Yang, K.; Stiefelhagen, R. Exploring Self-supervised Skeleton-based Action Recognition in Occluded Environments. arXiv 2025, arXiv:2309.12029. [Google Scholar] [CrossRef]
  35. Vernikos, I.; Spyrou, E. Skeleton Reconstruction Using Generative Adversarial Networks for Human Activity Recognition Under Occlusion. Sensors 2025, 25, 1567. [Google Scholar] [CrossRef]
  36. Song, Y.F.; Zhang, Z.; Shan, C.; Wang, L. Richly activated graph convolutional network for robust skeleton-based action recognition. IEEE Trans. Circuits Syst. Video Technol. 2021, 31, 1915–1925. [Google Scholar] [CrossRef]
  37. Chen, Y.; Wang, L.; Hu, D.; Cheng, H. Multi-view graph convolution network for the recognition of human action with spatial and temporal occlusion problems. J. Vis. Commun. Image Represent. 2023, 97, 103957. [Google Scholar] [CrossRef]
  38. Qiu, L.; Zhang, X.; Li, Y.; Li, G.; Wu, X.; Xiong, Z.; Han, X.; Cui, S. Peeking into occluded joints: A novel framework for crowd pose estimation. In Proceedings of the European Conference on Computer Vision (ECCV); Springer: Cham, Switzerland, 2020; pp. 488–504. [Google Scholar] [CrossRef]
  39. Wang, X.; Gao, L.; Dai, Y. Semantic aware transfer with instance-adaptive parsing for crowded scenes pose estimation. In Proceedings of the 29th ACM International Conference on Multimedia; ACM: New York, NY, USA, 2021; pp. 686–694. [Google Scholar]
  40. Li, D.; Shi, W. Partially occluded skeleton action recognition based on multi-stream fusion graph convolutional networks. In Proceedings of the Computer Graphics International (CGI); Springer: Cham, Switzerland, 2021; pp. 178–189. [Google Scholar]
  41. Guo, J.; Ji, Q.; Shan, G. Overcomplete graph convolutional denoising autoencoder for noisy skeleton action recognition. IET Image Process. 2024, 18, 233–246. [Google Scholar] [CrossRef]
  42. Gao, X.; Du, S.; Yang, Y. Glimpse and focus: Global and local-scale graph convolution network for skeleton-based action recognition. Neural Netw. 2023, 167, 551–558. [Google Scholar] [CrossRef] [PubMed]
  43. Liu, J.; Shahroudy, A.; Perez, M.; Wang, G.; Duan, L.Y.; Kot, A.C. NTU RGB+D 120: A large-scale benchmark for 3D human activity understanding. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 42, 2684–2701. [Google Scholar] [CrossRef]
  44. Zhang, P.; Lan, C.; Zeng, W.; Xue, J.; Zheng, N. Semantics-guided neural networks for efficient skeleton-based human action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); IEEE: Piscataway, NJ, USA, 2020; pp. 1109–1118. [Google Scholar] [CrossRef]
  45. Zhou, Y.; Yan, X.; Cheng, Z.Q.; Yan, Y.; Dai, Q.; Hua, X.S. BlockGCN: Redefine topology awareness for skeleton-based action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); IEEE: Piscataway, NJ, USA, 2024; pp. 2049–2058. [Google Scholar] [CrossRef]
  46. Chen, H.; Shen, Y.; Zhang, Y.; Pan, X.; Yang, Q. Skeleton-based action recognition through dual-granularity feature fusion with self-adapting graph convolution and multi-scale temporal convolution. Neurocomputing 2025, 639, 130261. [Google Scholar] [CrossRef]
  47. Chen, E.; Wang, X.; Guo, X. Masked reconstruction model of latent space vector quantization for human skeleton-based action recognition. Neurocomputing 2026, 666, 132126. [Google Scholar] [CrossRef]
Figure 1. Overall architecture of the proposed ROC-GCN. The framework consists of the Adaptive Body-Prior-Guided Drop (A-BPGD) module and the Occlusion Compensation Graph Convolution (OC-GCN) module. The input skeleton sequence passes through A-BPGD for spatiotemporal dropout, followed by OC-GCN for graph convolution and classification.
Figure 1. Overall architecture of the proposed ROC-GCN. The framework consists of the Adaptive Body-Prior-Guided Drop (A-BPGD) module and the Occlusion Compensation Graph Convolution (OC-GCN) module. The input skeleton sequence passes through A-BPGD for spatiotemporal dropout, followed by OC-GCN for graph convolution and classification.
Electronics 15 02311 g001
Figure 2. Architecture of the Drop S Block and Drop T Block. Each block contains a dual-path pooling layer for feature compression, two non-shared MLPs for attention map generation, a Bernoulli sampling stage for mask generation, and a masking operation applied to the input skeleton data.
Figure 2. Architecture of the Drop S Block and Drop T Block. Each block contains a dual-path pooling layer for feature compression, two non-shared MLPs for attention map generation, a Bernoulli sampling stage for mask generation, and a masking operation applied to the input skeleton data.
Electronics 15 02311 g002
Figure 3. Detailed structure of the Occlusion Compensation Graph Convolution (OC-GCN) module, including the LG-BPGA mechanism, feature-guided mechanism, and multi-hop mechanism.
Figure 3. Detailed structure of the Occlusion Compensation Graph Convolution (OC-GCN) module, including the LG-BPGA mechanism, feature-guided mechanism, and multi-hop mechanism.
Electronics 15 02311 g003
Figure 4. Architecture of the Local–Global Body-Prior-Guided Attention (LG-BPGA) mechanism. The skeleton is partitioned into five semantic body parts for local channel attention extraction, followed by fusion with global features to compensate for occluded regions.
Figure 4. Architecture of the Local–Global Body-Prior-Guided Attention (LG-BPGA) mechanism. The skeleton is partitioned into five semantic body parts for local channel attention extraction, followed by fusion with global features to compensate for occluded regions.
Electronics 15 02311 g004
Figure 5. NTU60 occlusion dataset. Green dots denote visible joints, red lines denote visible bones, and gray semi-transparent dots and lines denote occluded joints and bones.
Figure 5. NTU60 occlusion dataset. Green dots denote visible joints, red lines denote visible bones, and gray semi-transparent dots and lines denote occluded joints and bones.
Electronics 15 02311 g005
Figure 6. NTU120 occlusion dataset. Green dots denote visible joints, red lines denote visible bones, and gray semi-transparent dots and lines denote occluded joints and bones.
Figure 6. NTU120 occlusion dataset. Green dots denote visible joints, red lines denote visible bones, and gray semi-transparent dots and lines denote occluded joints and bones.
Electronics 15 02311 g006
Figure 7. Comparison of Recognition Accuracy on the NTU60/120 and Occlusion Datasets.
Figure 7. Comparison of Recognition Accuracy on the NTU60/120 and Occlusion Datasets.
Electronics 15 02311 g007
Figure 8. Accuracy of Each Modality on the NTU60/120 Occlusion Datasets.
Figure 8. Accuracy of Each Modality on the NTU60/120 Occlusion Datasets.
Electronics 15 02311 g008
Figure 9. Representative real-world missing-joint sample from NTU RGB+D 120 (S006C003P017R001A055, A55: Hugging, joint validity = 80.7%). Green dots and red lines denote successfully tracked joints and bones, respectively; red crosses mark joints that the Kinect sensor failed to track.
Figure 9. Representative real-world missing-joint sample from NTU RGB+D 120 (S006C003P017R001A055, A55: Hugging, joint validity = 80.7%). Green dots and red lines denote successfully tracked joints and bones, respectively; red crosses mark joints that the Kinect sensor failed to track.
Electronics 15 02311 g009
Figure 10. Baseline per-class accuracy on the NTU60 occlusion dataset.
Figure 10. Baseline per-class accuracy on the NTU60 occlusion dataset.
Electronics 15 02311 g010
Figure 11. ROC-GCN per-class accuracy on the NTU60 occlusion dataset.
Figure 11. ROC-GCN per-class accuracy on the NTU60 occlusion dataset.
Electronics 15 02311 g011
Table 1. The comprehensive comparison between the proposed ROC-GCN and representative skeleton-based action recognition methods on the NTU RGB+D 60 and NTU RGB+D 120 datasets under the Cross-Subject protocol.
Table 1. The comprehensive comparison between the proposed ROC-GCN and representative skeleton-based action recognition methods on the NTU RGB+D 60 and NTU RGB+D 120 datasets under the Cross-Subject protocol.
MethodYearBackboneParams (M)NTU60 X-Sub (%)NTU120 X-Sub (%)
ST-GCN [8]2018GCN3.181.570.7
2s-AGCN [9]2018GCN3.4888.582.9
SGN [44]2020GCN0.6989.079.2
CTR-GCN [10]2021GCN1.4692.488.9
BlockGCN [45]2024GCN1.393.190.3
MTGCN [46]2025GCN0.4590.580.8
RA-GCN [36]2020GCN6.2187.381.1
MSFGCN [40]2023GCN/87.282.8
SkeletonMAE [31]2023STTFormer1186.676.8
VQMVAE [47]2025STTFormer1087.979.8
BPG-GCN (Baseline) [14]2023GCN1.889.386.2
Ours2025GCN2.5190.086.8
Table 2. Performance Comparison Between ROC-GCN and Baseline Models on NTU60/120 Datasets (Top-1/Top-5). The symbol ↑ indicates the accuracy improvement of ROC-GCN over the baseline.
Table 2. Performance Comparison Between ROC-GCN and Baseline Models on NTU60/120 Datasets (Top-1/Top-5). The symbol ↑ indicates the accuracy improvement of ROC-GCN over the baseline.
MethodNTU60 X-Sub (%)NTU120 X-Sub (%)
Top-1Top-5Top-1Top-5
BPG-GCN89.36 ± 0.6198.28 ± 0.2886.24 ± 0.4797.33 ± 0.20
Ours89.77 ± 0.1798.46 ± 0.0286.72 ± 0.1097.64 ± 0.08
Improve↑ 0.41↑ 0.18↑ 0.48↑ 0.31
Table 3. Performance Comparison Between ROC-GCN and Baseline Models Across Modalities on NTU60/120 Datasets (Top-1/Top-5).
Table 3. Performance Comparison Between ROC-GCN and Baseline Models Across Modalities on NTU60/120 Datasets (Top-1/Top-5).
ModalityNTU 60 X-Sub (%)NTU 120 X-Sub (%)
BPG-GCN Ours Gain BPG-GCN Ours Gain
Joint86.56 ± 0.8386.76 ± 0.510.2082.80 ± 0.4883.05 ± 0.390.25
Bone84.07 ± 1.6284.12 ± 1.010.0577.87 ± 1.2078.46 ± 0.950.59
Joint Motion83.61 ± 1.4083.71 ± 0.870.1077.86 ± 0.7778.77 ± 0.560.91
Bone Motion81.19 ± 0.8381.90 ± 0.480.7175.20 ± 0.7176.43 ± 0.861.23
Table 4. Performance Comparison Between ROC-GCN and Baseline Models on NTU60/120 Occlusion Datasets (Top-1/Top-5). The symbol ↑ indicates the accuracy improvement of ROC-GCN over the baseline.
Table 4. Performance Comparison Between ROC-GCN and Baseline Models on NTU60/120 Occlusion Datasets (Top-1/Top-5). The symbol ↑ indicates the accuracy improvement of ROC-GCN over the baseline.
MethodNTU 60 X-Sub (%)NTU 120 X-Sub (%)
Top-1 Top-5 Top-1 Top-5
BPG-GCN78.74 ± 2.8796.50 ± 0.6976.58 ± 0.3093.70 ± 0.32
Ours79.72 ± 2.5196.81 ± 0.6077.31 ± 0.2294.30 ± 0.11
Improve↑ 0.98↑ 0.31↑ 0.73↑ 0.60
Table 5. Performance Comparison Between ROC-GCN and Baseline Models Across Modalities on NTU60/120 Occlusion Datasets (Top-1).
Table 5. Performance Comparison Between ROC-GCN and Baseline Models Across Modalities on NTU60/120 Occlusion Datasets (Top-1).
ModalityNTU 60 X-Sub (%)NTU 120 X-Sub (%)
BPG-GCN Ours Gain BPG-GCN Ours Gain
Joint74.44 ± 11.2979.76 ± 2.805.3271.73 ± 0.4071.60 ± 0.41−0.13
Bone70.55 ± 1.6973.36 ± 1.282.8171.61 ± 0.5371.93 ± 0.450.32
Joint Motion70.81 ± 1.3872.13 ± 1.221.3264.67 ± 0.8266.33 ± 0.721.66
Bone Motion69.88 ± 3.6671.16 ± 2.221.2864.45 ± 0.6065.61 ± 1.131.16
Table 6. Top-1/Top-5 accuracy (%) on the real missing-joint subset of NTU RGB+D 120 X-Sub.
Table 6. Top-1/Top-5 accuracy (%) on the real missing-joint subset of NTU RGB+D 120 X-Sub.
SubsetnBPG-GCN (Top-1/Top-5)ROC-GCN (Top-1/Top-5) Δ Top-1
Full real-missing set17948.60/68.1651.96/70.39+3.36
Spatial loss ( j < 0.95 )4454.55/79.5572.73/86.36+18.18
Spatial loss ( j < 0.80 )3855.26/78.9573.68/86.84+18.42
Spatial loss ( j < 0.70 )2857.14/82.1478.57/85.71+21.43
Table 7. Computational cost and inference efficiency comparison between ROC-GCN and the baseline.
Table 7. Computational cost and inference efficiency comparison between ROC-GCN and the baseline.
ModelParamsFLOPsLatencyFPS
(M) (G) (ms)
BPG-GCN (Baseline)1.81915.8617.6756.59
Ours2.5126.6624.4540.89
Table 8. The ablation results on NTU60, where each module is progressively added to the baseline.
Table 8. The ablation results on NTU60, where each module is progressively added to the baseline.
ConfigurationTop-1 (%)Top-5 (%)Gain (%)
Baseline88.6897.83
+A-BPGD Module89.2398.220.55
+LG-BPGA89.5698.520.33
+Multi-hop + Feature Guidance89.8498.460.28
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Jin, S.; Hu, K.; Shen, S.; Cai, Y.; Cai, C. A Graph Convolutional Network for Action Recognition in Occluded Skeleton Data. Electronics 2026, 15, 2311. https://doi.org/10.3390/electronics15112311

AMA Style

Jin S, Hu K, Shen S, Cai Y, Cai C. A Graph Convolutional Network for Action Recognition in Occluded Skeleton Data. Electronics. 2026; 15(11):2311. https://doi.org/10.3390/electronics15112311

Chicago/Turabian Style

Jin, Sicheng, Kai Hu, Shuai Shen, Yongkai Cai, and Chengxue Cai. 2026. "A Graph Convolutional Network for Action Recognition in Occluded Skeleton Data" Electronics 15, no. 11: 2311. https://doi.org/10.3390/electronics15112311

APA Style

Jin, S., Hu, K., Shen, S., Cai, Y., & Cai, C. (2026). A Graph Convolutional Network for Action Recognition in Occluded Skeleton Data. Electronics, 15(11), 2311. https://doi.org/10.3390/electronics15112311

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop