Search Results (161)

Skeleton-based action recognition has attracted considerable research interest because skeleton data are inherently robust to illumination changes, viewpoint variation, background clutter, and camera motion. Nevertheless, extracting informative representations from skeleton sequences remains a challenging problem, as it requires capturing both the spatial co-occurrence patterns among body joints and the fine-grained kinematic cues that distinguish different actions. In this paper, we propose a single-stream architecture that constructs an action-specific skeleton graph directly from motion and processes it with a kinematic-aware Transformer. Rather than relying on a fixed skeleton topology, a motion-guided dynamic-graph construction module infers a per-frame adjacency matrix from short-term motion cues through a differentiable edge predictor and Gumbel-Softmax sparsification, allowing the model to discover action-driven connections between distant joints that lack direct bone connectivity (e.g., coordinated hand motion during clapping). Each joint is described by kinematic node features that combine its 3D position, instantaneous velocity, and limb-angle encodings within a single descriptor, so that both motion dynamics and higher-order limb configurations are available to the spatial encoder from the outset. A graph-attention network (GAT) encodes the spatial configuration of every frame over the learned graph, and the resulting sequence of frame descriptors is processed by a Transformer encoder that models long-range temporal dependencies; a learnable classification token aggregates the sequence, and a multi-layer perceptron (MLP) produces the final action classification. The entire model is trained end-to-end from action labels alone. We conduct a comprehensive ablation study and evaluate the proposed method on the large-scale NTU RGB+D 60 and NTU RGB+D 120 benchmarks, where the results demonstrate that our approach achieves competitive performance compared to state-of-the-art architectures. Full article

Skeleton-based human action recognition is an important problem in applied vision systems, yet many existing approaches depend on a single skeleton descriptor or a single feature-learning mechanism. This restriction can weaken the representation of local body kinematics, long-range joint relations, and temporal dependencies within an action sequence. To address these limitations, this paper proposes HILF-GT (Hybrid Invariant Latent Feature Graph Transformer), a hybrid Graph Convolutional Network (GCN)-Transformer framework based on multiple spatio-temporal invariant latent features. The representation module constructs complementary structured tensors from skeleton graphs, inter-joint distances, adjacent-frame joint displacements, and inter-limb angles. Instead of transforming these descriptors into image-like maps for separate Convolutional Neural Network (CNN)-based classification, HILF-GT keeps their graph and temporal organization during learning. A local GCN branch models skeleton-aware kinematic patterns, whereas a graph-aware Transformer branch uses biased self-attention and cross-attention to capture dependencies among distant joints, frames, and latent-feature streams. A Perceiver-style latent bottleneck is further introduced to reduce the memory cost of global attention over frame-joint tokens. Experiments were conducted on four standard benchmark datasets, including NTU-RGB+D 60, NTU-RGB+D 120, NW-UCLA, and UTD-MHAD. The proposed method achieved 93.1% and 97.20% accuracy on the NTU-RGB+D 60 Cross-Subject and Cross-View protocols, 88.15% and 90.20% on the NTU-RGB+D 120 Cross-Subject and Cross-Setup protocols, 98.50% on NW-UCLA, and 97.50% on UTD-MHAD. Full article

Movement-based and physical activity programs are central tools in autism intervention, so recognizing the activities a child performs during therapy is valuable for objective progress tracking. Manual monitoring of these sessions is time-consuming and subjective, and raw videos raise privacy concerns because it shows identifiable children. We address autism therapeutic activity recognition from privacy-preserving 2D skeletons, and we focus on the practical difficulty of how several therapeutic activities differ only in subtle motion details. As a backbone, we adopt ProtoGCN, a graph convolutional network that represents each action as a combination of learnable motion prototypes. However, this contrastive backbone organizes all classes at once, so it does not enforce a margin between the few pairs that remain entangled after training. We therefore introduce a Refine–Confusable (RC) module, a training-only regularizer that pushes apart the empirically most-confused class pairs using a hinge-margin loss over momentum-updated class centroids. The module changes neither the backbone nor the inference cost. On the MMASD dataset, restricted to the ten-class 2D-skeleton configuration, the RC module improves the base model across random, session-independent, and subject-independent evaluation. The gain is largest on the strictest subject-independent split and a clip-level analysis confirms that this improvement is statistically significant. Under the protocol-matched holdout, the method reaches 96.30% accuracy with 0.959 macro-F1, surpassing recent 2D-skeleton baselines while keeping a lightweight and privacy-preserving modality. The improvements are modest, as expected on a small clinical dataset, and t-SNE and prototype visualizations show that the learned representation is discriminative and interpretable. Full article

Accurate boxing action recognition is critical for intelligent combat training, action quality assessment, and sports injury prevention. However, existing deep learning approaches face three key challenges: limited feature extraction for high-speed non-rigid boxing motions, weak robustness against background interference and occlusion, and performance instability from labor-intensive manual hyperparameter tuning. Furthermore, the original Centered Collision Optimizer (CCO), a biomimetic algorithm inspired by celestial collision dynamics, suffers from insufficient population diversity, poor adaptive regulation, and premature convergence in high-dimensional hyperparameter optimization tasks. To address these issues, this paper proposes a novel biomimetic optimization-driven boxing action recognition framework, where an Enhanced Adaptive Centered Collision Optimizer (EACCO) automatically optimizes the hyperparameters of a Multi-Scale Spatio-Temporal Adaptive ConvNeXt (MSTA-ConvNeXt) network. First, the MSTA-ConvNeXt backbone integrates multi-scale dynamic deformable convolution, a Bi-GRU spatio-temporal fusion module, and a dual-channel attention mechanism to enhance fine-grained feature extraction and temporal modeling. Second, three biomimetic improvements are introduced to CCO: Tent chaotic elite opposition-based initialization, adaptive nonlinear convergence factor with dynamic weight guidance, and adaptive Gaussian-Cauchy hybrid mutation, which balance exploration and exploitation and avoid local optima. Experiments on two public benchmark datasets show that the proposed framework achieves 96.1% accuracy, 95.9% precision, 95.7% recall, and 95.8% F1-score on the Boxing Jab Skeleton Dataset, and 95.4% accuracy, 95.2% precision, 94.9% recall, and 95.0% F1-score on the Olympic Boxing dataset, outperforming all state-of-the-art methods. Ablation studies validate the effectiveness of each EACCO component and confirm that this biomimetic hyperparameter optimization approach outperforms manual tuning and other popular optimizers. This work provides an effective biomimetic optimization solution for intelligent sports action recognition. Full article

Skeleton-based human action recognition (HAR) requires models that preserve the local kinematic structure of the human body while capturing long-range spatiotemporal dependencies under noisy or incomplete joint observations. Traditional Graph Convolutional Networks (GCNs) provide topology-aligned inductive bias but are often limited by local information aggregation from neighboring joints. In contrast, attention-based mechanisms capture global interactions, yet they may attend to spurious correlations when skeletal constraints are weakly enforced. This paper proposes Differential Hyperedge Attention-enhanced GCN (DHA-eGCN), a hybrid architecture that couples structure-aware Differential Hyperedge Attention with multi-scale temporal convolution for spatiotemporal skeleton sequence processing. DHA injects skeletal structure into attention via hop-distance relative positional encoding and hyperedge context tokens generated via joint-to-part pooling. It further employs differential attention to suppress shared noisy correlations and enhance interaction selectivity. To strengthen spatial grounding, an explicit GCN branch is added under partial- or full-depth configurations, where the first four or all ten layers are applied with graph convolutions. The model further employs an ensemble strategy that combines predictions from multiple complementary model instances. Our experiments on NTU RGB+D 60 under the X-Sub and X-View protocols, NTU RGB+D 120 under the X-Sub and X-Set protocols, and Northwestern-UCLA demonstrate that DHA-eGCN consistently outperforms or remains competitive with strong graph-based, transformer-based, and hybrid state-of-the-art methods based on the same four-stream architecture. The best configuration achieves 93.7% and 97.0% on NTU RGB+D 60 X-Sub and X-View, respectively; 90.9% and 91.9% on NTU RGB+D 120 X-Sub and X-Set, respectively; and 97.6% on Northwestern-UCLA. Full article

To address the issues of insufficient real-time performance and inadequate modeling of temporal features in traffic police gesture recognition, this paper proposes a method based on skeleton keypoints and hybrid temporal modeling. First, YOLOv11m-Pose is employed to detect human skeleton keypoints in video sequences, extracting reliable two-dimensional skeleton features. Second, this study designs a temporal modeling network that integrates a bidirectional long short-term memory (BiLSTM) with a Transformer. The BiLSTM models local temporal continuity and action transition features between adjacent frames, capturing short-term dynamic changes. The Transformer, through its self-attention mechanism, models global temporal dependencies and weights critical time steps to extract long-range discriminative information. Experimental results demonstrate that the proposed method achieved 98.91% for both Accuracy and F1-Score. In terms of Accuracy, it outperformed the BiLSTM and Transformer models by 2.43% and 7.67%, respectively. It outperforms most methods based on recurrent neural networks and feature fusion. Meanwhile, the model achieves an average inference time of just 1.3299 s per gesture sequence. Consequently, this approach strikes a favorable balance between recognition accuracy and real-time performance, demonstrating significant practical value. Full article

Human action recognition is a key research area in computer vision, where accurate recognition relies on effective modeling of both global and local spatiotemporal information. However, existing GCN-based methods often overemphasize the local topological connectivity of human skeletons. Moreover, their temporal modules fail to fully capture the evolution of action sequences, leading to critical instantaneous information being obscured by global representations. To address these problems, we propose an integrated framework termed MADS-GCN. In the spatial modeling stage, we introduce two parallel streams: the Physical Stream uses the adjacency matrix to constrain convolution and capture global structural patterns, while the Topological Stream leverages spatial attention to assign adaptive weights to joints, preserving discriminative local adaptive features. For temporal modeling, a channel-temporal attention mechanism is applied to adaptively refine feature maps, followed by a bidirectional GRU to capture multi-scale temporal patterns. Extensive experiments on NTU RGB+D60, Northwestern-UCLA, and our custom DanceBasic-Set demonstrate the effectiveness of MADS-GCN and indicate its applicability to dance action recognition scenarios. Full article

Skeleton-based human action recognition has achieved significant progress, but local occlusions and missing joints in complex environments (e.g., occlusion and low-light conditions) still degrade recognition accuracy and stability. Existing GCN-based methods aggregate features uniformly across joints and lack mechanisms to suppress unreliable observations or recover structural semantics under large-area occlusion. To address this, we propose a Robust Occlusion-Compensated Graph Convolutional Network (ROC-GCN) with two complementary components: an adaptive dropout module that suppresses spatiotemporal noise via attention-guided Bernoulli sampling with dynamic spatial–temporal fusion, and an Occlusion Compensation Graph Convolution Module that compensates occluded features through Local–Global Body-Prior-Guided Attention together with feature-guided and multi-hop aggregation. To enable systematic evaluation, we further construct two complementary occlusion benchmarks on NTU RGB+D 60/120 covering spatial-random and spatiotemporal-continuous occlusion, and additionally validate the model on a real-world missing-joint subset. On standard NTU60/120 X-Sub, ROC-GCN improves Top-1 accuracy by +0.41% and +0.48% over the baseline, with the Top-1 standard deviation reduced from 0.61 → 0.17 and 0.47 → 0.10. On the occlusion benchmarks, Top-1 accuracy further improves by +0.98% and +0.73%, and consistent gains are also observed on the real-world missing-joint validation, confirming improved robustness and training stability. Full article

While Graph Convolutional Networks (GCNs) have revolutionized skeleton-based action recognition, existing methods face a critical efficiency–accuracy dilemma: state-of-the-art approaches achieve high performance through computationally expensive multi-stream fusion (joint, bone, joint motion, and bone motion) and deep architectures, limiting real-world deployment on resource-constrained devices. We propose LST-AGCN (Lightweight Spatial–Temporal Attention Graph Convolutional Network), introducing three technical contributions that address this challenge: (1) Unified Attention Module (UAM)—a framework that integrates channel, spatial, and temporal attention through a single compact operation, significantly reducing attention parameters compared to separate attention mechanisms; (2) Depthwise Separable Attention Mechanism (DSAM)—a factorization using depthwise separable convolutions that achieves linear complexity reduction from $O\left(C^2\right)$ to $O\left(C\right)$ in attention operations; and (3) Efficient Topology-Aware Fusion (ETAF)—an adaptive Joint-wise Attention strategy that captures fine-grained spatial relationships without quadratic complexity growth. Extensive experiments on NTU RGB+D 60 and NTU RGB+D 120 datasets demonstrate that LST-AGCN achieves strong performance using only joint modality (86.14%/94.0% and 79.5%/82.0% Top-1 accuracy with 99.0% Top-5 on cross-view) while requiring 14.11 M parameters and 19.02 GFLOPs, delivering efficient inference suitable for edge deployment. Full article

In industrial environments, robust Temporal Action Localization (TAL) is essential; however, frequent occlusions often compromise the reliability of skeletal data, leading to negative transfer in multimodal fusion. To address this challenge, we propose a Gated Skeleton Refinement Module (Gated SRM), a universal front-end preprocessing module that explicitly incorporates OpenPose confidence scores into the network architecture. By applying these scores as a logarithmic bias within a self-attention mechanism, our method achieves soft suppression—dynamically attenuating the attention weights assigned to unreliable joints—before adaptively fusing the refined skeletal features with RGB representations through a learnable gating network. Extensive experiments on the heavily occluded IKEA ASM dataset demonstrate that our approach effectively prevents the catastrophic accuracy degradation typical of naive and established multimodal fusion strategies, improving the mean Average Precision (mAP) to 21.77%, maintaining parity with the RGB-only baseline while demonstrating superior robustness. Furthermore, the system maintains a practical end-to-end inference speed of approximately 9.2 frames per second (FPS), which is sufficient for monitoring macro-level industrial workflows. By prioritizing confidence-based data selection over data restoration, this sensor-metadata-driven architecture offers a robust and principled approach acting as a critical fail-safe and safety-net for real-world action recognition under occlusion. Full article

Skeleton-based action recognition is a critical technology for intelligent sports analysis. Although the human skeletal structure exhibits inherent bilateral symmetry, sensor noise on resource-constrained edge devices frequently induces geometric distortion and topological asymmetry. Consequently, achieving a balance between high accuracy and real-time performance remains a significant challenge. To this end, we propose EMS-GCN, an Efficient Multi-scale Shift Graph Convolutional Network that integrates geometric priors. Specifically, we design a Gaussian kernel-driven topology refinement module to mitigate structural noise inherent in sensor data. By leveraging geometric symmetry and Gaussian distances among nodes, this module dynamically constrains graph topology learning, thereby effectively rectifying the structural asymmetry and ambiguity induced by noise. Furthermore, we construct a Multi-scale Shift Linear Attention (MSLA) module to replace computationally intensive temporal convolutions. Leveraging temporal shift invariance, this module captures multi-scale contexts via parameter-free shift operations. Furthermore, we introduce a linear temporal attention mechanism to model global temporal dependencies with linear complexity, effectively resolving the information asymmetry inherent in long-range interactions. Finally, EMS-GCN incorporates a dual-branch attention structure to adaptively calibrate feature responses. Extensive experiments demonstrate that our model maintains high recognition accuracy with only 0.56 M parameters, representing a reduction of over 60% compared to mainstream baselines. These results validate the efficacy of leveraging geometric and temporal symmetries to enhance real-time sports analysis. Full article

Falls remain a critical safety concern in surveillance settings, yet monocular RGB methods often degrade in multi-person scenes with occlusion and loss of three-dimensional cues. This study proposes a neuro-symbolic framework that restores physically interpretable depth proxies from monocular video and fuses them with skeleton-based spatio-temporal inference for robust fall detection. The pipeline estimates per-frame depth and 2D skeletons, recovers world coordinates for key joints, and derives absolute neck height and vertical descent rate for rule-based adjudication, while a neural method operates on joint trajectories; final decisions combine both streams with a logical policy and short-horizon temporal consistency. Experiments in a realistic indoor testbed with multi-person activity compare three configurations—neural, symbolic, and fused. The fused neuro-symbolic method achieved an accuracy of 0.88 and an F1 score of 0.76 on the real surveillance test set, outperforming the neural method alone (accuracy 0.81, F1 0.64) and the symbolic method alone (accuracy 0.77, F1 0.35). Gains arise from complementary error profiles: depth-derived, rule-based cues suppress spurious positives on non-fall frames, while the neural stream recovers true falls near rule boundaries. These findings indicate that integrating monocular depth proxies with interpretable rules improves reliability without additional sensors, supporting deployment in complex, multi-person surveillance environments. Full article

We present a skeleton-based approach to baseball pitch type classification from broadcast video. We leverage Human Pose Estimation and an ST-GCN architecture, improved with a projection-gated temporal downsampler, to learn kinematic signatures of the pitcher’s body, adaptively selecting the most informative frames, enabling pitch type classification without the need for ball tracking. On the MLB-YouTube dataset, our proposed method reaches ~62% six-class accuracy, highlighting body mechanics as a practical biometric cue. Full article

Skeleton-based action recognition has achieved remarkable advances with graph convolutional networks (GCNs). However, most existing models process spatial and temporal information within a single coupled stream, which often obscures the distinct patterns of joint configuration and motion dynamics. This paper introduces the Dual-Path Cross-Attention Graph Convolutional Network (DPCA-GCN), an architecture that explicitly separates spatial and temporal modeling into two specialized pathways while maintaining rich bidirectional interaction between them. The spatial branch integrates graph convolution and spatial transformers to capture intra-frame joint relationships, whereas the temporal branch combines temporal convolution and temporal transformers to model inter-frame dependencies. A bidirectional cross-attention mechanism facilitates explicit information exchange between both paths, and an adaptive gating module balances their respective contributions according to the action context. Unlike traditional approaches that process spatial–temporal information sequentially, our dual-path design enables specialized processing while maintaining cross-modal coherence through memory-efficient chunked attention mechanisms. Extensive experiments on the NTU RGB+D 60 and NTU RGB+D 120 datasets demonstrate that DPCA-GCN achieves competitive joint-only accuracies of 88.72%/94.31% and 82.85%/83.65%, respectively, with exceptional top-5 scores of 96.97%/99.14% and 95.59%/95.96%, while maintaining significantly lower computational complexity compared to multi-modal approaches. Full article

The graph convolutional network (GCN) has become a mainstream technology in skeleton-based action recognition since it was first applied to this field. However, previous studies often overlooked the pivotal role of heuristic model initialization in the extraction of spatial features, impeding the model from achieving its optimal performance. To address this issue, a lightweight initialization-enhanced adaptive graph convolutional network (LI-AGCN) is proposed, which effectively captures spatiotemporal features while maintaining low computational complexity. LI-AGCN employs three coordinate-based input branches (CIB) to dynamically adjust graph structures, which facilitates the extraction of informative spatial features. In addition, the model incorporates a lightweight and multi-scale temporal module to extract temporal feature, and employs an attention module that considers the temporal, spatial, and channel dimensions simultaneously to enhance key features. Finally, the performance of our proposed model is evaluated on three large-scale public datasets: NTU RGB+D, NTU RGB+D 120, and UAV-Human. The experimental results demonstrate that the LI-AGCN achieves excellent comprehensive performances on these datasets, especially obtaining 90.03% accuracy on the cross-subject benchmark of the NTU RGB+D dataset with only 0.18 million parameters, showcasing the effectiveness of the model. Full article