Search Results (9)

A new approach for skeleton extraction has been designed to work directly with 3D point cloud data. It blends hierarchical segmentation with a multi-scale ensemble built on top of modified PointNet models. Outputs from three network variants trained at different spatial resolutions are aggregated using majority voting, unweighted averaging, and adaptive weighting, with the latter yielding the best performance. Each joint is set at the center of its part. A radius-based filter is used to remove any outliers, specifically, points that fall too far from where the joints are expected to be. When evaluated on benchmark datasets such as DFaust, CMU, Kids, and EHF, the model demonstrated strong segmentation accuracy (mIoU = 0.8938) and low joint localization error (MPJPE = 22.82 mm). The method generalizes well to an unseen dataset (DanceDB), maintaining strong performance across diverse body types and poses. Compared to benchmark methods such as L1-Medial, Pinocchio, and MediaPipe, our approach offers greater anatomical symmetry, joint completeness, and robustness in occluded or overlapping regions. Structural integrity is maintained by working directly with 3D data, without the need for 2D projections or medial-axis approximations. The visual assessment of DanceDB results indicates improved anatomical accuracy, even in the absence of quantitative comparison. The outcome supports practical applications in animation, motion tracking, and biomechanics. Full article

Predicting human future motion holds significant importance in the domains of autonomous driving and public safety. Kinematic features, including joint coordinates and velocity, are commonly employed in skeleton-based human motion prediction. Nevertheless, most existing approaches neglect the critical role of dynamic information and tend to degrade as the prediction length increases. To address the related constraints due to single-scale and fixed-joint topological relationships, this study proposes a novel method that incorporates joint torques estimated via Lagrangian equations as dynamic features of the human body. Specifically, the human skeleton is modeled as a multi-rigid body system, with generalized joint torques calculated based on the Lagrangian formula. Furthermore, to extract both kinematic and dynamic joint information effectively for predicting long-term human motion, we propose a Multiscale Mixed-Graph Neural Network (MS-MGNN). MS-MGNN can extract kinematic and dynamic joint features across three distinct scales: joints, limbs, and body parts. The extraction of joint features at each scale is facilitated by a single-scale mixed-graph convolution module. And to effectively integrate the extracted kinematic and dynamic features, a KD-fused Graph-GRU (Kinematic and Dynamics Fused Graph Gate Recurrent Unit) predictor is designed to fuse them. Finally, the proposed method exhibits superior motion prediction capabilities across multiple motions. In motion prediction experiments on the Human3.6 dataset, it outperforms existing approaches by decreasing the average prediction error by 9.1%, 12.2%, and 10.9% at 160 ms, 320 ms, and 400 ms for short-term prediction and 7.1% at 560 ms for long-term prediction. Full article

In action recognition, obtaining skeleton data from human poses is valuable. This process can help eliminate negative effects of environmental noise, including changes in background and lighting conditions. Although GCN can learn unique action features, it fails to fully utilize the prior knowledge of human body structure and the coordination relations between limbs. To address these issues, this paper proposes a Multi-level Topological Channel Attention Network algorithm: Firstly, the Multi-level Topology and Channel Attention Module incorporates prior knowledge of human body structure using a coarse-to-fine approach, effectively extracting action features. Secondly, the Coordination Module utilizes contralateral and ipsilateral coordinated movements in human kinematics. Lastly, the Multi-scale Global Spatio-temporal Attention Module captures spatiotemporal features of different granularities and incorporates a causal convolution block and masked temporal attention to prevent non-causal relationships. This method achieved accuracy rates of 91.9% (Xsub), 96.3% (Xview), 88.5% (Xsub), and 90.3% (Xset) on NTU-RGB+D 60 and NTU-RGB+D 120, respectively. Full article

In recent years, skeleton-based human action recognition has garnered significant research attention, with proposed recognition or segmentation methods typically validated on large-scale coarse-grained action datasets. However, there remains a lack of research on the recognition of small-scale fine-grained human actions using deep learning methods, which have greater practical significance. To address this gap, we propose a novel approach based on heatmap-based pseudo videos and a unified, general model applicable to all modality datasets. Leveraging anthropometric kinematics as prior information, we extract common human motion features among datasets through an ad hoc pre-trained model. To overcome joint mismatch issues, we partition the human skeleton into five parts, a simple yet effective technique for information sharing. Our approach is evaluated on two datasets, including the public Nursing Activities and our self-built Tai Chi Action dataset. Results from linear evaluation protocol and fine-tuned evaluation demonstrate that our pre-trained model effectively captures common motion features among human actions and achieves steady and precise accuracy across all training settings, while mitigating network overfitting. Notably, our model outperforms state-of-the-art models in recognition accuracy when fusing joint and limb modality features along the channel dimension. Full article

Ambient Assisted Living (AAL) systems are designed to provide unobtrusive and user-friendly support in daily life and can be used for monitoring frail people based on various types of sensors, including wearables and cameras. Although cameras can be perceived as intrusive in terms of privacy, low-cost RGB-D devices (i.e., Kinect V2) that extract skeletal data can partially overcome these limits. In addition, deep learning-based algorithms, such as Recurrent Neural Networks (RNNs), can be trained on skeletal tracking data to automatically identify different human postures in the AAL domain. In this study, we investigate the performance of two RNN models (2BLSTM and 3BGRU) in identifying daily living postures and potentially dangerous situations in a home monitoring system, based on 3D skeletal data acquired with Kinect V2. We tested the RNN models with two different feature sets: one consisting of eight human-crafted kinematic features selected by a genetic algorithm, and another consisting of 52 ego-centric 3D coordinates of each considered skeleton joint, plus the subject’s distance from the Kinect V2. To improve the generalization ability of the 3BGRU model, we also applied a data augmentation method to balance the training dataset. With this last solution we reached an accuracy of 88%, the best we achieved so far. Full article

Most approaches to action recognition based on pseudo-images involve encoding skeletal data into RGB-like image representations. This approach cannot fully exploit the kinematic features and structural information of human poses, and convolutional neural network (CNN) models that process pseudo-images lack a global field of view and cannot completely extract action features from pseudo-images. In this paper, we propose a novel pose-based action representation method called Optical Flow Pose Image (OFPI) in order to fully capitalize on the spatial and temporal information of skeletal data. Specifically, in the proposed method, an advanced pose estimator collects skeletal data before locating the target person and then extracts skeletal data utilizing a human tracking algorithm. The OFPI representation is obtained by aggregating these skeletal data over time. To test the superiority of OFPI and investigate the significance of the model having a global field of view, we trained a simple CNN model and a transformer-based model, respectively. Both models achieved superior outcomes. Because of the global field of view, especially in the transformer-based model, the OFPI-based representation achieved 98.3% and 94.2% accuracy on the KTH and JHMDB datasets, respectively. Compared with other advanced pose representation methods and multi-stream methods, OFPI achieved state-of-the-art performance on the JHMDB dataset, indicating the utility and potential of this algorithm for skeleton-based action recognition research. Full article

The research introduces a unique deep-learning-based technique for remote rehabilitative analysis of image-captured human movements and postures. We present a ploninomial Pareto-optimized deep-learning architecture for processing inverse kinematics for sorting out and rearranging human skeleton joints generated by RGB-based two-dimensional (2D) skeleton recognition algorithms, with the goal of producing a full 3D model as a final result. The suggested method extracts the entire humanoid character motion curve, which is then connected to a three-dimensional (3D) mesh for real-time preview. Our method maintains high joint mapping accuracy with smooth motion frames while ensuring anthropometric regularity, producing a mean average precision (mAP) of $0.950$ for the task of predicting the joint position of a single subject. Furthermore, the suggested system, trained on the MoVi dataset, enables a seamless evaluation of posture in a 3D environment, allowing participants to be examined from numerous perspectives using a single recorded camera feed. The results of evaluation on our own self-collected dataset of human posture videos and cross-validation on the benchmark MPII and KIMORE datasets are presented. Full article

Human action recognition (HAR) is the foundation of human behavior comprehension. It is of great significance and can be used in many real-world applications. From the point of view of human kinematics, the coordination of limbs is an important intrinsic factor of motion and contains a great deal of information. In addition, for different movements, the HAR algorithm provides important, multifaceted attention to each joint. Based on the above analysis, this paper proposes a HAR algorithm, which adopts two attention modules that work together to extract the coordination characteristics in the process of motion, and strengthens the attention of the model to the more important joints in the process of moving. Experimental data shows these two modules can improve the recognition accuracy of the model on the public HAR dataset (NTU-RGB + D, Kinetics-Skeleton). Full article

Quantitative functional assessment of Posture and Motion Analysis of the entire skeleton and spine is highly desirable. Nonetheless, in most studies focused on posture and movement biomechanics, the spine is only grossly depicted because of its required level of complexity. Approaches integrating pressure measurement devices with stereophotogrammetric systems have been presented in the literature, but spine biomechanics studies have rarely been linked to baropodometry. A new multi-sensor system called GOALS-E.G.G. (Global Opto-electronic Approach for Locomotion and Spine-Expert Gait Guru), integrating a fully genlock-synched baropodometric treadmill with a stereophotogrammetric device, is introduced to overcome the above-described limitations. The GOALS-EGG extends the features of a complete 3D parametric biomechanical skeleton model, developed in an original way for static 3D posture analysis, to kinematic and kinetic analysis of movement, gait and run. By integrating baropodometric data, the model allows the estimation of lower limb net-joint forces, torques and muscle power. Net forces and torques are also assessed at intervertebral levels. All the elaborations are completely automatised up to the mean behaviour extraction for both posture and cyclic-repetitive tasks, allowing the clinician/researcher to perform, per each patient, multiple postural/movement tests and compare them in a unified statistically reliable framework. Full article