Search Results (118)

This paper presents a human-in-the-loop interactive framework for skeleton-based posture recognition, designed to support model training and artistic education. A total of 4870 labeled images are used for training and validation, and 500 images are reserved for testing across five core posture categories: standing, sitting, jumping, crouching, and lying. From each image, comprehensive skeletal features are extracted, including joint coordinates, angles, limb lengths, and symmetry metrics. Multiple classification algorithms—traditional (KNN, SVM, Random Forest) and deep learning-based (LSTM, Transformer)—are compared to identify effective combinations of features and models. Experimental results show that deep learning models achieve superior accuracy on complex postures, while traditional models remain competitive with low-dimensional features. Beyond classification, the system integrates posture recognition with a visual recommendation module. Recognized poses are used to retrieve matched examples from a reference library, allowing instructors to browse and select posture suggestions for learners. This semi-automated feedback loop enhances teaching interactivity and efficiency. Among all evaluated methods, the Transformer model achieved the best accuracy of 92.7% on the dataset, demonstrating the effectiveness of our closed-loop framework in supporting pose classification and model training. The proposed framework contributes both algorithmic insights and a novel application design for posture-driven educational support systems. Full article

Resilient artificial intelligence (Resilient AI) is relevant in many areas where technology needs to adapt quickly to changing and unexpected conditions, such as in the medical, environmental, security, and agrifood sectors. In the case study involving the therapeutic rehabilitation of patients with motor problems, the Resilient AI system is crucial to ensure that systems can effectively respond to changes, maintain high performance, cope with uncertainties and complex variables, and enable the dynamic monitoring and adaptation of therapy in real time. The proposed system integrates advanced technologies, such as computer vision and deep learning models, focusing on non-invasive solutions for monitoring and adapting rehabilitation therapies. The system combines the Microsoft Kinect v3 sensor with MoveNet Thunder – SinglePose, a state-of-the-art deep-learning model for human pose estimation. Kinect’s 3D skeletal tracking and MoveNet’s high-precision 2D keypoint detection together improve the accuracy and reliability of postural analysis. The main objective is to develop an intelligent system that captures and analyzes a patient’s movements in real time using Motion Capture techniques and artificial intelligence (AI) models to improve the effectiveness of therapies. Computer vision tracks human movement, identifying crucial biomechanical parameters and improving the quality of rehabilitation. Full article

Human pose estimation is an important research direction in the field of computer vision, aiming to detect and locate key points of the human body from images or videos and infer human posture. It plays a significant role in many applications, such as action recognition, motion analysis, virtual reality, and human–computer interaction. As a popular research topic, it is often studied by beginners in deep learning. However, the task of human pose estimation is rather complex, and the mainstream datasets are huge. Even on high-end single-GPU devices, training models requires a considerable amount of time. To help beginners learn efficiently on devices with limited performance, this paper introduces the method of mixed-precision training into the model and combines it with early stopping to reduce the training time. The experimental results show that after introducing mixed-precision training, the training speed of the model was significantly improved and there was no significant decrease in model accuracy. Full article

Transformer has a wide range of applications in human posture estimation. It can model the global dependence relationship of images through the self-attention mechanism to obtain key human body information. However, Transformer consumes a lot of computation. An axial compression pose transformer (ACPose) method is proposed to reduce part of the computational cost of Transformer by the axial compression of the input matrix, while maintaining the global receptive field by feature fusion. A Local Enhancement Module is constructed to avoid the loss of too much feature information in the compression process. In the COCO dataset experiment, there was a significant reduction in computational cost compared to those of state-of-the-art transformer-based algorithms. Full article

Estimating human posture in crowded smart teaching environments is a fundamental technical challenge for measuring learners’ engagement levels. This work presents a model for detecting critical points in human posture using ECAv2-HRNet in crowded situations. The paper introduces a method called ECAv2Net, which combines a channel feature reinforcement method with the ECANet attention mechanism network, this innovation improves the performance of the network. Additionally, ECAv2Net is integrated into the high-resolution network HRNet to create ECAv2-HRNet. This fusion allows for the incorporation of more useful feature information without increasing the model parameters. The paper also presents a human posture dataset called GUET CLASS PICTURE, which is designed for dense scenes. The experimental results when using this dataset, as well as a public dataset, demonstrate the superior performance of the human posture estimation model based on ECAv2-HRNet proposed in this paper. Full article

Human pose comparison involves measuring the similarities in body postures between individuals to understand movement patterns and interactions, yet existing methods are often insufficiently robust and flexible. In this paper, we propose a RotJoint-based pipeline for pose similarity estimation that is both fine-grained and generalizable, as well as robust. Firstly, we developed a comprehensive benchmark for action ambiguity that intuitively and effectively evaluates the robustness of pose comparison methods against challenges such as body shape variations, viewpoint variations, and torsional poses. To address these challenges, we define a feature representation called RotJoints, which is strongly correlated with both the semantic and spatial characteristics of the pose. This parameter emphasizes the description of limb rotations across multiple dimensions, rather than merely describing orientation. Finally, we propose TemporalRotNet, a Transformer-based network, trained via supervised contrastive learning to capture spatial–temporal motion features. It achieves 93.7% accuracy on NTU-RGB+D close set action classification and 88% on the open set, demonstrating its effectiveness for dynamic motion analysis. Extensive experiments demonstrate that our RotJoint-based pipeline produces results more aligned with human understanding across a wide range of common pose comparison tasks and achieves superior performance in situations prone to ambiguity. Full article

Sensorimotor training on an unstable base of support is considered to lead to improvements in balance and coordination tasks. Here, we intend to lay the groundwork for generating cost-effective real-time kinematic feedback for coordination training on devices with an unstable base of support, such as Sensopros or slacklines, by establishing a model for estimating relevant tape kinematic data from angle measurements alone. To assess the accuracy of the model in a real-world setting, we record a convenience sample of three people performing ten exercises on the Sensopro Luna and compare the model predictions to motion capture data of the tape. The measured accuracy is reported for each target measure separately, namely the roll angle and $XYZ$-position of the tape segment directly below the foot. After the initial assessment of the model in its general form, we also propose how to adjust the model parameters based on preliminary measurements to adapt it to a specific setting and further improve its accuracy. The results show that the proposed method is viable for recording tape kinematic data in real-world settings, and may therefore serve as a performance indicator directly or form the basis for estimating posture and other measures related to human motor control in a more intricate training feedback system. Full article

Numerous studies leverage pose estimation to extract human keypoint data and then classify sitting postures. However, employing neural networks for direct keypoint classification often yields suboptimal results. Alternatively, modeling keypoints into other data representations before classification introduces redundant information and substantially increases inference time. In addition, most existing methods perform well only under a single fixed viewpoint, limiting their applicability in complex real-world scenarios involving unseen viewpoints. To better address the first limitation, we propose KeypointNet, which employs a decoupled feature extraction strategy consisting of a Keypoint Feature Extraction module and a Multi-Scale Feature Extraction module. In addition, to enhance multi-view recognition capability, we propose the Multi-View Simulation (MVS) algorithm, which augments the viewpoint information by first rotating keypoints and then repositioning the camera. Simultaneously, we propose the multi-view sitting posture (MVSP) dataset, designed to simulate diverse real-world viewpoints. The experimental results demonstrate that KeypointNet outperforms the other state-of-the-art methods on both the proposed MVSP dataset and the other public datasets, while maintaining a lightweight and efficient design. Ablation studies demonstrate the effectiveness of MVS and all KeypointNet modules. Furthermore, additional experiments highlight the superior generalization, small-sample learning capability, and robustness to unseen viewpoints of KeypointNet. Full article

Relating continuously collected inertial data to the activities or postures performed by the sensor wearer requires pattern recognition or machine-learning-based algorithms, accounting for the temporal and scale variability present in human movements. The objective of this study was to develop a sensor-based framework for the detection and measurement of high-flexion postures frequently adopted in occupational settings. IMU-based joint angle estimates for the ankle, knee, and hip were time and scale normalized prior to being input to a multi-dimensional Dynamic Time Warping (mDTW) distance-based Nearest Neighbour algorithm for the identification of twelve postures. Data from 50 participants were divided to develop and evaluate the mDTW model. Overall accuracies of 82.3% and 55.6% were reached when classifying movements from the testing and validation datasets, respectively, which increased to 86% and 74.6% when adjusting for imbalances between classification groups. The highest misclassification rates occurred between flatfoot squatting, heels-up squatting, and stooping, while the model was incapable of identifying sequences of walking based on a single stride template. The developed mDTW model proved robust in identifying high-flexion postures performed by participants both included and precluded from algorithm development, indicating its strong potential for the quantitative measurement of postural adoption in real-world settings. Full article

Kinect, with its intrinsic and accessible human motion capture capabilities, found widespread application in real-world scenarios such as rehabilitation therapy and robot control. Consequently, a thorough analysis of its previously under-examined motion capture accuracy is of paramount importance to mitigate the risks potentially arising from recognition errors in practical applications. This study employs a high-precision, marker-based motion capture system to generate ground truth human pose data, enabling an evaluation of Azure Kinect’s performance across a spectrum of tasks, which include both static postures and dynamic movement behaviors. Specifically, the cosine similarity for skeletal representation is employed to assess pose estimation accuracy from an application-centric perspective. Experimental results reveal that factors such as the subject’s distance and orientation relative to the Kinect, as well as self-occlusion, exert a significant influence on the fidelity of Azure Kinect’s human posture recognition. Optimal testing recommendations are derived based on the observed trends. Furthermore, a linear fitting analysis between the ground truth data and Azure Kinect’s output suggests the potential for performance optimization under specific conditions. This research provides valuable insights for the informed deployment of Kinect in applications demanding high-precision motion recognition. Full article

The construction industry is considered one of the most hazardous industries. The accidents associated with construction equipment are a leading cause of fatalities in the U.S., with one-quarter of all fatalities in the construction industry due to equipment-related incidents, including collisions, struck-by events, and rollovers. While close collaboration among multiple equipment and humans is common, conventional collision alert mechanisms for equipment usually rely on distance sensors with static thresholds, often resulting in too many false alarms, causing drivers’ ignorance. Considering the collaborative operation scenario, this research proposes and develops a dynamic-threshold alert system by recognizing hazardous events based on the types of nearby objects with their orientation or postures and their distances to the system carrier equipment based on image-based recognition and Sim2Real techniques. Two experiments were conducted, and the results show that the system successfully reduced a large number of false near-collision alarms for the collaboration scenarios. Although the accuracy of object recognition and image-based distance estimation is feasible for practical use, it is also easily degraded in the self-obstruction scenario or for equipment with large and movable parts due to incorrect recognition of the bounding boxes of the target objects. Full article

A rapidly growing body of experimental evidence in the literature shows that the effects of humans interacting with vibrating structures, other humans, and their surrounding environment can be critical for reliable estimation of structural vibrations. The Interaction-based Vibration Serviceability Assessment framework (I-VSA) was proposed by the authors in 2017 to address this, taking into account human-structure dynamic interactions (HSI) to simulate the structural vibrations experienced by each occupant/pedestrian. The I-VSA method, however, had limited provisions to simulate simultaneously multiple modes of structure in HSI, to simulate human-human and human-environment interactions, and the movement pattern of the occupants/pedestrians. This study proposes a new Agent-based Vibration Serviceability Assessment framework, termed AVSA, to address the following limitations: (a) allowing for multiple modes of structure to be simulated simultaneously, (b) to simulate effects of vibrations on gait parameters and walking pattern/routes, and (c) to simulate human-environment interactions, and movement patterns for any desired interior layout and use case. The AVSA framework was used to simulate the response and to assess the vibration serviceability of a lightweight floor under a combination of sitting and walking traffic, where three vertical modes of vibrations were engaged simultaneously. The results of the simulations show that for all tests, the experimental Cumulative Distribution Functions of the vibrations experienced by the participants are within the 95% confidence interval predicted by the AVSA method. The proposed method provides a generic and flexible framework to simulate simultaneously different interaction modalities, different human tasks and postures, and multiple modes of structure and the human body. Full article

Human pose estimation is an important research direction in the field of computer vision, which aims to accurately identify the position and posture of keypoints of the human body through images or videos. However, multi-person pose estimation yields false detection or missed detection in dense crowds, and it is still difficult to detect small targets. In this paper, we propose a Mamba-based human pose estimation. First, we design a GMamba structure to be used as a backbone network to extract human keypoints. A gating mechanism is introduced into the linear layer of Mamba, which allows the model to dynamically adjust the weights according to the different input images to locate the human keypoints more precisely. Secondly, GMamba as the backbone network can effectively solve the long-sequence problem. The direct use of convolutional downsampling reduces selectivity for different stages of information flow. We used slice downsampling (SD) to reduce the resolution of the feature map to half the original size, and then fused local features from four different locations. The fusion of multi-channel information helped the model obtain rich pose information. Finally, we introduced an adaptive threshold focus loss (ATFL) to dynamically adjust the weights of different keypoints. We assigned higher weights to error-prone keypoints to strengthen the model’s attention to these points. Thus, we effectively improved the accuracy of keypoint identification in cases of occlusion, complex background, etc., and significantly improved the overall performance of attitude estimation and anti-interference ability. Experimental results showed that the AP and AP50 of the proposed algorithm on the COCO 2017 validation set were 72.2 and 92.6. Compared with the typical algorithm, it was improved by 1.1% on AP50. The proposed method can effectively detect the keypoints of the human body, and provides stronger robustness and accuracy for the estimation of human posture in complex scenes. Full article

The Internet of Things (IoT) is gaining increasing attention in healthcare due to its potential to enable continuous monitoring of patients, both at home and in controlled medical environments. In this paper, we explore the integration of IoT with human-robotics in the context of motor rehabilitation for groups of patients performing moderate physical routines, focused on balance, stretching, and posture. Specifically, we propose the I-TROPHYTS framework, which introduces a step-change in motor rehabilitation by advancing towards more sustainable medical services and personalized diagnostics. Our framework leverages wearable sensors to monitor patients’ vital signs and edge computing to detect and estimate motor routines. In addition, it incorporates a humanoid robot that mimics the actions of a physiotherapist, adapting motor routines in real-time based on the patient’s condition. All data from physiotherapy sessions are modeled using an ontology, enabling automatic reasoning and planning of robot actions. In this paper, we present the architecture of the proposed framework, which spans four layers, and discuss its enabling components. Furthermore, we detail the current deployment of the IoT system for patient monitoring and automatic identification of motor routines via Machine Learning techniques. Our experimental results, collected from a group of volunteers performing balance and stretching exercises, demonstrate that we can achieve nearly 100% accuracy in distinguishing between shoulder abduction and shoulder flexion, using Inertial Measurement Unit data from wearable IoT devices placed on the wrist and elbow of the test subjects. Full article

This study addresses the challenges of human–robot interactions in real-time environments with adaptive field-programmable gate array (FPGA)-based accelerators. Predicting human posture in indoor environments in confined areas is a significant challenge for service robots. The proposed approach works on two levels: the estimation of human location and the robot’s intention to serve based on the human’s location at static and adaptive positions. This paper presents three methodologies to address these challenges: binary classification to analyze static and adaptive postures for human localization in indoor environments using the sensor fusion method, adaptive Simultaneous Localization and Mapping (SLAM) for the robot to deliver the task, and human–robot implicit communication. VLSI hardware schemes are developed for the proposed method. Initially, the control unit processes real-time sensor data through PIR sensors and multiple ultrasonic sensors to analyze the human posture. Subsequently, static and adaptive human posture data are communicated to the robot via Wi-Fi. Finally, the robot performs services for humans using an adaptive SLAM-based triangulation navigation method. The experimental validation was conducted in a hospital environment. The proposed algorithms were coded in Verilog HDL, simulated, and synthesized using VIVADO 2017.3. A Zed-board-based FPGA Xilinx board was used for experimental validation. Full article