Search Results (68)

A distinct typing interface using surface electromyography (sEMG) can facilitate silent, hands-free typing by interpreting muscle activity in relation to specific keystrokes. Character-level recognition poses greater challenges than coarse gesture recognition because it is sensitive to subtle temporal variations and overlapping muscle dynamics. Temporal features are essential for typing recognition because keypresses may differ in duration, force, and accompanying hand movements across users. This paper proposes TransTCNet, a two-stage deep neural network architecture with a causal convolutional layer for learning local features and a transformer-based component for learning long-range temporal interactions. We evaluated our network on a publicly available 26-class typing sEMG dataset acquired from 19 individuals. The model achieved a validation accuracy of 96.53%, exceeding the baseline models. Our study revealed generalization among participants, and the AUC values were also high (>0.994) across all classes. The model was highly reliable and exhibited high prediction confidence (>0.9), enabling us to achieve a high training accuracy (97.86%) for real-time filtering decisions. TransTCNet could be suitable for wearable and edge devices due to its efficient architecture and low inference cost. The model’s ability to consistently decode fine-grained neuromuscular signals across users makes it well-suited for real-time applications such as adaptive user interfaces, virtual and augmented reality, prosthetic control, and communication systems. Full article

Fitts’ law is a foundational model for predicting pointing performance and has been increasingly explored in immersive virtual reality (VR) environments. This paper presents a controlled experimental framework for deriving modality-specific Fitts’ law models in VR and evaluating their predictive transfer to applied interaction tasks. The framework comprises two scenarios. The first replicates a standardized ISO 9241 pointing task in a 3D virtual environment to derive predictive movement time models by systematically varying target distance (20–50 cm), target size (2.5–5 cm), and spatial configuration ($0^{\circ }$, ${45}^{\circ }$, ${90}^{\circ }$, ${135}^{\circ }$). The second simulates an applied warehouse-inspired task involving tool sorting and structured placement actions to evaluate the generalizability of the derived models in more ecologically valid VR interactions. Thirty-two participants completed all tasks using the Meta Quest 3 headset and two interaction modalities: a handheld controller and hand tracking with gesture recognition. Results show that Fitts’ law remains a strong predictor of movement time for 3D pointing in VR, with high linear fits for both the controller ($R^2=0.9615$) and hand tracking ($R^2=0.9668$). However, models derived from standardized pointing tasks showed limited transferability to applied object-manipulation scenarios, producing prediction errors of approximately 27–35% and systematically underestimating movement times. Additionally, both objective metrics and subjective evaluations indicated that controller-based interaction outperformed hand tracking in efficiency, accuracy, perceived workload, and usability. These findings highlight both the robustness and limitations of Fitts-based performance modeling in realistic VR interaction contexts. Full article

Flexible deformation sensors inevitably suffer from sensitivity degradation and severe measurement errors during long-term cyclic stretching due to structural fatigue. Traditional material-level optimizations are costly and lack dynamic adaptability. Herein, we propose an artificial intelligence (AI)-driven predict-and-compensate framework for the online calibration of flexible sensors. To overcome training sample scarcity, a generative adversarial network (GAN) performs temporal data augmentation. Subsequently, a hybrid deep learning framework integrating long short-term memory (LSTM) networks and a Sequence Attention mechanism is employed. This architecture accurately captures both local signal fluctuations and multiscale long-term decay trends, enabling precise multi-step prediction and output compensation. Experimental evaluations validate that this strategy significantly suppresses sensor response drift. Under cyclic loading, an initially substantial relative measurement error of 48.63% plummets to 7.16% post-calibration, with typical errors consistently reduced to the ~1% level. Furthermore, when deployed in a smart glove gesture recognition system, this method successfully restores the recognition accuracy from a fatigue-induced low of 75.73% (after 200 stretch cycles) back to 97.70%. This generative and attention-based deep learning paradigm offers robust, real-time error calibration, providing a highly viable solution for extending the long-term reliability and stability of flexible sensor systems. Full article

Arabic sign language (ArASL) is the primary communication medium for Deaf and hard-of-hearing people across Arabic-speaking communities. Most current ArASL recognition systems are based solely on visual features and do not incorporate linguistic or semantic information that could improve generalization and semantic grounding. This paper introduces CLIP-ArASL, a lightweight CLIP-style multimodal approach for static ArASL letter recognition that aligns visual hand gestures with bilingual textual descriptions. The approach integrates an EfficientNet-B0 image encoder with a MiniLM text encoder to learn a shared embedding space using a hybrid objective that combines contrastive and cross-entropy losses. This design supports supervised classification on seen classes and zero-shot prediction on unseen classes using textual class representations. The proposed approach is evaluated on two public datasets, ArASL2018 and ArASL21L. Under supervised evaluation, recognition accuracies of $99.25\pm 0.14\%$ and $91.51\pm 1.29\%$ are achieved, respectively. Zero-shot performance is assessed by withholding $20\%$ of gesture classes during training and predicting them using only their textual descriptions. In this setting, accuracies of $55.2\pm 12.15\%$ on ArASL2018 and $37.6\pm 9.07\%$ on ArASL21L are obtained. These results show that multimodal vision–language alignment supports semantic transfer and enables recognition of unseen classes. Full article

The adoption of active upper-limb prostheses with multiple degrees of freedom is largely lagging due to bulky designs and counterintuitive operation. Accurate gesture prediction with minimal sensors is key to enabling low-profile, user-friendly prosthetic devices. Wearable sensors, such as electromyography (EMG) and accelerometry (ACC) sensors, provide valuable signals for identifying patterns relating muscle activity and arm movement to specific gestures. This study investigates which sensor type (EMG or ACC) has the most valuable information to predict hand grasps and identifies the signal features contributing the most to grasp prediction performance. Using an open-source dataset, we trained two types of subject-specific classifiers (LDA & KNN) to predict 10 grasp types in 13 individuals with and 28 individuals without amputation. Having 4-fold cross-validation, LDA average accuracies using ACC only features (84.7%) were similar to combined ACC & EMG (88.3%) and much greater than with only EMG features (58.1%). Feature importance analysis showed that participants with amputation reached more than 80% accuracy using only three features, two of which were ACC-derived, while able-bodied participants required nine features, with greater reliance on EMG. These findings suggest that ACC is sufficient for robust grasp classification in individuals with amputation and can support simpler, more accessible prosthetic designs. Future work should focus on incorporating object and grip force detection alongside grasp recognition and testing model performance in real-time prosthetic control settings. Full article

Sign language recognition aims to capture and classify hand and arm motion signals to enable intuitive communication for individuals with hearing and speech impairments. This study proposes a real-time Chinese Sign Language (CSL) recognition framework that integrates a dual-stage segmentation strategy with a lightweight three-layer artificial neural network to achieve early gesture prediction before completion of motion sequences. The system was evaluated on a 21-class CSL dataset containing several highly similar gestures and achieved an accuracy of 91.5%, with low average inference latency per cycle. Furthermore, training set truncation experiments demonstrate that using only the first 50% of each gesture instance preserves model accuracy while reducing training time by half, thereby enhancing real-time efficiency and practical deployability for embedded or assistive applications. Full article

Electromyography (EMG) signals reflect hand motion intention and exhibit a certain degree of amplitude symmetry. Nowadays, recognition of hand motion intention based on EMG has enriched its burgeoning promotion in various applications, such as rehabilitation, prostheses, and intelligent supply chains. For instance, the motion intentions of humans can be conveyed to logistics equipment, thereby improving the level of intelligence in a supply chain. To enhance the recognition accuracy of multiple hand motion intentions, this paper proposes a hand motion intention recognition method that decodes EMG signals based on improved long short-term memory (LSTM). Firstly, we performed preprocessing and utilized overlapping sliding windows on EMG segments. Secondly, we chose LSTM and improved it so as to capture features and enable prediction of hand motion intention. Specifically, we introduced the optimal key hyperparameter combination in the LSTM model using a genetic algorithm (GA). We found that our proposed method achieved relatively high accuracy in detecting hand motion intention, with average accuracies of 92.0% (five gestures) and 89.7% (seven gestures), while the highest accuracy reached 100.0% (seven gestures). Our paper may provide a way to predict the motion intention of the human hand for intention communication. Full article

In the field of Hand Gesture Recognition (HGR), Electromyography (EMG) is used to detect the electrical impulses that muscles emit when a movement is generated. Currently, there are several HGR models that use EMG to predict hand gestures. However, most of these models have limited performance in real-time applications, with the highest recognition rate achieved being 65.78 ± 15.15%, without post-processing steps. Other non-generalizable models, i.e., those trained with a small number of users, achieved a window-based classification accuracy of 93.84%, but not in time-real applications. Therefore, this study addresses these issues by employing transformers to create a generalizable model and enhance recognition accuracy in real-time applications. The architecture of our model is composed of a Convolutional Neural Network (CNN), a positional encoding layer, and the transformer encoder. To obtain a generalizable model, the EMG-EPN-612 dataset was used. This dataset contains records of 612 individuals. Several experiments were conducted with different architectures, and our best results were compared with other previous research that used CNN, LSTM, and transformers. The findings of this research reached a classification accuracy of 95.25 ± 4.9% and a recognition accuracy of 89.7 ± 8.77%. This recognition accuracy is a significant contribution because it encompasses the entire sequence without post-processing steps. Full article

The widespread use of WiFi Internet-of-Things (IoT) devices has rendered them valuable tools for detecting information about the physical environment. Recent studies have demonstrated that WiFi Channel State Information (CSI) can detect physical events like movement, occupancy increases, and gestures. This paper empirically investigates the conditions under which WiFi sensing technology remains effective for keystroke detection. To achieve this timely goal of assessing whether it can raise any privacy concerns, experiments are conducted using commodity hardware to predict the accuracy of WiFi CSI in detecting keys pressed on a keyboard. Our novel results show that, in an ideal setting with a robotic arm, the position of a specific key can be predicted with $99\%$ accuracy using a simple machine learning classifier. Furthermore, human finger localisation over a key and actual key-press recognition is also successfully achieved, with $94\%$ and $89\%$ reduced accuracy values, respectively. Moreover, our detailed investigation reveals that to ensure high accuracy, the gap distance between each test object must be substantial, while the size of the test group should be limited. Finally, we show WiFi sensing technology has limitations in small-scale gesture recognition for generic settings where proper device positioning is crucial. Specifically, detecting keyed words achieves an overall accuracy of $94\%$ for the forefinger and $87\%$ for multiple fingers when only the right hand is used. Accuracy drops to $56\%$ when using both hands. We conclude WiFi sensing is effective in controlled indoor environments, but it has limitations due to the device location and the limited granularity of sensing objects. Full article

Language is that kind of expression by which effective communication with another can be well expressed. One may consider such as a connecting bridge for bridging communication gaps for the hearing- and speech-impaired, even though it remains as an advanced method for hand gesture expression along with identification through the various different unidentified signals to configure their palms. This challenge can be met with a novel Enhanced Convolutional Transformer with Adaptive Tuna Swarm Optimization (ECT-ATSO) recognition framework proposed for double-handed sign language. In order to improve both model generalization and image quality, preprocessing is applied to images prior to prediction, and the proposed dataset is organized to handle multiple dynamic words. Feature graining is employed to obtain local features, and the ViT transformer architecture is then utilized to capture global features from the preprocessed images. After concatenation, this generates a feature map that is then divided into various words using an Inverted Residual Feed-Forward Network (IRFFN). Using the Tuna Swarm Optimization (TSO) algorithm in its enhanced form, the provided Enhanced Convolutional Transformer (ECT) model is optimally tuned to handle the problem dimensions with convergence problem parameters. In order to solve local optimization constraints when adjusting the position for the tuna update process, a mutation operator was introduced. The dataset visualization that demonstrates the best effectiveness compared to alternative cutting-edge methods, recognition accuracy, and convergences serves as a means to measure performance of this suggested framework. Full article

Body posture dynamics have garnered significant attention in recent years due to their critical role in understanding the emotional states conveyed through human movements during social interactions. Emotions are typically expressed through facial expressions, voice, gait, posture, and overall body dynamics. Among these, body posture provides subtle yet essential cues about emotional states. However, predicting an individual’s gait and posture dynamics poses challenges, given the complexity of human body movement, which involves numerous degrees of freedom compared to facial expressions. Moreover, unlike static facial expressions, body dynamics are inherently fluid and continuously evolving. This paper presents an effective method for recognizing 17 micro-emotions by analyzing kinematic features from the GEMEP dataset using video-based motion capture. We specifically focus on upper body posture dynamics (skeleton points and angle), capturing movement patterns and their dynamic range over time. Our approach addresses the complexity of recognizing emotions from posture and gait by focusing on key elements of kinematic gesture analysis. The experimental results demonstrate the effectiveness of the proposed model, achieving a high accuracy rate of 91.48% for angle metric + DNN and 93.89% for distance + DNN on the GEMEP dataset using a deep neural network (DNN). These findings highlight the potential for our model to advance posture-based emotion recognition, particularly in applications where human body dynamics distance and angle are key indicators of emotional states. Full article

Hand gesture recognition is a prominent topic in the recent literature, with surface ElectroMyoGraphy (sEMG) recognized as a key method for wearable Human–Machine Interfaces (HMIs). However, sensor placement still significantly impacts systems performance. This study addresses sensor displacement by introducing a fast and low-impact orientation correction algorithm for sEMG-based HMI armbands. The algorithm includes a calibration phase to estimate armband orientation and real-time data correction, requiring only two distinct hand gestures in terms of sEMG activation. This ensures hardware and database independence and eliminates the need for model retraining, as data correction occurs prior to classification or prediction. The algorithm was implemented in a hand gesture HMI system featuring a custom seven-channel sEMG armband with an Artificial Neural Network (ANN) capable of recognizing nine gestures. Validation demonstrated its effectiveness, achieving $93.36$% average prediction accuracy with arbitrary armband wearing orientation. The algorithm also has minimal impact on power consumption and latency, requiring just an additional 500 μW and introducing a latency increase of 408 μs. These results highlight the algorithm’s efficacy, general applicability, and efficiency, presenting it as a promising solution to the electrode-shift issue in sEMG-based HMI applications. Full article

Causal machine learning is an approach that combines causal inference and machine learning to understand and utilize causal relationships in data. In current research and applications, traditional machine learning and deep learning models always focus on prediction and pattern recognition. In contrast, causal machine learning goes a step further by revealing causal relationships between different variables. We explore a novel concept called Double Machine Learning that embraces causal machine learning in this research. The core goal is to select independent variables from a gesture identification problem that are causally related to final gesture results. This selection allows us to classify and analyze gestures more efficiently, thereby improving models’ performance and interpretability. Compared to commonly used feature selection methods such as Variance Threshold, Select From Model, Principal Component Analysis, Least Absolute Shrinkage and Selection Operator, Artificial Neural Network, and TabNet, Double Machine Learning methods focus more on causal relationships between variables rather than correlations. Our research shows that variables selected using the Double Machine Learning method perform well under different classification models, with final results significantly better than those of traditional methods. This novel Double Machine Learning-based approach offers researchers a valuable perspective for feature selection and model construction. It enhances the model’s ability to uncover causal relationships within complex data. Variables with causal significance can be more informative than those with only correlative significance, thus improving overall prediction performance and reliability. Full article

Surface electromyography (sEMG) signals reflect the local electrical activity of muscle fibers and the synergistic action of the overall muscle group, making them useful for gesture control of myoelectric manipulators. In recent years, deep learning methods have increasingly been applied to sEMG gesture recognition due to their powerful automatic feature extraction capabilities. sEMG signals contain rich local details and global patterns, but single-scale convolutional networks are limited in their ability to capture both comprehensively, which restricts model performance. This paper proposes a deep learning model based on multi-scale feature fusion—MS-CLSTM (MS Block-ResCBAM-Bi-LSTM). The MS Block extracts local details, global patterns, and inter-channel correlations in sEMG signals using convolutional kernels of different scales. The ResCBAM, which integrates CBAM and Simple-ResNet, enhances attention to key gesture information while alleviating overfitting issues common in small-sample datasets. Experimental results demonstrate that the MS-CLSTM model achieves recognition accuracies of 86.66% and 83.27% on the Ninapro DB2 and DB4 datasets, respectively, and the accuracy can reach 89% in real-time myoelectric manipulator gesture prediction experiments. The proposed model exhibits superior performance in sEMG gesture recognition tasks, offering an effective solution for applications in prosthetic hand control, robotic control, and other human–computer interaction fields. Full article

Uncertainty-aware soft sensors in sign language recognition (SLR) integrate methods to quantify and manage the uncertainty in their predictions. This is particularly crucial in SLR due to the variability in sign language gestures and differences in individual signing styles. Managing uncertainty allows the system to handle variations in signing styles, lighting conditions, and occlusions more effectively. While current techniques for handling uncertainty in SLR systems offer significant benefits in terms of improved accuracy and robustness, they also come with notable disadvantages. High computational complexity, data dependency, scalability issues, sensor and environmental limitations, and real-time constraints all pose significant hurdles. The aim of the work is to develop and evaluate a Type-2 Neutrosophic Hidden Markov Model (HMM) for SLR that leverages the advanced uncertainty handling capabilities of Type-2 neutrosophic sets. In the suggested soft sensor model, the Foot of Uncertainty (FOU) allows Type-2 Neutrosophic HMMs to represent uncertainty as intervals, capturing the range of possible values for truth, falsity, and indeterminacy. This is especially useful in SLR, where gestures can be ambiguous or imprecise. This enhances the model’s ability to manage complex uncertainties in sign language gestures and mitigate issues related to model drift. The FOU provides a measure of confidence for each recognition result by indicating the range of uncertainty. By effectively addressing uncertainty and enhancing subject independence, the model can be integrated into real-life applications, improving interactions, learning, and accessibility for the hearing-impaired. Examples such as assistive devices, educational tools, and customer service automation highlight its transformative potential. The experimental evaluation demonstrates the superiority of the Type-2 Neutrosophic HMM over the Type-1 Neutrosophic HMM in terms of accuracy for SLR. Specifically, the Type-2 Neutrosophic HMM consistently outperforms its Type-1 counterpart across various test scenarios, achieving an average accuracy improvement of 10%. Full article