Search Results (94)

A sign language interpretation system aims to translate sign gestures into spoken or written language in real time, enabling signers and non-signers to communicate in their familiar linguistic forms. However, vision-based approaches suffer from hand occlusion, lighting variability, and complex backgrounds, while Deep Neural Network (DNN)-based methods incur heavy computational costs that hinder real-time use on resource-constrained platforms. In this paper, we propose sign language gloves and a lightweight motion retrieval method for real-time sign language interpretation that runs on mobile devices and embedded systems. The sign language gloves integrate flex sensors, an inertial measurement unit (IMU), and pressure sensors to accurately capture gesture features, including finger bending angles, hand orientation, movement trajectories, and fingertip contacts with body parts, enabling recognition of touch-based gestures. For the motion retrieval method, we build a comprehensive gesture dataset with the gloves and perform feature analysis on each sign language gesture to avoid redundant information in the dataset. During interpretation, our system employs a feature-labeling mechanism to ensure gesture distinguishability and a gesture retrieval algorithm to evaluate movement continuity and similarity. This allows the system to identify corresponding feature labels and consolidate them into complete sign language vocabulary entries. The proposed motion retrieval method is characterized by low computational complexity and a well-defined data structure. This makes it suitable for integration into embedded systems, offering real-time performance and high portability for practical deployment. In our experiments, the proposed system achieved an average recognition accuracy of $92\%$ on a gesture dataset covering 300 sign language words. Full article

Static, one-time authentication mechanisms such as passwords and PINs are increasingly inadequate for protecting mobile devices throughout an active session. Behavioral biometric continuous authentication (BBCA) addresses this gap by passively monitoring user-specific interaction patterns—keystroke dynamics, touch and swipe gestures, gait, and motion—to verify identity on an ongoing basis. This systematic review synthesizes 80 studies selected via a PRISMA-compliant protocol from IEEE Xplore, ACM Digital Library, Scopus, ScienceDirect, Web of Science, and SpringerLink (2017–2025). We examine behavioral and multimodal biometric modalities, machine learning approaches ranging from classical classifiers to deep sequence and transformer architectures, and their integration with intelligent personal agents, wearable devices, and IoT/edge infrastructures. Security analyses cover spoofing, adversarial and generative attacks, mimicry, and model-level threats including membership inference and reconstruction. Privacy-preserving mechanisms—cancelable biometrics, Bloom filter encodings, zero-knowledge proof protocols, federated learning, and blockchain-based identity management—are evaluated against practical trade-offs in energy consumption and latency on resource-constrained devices. Key research gaps are identified: the absence of standardized adversarial benchmarks, lack of end-to-end pipeline evaluations under simultaneous adversarial and privacy threat models, and limited user-centered studies on consent and acceptance of privacy-preserving mechanisms under frameworks such as GDPR. Recommended future directions combine adaptive multimodal fusion, privacy-preserving cryptography, energy-aware modality selection, and interdisciplinary human-centered evaluation to advance practical, resilient continuous authentication for mobile and assistant-enriched environments. Full article

The majority of currently available hand kinematic databases have been gathered using expensive marker-based systems or are restricted to a particular gesture-recognition task, failing to capture the dynamic nature of joints when the hand is engaged with an object. To address this gap, we introduce the RGB-based Hand Joint Kinematics (RGB-HJK) dataset, a publicly available collection of continuous, frame-level 3D joint angle trajectories, recorded while ten healthy adults (six male, four female; age $25.8\pm 3.2$ years; BMI $22.8\pm 2.0$ kg/m²) performed five standardized object interaction grasps: Power Grasp (cylindrical bottle), Tripod Grasp (pen), Static Power Hold (smartphone), Precision Pinch (thin paper), and Lateral Pinch (book). Data were collected using a standard RGB camera and the MediaPipe Hands markerless pipeline at $26.95\pm 0.29$ Hz, a rate that was stable across all subjects. Each participant completed five trials for each grasp type. After filtering using active hold, 28,111 validated frames remained, with a 100% detection rate for all 250 trials. Intra-subject repeatability was good (mean SD $\le 7.9°$ across all joint grasp combinations) and inter-subject variability was within the range expected based on normal anatomical diversity. Importantly, kinematic validation of the Index Proximal Interphalangeal (PIP) joint (61.8° ± 18.4°) showed values consistent with ranges reported in previous studies using instrumented gloves and depth sensors. Principal Component Analysis (PCA) confirmed clear linear separability among the five grasp configurations. Unlike existing datasets, the RGB-HJK method does not compromise the natural sense of touch and is free of hardware occlusions, thereby providing an easily accessible ecological baseline. Full article

This paper provides a comprehensive survey of Human–Robot Interaction (HRI) for indoor mobile robots operating in human-centered environments such as hospitals, laboratories, offices, and homes. We review interaction modalities—including speech, gesture, touch, visual, and multimodal interfaces—and examine key user experience factors such as usability, trust, and social acceptance. Implementation challenges are discussed, encompassing safety, privacy, and regulatory considerations. Representative case studies, including healthcare and domestic platforms, highlight design trade-offs and integration lessons. We identify critical technical challenges, including robust perception, reliable multimodal fusion, navigation in dynamic spaces, and constraints on computation and power. Finally, we outline future directions, including embodied AI, adaptive context-aware interactions, and standards for safety and data protection. This survey aims to guide the development of indoor mobile robots capable of collaborating with humans naturally, safely, and effectively. Full article

Background: People living with dementia in nursing homes commonly experience progressive impairments in cognition, communication, and functional ability, contributing to neuropsychiatric symptoms and reduced quality of life. As verbal communication declines, non-verbal communication (NVC) including facial expressions, gestures, eye contact, posture, and touch becomes increasingly important for maintaining meaningful interactions. Objectives: This study aims to explore current NVC practices between nursing home staff and residents living with dementia. Methods: A mixed methods, cross-sectional design was employed. NH staff completed an anonymous online questionnaire consisting of 13 items assessing NVC use and demographic characteristics. Quantitative items were rated using Likert scales, and qualitative responses were analysed using Giorgi’s phenomenological approach. Results: Quantitative findings showed that residents most frequently relied on facial expressions, reported as used very often in 24 of 33 NHs, followed by eye contact in 17 NHs and touch in 16 NHs. NH staff also reported extensive use of NVC during care interactions, particularly facial expressions (very often in 79% of NHs), eye contact (82%), and hand gestures (76%). Qualitative findings underscored the central role of NVC in interpreting residents’ needs, fostering emotional connection, and managing behavioural and psychological symptoms of dementia through subtle cues, visual prompts, and individualised strategies. Conclusions: Overall, the findings demonstrate that NVC is a fundamental component of communication and care delivery in dementia settings and highlight the need for structured training interventions to support staff in recognising and responding effectively to non-verbal signals. Full article

We present FingerType, a one-handed text input method based on thumb-to-finger gestures. FingerType detects tap events from 3D hand data using a Temporal Convolutional Network (TCN) and decodes the tap sequence into words with an n-gram language model. To inform the design, we examined thumb-to-finger interactions and collected comfort ratings of finger regions. We used these results to design an improved T9-style key layout. Our system runs at 72 frames per second and reaches 94.97% accuracy for tap detection. We conducted a six-block user study with 24 participants and compared FingerType with controller input and touch input. Entry speed increased from 5.88 WPM in the first practice block to 10.63 WPM in the final block. FingerType also supported more eyes-free typing: attention on the display panel within ±15° of head-gaze was 84.41%, higher than touch input (69.47%). Finally, we report error patterns and WPM learning curves, and a model-based analysis suggests improving gesture recognition accuracy could further increase speed and narrow the gap to traditional VR input methods. Full article

Sensory mapping in architecture lacks a guiding theoretical model, leaving practitioners without a clear way to relate spatial design to embodied experience. This study introduces a structured methodology that links phenomenological observation with affordance theory and sensory semiotics, framing sensory data within architectural contexts. Fieldwork in fourteen courtyard houses of Damascus had residents trace their movements on simplified floor plans, switching colors as sight, sound, touch, smell and taste became dominant. The analysis reveals that visitors pass through a narrow entry corridor, enter the courtyard, and converge at the central fountain, which emerges as a focal point for multiple senses. Residents consistently trace tactile interactions along the fountain’s stone rim and at raised benches in the liwan (open space). Gustatory (taste-related, food-linked) markers appear along the route from kitchen thresholds toward the fountain, suggesting how food preparation and communal gathering overlap. Using 28 sensory maps and a three-level analytical process, comparison, synthesis, and spatial interpretation, the study produced a unified sensory map of the Damascene courtyard house. This pattern highlights how sequential spatial arrangements shape sensory engagement and suggests conservation strategies that preserve these experiential pathways. Architects and conservators can reinforce welcome gestures at thresholds and design water features and planting schemes that invite lingering. The proposed methodology fills the theoretical gap and offers clear guidelines for crafting spaces that respond to human perception. Full article

This study presents a multimodal framework that uses smartphone motion sensors and generative AI to create audio comics from live news headlines. The system operates without direct touch or voice input, instead responding to simple hand-wave gestures. The system demonstrates potential as an alternative input method, which may benefit users who find traditional touch or voice interaction challenging. In the experiments, we investigated the generation of comics on based on the latest tech-related news headlines using Really Simple Syndication (RSS) on a simple hand wave gesture. The proposed framework demonstrates extensibility beyond comic generation, as various other tasks utilizing large language models and multimodal AI could be integrated by mapping them to different hand gestures. Our experiments with open-source models like LLaMA, LLaVA, Gemma, and Qwen revealed that LLaVA delivers superior results in generating panel-aligned stories compared to Qwen3-VL, both in terms of inference speed and output quality, relative to the source image. These large language models (LLMs) collectively contribute imaginative and conversational narrative elements that enhance diversity in storytelling within the comic format. Additionally, we implement an AI-in-the-loop mechanism to iteratively improve output quality without human intervention. Finally, AI-generated audio narration is incorporated into the comics to create an immersive, multimodal reading experience. Full article

This paper introduces the Gesture-Based Physical Stability Classification and Rehabilitation System (GPSCRS), a low-cost, non-invasive solution for evaluating physical stability using an Arduino microcontroller and the DFRobot Gesture and Touch sensor. The system quantifies movement smoothness, consistency, and speed by analyzing “up” and “down” hand gestures over a fixed period, generating a Physical Stability Index (PSI) as a single metric to represent an individual’s stability. The system focuses on a temporal analysis of gesture patterns while incorporating placeholders for speed scores to demonstrate its potential for a comprehensive stability assessment. The performance of various machine learning and deep learning models for gesture-based classification is evaluated, with neural network architectures such as Transformer, CNN, and KAN achieving perfect scores in recall, accuracy, precision, and F1-score. Traditional machine learning models such as XGBoost show strong results, offering a balance between computational efficiency and accuracy. The choice of model depends on specific application requirements, including real-time constraints and available resources. The preliminary experimental results indicate that the proposed GPSCRS can effectively detect changes in stability under real-time conditions, highlighting its potential for use in remote health monitoring, fall prevention, and rehabilitation scenarios. By providing a quantitative measure of stability, the system enables early risk identification and supports tailored interventions for improved mobility and quality of life. Full article

Unlike traditional control systems that rely on physical input devices, facial gesture-based interaction offers a contactless and intuitive method for operating autonomous systems. Recent advances in computer vision and deep learning have enabled the use of facial expressions and movements for command recognition in human–robot interaction. In this work, we propose a lightweight, real-time facial gesture recognition method, GhostNet-BiLSTM-Attention (GBA), which integrates GhostNet and BiLSTM with an attention mechanism, is trained on the FaceGest dataset, and is integrated with a 3D robot simulation in Unity. The system is designed to recognize predefined facial gestures such as head tilts, eye blinks, and mouth movements with high accuracy and low inference latency. Recognized gestures are mapped to specific robot commands and transmitted to a Unity-based simulation environment via socket communication across machines. This framework enables smooth and immersive robot control without the need for conventional controllers or sensors. Real-time evaluation demonstrates the system’s robustness and responsiveness under varied user and lighting conditions, achieving a classification accuracy of 99.13% on the FaceGest dataset. The GBA holds strong potential for applications in assistive robotics, contactless teleoperation, and immersive human–robot interfaces. Full article

Adaptors, self-touching movements that supposedly lack communicative significance, have often been overlooked by researchers focusing on co-speech gestures. A significant complication in their study arises from the somewhat ambiguous definition of adaptors. Examples of these movements include self-manipulations like scratching a leg, bringing a hand to the mouth or head, and fidgeting, nervous tics, and micro hand or finger movements. Research rooted in psychology indicates a link between adaptors and negative emotional states. However, psycholinguistic approaches suggest that these movements might be related to the communicative task. This study analyzes adaptors in forty Cantonese speakers of English as a second language in monologues and dialogues in face-to-face and online contexts, revealing that adaptors serve functions beyond emotional expression. Our data indicate that adaptors might have cognitive functions. We also identify micro-movements, flutter-like adaptors or “flutters” for short, that may have interactive functions conveying engagement. These findings challenge the traditional view of adaptors as purely non-communicative. Participants’ self-reports corroborate these interpretations, highlighting the complexity and individual variability in adaptor use. This study advocates for the inclusion of adaptors in gesture analysis, which may enrich understanding of gesture–speech integration and cognitive and emotional processes in communication. Full article

This study investigated whether self-adaptor behaviours, defined as non-communicative bodily actions such as touching one’s face or clasping one’s hands, facilitate lexical retrieval. Sixty Japanese-speaking adults were assigned to one of three conditions: a self-adaptor condition (instructed to hold their cheeks), a suppression condition (hand movements inhibited), and a control condition in which participants were allowed to move their hands freely. Participants completed a lexical retrieval task designed to elicit tip-of-the-tongue (TOT) states, followed by a recall task. Responses were categorised as successful retrieval, retrieval failure, TOT success, or TOT failure. Results showed that the self-adaptor condition achieved significantly more correct responses than the suppression condition, indicating that self-adaptor behaviour supports lexical access during retrieval difficulty. However, self-adaptor behaviour did not enhance the resolution of TOT states or improve recall performance. Video-based analysis further revealed that gestures increased markedly during TOT states, while self-adaptor behaviours were more frequent overall. These findings suggest that self-adaptor behaviour contributes to general lexical retrieval processes but does not directly facilitate the resolution of TOT states. The distinction between the functions of spontaneous bodily movements is therefore essential. This study extends the understanding of how embodied behaviours interact with verbal processes and underscores the cognitive significance of non-communicative bodily actions in speech production. Full article

The increasing integration of 3D technologies and machine learning is fundamentally reshaping mineral sciences and cultural heritage, establishing the foundation for an emerging “Mineralogy 4.0” framework. However, public engagement with digital 3D collections is often limited by complex or costly interfaces, such as VR/AR systems and traditional touchscreen kiosks, creating a clear need for more intuitive, accessible, and more engaging and inclusive solutions. This paper presents PETRA, an open-source, gesture-controlled system for exploring 3D rocks and minerals. Developed in the TouchDesigner environment, PETRA utilizes a standard webcam and the MediaPipe framework to translate natural hand movements into real-time manipulation of digital specimens, requiring no specialized hardware. The system provides a customizable, node-based framework for creating touchless, interactive exhibits. Successfully evaluated during a “Long Night of Museums” public event with 550 visitors, direct qualitative observations confirmed high user engagement, rapid instruction-free learnability across diverse age groups, and robust system stability in a continuous-use setting. As a practical case study, PETRA demonstrates that low-cost, webcam-based gesture control is a viable solution for creating accessible and immersive learning experiences. This work offers a significant contribution to the fields of digital mineralogy, human–machine interaction, and cultural heritage by providing a hygienic, scalable, and socially engaging method for interacting with geological collections. This research confirms that as digital archives grow, the development of human-centered interfaces is paramount in unlocking their full scientific and educational potential. Full article

This paper examines a previously unknown anonymous Hebrew letter inserted into a postwar edition of Shem HaGedolim, found in the library of the Jewish University in Budapest. The letter, composed in Győr in 1947, consists almost entirely of passages copied from Tiferet Chayim, a hagiographic genealogy of the Sanz Hasidic dynasty. Although derivative in content, the letter’s form and placement suggest it was not meant for transmission but instead served as a private act of mourning and historiographical preservation. By situating the letter within the broader context of post-Holocaust Jewish and Hasidic memory practices, including yizkor books, rabbinic memoirs, and grassroots commemorative writing, this study proposes that the document constitutes a “micro-yizkor”: a bibliographic ritual that aimed to re-inscribe lost tzaddikim into sacred memory. Drawing on theories of trauma, religious coping, and bereavement psychology, particularly the Two-Track Model of Bereavement, the paper examines the letter as both a therapeutic and historiographical gesture. The author’s meticulous copying, selective omissions, and personalized touches (such as modified honorifics and emotive phrases) reflect an attempt to maintain spiritual continuity in the wake of communal devastation. Engaging scholarship by Michal Shaul, Lior Becker, Gershon Greenberg, and others, the analysis demonstrates how citation, far from being a passive act, functions here as an instrument of resistance, memory, and redemptive reconstruction. The existence of such a document can also be examined through the lens of Maurice Rickards’ insights, particularly his characterization of the “compulsive note” as a salient form of ephemera, materials often inserted between the pages of books, which pose unique challenges for interpreting the time capsule their authors sought to construct. Ultimately, the paper argues that this modest and anonymous document offers a rare window into postwar Ultra-orthodox religious subjectivity. It challenges prevailing assumptions about Hasidic silence after the Holocaust and demonstarates how even derivative texts can serve as potent sites of historical testimony, spiritual resilience, and bibliographic mourning. The letter thus sheds light on a neglected form of Hasidic historiography, one authored not by professional historians, but by the broken-hearted, writing in the margins of sacred books. Full article

As gesture-based interactions expand across traditional 2D screens and immersive XR platforms, designing intuitive input modalities tailored to specific contexts becomes increasingly essential. This study explores how users cognitively and experientially engage with gesture-based interactions in two distinct environments: a lean-back 2D television interface and an immersive XR spatial environment. A within-subject experimental design was employed, utilizing a gesture-recognizable smart ring to perform tasks using three gesture modalities: (a) Surface-Touch gesture, (b) mid-air gesture, and (c) micro finger-touch gesture. The results revealed clear, context-dependent user preferences; Surface-Touch gestures were preferred in the 2D context due to their controlled and pragmatic nature, whereas mid-air gestures were favored in the XR context for their immersive, intuitive qualities. Interestingly, longer gesture execution times did not consistently reduce user satisfaction, indicating that compatibility between the gesture modality and the interaction environment matters more than efficiency alone. This study concludes that successful gesture-based interface design must carefully consider the contextual alignment, highlighting the nuanced interplay among user expectations, environmental context, and gesture modality. Consequently, these findings provide practical considerations for designing Natural User Interfaces (NUIs) for various interaction contexts. Full article