sensors-logo

Journal Browser

Journal Browser

Detecting, Measuring, and Modelling Speech and Body Movements in Virtual Reality and Human–Robot Interaction

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Biomedical Sensors".

Deadline for manuscript submissions: 1 July 2025 | Viewed by 638

Special Issue Editors


E-Mail Website
Guest Editor
Department of Mechanical and Electrical Engineering, University of Southern Denmark, 5230 Sonderborg, Denmark
Interests: speech prosody; speech perception; speech technology
Special Issues, Collections and Topics in MDPI journals

E-Mail Website
Guest Editor
Department of Mechanical and Electrical Engineering, University of Southern Denmark, 5230 Sonderborg, Denmark
Interests: virtual reality; public speaking

Special Issue Information

Dear Colleagues,

Voice and body movements are essential in human-to-human oral communication. With the use of virtual reality and human–robot interaction in increasingly diverse and demanding situations, nonverbal elements have become critical to the development of these technologies. The integration of natural language processing and machine learning has enhanced the intuitiveness and efficiency of these interactions, thereby promoting their widespread adoption, though there is still a lot of research to be conducted.

This Special Issue focuses on the detection, measurement, and modeling of speech and body movements within the contexts of virtual reality environments and human–robot interaction. It explores various strategies, including but not limited to the following: voice analysis, assessment, and elicitation techniques in virtual speech performances; the development and application of different voice synthesis methods tailored to various roles of voice assistants; the definition of optimal nonverbal behaviors for robotic interfaces; the development of mathematical and machine-learning models for the detection, measurement, and evaluation of vocal and bodily performances; and advanced voice signal processing methods to enhance the auditory and interaction design of robots.

Dr. Oliver Niebuhr
Dr. Ïo Valls-Ratés
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • voice
  • speech
  • acoustics
  • nonverbals
  • virtual reality
  • human–robot interaction
  • voice assistants
  • machine learning

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • Reprint: MDPI Books provides the opportunity to republish successful Special Issues in book format, both online and in print.

Further information on MDPI's Special Issue policies can be found here.

Published Papers (1 paper)

Order results
Result details
Select all
Export citation of selected articles as:

Research

18 pages, 1082 KiB  
Article
ITap: Index Finger Tap Interaction by Gaze and Tabletop Integration
by Jeonghyeon Kim, Jemin Lee, Jung-Hoon Ahn and Youngwon Kim
Sensors 2025, 25(9), 2833; https://doi.org/10.3390/s25092833 - 30 Apr 2025
Viewed by 293
Abstract
This paper presents ITap, a novel interaction method utilizing hand tracking to create a virtual touchpad on a tabletop. ITap facilitates touch interactions such as tapping, dragging, and swiping using the index finger. The technique combines gaze-based object selection with touch gestures, while [...] Read more.
This paper presents ITap, a novel interaction method utilizing hand tracking to create a virtual touchpad on a tabletop. ITap facilitates touch interactions such as tapping, dragging, and swiping using the index finger. The technique combines gaze-based object selection with touch gestures, while a pinch gesture performed with the opposite hand activates a manual mode, enabling precise cursor control independently of gaze direction. The primary purpose of this research is to enhance interaction efficiency, reduce user fatigue, and improve accuracy in gaze-based object selection tasks, particularly in complex and cluttered XR environments. Specifically, we addressed two research questions: (1) How does ITap’s manual mode compare with the traditional gaze + pinch method regarding speed and accuracy in object selection tasks across varying distances and densities? (2) Does ITap provide improved user comfort, naturalness, and reduced fatigue compared to the traditional method during prolonged scrolling and swiping tasks? To evaluate these questions, two studies were conducted. The first study compared ITap’s manual mode with the traditional gaze + pinch method for object selection tasks across various distances and in cluttered environments. The second study examined both methods for scrolling and swiping tasks, focusing on user comfort, naturalness, and fatigue. The findings revealed that ITap outperformed gaze + pinch in terms of object selection speed and error reduction, particularly in scenarios involving distant or densely arranged objects. Additionally, ITap demonstrated superior performance in scrolling and swiping tasks, with participants reporting greater comfort and reduced fatigue. The integration of gaze-based input and touch gestures provided by ITap offers a more efficient and user-friendly interaction method compared to the traditional gaze + pinch technique. Its ability to reduce fatigue and improve accuracy makes it especially suitable for tasks involving complex environments or extended usage in XR settings. Full article
Show Figures

Figure 1

Back to TopTop