Designing for Multi-Modal Human–Computer Interaction: Integrating Voice, Gesture, and Touch

Dear Colleagues,

Advances in sensing technologies, embedded systems, and artificial intelligence have significantly expanded the landscape of interactive systems. As digital environments increasingly rely on complex input mechanisms, the integration of voice, gesture, and touch has emerged as a key frontier in human–computer interaction. Multimodal interfaces enable more expressive, accessible, and adaptive communication between humans and machines, offering rich possibilities for real-time control, contextual understanding, and natural interaction.

Despite these advances, several challenges remain unresolved. These include the development of reliable cross-sensory fusion algorithms, the synchronization of input streams, latency and noise reduction, improving the interpretability of multimodal signals, improving the ergonomics of gesture-based communication, and the deployment of multimodal interaction in resource-constrained environments. Moreover, there is a growing need for systematic evaluation frameworks that measure performance not only at the algorithmic level but also in terms of user experience, responsiveness, error tolerance, and interaction fluidity.

This Special Issue aims to provide a platform for cutting-edge research on technologies, algorithms, architectures, and design strategies for multimodal interaction systems. We welcome submissions that contribute engineering advances, design principles, empirical findings, evaluation methodologies, and application-focused innovations in the domain of multimodal HCI.

We invite researchers to submit original research papers, technical/system papers, case studies, and review articles (including systematic, scoping, or state-of-the-art reviews).

Topics of Interest

We seek contributions in areas including, but not limited to, the following:

A. Multimodal Technologies and Signal Processing

• Architectures for combining voice, gesture, and touch input;
• Real-time multimodal fusion and synchronization (audio–tactile, audio–visual, visual–haptic);
• Speech recognition under noise, stress, and dynamic conditions;
• Gesture and body-movement tracking using cameras, IMUs, EMG, or depth sensors;
• Haptic rendering, tactile feedback modelling, and force-based interaction.

B. Machine Learning for Multimodal Interaction

• Deep learning approaches for multimodal intent recognition;
• Transformer architectures for cross-sensory understanding;
• Edge AI for on-device multimodal processing;
• Sensor-level data augmentation and cross-modal transfer learning.

C. Systems, Prototypes, and Applications

• Multimodal interfaces for XR (AR/VR/MR), robotics, and autonomous systems;
• Cross-device interaction (smartphones, wearables, environmental sensors);
• Multimodal control for smart homes, manufacturing, automotive environments;
• Assistive technologies integrating gesture, speech, and touch.

D. Design, Evaluation, and User Experience

• Interaction design frameworks for multimodal systems;
• Robustness, responsiveness, latency, and failure-mode analysis;
• User studies and performance evaluation metrics;
• Ergonomics of gesture-based interfaces.

E. Tools, Benchmarks, and Datasets

• Novel multimodal datasets;
• Evaluation toolkits and benchmarking methodologies;
• Standardization efforts and reproducibility frameworks.

Prof. Dr. Tihomir Orehovački
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 250 words) can be sent to the Editorial Office for assessment.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Electronics is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.