Submit to Special Issue Submit Abstract to Special Issue Review for Sensors Propose a Special Issue

Journal Menu

Journal Browser

► Journal Browser

Advanced Sensors and AI Integration for Human–Robot Teaming

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Benefits of Publishing in a Special Issue
Published Papers

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Sensors and Robotics".

Deadline for manuscript submissions: 25 January 2027 | Viewed by 21680

Share This Special Issue

Special Issue Editors

Prof. Dr. Nader Jalili

E-Mail Website
Guest Editor

Mary and Richard Templeton Dean of Lyle School of Engineering, Professor of Mechanical Engineering, Southern Methodist University (SMU), Dallas, TX, USA
Interests: automation; human–robot collaboration/integration and augmentation; micro nano-electromechanical sensors and actuators; smart manufacturing systems; human motor control; EEG; machine learning

Dr. Soroush Korivand

E-Mail Website
Guest Editor

Michael W. Hall School of Mechanical Engineering, Mississippi State University, Starkville, MS, USA
Interests: reinforcement learning; AI in rehabilitation; cyber-human systems; robotics
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

This Special Issue on "Advanced Sensors and AI Integration for Human–Robot Teaming" aims to explore the latest advancements in sensor technologies and their integration with artificial intelligence (AI) to enhance human–robot collaboration. As the fields of robotics and AI continue to evolve, the need for sophisticated sensors that can provide precise, real-time data is paramount. This issue will cover innovative sensor designs, data acquisition methods, and AI algorithms that enable seamless interaction between humans and robots.

This Special Issue invites contributions that address the challenges and opportunities in developing sensors that are not only accurate and reliable but also capable of being integrated into complex AI systems. Topics of interest include, but are not limited to, sensor fusion, machine learning for sensor data interpretation, real-time data processing, and the use of sensors in autonomous systems. By bringing together research from these diverse areas, this issue aims to provide a comprehensive overview of how advanced sensor technologies and AI can work together to improve the efficiency, safety, and functionality of human–robot teams.

Prof. Dr. Nader Jalili
Dr. Soroush Korivand
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 250 words) can be sent to the Editorial Office for assessment.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

human–robot teaming
sensor fusion
AI integration
real-time data processing
machine learning
autonomous systems
robotics
data acquisition
sensor technologies
intelligent systems

Benefits of Publishing in a Special Issue

Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
Reprint: MDPI Books provides the opportunity to republish successful Special Issues in book format, both online and in print.

Further information on MDPI's Special Issue policies can be found here.

Published Papers (9 papers)

Download All Papers

Order results

Result details

Show export options Show export options

Select all

Export citation of selected articles as:

Research

Jump to: Review

35 pages, 27039 KB

Open AccessArticle

A Complete Grocery Pick-and-Pack Application Using a Computationally Lightweight Vision-Based Mobile Manipulator

by Thanavin Mansakul, Gilbert Tang, Phil Webb, Jamie Rice, Daniel Oakley and James Fowler

Sensors 2026, 26(9), 2860; https://doi.org/10.3390/s26092860 - 3 May 2026

Viewed by 1125

Abstract

Mobile manipulators have become essential platforms for autonomous tasks that demand high-quality performance and efficient operational processes. This paper presents a complete grocery pick-and-pack system for a mobile manipulator, integrating a graphical user interface (GUI) with an end-to-end vision-based grasp detection pipeline designed for lightweight computation. The system is evaluated on the Grocery Pick-and-Pack Benchmark (Level-3), the most challenging level due to deformable objects, dimensional constraints, and strict grasp-point requirements. Experimental results demonstrate an average success rate of 92% across five item classes, with the deformable sweet bag the most challenging at 60% and an average execution time of 7.5 s on an edge device. The system achieves strong computational efficiency, reflected by a compute-to-speed ratio (CSR) of 0.008, with a total model size of only 30.9 MB. Performance is further validated across multiple hardware platforms and under real competition scenarios in the European Robotics League 2025. The findings highlight the practical impact of lightweight, vision-based mobile manipulation and provide insights into current challenges and future research directions for autonomous robotic applications. Full article

(This article belongs to the Special Issue Advanced Sensors and AI Integration for Human–Robot Teaming)

► Show Figures

Figure 1

15 pages, 2219 KB

Open AccessArticle

One Patch Is All You Need: Joint Surface Material Reconstruction and Classification from Minimal Visual Cues

by Sindhuja Penchala, Gavin Money, Gabriel Marques, Samuel Wood, Jessica Kirschman, Travis Atkison, Shahram Rahimi and Noorbakhsh Amiri Golilarz

Sensors 2026, 26(7), 2083; https://doi.org/10.3390/s26072083 - 27 Mar 2026

Viewed by 458

Abstract

Understanding material surfaces from sparse visual cues is critical for applications in robotics, simulation and material perception. However, most existing methods rely on dense or full scene observations, limiting their effectiveness in constrained or partial view environments. This gap highlights the need for models capable of inferring surfaces’ properties from extremely limited visual information. To address this challenge, we introduce SMARC, a unified model for Surface MAterial Reconstruction and Classification from minimal visual input. By giving only a single 10% contiguous patch of the image, SMARC recognizes and reconstructs the full RGB surface while simultaneously classifying the material category. Our architecture combines a Partial Convolutional U-Net with a classification head, enabling both spatial inpainting and semantic understanding under extreme observation sparsity. We compared SMARC against five models including convolutional autoencoders, Vision Transformer (ViT), Masked Autoencoder (MAE), Swin Transformer and DETR using the Touch and Go dataset of real-world surface textures. SMARC achieves the highest performance among the evaluated methods with a PSNR of 17.55 dB and a surface classification accuracy of 85.10%. These results validate the effectiveness of SMARC in relation to surface material understanding and highlight its potential for deployment in robotic perception tasks where visual access is inherently limited. Full article

(This article belongs to the Special Issue Advanced Sensors and AI Integration for Human–Robot Teaming)

► Show Figures

Figure 1

17 pages, 561 KB

Open AccessArticle

Multimodal Shared Autonomy for Heavy-Load UAV Operations with Physics-Aware Cooperative Control

by Xu Gao, Jingfeng Wu, Yuchen Wang, Can Cao, Lihui Wang, Bowen Wang and Yimeng Zhang

Sensors 2026, 26(6), 1997; https://doi.org/10.3390/s26061997 - 23 Mar 2026

Viewed by 519

Abstract

Heavy-load unmanned aerial vehicles (UAVs) are increasingly being applied in logistics, infrastructure installation, and emergency response missions, where complex payload dynamics and unstructured environments pose significant challenges to safe and efficient operation. Conventional manual teleoperation interfaces, such as dual-joystick control, impose a high cognitive workload and provide limited support for expressing high-level operator intent, while fully autonomous solutions remain difficult to deploy reliably under real-world uncertainty. To address these limitations, this paper proposes the Multimodal Fusion Cooperation Network (MFCN), an end-to-end shared autonomy framework that integrates speech commands, visual gestures, and haptic cues through cross-modal feature fusion to infer operator intent in real time. The fused intent representation is translated into dynamically feasible control commands by a cooperative control policy with embedded physics-aware constraints to suppress payload oscillations and ensure flight stability. Extensive semi-physical simulations and real-world experiments demonstrate that the MFCN significantly improves the task success rate, positioning accuracy, and payload stability while reducing the task completion time and operator cognitive workload compared with manual, unimodal, and heuristic multimodal baselines. Full article

(This article belongs to the Special Issue Advanced Sensors and AI Integration for Human–Robot Teaming)

► Show Figures

Figure 1

18 pages, 2996 KB

Open AccessArticle

A Multimodal Agentic AI Framework for Intuitive Human–Robot Collaboration

by Xiaoyun Liang and Jiannan Cai

Sensors 2026, 26(6), 1958; https://doi.org/10.3390/s26061958 - 20 Mar 2026

Viewed by 4093

Abstract

Widespread acceptance of collaborative robots in human-involved scenarios requires accessible and intuitive interfaces for lay workers and non-expert users. Existing interfaces often rely on users to plan and issue low-level commands, necessitating extensive knowledge of robot control. This study proposes a multimodal agentic AI framework integrating natural user interfaces (NUIs) to foster effortless human-like partnerships in human–robot collaboration (HRC), which enhance intuitiveness and operational efficiency. First, it allows users to instruct robots using plain language verbally, coupled with gaze, revealing objects precisely. Second, it offloads users’ workload for robot motion planning by understanding context and reasoning task decomposition. Third, coordinating with AI agents built on large language models (LLMs), the system interprets users’ requests effectively and provides feedback to establish transparent communication. This proof-of-concept study included experiments to demonstrate a practical implementation of the agentic AI framework on a mobile manipulation robot in the collaborative task of human–robot wood assembly. Seven participants were recruited to interact with this AI-integrated agentic robotic system. Task performance and user experience metrics were measured in terms of completion time, intervention rate, NASA TLX survey for workload, and valuable insights of practical applications were summarized through a qualitative analysis. This study highlights the potential of NUIs and agentic AI-embodied robots to overcome existing HRC barriers and contributes to improving HRC intuitiveness and efficiency. Full article

(This article belongs to the Special Issue Advanced Sensors and AI Integration for Human–Robot Teaming)

► Show Figures

Figure 1

16 pages, 3708 KB

Open AccessArticle

Myoelectric and Inertial Data Fusion Through a Novel Attention-Based Spatiotemporal Feature Extraction for Transhumeral Prosthetic Control: An Offline Analysis

by Andrea Tigrini, Alessandro Mengarelli, Ali H. Al-Timemy, Rami N. Khushaba, Rami Mobarak, Mara Scattolini, Gaith K. Sharba, Federica Verdini, Ennio Gambi and Laura Burattini

Sensors 2025, 25(18), 5920; https://doi.org/10.3390/s25185920 - 22 Sep 2025

Cited by 1 | Viewed by 950

Abstract

This study proposes a feature extraction scheme that fuses accelerometric (ACC) and electromyographic (EMG) data to improve shoulder movement identification in individuals with transhumeral amputation, in whom the clinical need for intuitive control strategies enabling reliable activation of full-arm prostheses is underinvestigated. A novel spatiotemporal warping feature extraction architecture was employed to realize EMG and ACC information fusion at the feature level. EMG and ACC data were collected from six participants with intact limbs and four participants with transhumeral amputation using an NI USB-6009 device at 1000 Hz to support the proposed feature extraction scheme. For each participant, a leave-one-trial-out (LOTO) training and testing approach was used for developing pattern recognition models for both the intact-limb (IL) and amputee (AMP) groups. The analysis revealed that the introduction of ACC information has a positive impact when using windows of length (WLs) lower than 150 ms. A linear discriminant analysis (LDA) classifier was able to exceed the accuracy of 90% in each WL condition and for each group. Similar results were observed for an extreme learning machine (ELM), whereas k-nearest neighbors (kNN) and an autonomous learning multi-model classifier showed a mean accuracy of less than 87% for both IL and AMP groups at different WLs, guaranteeing applicability over a large set of shallow pattern-recognition models that can be used in real scenarios. The present work lays the groundwork for future studies involving real-time validation of the proposed methodology on a larger population, acknowledging the current limitation of offline analysis. Full article

(This article belongs to the Special Issue Advanced Sensors and AI Integration for Human–Robot Teaming)

► Show Figures

Figure 1

16 pages, 5201 KB

Open AccessArticle

Robotic Fast Patch Clamp in Brain Slices Based on Stepwise Micropipette Navigation and Gigaseal Formation Control

by Jinyu Qiu, Qili Zhao, Ruimin Li, Yuzhu Liu, Biting Ma and Xin Zhao

Sensors 2025, 25(4), 1128; https://doi.org/10.3390/s25041128 - 13 Feb 2025

Cited by 4 | Viewed by 2048

Abstract

The patch clamp technique has become the gold standard for neuron electrophysiology research in brain science. Brain slices have been widely utilized as the targets of the patch clamp technique due to their higher optical transparency compared to a live brain and their intercellular connectivity in comparison to cultured single neurons. However, the narrow working space, small scope, and depth of the field of view make the positioning of the operation’s micropipette to the target neuron a time-consuming task reliant on a high level of experience, significantly slowing down operation of the patch clamp technique in brain slices. Further, the current poor controllability in gigaseal formation, which is the key to electrophysiology signal recording, significantly lowers the patch clamp success rate. In this paper, a stepwise navigation of the micropipette is conducted to accelerate the positioning process of the micropipette tip to the target neuron in the brain slice. Then, a fuzzy proportional–integral–derivative controller is designed to control the gigaseal formation process along a designed resistance curve. The experimental results demonstrate an almost doubled patch clamp technique speed, with a 25% improvement in the success rate compared to the conventional manual method. The above advantages may promote the application of our method in brain science research based on brain slice platforms. Full article

(This article belongs to the Special Issue Advanced Sensors and AI Integration for Human–Robot Teaming)

► Show Figures

Figure 1

23 pages, 4654 KB

Open AccessArticle

Effective Acoustic Model-Based Beamforming Training for Static and Dynamic Hri Applications

by Alejandro Luzanto, Nicolás Bohmer, Rodrigo Mahu, Eduardo Alvarado, Richard M. Stern and Néstor Becerra Yoma

Sensors 2024, 24(20), 6644; https://doi.org/10.3390/s24206644 - 15 Oct 2024

Cited by 1 | Viewed by 4280

Abstract

Human–robot collaboration will play an important role in the fourth industrial revolution in applications related to hostile environments, mining, industry, forestry, education, natural disaster and defense. Effective collaboration requires robots to understand human intentions and tasks, which involves advanced user profiling. Voice-based communication, rich in complex information, is key to this. Beamforming, a technology that enhances speech signals, can help robots extract semantic, emotional, or health-related information from speech. This paper describes the implementation of a system that provides substantially improved signal-to-noise ratio (SNR) and speech recognition accuracy to a moving robotic platform for use in human–robot interaction (HRI) applications in static and dynamic contexts. This study focuses on training deep learning-based beamformers using acoustic model-based multi-style training with measured room impulse responses (RIRs). The results show that this approach outperforms training with simulated RIRs or matched measured RIRs, especially in dynamic conditions involving robot motion. The findings suggest that training with a broad range of measured RIRs is sufficient for effective HRI in various environments, making additional data recording or augmentation unnecessary. This research demonstrates that deep learning-based beamforming can significantly improve HRI performance, particularly in challenging acoustic environments, surpassing traditional beamforming methods. Full article

(This article belongs to the Special Issue Advanced Sensors and AI Integration for Human–Robot Teaming)

► Show Figures

Figure 1

Review

Jump to: Research

32 pages, 2874 KB

Open AccessReview

Survey on Reconnaissance Autonomous Robotic Systems for Disaster Management

by Sahaj Sinha, Sinjae Lee and Saurabh Singh

Sensors 2026, 26(5), 1659; https://doi.org/10.3390/s26051659 - 5 Mar 2026

Viewed by 796

Abstract

Systems that operate in dangerous environments are becoming essential in case of emergencies. This survey reviews the latest ground reconnaissance robots using computer vision (CV), machine learning (ML), MCU-based control, LoRa communication, DC motors, and dual-power systems. The analysis includes hardware and algorithms, and their performance in the field and lab. There has been clear progress in navigation, sensor fusion, and situational awareness. The main challenges which remain include the use of energy and standardization of benchmarks. This survey focuses exclusively on Unmanned Ground Vehicles (UGVs) for disaster reconnaissance, examining recent advances in hardware, software, and autonomy. The survey highlights the improvements in navigation, sensor fusion, and intelligence, and identifies remaining challenges such as energy limitations, robustness in harsh conditions, and the lack of standardized benchmarks. The analysis synthesizes findings from over 190 recent studies (2020–2025) in ground-based disaster robotics, providing a comprehensive overview of current capabilities and research gaps. It encapsulates all issues with their remedy for future disaster-response systems. Full article

(This article belongs to the Special Issue Advanced Sensors and AI Integration for Human–Robot Teaming)

► Show Figures

Figure 1

51 pages, 4751 KB

Open AccessReview

Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomy

by Vinit Mehta, Charu Sharma and Karthick Thiyagarajan

Sensors 2025, 25(20), 6394; https://doi.org/10.3390/s25206394 - 16 Oct 2025

Cited by 6 | Viewed by 6229

Abstract

With the rapid advancement of artificial intelligence and robotics, the integration of Large Language Models (LLMs) with 3D vision is emerging as a transformative approach to enhancing robotic sensing technologies. This convergence enables machines to perceive, reason, and interact with complex environments through natural language and spatial understanding, bridging the gap between linguistic intelligence and spatial perception. This review provides a comprehensive analysis of state-of-the-art methodologies, applications, and challenges at the intersection of LLMs and 3D vision, with a focus on next-generation robotic sensing technologies. We first introduce the foundational principles of LLMs and 3D data representations, followed by an in-depth examination of 3D sensing technologies critical for robotics. The review then explores key advancements in scene understanding, text-to-3D generation, object grounding, and embodied agents, highlighting cutting-edge techniques such as zero-shot 3D segmentation, dynamic scene synthesis, and language-guided manipulation. Furthermore, we discuss multimodal LLMs that integrate 3D data with touch, auditory, and thermal inputs, enhancing environmental comprehension and robotic decision-making. To support future research, we catalog benchmark datasets and evaluation metrics tailored for 3D-language and vision tasks. Finally, we identify key challenges and future research directions, including adaptive model architectures, enhanced cross-modal alignment, and real-time processing capabilities, which pave the way for more intelligent, context-aware, and autonomous robotic sensing systems. Full article

(This article belongs to the Special Issue Advanced Sensors and AI Integration for Human–Robot Teaming)

► Show Figures

Journal Menu

Journal Browser

Advanced Sensors and AI Integration for Human–Robot Teaming

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Published Papers (9 papers)

Research

Review

Further Information

Guidelines

MDPI Initiatives

Follow MDPI