Robot Perception in the Era of Foundation Models: Progress, Challenges and Future Trends

Dear Colleagues,

Robot perception plays a fundamental role in enabling intelligent robotic systems to understand and interact with complex environments. Traditionally, perception in robotics has relied on task-specific models and carefully designed pipelines for processing sensory data such as images, depth measurements, tactile signals, and other physical or biological signals. While these approaches have achieved notable progress, they often struggle to generalize across diverse scenarios, sensing modalities, and operational conditions.

The recent emergence of foundation models, including large-scale vision, language, and multimodal models trained on massive datasets, is transforming the landscape of robot perception. These models demonstrate remarkable capabilities in representation learning, cross-modal reasoning, and knowledge transfer, offering new opportunities for robots to perceive and interpret their surroundings with unprecedented robustness and flexibility. By leveraging foundation models, robot perception systems can potentially move beyond narrowly specialized solutions toward more unified and scalable frameworks capable of integrating information from heterogeneous sensory sources.

At the same time, the integration of foundation models into robotic perception introduces significant challenges. These include efficient adaptation of large models to resource-constrained robotic platforms, reliable multimodal sensing and fusion, domain adaptation between simulated and real-world environments, real-time perception requirements, and ensuring safety and robustness in dynamic and uncertain environments. Addressing these issues requires advances not only in algorithms and learning paradigms, but also in sensing technologies, system integration, and application-driven evaluation.

This Special Issue aims to provide a platform for researchers and practitioners to present recent progress, emerging challenges, and future trends in robot perception in the era of foundation models. We welcome contributions that explore novel methodologies, system frameworks, theoretical insights, and practical applications that advance the capabilities of intelligent perception systems for robotics and related fields. Topics of interest include, but are not limited to, vision-based perception, multimodal sensing and fusion, large-model-based perception frameworks, learning from limited supervision, perception for human-robot interaction, bio-signal-based perception, and real-world deployment of perception systems in domains such as healthcare, manufacturing, autonomous systems, and intelligent environments.

Through this Special Issue, we aim to highlight innovative ideas and interdisciplinary approaches that will shape the next generation of robot perception technologies and contribute to the development of more capable, adaptive, and trustworthy robotic systems.

Dr. Shuwei Shao
Dr. Chenglong Zhang
Dr. Cheng Shen
Dr. Zilin Liang
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 250 words) can be sent to the Editorial Office for assessment.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Journal of Imaging is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

• robot perception
• foundation models
• multimodal perception
• vision–language models
• large-scale pretrained models
• multisensor data fusion
• bio-signal analysis (EEG, EMG)
• intelligent sensing systems
• embodied intelligence
• human–robot interaction
• autonomous systems
• AI for robotics