Multimodal Scene Perception
This special issue belongs to the section "AI-Driven Innovations".
Special Issue Information
Dear Colleagues,
With the rapid development of the Internet of Things (IoT), autonomous driving, and embodied intelligence, scene perception is evolving from mere “signal detection” to in-depth “semantic understanding.” Single-modality perception can no longer meet the demands of complex and dynamic environments. The fusion of multimodal data (RGB-T, LiDAR, point clouds, panoramic images, BeiDou data, etc.) provides a solid foundation for high-precision perception.
Meanwhile, the rise in Large Language Models (LLMs) and Large Multimodal Models (LMMs) has brought revolutionary breakthroughs to scene perception. In particular, Multimodal Foundation Models and Generative AI have emerged as pivotal paradigms, leveraging powerful common-sense reasoning, zero-shot generalization, and semantic alignment capabilities to address bottlenecks in traditional perception models, such as long-tail scenarios, open-vocabulary detection, and complex interactive understanding. Furthermore, the advent of AI Agents has transformed passive perception into active cognition, enabling systems to autonomously plan, reason, and execute tasks in real-world environments. Together, these advancements have become a focal point in both academia and industry, driving the transition from static scene understanding to dynamic, goal-oriented intelligence.
This Special Issue aims to explore the deep integration of multimodal hardware perception, large model cognitive capabilities, and multimodal data analysis. We welcome high-quality submissions covering sensor data processing, 3D vision, large model empowerment, and data mining.
Topics of Interest
This Special Issue covers, but is not limited to, the following topics. Cross-modality and cross-disciplinary integrated research is particularly encouraged.
- Large Language Model (LLM) Empowered Scene Perception
- Language-driven Visual Perception: Open-vocabulary object detection and segmentation based on natural language prompts.
- Scene Semantic Understanding and Reasoning: Utilizing LLMs for logical reasoning, causal analysis, and anomaly explanation in complex scenes.
- Multimodal Fusion Large Models: Pre-training and fine-tuning techniques for Vision–Language–LiDAR models.
- Embodied Interaction for Perception Models: Instruction following and task planning of large models in robot navigation and human–computer interaction.
- Few-shot and Zero-shot Perception: Utilizing large model prior knowledge to address data scarcity in extreme environments (e.g., extremely low light, occlusion).
- Multimodal Data Analysis and Mining
- Cross-modal Data Association and Alignment: Semantic alignment and joint representation learning of heterogeneous data (e.g., images, text, audio, sensor signals).
- Multimodal Anomaly Detection: Complex system fault diagnosis and abnormal behavior recognition based on multi-source data fusion.
- Time-series Multimodal Data Analysis: Spatiotemporal data modeling, prediction, and trend analysis in dynamic scenes.
- Multimodal Knowledge Graphs: Knowledge extraction, reasoning, and graph construction integrating vision, text, and structured data.
- Data Privacy and Security: Federated learning, differential privacy, and adversarial defense mechanisms in multimodal data analysis.
- Multimodal Object Detection and Recognition
- All-weather object detection based on RGB-T (Visible–Infrared) fusion.
- Cross-modal feature alignment, denoising, and fusion network design.
- Small object and occluded object detection in complex backgrounds.
- 3D Reconstruction and Point Cloud Data Processing
- Fine-grained 3D reconstruction based on LiDAR and vision fusion.
- Point cloud semantic segmentation and instance segmentation in large-scale outdoor scenes.
- Point cloud registration, denoising, and completion in dynamic scenes.
- Applications of Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) in scene reconstruction.
- Panoramic Perception and Image Stitching
- High-resolution panoramic image generation via multi-sensor joint calibration.
- Registration and stitching of heterogeneous images (e.g., visible light and infrared, satellite and UAV imagery).
- Visual SLAM and localization technologies based on panoramic images.
- BeiDou Navigation and Multi-source Spatiotemporal Fusion
- Deeply coupled navigation and positioning using BeiDou/GNSS integrated with vision/radar.
- Localization technology based on scene recognition in GNSS-denied environments.
- Synchronization, fusion, and mining of multi-source spatiotemporal data (spatiotemporal big data).
- Multimodal Foundation Models and Generative AI
- Novel architectures for foundation models,
- Generative approaches for multimodal data synthesis,
- Simulation and reconstruction, as well as content generation techniques that enhance scene understanding.
- AI Agents
- The development of autonomous agents capable of perceiving, reasoning, and acting in complex environments;
- Multi-agent collaboration and the integration of large models into embodied AI systems for decision-making and task planning.
Dr. Mingwei Cao
Guest Editor
Manuscript Submission Information
Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the Special Issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 250 words) can be sent to the Editorial Office for assessment.
Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-anonymized peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Computers is an international peer-reviewed open access monthly journal published by MDPI.
Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.
Keywords
- scene perception
- multimodal data analysis
- object detection and recognition
- panoramic perception
- BeiDou navigation
- AI
- agent
Benefits of Publishing in a Special Issue
- Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
- Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
- Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
- External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
- Reprint: MDPI Books provides the opportunity to republish successful Special Issues in book format, both online and in print.

