Sign in to use this feature.

Years

Between: -

Subjects

remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline

Journals

remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline

Article Types

Countries / Regions

remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline

Search Results (847)

Search Parameters:
Keywords = vision-language model

Order results
Result details
Results per page
Select all
Export citation of selected articles as:
27 pages, 1808 KB  
Article
Role of Generative Artificial Intelligence in Transforming Construction Safety Training
by Thamali Sarathchandra, Giphy George, Udara Ranasinghe, Madduma Kaluge Chamitha Sanjani Wijewickrama and David J. Edwards
Buildings 2026, 16(13), 2686; https://doi.org/10.3390/buildings16132686 (registering DOI) - 7 Jul 2026
Abstract
The construction industry continues to face high levels of accidents despite the use of various safety training approaches, highlighting the need for more effective and responsive methods. This study examines the role of Generative Artificial Intelligence (GenAI) in potentially improving construction safety training [...] Read more.
The construction industry continues to face high levels of accidents despite the use of various safety training approaches, highlighting the need for more effective and responsive methods. This study examines the role of Generative Artificial Intelligence (GenAI) in potentially improving construction safety training by exploring the development of training practices and identifying the shortcomings of existing approaches. A systematic literature review (SLR) was conducted to analyse safety training methods and emerging GenAI applications, followed by validation interviews with industry experts in South Australia to ensure practical relevance. Emergent findings show that safety training has progressed through three main stages: instructor-led, digital and GenAI-enabled. However, instructor-led and digital approaches remain limited by non-interactive learning, limited flexibility to different learner needs, lack of real-time feedback and weak alignment with actual site conditions. In contrast, GenAI offers opportunities to support more interactive, personalised and context-aware training through technologies such as large language models (LLMs), adaptive learning systems, computer vision and scenario generation. Despite these benefits, significant challenges related to data quality, system reliability, ethical concerns and organisational readiness continue to affect implementation. Based on these findings, the study develops an integrated framework that links training evolution, key challenges and GenAI capabilities, providing practical guidance to improve safety training in construction. Full article
Show Figures

Figure 1

23 pages, 2149 KB  
Article
Hierarchical Vision–Language Fusion with Structural Constraint Reasoning for Robust Multi-Jurisdiction License Plate Recognition
by Safa Issaoui, Sarah A. Alzakari, Issra Saidi, Ridha Ejbali and Amina Serir
Appl. Sci. 2026, 16(13), 6792; https://doi.org/10.3390/app16136792 - 6 Jul 2026
Abstract
Automatic License Plate Recognition (ALPR) in unconstrained traffic environments requires simultaneously addressing two fundamental challenges: reliable localization of small and degraded license plates and accurate decoding of visually ambiguous character sequences. This paper presents a hierarchical multi-stage framework that combines deep-learning-based detection, geometric [...] Read more.
Automatic License Plate Recognition (ALPR) in unconstrained traffic environments requires simultaneously addressing two fundamental challenges: reliable localization of small and degraded license plates and accurate decoding of visually ambiguous character sequences. This paper presents a hierarchical multi-stage framework that combines deep-learning-based detection, geometric normalization, dual-channel recognition, and structured post-correction to improve recognition robustness under diverse real-world conditions. A systematic ablation study involving five configurations (A0–A4) demonstrates the effectiveness of the proposed architecture across three benchmark datasets. On the UC3M-LP dataset, exact-match accuracy increases from 45.2% to 88.3%, while achieving 91.6% partial accuracy and a zero detection-miss rate. The framework further attains 95% exact-match accuracy on controlled European license plate crops and 93% on a large-scale custom dataset. In addition, we identify systematic evaluation artifacts in partially annotated benchmarks, showing that truncated ground-truth labels can underestimate genuine character-level improvements. The proposed framework supports multiple license plate formats through a configurable structural template library, and preliminary experiments on a small Arabic-script subset suggest potential extensibility without full model retraining. To ensure full reproducibility, all source code and evaluation resources are publicly released. Full article
Show Figures

Figure 1

52 pages, 3537 KB  
Article
An Interpretable Vision-Language Framework for Evaluating the Uncanny Valley Effect of XR Humanoid Characters
by Xiner Li, Yi Xiao, Jinhao Qiao, Yan Zheng and Chi-Sing Leung
Electronics 2026, 15(13), 2959; https://doi.org/10.3390/electronics15132959 - 6 Jul 2026
Abstract
As AI-generated humanoid characters are increasingly used in virtual, augmented, and mixed reality applications, evaluating the Uncanny Valley Effect (UVE) is crucial for immersive user experience. Existing evaluation methods map visual features to affective scores, offering limited interpretability regarding which visual cues are [...] Read more.
As AI-generated humanoid characters are increasingly used in virtual, augmented, and mixed reality applications, evaluating the Uncanny Valley Effect (UVE) is crucial for immersive user experience. Existing evaluation methods map visual features to affective scores, offering limited interpretability regarding which visual cues are associated with affinity judgments. Among the theoretical perspectives proposed to explain the UVE, perceptual conflict provides a visual-cue-oriented perspective for analyzing whether local-feature realism supports a coherent overall human-likeness impression and how this is reflected in affinity judgments, yet this perspective is rarely incorporated into interpretable UVE assessment. Thus, we propose UVE-Perception Chain-of-Thought (UVE-PCoT), a vision-language framework for interpretable UVE evaluation from a perceptual-conflict-oriented perspective. UVE-PCoT organizes assessment through a structured perceptual decomposition, including assessments of overall human-likeness, local-feature realism, perceptual conflict, and affinity. To provide supervision, we construct UVE-R, a structured rationale dataset with image-grounded, rating-consistent rationales linking visual cue observations, cue-level inconsistency analysis, and affinity judgments. Results show that UVE-PCoT improves affinity prediction and cue-level explanation over general-purpose multimodal large language models and ablations. Our approach operationalizes this perceptual-conflict-oriented perspective into an interpretable framework, advancing UVE evaluation from black-box scoring to explanatory analysis and providing cue-level insights for XR character assessment and revision. Full article
34 pages, 22783 KB  
Article
An Explainable Multimodal Framework for Cyclist Safety Perception in Mixed Traffic Environments
by Chia-Yen Chiang, Meihui Wang, Yasmin Fathy, Mona Jaber and Ahmed M. Abdelmoniem
Appl. Sci. 2026, 16(13), 6690; https://doi.org/10.3390/app16136690 - 3 Jul 2026
Viewed by 196
Abstract
Despite growing policy support for active travel, the fatality rate of vulnerable road users has remained persistently high in recent years, while the emergence of autonomous vehicles has further increased the complexity of mixed traffic environments. Interactions between cyclists and motorized vehicles are [...] Read more.
Despite growing policy support for active travel, the fatality rate of vulnerable road users has remained persistently high in recent years, while the emergence of autonomous vehicles has further increased the complexity of mixed traffic environments. Interactions between cyclists and motorized vehicles are a major contributor to these fatalities, highlighting the urgent need for effective cyclist protection strategies. As one of the most widely adopted active transport modes, cycling safety cannot be assessed solely through crash statistics; understanding cyclists’ perceived safety is equally critical, as it reflects how infrastructure design and dynamic traffic conditions influence cycling behavior. In this study, we propose a cyclist safety perception framework that combines vision–language models with interpretable machine learning to analyze perceived safety in mixed traffic scenarios. A vision–language model is employed to generate semantic descriptions of traffic scenes, while an Explainable Boosting Machine quantifies both individual and interactive contributions of traffic-related features. By integrating visual information with road attributes extracted from OpenStreetMap, the proposed framework achieves a binary safety classification accuracy of 71% and a mean absolute error of 1.01 on a safety score scale ranging from 1 to 9. The results demonstrate the potential of combining multimodal perception and explainable models to support cyclist-centered safety assessment and inform sustainable and intelligent transportation system design. More specifically, the results show that protected cycling infrastructure is the most significant factor in improving perceived safety, whereas road construction has the opposite effect. Full article
(This article belongs to the Special Issue Advances in Intelligent Transportation and Sustainable Mobility)
Show Figures

Figure 1

22 pages, 7333 KB  
Article
Bloom or Bluff? Benchmarking Vision–Language Models Against Classical Machine Learning for Harmful Algal Bloom Detection from Satellite Imagery
by Harsh Deep Singh Narula
Remote Sens. 2026, 18(13), 2147; https://doi.org/10.3390/rs18132147 - 2 Jul 2026
Viewed by 133
Abstract
In recent years, there has been growing interest in applying vision–language models (VLMs) to quantitative remote sensing. This study evaluates whether three commercial VLMs (GPT-4o, GPT-5.5, and Claude Sonnet 4.6) can detect and classify the severity of harmful algal blooms (HABs) from Sentinel-2 [...] Read more.
In recent years, there has been growing interest in applying vision–language models (VLMs) to quantitative remote sensing. This study evaluates whether three commercial VLMs (GPT-4o, GPT-5.5, and Claude Sonnet 4.6) can detect and classify the severity of harmful algal blooms (HABs) from Sentinel-2 satellite imagery of western Lake Erie and compares them against classical machine learning classifiers (Random Forest (RF), Support Vector Machine (SVM), and eXtreme Gradient Boosting (XGBoost)) trained on both a three-band red, green, blue (RGB) composite representation of the imagery and a 10-band multi-spectral reflectance representation. Forty bloom events identified from the National Oceanic and Atmospheric Administration (NOAA) Harmful Algal Bloom Operational Forecast System (HAB-OFS) severity assessments were assembled into the evaluation dataset, spanning seven bloom seasons (2019–2025). For binary bloom detection, the VLMs did not match the classical RGB classifiers; their F1 scores (0.69–0.75) fell below the best RGB classifier (Random Forest, 0.76) and below a trivial always-present baseline (F1 = 0.77), and they carried false positive rates of 73–93% on bloom-absent images, against 27–40% for the RGB classifiers. The VLMs reached high recall by labeling most scenes as bloom-positive, which makes them operationally unreliable in this configuration. For severity classification, the VLMs assigned 60–70% of their predictions to the “moderate” category regardless of actual conditions and identified at most one of the two severe blooms, whereas the classical classifiers tracked the ground-truth distribution and delivered two to nearly three times the exact-match accuracy (0.44–0.59 vs. 0.20–0.225). The strongest method across all metrics was the multi-spectral SVM (F1 = 0.833, false positive rate 27%, accuracy 0.795). Switching the same SVM from RGB to multi-spectral features raised accuracy from 0.675 to 0.795, a 12-percentage-point gain that measures the spectral information carried by red-edge and shortwave infrared bands that are accessible through multi-spectral sensors but unavailable to standard VLM vision encoders. Feature-importance analysis showed that the multi-spectral classifiers ranked chlorophyll-specific indices, the Normalized Difference Chlorophyll Index (NDCI) and the Floating Algae Index (FAI), among their top predictors, the same signatures used in established operational algorithms, while the RGB classifiers relied on red-channel variability and green-dominant pixel fractions because RGB inputs cannot compute those indices. Two compounded limitations therefore constrain off-the-shelf VLMs for aquatic remote sensing: the limited spectral information available through standard RGB channels and a mismatch between the land-dominated training distributions of these models and aquatic optical conditions. Domain-specific classifiers operating on multi-spectral data remain the more suitable tools for continued development of HAB monitoring and water-quality retrieval. Full article
Show Figures

Figure 1

26 pages, 2389 KB  
Article
LLMs in Automated Assessment: A Role-Based Taxonomy and Framework for Controlled Educational Integration
by Anastasia Vangelova and Veska Gancheva
Appl. Sci. 2026, 16(13), 6617; https://doi.org/10.3390/app16136617 - 2 Jul 2026
Viewed by 156
Abstract
Large language models (LLMs) are reshaping the automated assessment of open-ended student responses. Compared with earlier rule-based, statistical, and feature-engineered approaches, they enable a deeper interpretation of meaning, context, and argumentation. This development can be understood as a fifth generation of automated scoring [...] Read more.
Large language models (LLMs) are reshaping the automated assessment of open-ended student responses. Compared with earlier rule-based, statistical, and feature-engineered approaches, they enable a deeper interpretation of meaning, context, and argumentation. This development can be understood as a fifth generation of automated scoring systems, but it also raises a new question: not only what LLMs can do, but also how they can be deployed in education in a controlled and reliable manner. This paper presents a role-based taxonomy that distinguishes between generative LLMs used as direct virtual graders, encoder transformers used as semantic tools, and intermediate text-to-text models used in more formalized assessment tasks. It also discusses the main limitations of standalone LLM graders, including hallucinations, probabilistic instability, limited interpretability, bias, and weak grounding in domain-specific content. To address these issues, the paper presents a developed framework implemented in an integrated assessment system built on role prompting, rubric-constrained grading, Retrieval-Augmented Generation (RAG), structured machine-readable outputs, workflow orchestration, and LMS integration. The framework is further extended to multimodal assessment through vision-based evaluation of visual artifacts such as UML state diagrams. The main contribution of the paper is not only a conceptual framework, but also its realization in a working integrated system for automated assessment in a more traceable, pedagogically grounded, and institutionally reliable way. Full article
(This article belongs to the Special Issue Application of Semantic Web Technologies for E-Learning)
Show Figures

Figure 1

34 pages, 19823 KB  
Article
An Agentic AI System for Roof Design Compliance Using Computer Vision, Retrieval-Augmented Generation and Large Language Models
by Nawari O. Nawari and Oluwatoyin O. Lawal
Buildings 2026, 16(13), 2637; https://doi.org/10.3390/buildings16132637 - 2 Jul 2026
Viewed by 179
Abstract
Designers, engineers, and building officials face increasing pressure to accelerate and improve the accuracy of design review for buildings and infrastructure. Roof assemblies and rooftop structures are particularly challenging due to the complexity and fragmentation of regulatory requirements, especially in jurisdictions such as [...] Read more.
Designers, engineers, and building officials face increasing pressure to accelerate and improve the accuracy of design review for buildings and infrastructure. Roof assemblies and rooftop structures are particularly challenging due to the complexity and fragmentation of regulatory requirements, especially in jurisdictions such as Florida, where compliance must be verified across both the residential and commercial volumes of the Florida Building Code (FBC). The resulting review process is technically demanding and time-intensive, imposing significant cognitive and operational burdens on practitioners and under-resourced public agencies. To address these challenges, this study proposes and evaluates an agentic artificial intelligence (AI) framework for automated code compliance checking of roof assemblies and rooftop structures. The framework employs a multi-agent architecture in which specialized AI agents collaboratively interpret regulatory provisions and evaluate roof design parameters across four core modules: data preprocessing and code ingestion, rule-based and semantic analysis, results visualization, and iterative validation. YOLO11m-seg and Mask R-CNN were used for element detection and segmentation, and the system was developed using 150 design projects, including roof plans, section details, and specifications. Four large language models from two families (Mistral and GPT) were comparatively evaluated on standardized compliance tasks. The framework was then tested on a held-out portfolio of 15 distinct roof-design projects comprising 60 code-compliance decisions derived from the FBC 2023, with performance measured by precision, recall, F1-score, and accuracy. GPT-5.4 achieved the highest overall performance (F1 = 0.97; accuracy = 97%). Because the reasoning and vision components were evaluated separately rather than as an integrated end-to-end pipeline, and the scope was limited to one jurisdiction and two drawing types, broader code coverage and production-setting validation are needed before claims of generality. Nonetheless, the results suggest that agentic AI can meaningfully support compliance review and reduce reviewer burden in roof-design permitting. Full article
(This article belongs to the Section Construction Management, and Computers & Digitization)
Show Figures

Figure 1

18 pages, 2842 KB  
Article
CollectivIA: Two-Pipeline Multilingual Legal RAG for Moroccan Territorial Governance with LLM-Assisted and Regex-Based Chunking
by Firiel Zouak, Omar El Beqqali and Jamal Riffi
Big Data Cogn. Comput. 2026, 10(7), 212; https://doi.org/10.3390/bdcc10070212 - 29 Jun 2026
Viewed by 207
Abstract
Retrieval grounding is crucial for high-stakes administrative applications, since large language models remain prone to hallucinations when addressing legal questions. This problem is particularly relevant in Moroccan territorial governance, where official legislative PDFs have highly heterogeneous digital quality, user interactions often occur in [...] Read more.
Retrieval grounding is crucial for high-stakes administrative applications, since large language models remain prone to hallucinations when addressing legal questions. This problem is particularly relevant in Moroccan territorial governance, where official legislative PDFs have highly heterogeneous digital quality, user interactions often occur in Moroccan Darija, and the legal corpus is bilingual Arabic–French. This paper presents CollectivIA, a multilingual Retrieval-Augmented Generation system implemented for Moroccan territorial governance law. The system supports queries in French, Arabic, and Moroccan Darija and indexes 2272 article-level segments from sixteen official legislative documents. We compare two end-to-end retrieval pipelines: an LLM-assisted semantic chunking architecture using Gemini and ChromaDB and a regex-based chunking architecture using FAISS. Based on an expanded multilingual benchmark of 150 legal queries, with 50 queries per language group, the LLM-assisted pipeline achieved higher RAGAS scores than the regex-based pipeline, particularly improving Context Precision from 0.315 to 0.818. The multimodal Vision fallback successfully recovered 456 articles, which remained inaccessible under the regex-based pipeline. Overall, the LLM-assisted pipeline yielded legal boundaries with greater coherence and retrieved contexts with higher focus, while the regex-based design maintained a broader source diversity. These results suggest that LLM-assisted semantic chunking with multimodal fallback is a promising approach to enhance multilingual legal RAG over heterogeneous Moroccan legal corpora. Full article
(This article belongs to the Special Issue Artificial Intelligence (AI) and Natural Language Processing (NLP))
Show Figures

Figure 1

19 pages, 1460 KB  
Article
RAG-Enhanced Vision–Language Framework and Dataset for Railway Signal Cognition and Safety Reasoning
by Qunbo Wang, Shiyi Xiong, Jiawei Li, Weiliang Li, Chu Huang, Sen Zhang, Xize Guo, Chao Fan and Wenjun Wu
Computers 2026, 15(7), 416; https://doi.org/10.3390/computers15070416 - 29 Jun 2026
Viewed by 209
Abstract
Railway scene understanding is critical for ensuring train operational safety and advancing intelligent railway systems. Existing railway vision methods mainly focus on perception and classification, while lacking regulation-guided semantic reasoning capabilities in complex environments. To address these limitations, this paper proposes a retrieval-augmented [...] Read more.
Railway scene understanding is critical for ensuring train operational safety and advancing intelligent railway systems. Existing railway vision methods mainly focus on perception and classification, while lacking regulation-guided semantic reasoning capabilities in complex environments. To address these limitations, this paper proposes a retrieval-augmented generation (RAG)-enhanced vision–language framework for railway signal cognition and safety reasoning. The proposed method integrates railway signal perception, regulatory knowledge retrieval, and multi-modal reasoning to improve factual consistency, reasoning reliability, and operational interpretability. In addition, a dedicated railway signal dataset comprising 500 standardized railway scene images with structured QA annotations is constructed to support regulation-oriented multi-modal recognition evaluation. Experimental results show that the proposed framework improves reasoning accuracy from 28.40% to 67.20% with an average end-to-end inference latency of 11.31 s per sample, and the inference speed can be further improved by adjusting experimental configurations to trade off between efficiency and accuracy, demonstrating the potential of RAG-enhanced architectures as a foundational step toward reliable multi-modal cognition in intelligent railway systems. Full article
Show Figures

Figure 1

35 pages, 1355 KB  
Article
Robustness of Large Vision Language Model Features Under Wireless Channel Degradation for Medical Visual Question Answering
by Merve Güllü and Necaattin Barışçı
Appl. Sci. 2026, 16(13), 6425; https://doi.org/10.3390/app16136425 - 27 Jun 2026
Viewed by 140
Abstract
Deploying medical visual question answering (VQA) systems over wireless networks introduces a fundamental challenge: channel-induced image degradation may corrupt the visual representations extracted by large vision-language models (VLMs), leading to unreliable diagnostic decisions. We investigate the robustness of frozen LLaVA-1.6, BLIP-2, and BioViL-T [...] Read more.
Deploying medical visual question answering (VQA) systems over wireless networks introduces a fundamental challenge: channel-induced image degradation may corrupt the visual representations extracted by large vision-language models (VLMs), leading to unreliable diagnostic decisions. We investigate the robustness of frozen LLaVA-1.6, BLIP-2, and BioViL-T hidden-state features under additive white Gaussian noise (AWGN), Rayleigh fading, and six combined JPEG-compression-plus-channel conditions (quality factors q{20,50,70}) across signal-to-noise ratios (SNRs) from 5 to +20 dB. A lightweight MLP classifier is trained exclusively on clean features and evaluated on channel-degraded features, enabling controlled analysis of representation robustness without retraining. We introduce the Feature Robustness Score (FRS), defined as the difference between cosine similarity and normalized L2 drift of clean versus degraded features, together with a validation-set FRS threshold analysis as a label-free retraining criterion. A wavelet sub-band energy analysis further characterizes the spectral distribution of channel-induced feature drift. Experiments on PathVQA and VQA-RAD reveal four key findings: (1) LLaVA-1.6 features maintain cosine similarity above 0.98 across all eight channel conditions and all SNR levels, with statistically significant MLP gains at every tested point (p<0.05, McNemar’s test); (2) BLIP-2 and BioViL-T features are less stable but still support consistent MLP improvements, with BioViL-T performing competitively on VQA-RAD, suggesting domain alignment matters; (3) JPEG compression quality (q=20,50,70) has negligible impact on feature drift, establishing VLM features as JPEG quality-invariant; and (4) wavelet analysis confirms that channel noise primarily affects high-frequency detail bands while preserving low-frequency semantic content. Full article
(This article belongs to the Special Issue Deep Learning and Its Applications in Natural Language Processing)
Show Figures

Figure 1

25 pages, 2850 KB  
Article
Collaborative Vision-and-Language Navigation for UAVs in Low-Altitude Urban Space Leveraging Embodied Multi-Agent Systems
by Dongyang Wang, Jiankun Shi, Yantao Lu, Jinchao Chen and Chenglie Du
Drones 2026, 10(7), 491; https://doi.org/10.3390/drones10070491 (registering DOI) - 27 Jun 2026
Viewed by 152
Abstract
Large vision–language models have advanced embodied navigation by integrating visual perception with natural-language reasoning. However, vision-and-language navigation (VLN) for unmanned aerial vehicles in low-altitude urban airspaces remains challenging due to occluded views, dynamic layouts, limited communication bandwidth, and partial observability. Existing methods mainly [...] Read more.
Large vision–language models have advanced embodied navigation by integrating visual perception with natural-language reasoning. However, vision-and-language navigation (VLN) for unmanned aerial vehicles in low-altitude urban airspaces remains challenging due to occluded views, dynamic layouts, limited communication bandwidth, and partial observability. Existing methods mainly focus on single-agent egocentric navigation and lack explicit modeling of uncertainty and inter-agent dependencies in collaborative multi-UAV settings. We propose Collaborative Low-Altitude Space Navigation (Co-LASN), a dynamic Bayesian network-based framework for collaborative VLN in embodied multi-agent systems. Co-LASN jointly models environmental dynamics, linguistic constraints, and inter-agent dependencies in a unified probabilistic representation, allowing each UAV to update its belief state and incorporate information from neighboring agents when making navigation decisions. Experiments on a low-altitude subset of the HaL-13k benchmark show that, under the evaluated simulation protocol, Co-LASN achieves higher navigation metrics than single-agent and partially collaborative baselines. In the 3-agent setting, Co-LASN increases the any-success rate (ASR) from 12.37% to 15.23% and reduces the min navigation error (MNE) from 99.86 to 89.46. These results demonstrate the relative effectiveness of belief-aware collaboration within the evaluated simulation setting. Full article
Show Figures

Figure 1

29 pages, 2114 KB  
Systematic Review
Do Multimodal Vision-Language Models Enhance the Medical Diagnostic Process? A Systematic Review
by Lattawat Eauchai, Laura Otálora González, Yifan Shi, Michele T. McGinnis, Alexander Yovchev, Svetlana Herasevich, Brian W. Pickering and Vitaly Herasevich
Healthcare 2026, 14(13), 1877; https://doi.org/10.3390/healthcare14131877 - 26 Jun 2026
Viewed by 255
Abstract
Background/Objectives: Novel vision-language models (VLMs) can integrate patient textual data with image data to support medical diagnosis. Recent studies reported conflicting results regarding the performance of multimodal VLMs compared to other models and physician performance. This systematic review aims to assess the [...] Read more.
Background/Objectives: Novel vision-language models (VLMs) can integrate patient textual data with image data to support medical diagnosis. Recent studies reported conflicting results regarding the performance of multimodal VLMs compared to other models and physician performance. This systematic review aims to assess the diagnostic performance of multimodal VLMs integrating both patient textual and image data across diverse real-world hospital settings. Methods: We performed comprehensive searches of eight resources, including Embase, MEDLINE, and SCOPUS, on 17 December 2025. Eligible studies reporting diagnostic performance of VLMs integrating both image and patient history textual data from real-world adult patients compared to that of other models and physicians were included. The review adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The Prediction model study Risk Of Bias Assessment Tool + AI (PROBAST + AI) was used to assess the quality and risk of bias. The study protocol was registered in the PROSPERO database (CRD420251244054). This review received no external funding. Results: We screened 11,026 records, of which 18 studies met the inclusion criteria. Six studies comparing multimodal and unimodal models demonstrated the consistent superiority of the multimodal models. Four studies evaluating VLM accuracy as standalone agents compared with physician performance reported conflicting evidence. One study assessing VLMs as a clinical copilot demonstrated higher accuracy from the group of physicians using VLM assistance. A meta-analysis could not be performed due to the heterogeneity across study populations and outcomes. The majority of the studies were assessed as having a high risk of bias due to dataset quality. Primary limitations identified across studies include small sample size, a lack of external validation, and the need for prospective clinical deployment studies. No study provided documented considerations regarding model safety or data security. Conclusions: This systematic review suggests that multimodal VLMs consistently outperform unimodal models with access to only image or text. While model performance as standalone agents compared to humans remains inconclusive, a copilot model has demonstrated high diagnostic accuracy. Given substantial methodological concerns across studies, cautious interpretation is required, No firm clinical recommendation can be made regarding the use of standalone VLMs. Further research employing high-quality datasets is needed to ensure the reliability and clinical applicability of future VLMs. Full article
Show Figures

Figure 1

29 pages, 1051 KB  
Article
Benchmarking Multimodal Mathematical Reasoning: Prompt Effects, Modality Gaps, and Failure Modes
by Gökan Görer, Maria Osipenko and Thomas Knispel
Metrics 2026, 3(3), 11; https://doi.org/10.3390/metrics3030011 - 26 Jun 2026
Viewed by 235
Abstract
Large language models and vision–language models already achieve strong results on reasoning tasks, but their reliability under controlled assessment-style conditions remains insufficiently characterized. This paper presents a benchmark study of multimodal multiple-choice mathematical reasoning using 324 Austrian Mathematical Kangaroo competition problems (2022–2024), including [...] Read more.
Large language models and vision–language models already achieve strong results on reasoning tasks, but their reliability under controlled assessment-style conditions remains insufficiently characterized. This paper presents a benchmark study of multimodal multiple-choice mathematical reasoning using 324 Austrian Mathematical Kangaroo competition problems (2022–2024), including both text-only and diagram-dependent items. We evaluate five state-of-the-art models under a controlled protocol that isolates two factors: input modality and prompt format. We compare a strict short-answer condition requiring a single option label (one_liner) with a structured condition eliciting step-by-step reasoning and an explicit final answer (full) while enforcing deterministic decoding and rule-based answer extraction. Performance is assessed using accuracy, abstention rates, and contest-style scoring, supported by paired and unpaired statistical analyses and a structured error taxonomy. The results show that prompt format is the primary driver of performance: structured prompting yields substantial gains across all the models, particularly on text-only items. In contrast, visual-text problems remain consistently harder, with a robust performance gap that persists across prompting conditions, indicating persistent limitations in visual grounding. Model comparisons are additionally influenced by response strategies, especially abstention behavior under strict output constraints. An error analysis reveals systematic failure modes, including constraint violations, inappropriate strategy selection, and diagram misinterpretation, alongside structured biases in multiple-choice selection under constrained prompting. Overall, the findings demonstrate that measured performance is highly sensitive to the interaction between prompt format and input modality. This underscores the importance of treating prompting, decoding, and answer extraction as integral components of evaluation in assessment-oriented settings, where reliability and reproducibility are central. Full article
Show Figures

Figure 1

19 pages, 1360 KB  
Article
Efficient Image-Only Inference for Multimodal Crop Disease Recognition via Modal Dropout and Adaptive Multi-Task Loss Learning
by Jianlin Qiu, Depeng Gao, Shuxi Chen and Wenjie Liu
Sensors 2026, 26(13), 4052; https://doi.org/10.3390/s26134052 - 25 Jun 2026
Viewed by 220
Abstract
Crop leaf diseases cause 10–40% annual yield losses, yet timely field diagnosis remains difficult. Vision-language models (VLMs) lift recognition accuracy with rich textual descriptions, but multimodal pipelines are too slow for real-time field use because they require text processing at inference. We present [...] Read more.
Crop leaf diseases cause 10–40% annual yield losses, yet timely field diagnosis remains difficult. Vision-language models (VLMs) lift recognition accuracy with rich textual descriptions, but multimodal pipelines are too slow for real-time field use because they require text processing at inference. We present MTL-AWL, a framework built on a training–inference asymmetry: VLM text serves as privileged training-time supervision, and two coupled mechanisms—one retaining VLM semantics in the image encoder and one exploiting them—enable image-only deployment at multimodal accuracy. A modal-dropout strategy (p=0.6) intermittently masks the VLM text sequence during training, forcing the image encoder to retain cross-modal representations independently. An adaptive multi-task loss jointly optimizes InfoNCE contrastive alignment, attention diversity, and modality consistency under learnable softmax weights, consistently converging to a dominant contrastive weight (55% on soybean, 68% on PlantDoc)—identifying cross-modal alignment as the primary mechanism of VLM knowledge transfer. At inference, the model reaches 818 FPS (3.7× faster than multimodal methods) at only 0.41% accuracy cost, attaining 99.30%/98.89% (multimodal/image-only) on soybean and 72.65%/68.80% on PlantDoc—compact enough for real-time, offline field screening. Full article
Show Figures

Figure 1

19 pages, 3177 KB  
Article
Small Models, Big Cities: A Low-Cost AI Pipeline for Urban Regulatory Document Analysis in Metropolitan Planning
by Francisco Vergara-Perucich
Urban Sci. 2026, 10(7), 352; https://doi.org/10.3390/urbansci10070352 - 25 Jun 2026
Viewed by 197
Abstract
Background: Urban planning documents at metropolitan scale typically demand large, cloud-hosted language models that limit their adoption in Global South contexts. This study deploys Moondream, a 1.7-billion-parameter vision-language model (VLM) runnable locally via Ollama, for extracting geographic knowledge from Planes Reguladores Comunales (PRCs) [...] Read more.
Background: Urban planning documents at metropolitan scale typically demand large, cloud-hosted language models that limit their adoption in Global South contexts. This study deploys Moondream, a 1.7-billion-parameter vision-language model (VLM) runnable locally via Ollama, for extracting geographic knowledge from Planes Reguladores Comunales (PRCs) across 29 processed Gran Santiago municipalities. The pipeline combines native PDF text extraction, keyword-based multi-label classification across six thematic axes, and VLM-based optical character recognition and cartographic interpretation. Results: The pipeline processes 2289 PRC articles in 4.3 min at an estimated energy cost of 0.000866 kWh and zero marginal monetary cost. Zoning (53.3%) and land use (43.1%) dominate PRC content, while social housing provisions appear in only 4.0% of articles; normative gap analysis identifies five municipalities where social housing is entirely absent from regulatory text. A comparative evaluation of Moondream against keyword baseline on an 88-article validation sample yields macro-F1 = 0.355 and mean Cohen’s κ = 0.004, confirming that generalist VLMs require domain fine-tuning for specialized legal text. It is argued that the cost asymmetry between industrial-scale and small-model approaches constitutes an epistemic asymmetry with direct consequences for the geographic distribution of urban data infrastructure. Full article
(This article belongs to the Special Issue Advances in Urban Planning and the Digitalization of City Management)
Show Figures

Figure 1

Back to TopTop