Abstract
Tomato leaf diseases exhibit subtle inter-class differences and substantial intra-class variability, making accurate identification challenging for conventional deep learning models, especially under real-world conditions with diverse lighting, occlusion, and growth stages. Moreover, most existing approaches rely solely on visual features and lack the ability to incorporate semantic descriptions or expert knowledge, limiting their robustness and interpretability. To address these issues, we propose LPDiag, a multimodal prototype-attention diagnostic framework that integrates large language models (LLMs) for fine-grained recognition of tomato diseases. The framework first employs an LLM-driven semantic understanding module to encode symptom-aware textual embeddings from disease descriptions. These embeddings are then aligned with multi-scale visual features extracted by an enhanced Res2Net backbone, enabling cross-modal representation learning. A set of learnable prototype vectors, combined with a knowledge-enhanced attention mechanism, further strengthens the interaction between visual patterns and LLM prior knowledge, resulting in more discriminative and interpretable representations. Additionally, we develop an interactive diagnostic system that supports natural-language querying and image-based identification, facilitating practical deployment in heterogeneous agricultural environments. Extensive experiments on three widely used datasets demonstrate that LPDiag achieves a mean accuracy of 98.83%, outperforming state-of-the-art models while offering improved explanatory capability. The proposed framework offers a promising direction for integrating LLM-based semantic reasoning with visual perception to enhance intelligent and trustworthy plant disease diagnostics.
1. Introduction
In real field scouting, a farmer may capture two smartphone images that appear nearly indistinguishable—both showing small brown spots and mild yellowing—yet the correct diagnoses can differ (e.g., early blight vs. septoria leaf spot), leading to very different treatment decisions. Conversely, the same disease category may present substantially different visual patterns under varying illumination, growth stages, and occlusion, resulting in high intra-class variability and unreliable predictions for purely data-driven systems.
Plant disease recognition has become a cornerstone of intelligent agriculture, offering the potential to improve crop management, secure food production, and reduce economic losses. The widespread availability of plant disease datasets [1,2,3,4] and the rapid evolution of deep learning technologies have accelerated the development of automated diagnostic systems for crops such as tomatoes. Early approaches—ranging from classical machine learning to Convolutional Neural Network (CNN)-based models—have achieved substantial progress in image-based diagnosis [5,6,7]. More advanced architectures, including multi-scale networks [8], transformer-driven frameworks [9,10], and multimodal fusion systems [11], further enhance feature representation, improve robustness to illumination and background variability, and strengthen discrimination between subtle lesions.
Despite these advancements, purely vision-based models still encounter significant limitations when addressing the diverse symptom manifestations of tomato diseases across various developmental stages [12]. Lesions from different diseases may appear visually similar, while the same disease may present heterogeneous patterns depending on growth phase, environment, or imaging conditions. Although CNNs, attention mechanisms, and multi-resolution strategies improve the capture of low-level and mid-level features, these models inherently lack the ability to integrate semantic, causal, or expert-level knowledge that human agronomists rely on for differential diagnosis. This limitation leads to misclassification among visually confounding diseases and restricts generalization in real-world agricultural scenarios.
This gap originates from a fundamental limitation of deep visual models: they interpret images solely through statistical features rather than through conceptual or knowledge-driven reasoning. As a result, they fail to encode symptom descriptions, interpret disease evolution, or capture high-level associations between visual cues and biological meanings. Recent works attempt to incorporate knowledge graphs or context-aware mechanisms to enrich visual learning [13], yet these methods remain constrained by the fixed structure and limited expressiveness of manually designed knowledge sources.
The emergence of large language models (LLMs) offers an unprecedented opportunity to address this long-standing semantic bottleneck. LLMs exhibit strong capabilities in natural-language understanding, knowledge retrieval, causal reasoning, and multi-step interpretation—competencies that are crucial for agricultural diagnostics. Groundbreaking studies such as AgroGPT [14], crop-oriented multimodal LLMs [15], and multimodal veterinary diagnosis systems [16] demonstrate that combining visual models with LLMs enables richer semantic grounding, enhances interpretability, and moves agricultural AI toward expert-level decision support.
Within this emerging landscape, existing LLM-based agricultural models generally follow three representative directions: (i) Agriculture-adapted vision–language models that enhance visual features with domain prompts and fine-tuning (e.g., AgriVLM [17]); (ii) Expert-tuned VLMs that explicitly inject agricultural expertise into the vision–language pipeline (e.g., AgroGPT [14]); and (iii) LLM-assisted diagnostic systems that generate professional descriptions and recommendations for crop disease identification (e.g., LLMI-CDP [15]). Compared with these works, LPDiag is positioned as a multimodal prototype-learning framework that (1) explicitly models prototype-level reasoning to bridge LLM semantic priors with representative lesion patterns, (2) enforces visual–semantic alignment via a prototype-guided attention mechanism for fine-grained discrimination under high inter-class similarity, and (3) integrates uncertainty estimation to support safer deployment under open-world and ambiguous scenarios.
However, as Figure 1 shows, existing agricultural multimodal LLM frameworks still exhibit key limitations:
Figure 1.
The challenges in AI for classification of tomato leaf diseases.
- Most models are not tailored to unknown disease classification, especially for crops like tomato, where lesions are highly similar.
- Current architectures lack prototype-level reasoning mechanisms that can link semantic knowledge with representative lesion patterns.
- Few models explicitly address visual–semantic alignment, resulting in suboptimal integration between image features and LLM-generated textual knowledge.
- Interactive diagnosis remains underexplored, with limited systems enabling domain-level analysis, natural language queries, or interpretable reporting.
To overcome these issues, we propose a novel LLM-Enhanced Multimodal Prototype Learning Framework for Intelligent Tomato Leaf Disease Diagnosis, called LPDiag, which integrates an LLM with deep visual representations. This framework achieves an accurate, interpretable, and semantically grounded disease diagnosis. Specifically, LPDiag adopts a multi-scale Res2Net backbone to extract hierarchical visual features from tomato leaf images, providing rich representations of lesion patterns at different spatial resolutions. Meanwhile, textual disease knowledge generated by a large language model is introduced as semantic guidance. To effectively bridge the semantic gap between visual and textual modalities, a prototype-guided attention mechanism is designed, which aligns visual features with LLM-derived disease prototypes and enables fine-grained disease discrimination. Furthermore, LPDiag incorporates a multimodal fusion network with an uncertainty estimation module, allowing the model to produce calibrated confidence scores for both known and unknown disease categories. This design improves the reliability of diagnostic results in real-world agricultural scenarios where disease manifestations are diverse and open-set conditions are common. In addition, we develop an interactive multimodal diagnostic system that supports natural-language querying and image-based identification. This system enables users to obtain interpretable diagnostic explanations and disease-related knowledge through human–machine interaction, thereby enhancing the practical usability of the proposed framework in agricultural disease management. Experimental results demonstrate that LPDiag effectively integrates visual and semantic information, achieves precise lesion localization and robust disease recognition, and exhibits strong interpretability and interaction capability, highlighting its potential for intelligent tomato disease diagnosis in practical agricultural applications.
2. Related Work
This section reviews the relevant literature on CNNs for plant disease recognition, multimodal data and vision–language models, and uncertainty evaluation of neural models.
2.1. Deep Learning for Plant Disease Diagnosis
Deep learning has substantially advanced automated plant disease recognition, particularly through CNNs and Vision Transformers (ViTs). CNN-based models and their improved variants have demonstrated strong capabilities in hierarchical feature extraction, while Transformer-based architectures have further enhanced global modeling and multi-scale representation learning [16,18].
For example, Islam et al. [19] proposed an Attention-based Dilated CNN with Logistic Regression (ADCLR), where dilated convolutions and attention mechanisms expand the receptive field and improve multi-scale context aggregation for tomato leaf disease recognition. Hossain et al. [20] conducted a comparative analysis across EANet, MaxViT, CCT, and PVT, showing that MaxViT achieved the best multi-scale attention aggregation with an accuracy of 97%.
Despite their high performance, these deep visual models remain predominantly data-driven. Their reliance on visual features alone often limits their ability to incorporate domain knowledge, interpret fine-grained lesion characteristics, or generalize across developmental stages. This semantic gap significantly constrains diagnostic reliability in real-world agricultural environments.
2.2. Multimodal Data Fusion and Vision–Language Models
Vision–language models (VLMs) such as CLIP [21] and BLIP [22] have revealed the power of aligning visual and textual representations to enhance robustness, transferability, and semantic reasoning. However, the application of such multimodal frameworks in agriculture is still emerging.
Deng et al. [23] introduced FCMNet, integrating Fourier-domain frequency priors with spatial–channel attention to improve noise robustness and lesion localization. With multimodal fusion of global and local features, the framework demonstrated more stable disease identification under complex conditions.
More recently, the emergence of LLMs—including GPT-4 and LLaMA-3—has enabled unprecedented abilities in zero-shot reasoning, open-domain knowledge retrieval, and contextual semantic understanding. Several studies have begun exploring their use in agriculture. Yu et al. [17] proposed AgriVLM, a visual–language framework fine-tuned on large-scale agricultural datasets for multimodal analysis. Wang et al. [15] developed LLMI-CDP based on VisualGLM to support crop disease identification and provide professional prevention advice. Liu et al. [24] leveraged LLMs to generate detailed textual descriptions—global, local, and color–texture—to guide cross-modal disease recognition. Ref. [25] proposed a DIMPCNET network, combining enhanced data, noise removal, and attention mechanisms, which achieves 94.44% accuracy in tomato leaf disease classification despite challenges from complex backgrounds and subtle disease differences.
These works collectively highlight the promise of multimodal learning for bridging the gap between visual signals and domain-specific semantics. Cross-modal alignment, feature interaction, and LLM-driven knowledge enhancement have shown clear potential to improve interpretability, robustness, and generalization [26]. Additionally, advances in reliability analysis [27] and model optimization [28] have further accelerated the practical deployment of multimodal systems in agriculture.
2.3. Attention Mechanisms
Attention mechanisms have become integral to agricultural AI by enabling models to dynamically prioritize task-relevant features, thereby mitigating information redundancy and improving prediction accuracy.
Recent studies have integrated attention mechanisms into lightweight CNNs and Transformer-based architectures to balance accuracy and computational efficiency. Channel and spatial attention modules [29,30] have been shown to improve sensitivity to lesion regions in leaf disease images. Sun et al. introduced an attention-enhanced deep learning architecture that improves tea disease detection by optimizing multiscale feature expression and semantic fusion, reaching 92.9% accuracy [31]. Ye et al. proposed M2CNet, which combines lightweight convolution with multi-head attention to jointly model local lesion details and global contextual dependencies, achieving strong performance on benchmark plant disease datasets with a compact parameter size [32]. Karimanzira et al. further introduced cascaded group attention within a Vision Transformer framework to alleviate class imbalance and improve robustness for tomato disease recognition under heterogeneous data distributions [12]. Attention mechanisms have also been extensively explored in broader agricultural applications, particularly in remote sensing and crop monitoring. Attention-enhanced recurrent and Transformer models have been applied to NDVI and multispectral time-series data, enabling adaptive weighting of critical temporal segments and improving crop growth modeling and yield estimation accuracy [33]. These results highlight the general effectiveness of attention in capturing long-range dependencies and heterogeneous feature importance in agricultural data.
Despite these advances, most existing attention-based disease diagnosis models remain purely data-driven and rely on implicitly learned feature weights. Their decision processes therefore lack semantic interpretability and explicit integration of agronomic knowledge, which limits their practical value in expert-oriented agricultural decision support systems. Attention weights learned solely from visual statistics do not necessarily correspond to biologically meaningful symptom patterns, especially under limited, imbalanced, or non-IID data conditions commonly encountered in real farming scenarios. To address this, prototype learning mechanisms, such as semantic-guided attention (SEGA) [34], have emerged as a robust framework. By leveraging class-specific prototypes to supervise visual feature extraction, this paradigm facilitates a coherent synthesis of deep visual perception and symbolic reasoning. This theoretical synergy underpins the prototype-guided attention strategy developed in the current study.
2.4. Uncertainty Estimation
Reliable plant disease diagnosis requires not only accurate visual classification but also well-calibrated uncertainty estimation to support trustworthy decision-making in real-world agricultural environments. Conventional neural networks often produce overconfident predictions, especially under data noise, distribution shifts, and visually ambiguous symptoms, limiting their reliability in practical agricultural scenarios. To address these challenges, Dirichlet-based evidential uncertainty estimation has emerged as an effective probabilistic framework, enabling the joint modeling of class probabilities and epistemic/aleatoric uncertainty through the parameterization of a Dirichlet distribution.
Recent developments in Evidential Deep Learning (EDL) provide more robust and theoretically grounded uncertainty modeling. Deng et al. [35] introduced a Fisher-information–guided evidential learning strategy that improves training stability and enhances robustness under distribution shifts. Yoon et al. [36] further proposed density-aware evidential learning for more reliable out-of-distribution detection, a problem frequently encountered in open-field agricultural disease monitoring. Chen et al. [37] revisited epistemic evidence formation, offering deeper insight into uncertainty decomposition and training dynamics, while Sensoy et al. [38] proposed a novel approach that treats neural network predictions as subjective opinions within a Dirichlet distribution framework, enabling explicit modeling of prediction confidence through evidence collection and demonstrating unprecedented success in out-of-distribution detection and adversarial robustness. In addition, risk-aware evidential classification frameworks [39] demonstrated that Dirichlet-based uncertainty can effectively prevent high-risk mispredictions, which is crucial in high-stakes agricultural applications. Damola et al. [40] proposed a Multi-Expert Gating Network that aggregates uncertainty estimates and predictions from multiple AI experts via EDL models trained with diverse ground truths and modeling strategies.
In plant disease diagnosis, where many leaf diseases exhibit subtle inter-class differences and high visual similarity, Dirichlet-based uncertainty provides a principled mechanism to quantify prediction confidence and mitigate ambiguous or noisy visual evidence. Moreover, when combined with multimodal vision–language systems, uncertainty signals can guide more stable cross-modal alignment by down-weighting unreliable visual cues and leveraging semantic priors—an essential capability for trustworthy multimodal agricultural diagnostic systems.
3. Methodology
3.1. Overall Architecture
The overall framework of the proposed LPDiag is shown in Figure 2. This architecture aims to achieve intelligent diagnosis of tomato leaf diseases by integrating multimodal information from both images and text. The model primarily comprises two parallel branches: an image feature extraction branch and a text feature extraction branch. The image branch employs a Res2Net encoder () to extract visual features () from the input images. Concurrently, the text branch leverages a frozen LLM to generate textual prompts and subsequently uses a Clip text encoder () to extract corresponding text features (). The core of our model lies in its multimodal prototype-guided alignment mechanism. First, undergoes visual–language alignment refinement via a Prototype-Guided Attention (PGA) module. This process aims to align the textual semantics with the visual context, thereby generating Aligned Embeddings. These Aligned Embeddings are then fused with the image features () to collectively form the ultimate unified feature (). Subsequently, is used to compute cosine similarities with a set of learnable category prototypes (), quantifying the degree of similarity between the feature and each disease category. Finally, these integrated similarity features are fed into a trainable Fully Connected (FC) Layer, which outputs the final disease prediction results. This framework fully leverages complementary information from visual and textual modalities, significantly enhancing the model’s diagnostic capabilities through a prototype-guided mechanism.
Figure 2.
Overview of the proposed multimodal prototype-guided learning network integrated with large language models.
3.2. Multiscale Visual Feature Extraction
We employ a Res2Net-based backbone to construct a multi-scale feature pyramid. The output feature maps encode hierarchical representations of disease symptoms such as spots, blights, and leaf texture.
As shown in Table 1, Res2Net significantly enhances fine-grained image recognition capabilities through its multi-scale feature extraction mechanism, with its core innovation lying in the design of the bottleneck module. This module employs a feature-splitting strategy to divide input features into four independent branches (scale = 4) along the channel dimension. Each branch processes visual information at different scales in parallel: three branches extract features via 3 × 3 convolutions, while the fourth branch preserves the original features. The key parameter baseWidth = 26 controls the base width, calculated as width = planes × (baseWidth/64.0), ensuring the network maintains a reasonable feature dimension across varying depths. During multi-scale processing, feature reuse is achieved through residual connections between branches: the input to the (i + 1) th branch is the sum of the original split features and the output features from the ith branch, enhancing feature flow and information transfer. After multi-scale branch processing, all features are concatenated along the channel dimension, followed by 1 × 1 convolutions for feature fusion and dimension adjustment. The complete network employs a (3, 4, 6, 3) layer configuration, with Layers 1 to 4 processing 64, 128, 256, and 512 channels respectively, ultimately outputting a 2048-dimensional feature vector. The network is initialized using ImageNet pre-trained weights and supports loading official models.
Table 1.
Parameters of Res2Net Backbone Layer.
3.3. LLM-Based Semantic Embedding and Alignment
To bridge the gap between low-level visual patterns and high-level pathological concepts, we implement a multimodal alignment strategy leveraging Large Language Models (LLMs). This process begins with the generation of structured semantic descriptors from input tomato leaf images, which are subsequently projected into a shared latent space to guide visual perception.
3.3.1. Expert-Driven Prompt Design
To Draw inspiration from the cognitive hierarchy used in expert plant diagnosis [41], we employ a pathology-driven prompt strategy to ensure the generation of discriminative textual features. As detailed in Table 2, the prompt is formulated to emulate a professional botanical examination, directing the LLM to perform “linguistic re-engineering” by focusing on five critical diagnostic dimensions: (i) morphological traits, (ii) chromatic distribution, (iii) lesion architecture, (iv) tissue degradation patterns, and (v) potential disease indicators. This structured constraint ensures that the generated text provides a dense and standardized semantic representation, effectively filtering out non-diagnostic environmental noise.
Table 2.
Text prompt for Tomato Disease Diagnosis using LLM Models.
3.3.2. Semantic Embedding and Modal Alignment
To transform these rich natural language descriptions into a format processable by the neural network, we utilize a CLIP-based [21] text encoder . Specifically, we employ the CLIP RN50 architecture as our foundational framework, leveraging its dual-stream capability that aligns a ResNet-based vision encoder with a Transformer-based text encoder. The generated descriptions are encoded into textual embeddings , where N denotes the token count and represents the embedding dimension.
3.4. Prototype-Guided Attention
In complex tomato leaf disease diagnosis, solely relying on single-modality features often fails to achieve robust performance. Leveraging LLMs to generate textual prompts has emerged as a promising avenue for providing rich, semantically meaningful context. To bridge the semantic gap between visual and textual information and to imbue our model with a more interpretable and robust understanding of disease patterns, we introduce a novel PGA mechanism, designed to facilitate robust multimodal knowledge fusion.
The PGA module plays a crucial role in our framework by addressing two fundamental challenges: (1) directly fusing LLM-generated text features with visual features can lead to suboptimal alignment due to the heterogeneity of the two modalities, and (2) simple concatenation or element-wise fusion operations fail to capture the complex semantic relationships between disease patterns. The prototype-guided mechanism overcomes these limitations by introducing learnable prototype vectors that serve as semantic anchors, mediating the interaction between visual and textual representations. This enables the model to focus on disease-specific features while filtering out irrelevant background information.
As shown in Figure 3 and detailed in Algorithm 1, the PGA module employs a two-layer cross-attention mechanism, which guides the learning of image features through learnable prototype vectors. This includes a two-layer Multi-Layer Perceptron (MLP) structure (512→1024→512) with GELU activation and 0.1 dropout regularization. This module achieves interaction between prototypes and image features through the first attention layer. After MLP enhances feature representation, the second attention layer integrates textual information, ultimately outputting 512-dimensional multimodal fusion features.
| Algorithm 1 Prototype-Guided Attention (PGA) Module |
|
Figure 3.
The detailed architecture of the PGA module.
We initialize a set of K learnable prototype vectors, where K equals the number of categories, denoted as . These prototype vectors employ a standard normal distribution for random initialization. This approach avoids prior bias, enabling the model to autonomously learn optimal prototype representations from plant disease data. Additionally, the standard normal distribution exhibits favorable gradient flow properties, facilitating back propagation optimization. Prototype vectors serve as dynamic representations of key disease patterns and semantic anchors. To capture image-specific contextual information, these prototype vectors are first expanded to match the current batch size and then used as queries . Subsequently, we utilize a cross-attention mechanism to interact with the visual embeddings (where B is the batch size and is the visual embedding sequence length). Formally, we treat the prototype vectors as the query , and the visual embeddings as the key and value . The prototype-guided attention mechanism is computed as follows:
where D is the embedding dimension. This step refines the initial prototypes into image-aware prototypes , allowing them to focus on the most relevant visual information in .
Following this, the image-aware prototypes are further refined through an MLP and Softmax activation to derive attention weights that emphasize discriminative prototype elements. The prototype-enhanced representation is obtained as follows:
where ⊙ denotes the Hadamard product. Subsequent Layer Normalization is applied to to stabilize the representation. Each element in now encapsulates an enriched visual context.
To integrate the semantic knowledge from the LLM, we prepare key–value pairs for a second cross-attention. Here, a key–value pair in attention mechanisms consists of reference vectors (keys and values) used for information retrieval. These pairs are formed by concatenating the initial prototypes with the image-aware prototype embeddings , leading to . derived from the LLM via the serves as the query . The cross-attention between textual queries and prototype-enriched visual context is then computed as:
where represents the text features complemented by multimodal knowledge. This mechanism effectively aligns the textual semantics with the visually informed and prototype-structured representations.
3.5. Uncertainty Evaluation
To enhance model reliability without compromising crop disease classification performance, this study introduces an Uncertainty Estimation (UE) module. Leveraging features extracted by the encoder, the UE module directly generates crop disease classification results accompanied by uncertainty scores.
Specifically, consider the model of a given client as an example. Suppose the client’s classification task involves K classes. For a given diseased leaf image input, the UE module outputs non-negative values whose sum equals 1. Among them, the value corresponding to the i-th class represents the predicted probability for that class, while u denotes the overall uncertainty score, satisfying the constraint (4):
The computation process of the UE module proceeds as follows: First, a local Fully Connected (FC) layer learns class-specific features relevant to the client’s classification categories. These features are then processed by the Softplus activation function to obtain evidence , ensuring all feature values are positive. Following the method in [42], the evidence E is re-parameterized into Dirichlet concentration parameters as , where , with and denoting the Dirichlet parameter and the evidence for the k-th class, respectively.
Next, the belief mass and the corresponding uncertainty score u are computed as:
where S is the Dirichlet strength, defined as:
This formulation implies that the probability assigned to class k is positively correlated with the amount of evidence supporting that class, whereas lower total evidence results in a higher uncertainty score u.
4. Experiments
In this section, we first introduce the experimental datasets used in the experiments, then provide a detailed description of our proposed method, and finally introduce the evaluation results of the experiments.
4.1. Datasets
We evaluate the model on three public datasets, as shown in Table 3: PlantVillage [3], CCMT [2] and TaiwanTomato [4]. Performance is measured based on Accuracy, Precision, Recall, F1-score, and AUC. To ensure scientific rigor, we validated the statistical robustness of LPDiag using a dual-test strategy: McNemar’s test was employed for Accuracy to analyze paired prediction error distributions, while Bootstrap resampling (1000 iterations) was used for AUC, Precision, Recall, and F1-score to derive 95% confidence intervals (CI). A p-value of less than 0.05 was considered the criterion for statistical significance, confirming that the superiority of LPDiag over baseline models is not the result of experimental noise or random variation.
Table 3.
The datasets of the experiment.
The tomato leaf disease dataset employed in this study consists of three primary sources: a self-collected private dataset and publicly accessible open datasets.
4.2. Experimental Setup
4.2.1. Training Hyperparameters
The model is trained using the Adam optimizer with an initial learning rate of and weight decay of . The learning rate is reduced by a factor of 0.5 when the validation loss plateaus for 5 epochs (patience = 5). Batch size is set to 64 for all experiments. The maximum number of training epochs is 100, with early stopping triggered if the validation accuracy does not improve for 10 consecutive epochs. For the uncertainty estimation module, the Dirichlet prior concentration parameter is initialized to 1.0, and the uncertainty threshold for out-of-distribution detection is set to 0.31, which is determined through validation on a held-out set.
As Table 4 shows, the model is implemented in PyTorch 1.10 and trained on an NVIDIA RTX 4090 GPU.
Table 4.
Detailed hardware and software configurations.
4.2.2. Data Preprocessing and Augmentation
In this experiment, image preprocessing was conducted on all datasets. As Figure 4 shows, all images were uniformly resized to 256 × 256 pixels to ensure consistency at first. Subsequently, random horizontal flipping with a probability of 0.5 was applied to enhance the model’s symmetry recognition capability, followed by random rotation within the range of ±15 degrees to simulate the natural posture variations in plant leaves. Furthermore, color jittering was utilized to adjust brightness (±0.2), contrast (±0.2), saturation (±0.2), and hue (±0.1) to simulate diverse shooting environmental conditions. Finally, the images were converted into tensors and normalized using the standardization parameters of the ImageNet pre-trained model (mean [0.485, 0.456, 0.406], standard deviation [0.229, 0.224, 0.225]) to ensure compatibility with the pre-trained weights of Res2Net. Regarding the textual component, the LLM employed in this study is ChatGPT-4.1.
Figure 4.
Illustration of the tomato disease leaf image pre-processing.
4.3. Comparative Results
Comparative experiments were conducted between the proposed LPDiag and baseline methods (ResNet series, MobileNetv2, Res2Net) and the state-of-the-art multimodal framework BLIP [22], across three datasets: TaiwanTomato [4], PlantVillage [3], and CCMT [2]. As presented in Table 5, LPDiag demonstrates superior comprehensive performance, consistently surpassing baselines in Accuracy, Precision, Recall, and F1 score across all datasets. Notably, LPDiag achieves an overall accuracy exceeding 91.42% on all three datasets. Specifically, on the PlantVillage [3] dataset, the model delivers exceptional results with an accuracy of 98.83% and an F1 score of 98.65%. Similarly, on the CCMT [2] dataset, it maintains the lead with an AUC of 99.08% and an accuracy of 94.53%. Although ResNet101 exhibits a marginally higher AUC on the Taiwan dataset, LPDiag significantly outperforms it in the critical metrics of accuracy (91.42% vs. 88.17%) and F1 score (91.05% vs. 87.39%). Regarding computational efficiency, while LPDiag incurs a slight increase in inference time compared to lightweight models like MobileNetv2, it remains comparable to ResNet101 and Res2Net. These findings confirm that integrating LLM-derived semantics not only enhances recognition robustness and interpretability but also effectively balances computational cost with high-precision diagnosis in multimodal tasks.
Table 5.
Comparison of results of different methods on various datasets. Results are reported as mean confidence interval.
4.4. Ablation Study
As shown in Table 6, we conduct ablation experiments by removing key modules (prototype attention or LLM fusion). Each removal leads to a noticeable performance drop, verifying the contribution of each component.
Table 6.
Ablation results of the proposed components across three datasets.
While the baseline augmented with LLM (Backbone + LLM) demonstrates competitive performance, the integration of the PGA module consistently yields the superior diagnostic accuracy. The initial performance trade-off observed (where LLM integration improved ACC but slightly reduced AUC) highlights the limitations of naive feature concatenation. The slight decrement in AUC following the integration of LLM components suggests a polarization effect in semantic–visual fusion. While LLM-derived textual priors significantly sharpen the decision boundary, leading to improved ACC and F1 scores, they may constrain the model’s probabilistic ranking flexibility, which is reflected in the marginal AUC fluctuation. This observation directly motivates the design of our PGA module, which acts as a strategic mediator to harmonize semantic and visual modalities. As shown in the final row of each dataset, the PGA successfully restores and exceeds the original AUC performance while maintaining the high accuracy gained from the LLM. Specifically, on the TaiwanTomato [4] dataset, the inclusion of PGA elevates the Accuracy from 88.98% to 91.42% and the F1-score from 88.15% to 91.05%. A similar trend is observed in the PlantVillage [3] dataset, where Accuracy reaches a peak of 98.83%.
As illustrated in the heatmaps (Figure 5), the baseline Res2Net architecture establishes a foundational capability for feature extraction, successfully activating across the general leaf regions. However, its attention distribution appears relatively dispersed, often failing to strictly distinguish between specific pathological lesions, healthy tissue, and background noise. With the integration of the LLM, the model incorporates semantic guidance, initiating a shift in attention towards more symptom-relevant areas. Most notably, the introduction of the PGA module demonstrates a decisive improvement in feature localization. By leveraging prototype vectors to guide the attention mechanism, PGA effectively suppresses irrelevant background information and tightly focuses the high-activation regions on the most discriminative disease symptoms (such as specific spots, mold, and discoloration). This visual evidence confirms that, rather than merely recognizing the leaf morphology, the proposed method precisely targets the critical pathological features essential for fine-grained diagnosis.
Figure 5.
Comparison of heat maps generated by different solution.
In summary, by comprehensively analyzing the heat map performances of each component, this study visually showcases the significant advantages of Res2Net, LLM, and PGA working synergistically. Specifically, Res2Net establishes a solid feature foundation, LLM deepens semantic understanding, and PGA ensures precise feature localization. Together, they contribute to the comprehensive enhancement of feature completeness, semantic precision, and diagnostic accuracy in complex leaf recognition tasks.
To evaluate the impact of prompt design, we compared “Minimalist” and “Expert” styles [41] on the PlantVillage dataset. The results in Table 7 show that LPDiag is highly robust, achieving 96.12% AUC even with minimal instructions. The “Expert” prompt improved performance to 98.83% by providing domain-specific pathological anchors. This confirms that precise semantic guidance effectively enhances multimodal feature alignment within the PGA module.
Table 7.
Ablation study of different prompt styles on LPDiag performance using the PlantVillage dataset.
4.5. Visualizing Feature Space Transformation
By employing the t-SNE [43] algorithm to visualize the feature distributions as depicted in Figure 6, a pronounced discrepancy emerges between the initial text features and the subsequent aligned features across the three datasets. The visualization provides a clear illustration of the evolution of the feature space from a disordered state to a highly structured and discriminative representation.
Figure 6.
t-SNE visualization of feature distributions across TaiwanTomato (left column), PlantVillage (middle column), and CCMT (right column) datasets.
Specifically, as illustrated in the top row, the raw text features extracted directly from LLM prompts manifest a chaotic and heavily overlapping distribution. Samples from disparate disease categories (indicated by different colors) are intermingled, exhibiting indistinct boundaries, thereby forming amalgamated clusters. This phenomenon suggests that, in the absence of visual context alignment, text embeddings alone lack sufficient discriminability to distinguish between fine-grained disease types, resulting in significant semantic confusion.
Conversely, the bottom row demonstrates the significant impact of the alignment mechanism. Subsequent to the passage through the alignment module, a fundamental transformation is undergone by the feature distribution. The previously entangled points are reorganized into distinct, well-separated clusters. Each cluster corresponds to a specific disease category, exhibiting high intra-class compactness and minimal inter-class overlap. This distinct separation indicates that the model has effectively aligned textual semantics with visual features, thereby learning a robust feature space in which disease categories can be distinctly distinguished.
Regarding the uniformity of feature distribution patterns, the aligned features demonstrate a remarkable structural consistency across diverse datasets. While the raw text features appear as unstructured and randomized clouds with varying densities across datasets, the aligned features across TaiwanTomato [4], PlantVillage [3], and CCMT [2] converge into a unified topological pattern characterized by clear, well-separated clusters. This observation confirms that the model has successfully learned a generalized representation logic independent of specific data sources. It consistently enforces a stable and uniform structure in the feature space, ensuring that discriminative patterns are preserved regardless of dataset variations.
4.6. Advantage of Uncertainty Estimation
To thoroughly analyze the uncertainty assessment mechanism of deep learning models, as illustrated in the figure, LPDiag employs a specific uncertainty threshold to differentiate between reliable and unreliable predictions, categorized as follows:
- Reliable results refer to cases where the model exhibits high confidence in its prediction and the actual classification is correct.
- Unreliable results denote instances where the model’s prediction confidence is low, necessitating further validation.
- Unreliable yet correct results represent a special case where, despite being misclassified as “unreliable” by the model, the prediction is actually accurate.
Figure 7 presents a comparative visualization of the uncertainty estimation mechanism between a traditional AI model and the proposed LPDiag across three distinct scenarios.
Figure 7.
Result of uncertainty estimation with random samples.
For in-distribution samples, such as (a) Early Blight and (b) Healthy leaves, the traditional AI model tends to output extremely high confidence scores (1.0). Similarly, LPDiag correctly classifies these samples with high probabilities (0.79 and 0.85, respectively). Crucially, the generated Uncertainty Scores (US) are 0.21 and 0.15, both falling below the preset uncertainty threshold of 0.31. Consequently, these predictions are validated as “Reliable Correct Results”, demonstrating that LPDiag maintains high confidence for familiar data.
The advantage of LPDiag is most prominent in the (c) Out-of-Distribution (OOD) scenario, where an image of “Black Mold” (a category unseen during training) is input. The traditional AI model exhibits the dangerous “overconfidence” phenomenon, incorrectly classifying the unknown disease as “Bacterial Spot (BS)” with a high confidence of 0.78, leading to an “Incorrect Result.” In contrast, while LPDiag also assigns a probability to BS, its Uncertainty Estimation module generates a high Uncertainty Score of 0.39, which exceeds the safety threshold (0.31). This triggers an alert, correctly flagging the prediction as “Unreliable.” By effectively filtering out such high-risk, unknown samples, LPDiag avoids the silent failures typical of traditional models, significantly enhancing the system’s safety and robustness in open-world clinical applications.
Compared to traditional methods like Monte Carlo Dropout [44] and Deep Ensembles, our Dirichlet-based UE module offers significant advantages in efficiency and interpretability. Unlike sampling-based approaches that require multiple forward passes or maintaining multiple model sets, our module operates via deterministic single-pass inference, drastically reducing the computational and energy costs for mobile agricultural devices. Furthermore, while traditional methods often rely on predictive entropy, our approach explicitly decouples vacuity of evidence (u) from classification probability through the Dirichlet distribution. This provides a more interpretable reliability metric for identifying out-of-distribution crop diseases without the prohibitive communication overhead typically associated with ensemble-based Federated Learning.
5. Discussion
5.1. Computational Efficiency and Deployment Feasibility
Analyzing the computational complexity in terms of parameters and FLOPs provides a comprehensive evaluation of the deployment feasibility of LPDiag on edge devices. As illustrated in the efficiency–accuracy trade-off analysis (Figure 8), a significant gap in computational overhead exists between the multimodal BLIP model and our proposed framework. While BLIP involves a massive footprint of 199.07 M parameters and 20.45 GFLOPs, LPDiag achieves a superior balance by maintaining a manageable complexity of 35.71 M parameters and 6.23 GFLOPs. This resource requirement is comparable to standard CNN backbones such as ResNet50 (23.52 M, 5.40 GFLOPs). Coupled with an average inference latency of 17.70 ms, these results demonstrate that LPDiag effectively leverages LLM-driven semantic knowledge while remaining well-suited for real-time diagnostic deployment on resource-constrained agricultural platforms.
Figure 8.
Comparison of computational cost, parameter size, and accuracy across different models on the PlantVillage dataset.
5.2. Overall Performance Comparison
As shown in Table 8, the classification performance of different models on the tomato leaf disease dataset was compared.
Table 8.
Comparison of classification performance on tomato leaf disease datasets.
The experiment compared ResNet50, ViT-Base, CLIP-ViT, and the improved algorithm LPDiag. The results show that the improved algorithm LPDiag significantly outperforms other models in terms of three metrics: Accuracy, F1-score, and AUC. ResNet50 has an accuracy of 96.8%, an F1-score of 0.962, and an AUC of 0.975; ViT-Base has an accuracy of 97.4%, an F1-score of 0.968, and an AUC of 0.982; CLIP-ViT has an accuracy of 98.5%, an F1-score of 0.982, and an AUC of 0.991; and LPDiag has a high accuracy of 99.2%, an F1-score of 0.991, and an AUC of 0.997, showing obvious advantages and highlighting the excellent performance of the improved algorithm in the tomato leaf disease classification task.
5.3. Case Studies: Successful and Failed Predictions
The LPDiag model (as illustrated in Figure 9) demonstrates exceptional capability in disease identification, accurately determining the type of disease and providing interpretable diagnostic evidence. The predictive results for diseases such as Bacterial Spot are highly consistent with actual professional diagnoses. This indicates that the model excels in visual feature extraction and multimodal information fusion, particularly in integrating image data with agricultural expertise. It effectively establishes a connection between the visual patterns of diseases and their underlying agricultural logic, ensuring high recognition accuracy.
Figure 9.
Visualization of successful prediction from system.
Through heatmap visualization (as shown in the center of Figure 10), the system intuitively reveals the key visual regions influencing the model’s decision-making process. This provides experts with easily understandable supporting evidence to validate the model’s focus, such as the distribution of lesions on the leaf surface.
Figure 10.
Visualization of failed prediction from system.
However, the failure cases in Figure 10 reveal core challenges in semantic–visual alignment for ambiguous samples. While the heatmap accurately localizes the leaf center, the low confidence (0.4010) and high uncertainty (0.5990) for “Leaf Mold” indicate a “semantic shift” in the latent space. This confusion typically occurs when the visual encoder prioritizes generic necrotic textures over class-specific semantic nuances, causing Leaf Mold features to overlap with visually similar conditions like early-stage Early Blight. To mitigate this, future work could incorporate Contrastive Prototype Learning to sharpen decision boundaries by pulling LLM semantic priors closer to their respective visual prototypes. Additionally, implementing an Uncertainty-aware Decoder would allow the model to dynamically weigh semantic information when visual evidence is ambiguous.
This instance highlights that while the heatmap indicates where the model is looking, the visual features alone may be insufficient for a confident diagnosis in complex scenarios. Therefore, enhancing the learning of “fine-grained” features and optimizing the cross-modal fusion mechanism—especially by strengthening the interaction between visual content and domain-specific textual knowledge—is crucial. Future work will focus on introducing structured semantic networks to better align these heterogeneous cues, ultimately reducing uncertainty and improving the system’s robustness in real-world agricultural applications.
5.4. Practical Considerations for Field Deployment
In actual field applications, relying on the commercial APIs can be difficult due to unstable internet in rural areas. To solve this, we are looking at replacing the online API with lightweight, open-source models like Llama-3-8B or DeepSeek-V3. Since these Large Language Models have already absorbed a vast range of general and domain-specific knowledge during their extensive pre-training, they can function as robust, offline diagnostic engines. By compressing these models to run directly on local hardware like an NVIDIA Jetson, the system can provide expert-level reasoning without a persistent connection. This not only cuts down the cost of API calls but also ensures that the pre-embedded knowledge is immediately accessible to farmers, making the diagnosis faster and more reliable.
6. Conclusions
This study presents LPDiag, a novel Multimodal Prototype Attention Network designed to bridge the gap between high-level linguistic reasoning and low-level visual perception in tomato leaf disease diagnosis. By synergistically integrating Large Language Models (LLMs) with a Prototype-Guided Attention (PGA) module, the framework achieves robust alignment between visual symptoms and semantic knowledge, yielding a state-of-the-art mean accuracy of 98.83%. Beyond performance, the system enhances diagnostic trustworthiness; qualitative heatmap analysis confirms that the PGA module serves as a “semantic anchor,” shifting focus from general morphology to fine-grained lesion characteristics. Furthermore, the inclusion of uncertainty estimation enables the model to quantify predictive confidence, a critical feature for flagging out-of-distribution samples and ensuring reliability in unpredictable open-world agricultural environments.
Despite these advancements, future work will transition from general-purpose LLMs to lightweight, domain-specific small language models (SLMs) and integrate knowledge graphs to improve on-device efficiency. By mapping prototype vectors to explicit botanical ontologies, we aim to evolve the framework from correlation-based learning toward knowledge-driven causal reasoning. Ultimately, this research offers a scalable, explainable framework for knowledge-aware AI, marking a significant step toward autonomous and intelligent plant protection systems.
Author Contributions
Conceptualization, H.D., D.F. and X.Q.; methodology, H.D. and Q.C.; software, H.D., X.Q. and D.F.; validation, D.F. and X.Q.; formal analysis, H.D. and J.Y.; investigation, J.L.; resources, M.H.; data curation, D.F., H.D. and X.Q.; writing—original draft, X.Q. and H.D.; writing—review and editing, H.D., M.H., J.Y., J.L., C.Y., R.L., Q.C. and R.C.; visualization, D.F. and X.Q.; supervision, R.L., M.H., C.Y., Q.C. and R.C.; project administration, Q.C. and R.C.; funding, R.L., Q.C. and R.C. All authors have read and agreed to the published version of the manuscript.
Funding
This work was supported in part by the Fujian Key Laboratory of Big Data Application and Intellectualization for Tea Industry (Wuyi University) (Grant No. FKLBDAITI202502), the National Natural Science Foundation of China (Grant No. 62301160), the Natural Science Foundation of Fujian Province (Grant No. 2022J01607), the Nanping Science and Technology Plan Project (Grant No. N2024Z014), and the Guiding project of the Fujian Provincial Science and Technology Plan (Grant Nos. 2025Y0066, 2025Y0069).
Data Availability Statement
The original data presented in this study were sourced from three publicly available datasets: (1) The PlantVillageTomato and TaiwanTomato Datasets: open-access datasets of tomato leaves for disease diagnosis, available at [https://doi.org/10.17632/ngdgg79rzb.1]. (2) CCMT: open-access Dataset for Crop Pest and Disease Detection, available at [https://doi.org/10.17632/bwh3zbpkpv.1].
Acknowledgments
We acknowledge the Key Laboratory of Smart Agriculture and Forestry of Fujian Province Universities for providing the platform for data collection and analysis.
Conflicts of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as potential conflicts of interest.
References
- Piccialli, F.; Della Bruna, C.; Chiaro, D.; Qi, P.; Savoia, M. AGRIFOLD: AGRIculture Federated learning for Optimized Leaf disease Detection. Expert Syst. Appl. 2025, 289, 128371. [Google Scholar] [CrossRef] [Scilit]
- Mensah, P.K.; Akoto-Adjepong, V.; Adu, K.; Ayidzoe, M.A.; Bediako, E.A.; Nyarko-Boateng, O.; Boateng, S.; Donkor, E.F.; Bawah, F.U.; Awarayi, N.S.; et al. CCMT: Dataset for crop pest and disease detection. Data Brief 2023, 49, 109306. [Google Scholar] [CrossRef] [Scilit]
- Hughes, D.; Salathé, M. An open access repository of images on plant health to enable the development of mobile disease diagnostics. arXiv 2015. [Google Scholar] [CrossRef] [Scilit]
- Huang, M.; Chang, Y. Dataset of Tomato Leaves, Version 1; Mendeley Data. 2020. Available online: https://data.mendeley.com/datasets/ngdgg79rzb/1 (accessed on 11 December 2025). [CrossRef]
- Nawaz, M.; Nazir, T.; Javed, A.; Masood, M.; Rashid, J.; Kim, J.; Hussain, A. A robust deep learning approach for tomato plant leaf disease localization and classification. Sci. Rep. 2022, 12, 18568. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Zhao, S.; Peng, Y.; Liu, J.; Wu, S. Tomato leaf disease diagnosis based on improved convolution neural network by attention module. Agriculture 2021, 11, 651. [Google Scholar] [CrossRef] [Scilit]
- Li, M.; Zhou, G.; Chen, A.; Li, L.; Hu, Y. Identification of tomato leaf diseases based on LMBRNet. Eng. Appl. Artif. Intell. 2023, 123, 106195. [Google Scholar] [CrossRef] [Scilit]
- Assaduzzaman, M.; Bishshash, P.; Nirob, M.A.S.; Al Marouf, A.; Rokne, J.G.; Alhajj, R. XSE-TomatoNet: An explainable AI based tomato leaf disease classification method using EfficientNetB0 with squeeze-and-excitation blocks and multi-scale feature fusion. MethodsX 2025, 14, 103159. [Google Scholar] [CrossRef] [Scilit]
- Barman, U.; Sarma, P.; Rahman, M.; Deka, V.; Lahkar, S.; Sharma, V.; Saikia, M.J. Vit-SmartAgri: Vision transformer and smartphone-based plant disease detection for smart agriculture. Agronomy 2024, 14, 327. [Google Scholar] [CrossRef] [Scilit]
- Nishankar, S.; Pavindran, V.; Mithuran, T.; Nimishan, S.; Thuseethan, S.; Sebastian, Y. ViT-RoT: Vision Transformer-Based Robust Framework for Tomato Leaf Disease Recognition. AgriEngineering 2025, 7, 185. [Google Scholar] [CrossRef] [Scilit]
- Ametefe, D.S.; Sarnin, S.S.; Ali, D.M.; Caliskan, A.; Caliskan, I.T.; Aliu, A.A.; John, D. Enhancing leaf disease detection accuracy through synergistic integration of deep transfer learning and multimodal techniques. Inf. Process. Agric. 2024, 15, 279–299. [Google Scholar] [CrossRef] [Scilit]
- Karimanzira, D. Context-aware tomato leaf disease detection using deep learning in an operational framework. Electronics 2025, 14, 661. [Google Scholar] [CrossRef] [Scilit]
- Djenouri, Y.; Belbachir, A.N.; Michalak, T.; Belhadi, A.; Srivastava, G. A Knowledge-Enhanced Object Detection for Sustainable Agriculture. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2024, 12, 279–299. [Google Scholar] [CrossRef] [Scilit]
- Awais, M.; Alharthi, A.H.S.A.; Kumar, A.; Cholakkal, H.; Anwer, R.M. AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning. In Proceedings of the 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Tucson, AZ, USA, 28 February–4 March 2025; pp. 5687–5696. [Google Scholar] [CrossRef] [Scilit]
- Wang, Y.; Wang, F.; Chen, W.; Lv, B.; Liu, M.; Kong, X.; Zhao, C.; Pan, Z. A large language model for multimodal identification of crop diseases and pests. Sci. Rep. 2025, 15, 21959. [Google Scholar] [CrossRef] [Scilit]
- Wen, H.; Shi, H.; Yu, J.; Fan, Z.; Dai, H.; Jiang, L.; Song, Q. An intelligent diagnostic method for porcine gastrointestinal infectious diseases based on multimodal AI and large language model. Front. Vet. Sci. 2025, 12, 1660745. [Google Scholar] [CrossRef] [Scilit]
- Yu, P.; Lin, B. A Framework for Agricultural Intelligent Analysis Based on a Visual Language Large Model. Appl. Sci. 2024, 14, 8350. [Google Scholar] [CrossRef] [Scilit]
- Zheng, Z.; Wang, Y.; Huang, Y.; Song, S.; Yang, M.; Tang, B.; Xiong, F.; Li, Z. Attention heads of large language models. Patterns 2025, 6, 101176. [Google Scholar] [CrossRef] [Scilit]
- Islam, M.S.; Sultana, S.; Farid, F.A.; Islam, M.N.; Rashid, M.; Bari, B.S.; Hashim, N.; Husen, M.N. Multimodal hybrid deep learning approach to detect tomato leaf disease using attention based dilated convolution feature extractor with logistic regression classification. Sensors 2022, 22, 6079. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Hossain, S.; Tanzim Reza, M.; Chakrabarty, A.; Jung, Y.J. Aggregating different scales of attention on feature variants for tomato leaf disease diagnosis from image data: A transformer driven study. Sensors 2023, 23, 3751. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Radford, A.; Kim, J.W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning, PmLR, Virtual, 18–24 July 2021; pp. 8748–8763. [Google Scholar]
- Li, J.; Li, D.; Xiong, C.; Hoi, S. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Proceedings of the International Conference on Machine Learning, PMLR, Virtual/Online, 17–21 July 2022; pp. 12888–12900. [Google Scholar]
- Deng, S.; Zhu, J.; Hu, Y.; He, M.; Xia, Y. Tomato Leaf Disease Identification Framework FCMNet Based on Multimodal Fusion. Plants 2025, 14, 2329. [Google Scholar] [CrossRef] [Scilit]
- Liu, W.; Wu, G.; Wang, H.; Ren, F. Cross-Modal Data Fusion via Vision-Language Model for Crop Disease Recognition. Sensors 2025, 25, 4096. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Peng, D.; Li, W.; Zhao, H.; Zhou, G.; Cai, C. Recognition of tomato leaf diseases based on DIMPCNET. Agronomy 2023, 13, 1812. [Google Scholar] [CrossRef] [Scilit]
- Chen, C.; Du, Y.; Fang, Z.; Wang, Z.; Luo, F.; Li, P.; Yan, M.; Zhang, J.; Huang, F.; Sun, M.; et al. Model composition for multimodal large language models. arXiv 2024, arXiv:2402.12750. [Google Scholar] [CrossRef] [Scilit]
- Baldassini, F.B.; Nguyen, H.H.; Chang, C.C.; Echizen, I. Cross-Attention watermarking of Large Language Models. In ICASSP 2024–2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea, 14–19 April 2024; IEEE: New York, NY, USA, 2024; pp. 4625–4629. [Google Scholar] [CrossRef] [Scilit]
- Liu, X.; Zhu, Y.; Lan, Y.; Yang, C.; Qiao, Y. Safety of multimodal large language models on images and texts. arXiv 2024, arXiv:2402.00357. [Google Scholar] [CrossRef] [Scilit]
- Sharma, U.; Artacho, B.; Savakis, A. GourmetNet: Food Segmentation Using Multi-Scale Waterfall Features with Spatial and Channel Attention. Sensors 2021, 21, 7504. [Google Scholar] [CrossRef] [Scilit]
- Lee, D.; Jang, K.; Cho, S.Y.; Lee, S.; Son, K. A Study on the Super Resolution Combining Spatial Attention and Channel Attention. Appl. Sci. 2023, 13, 3408. [Google Scholar] [CrossRef] [Scilit]
- Sun, Y.; Jiang, M.; Guo, H.; Zhang, L.; Yao, J.; Wu, F.; Wu, G. Multiscale Tea Disease Detection with Channel–Spatial Attention. Sustainability 2024, 16, 6859. [Google Scholar] [CrossRef] [Scilit]
- Ye, H.; Zhou, Z.; Wang, Y.; Chen, W.; Miao, L.; Li, J. M2CNet: An Infrared and Visible Image Fusion Method Based on Dual Marginal Contrastive Learning. In 2024 36th Chinese Control and Decision Conference (CCDC), Xi’an, China, 25–27 May 2024; IEEE: New York, NY, USA, 2024; pp. 396–402. [Google Scholar] [CrossRef] [Scilit]
- Zhang, Y.; Huang, S.; Zhou, G.; Hu, Y.; Li, L. Identification of tomato leaf diseases based on multi-channel automatic orientation recurrent attention network. Comput. Electron. Agric. 2023, 205, 107605. [Google Scholar] [CrossRef] [Scilit]
- Yang, F.; Wang, R.; Chen, X. SEGA: Semantic Guided Attention on Visual Prototype for Few-Shot Learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 3–8 January 2022; pp. 1056–1066. [Google Scholar]
- Deng, D.; Chen, G.; Yu, Y.; Liu, F.; Heng, P.A. Uncertainty estimation by fisher information-based evidential deep learning. In Proceedings of the International Conference on Machine Learning, PMLR, Honolulu, HI, USA, 23–29 July 2023; pp. 7596–7616. Available online: https://proceedings.mlr.press/v202/deng23b.html (accessed on 11 December 2025).
- Yoon, S.; Kim, E. Density aware evidential deep learning for out-of-distribution detection. In Proceedings of the 41st International Conference on Machine Learning, PMLR, Vienna, Austria, 21–27 July 2024; Volume 235. Available online: https://proceedings.mlr.press/v235/yoon24a.html (accessed on 11 December 2025).
- Chen, M.; Gao, J.; Xu, C. Revisiting essential and nonessential settings of evidential deep learning. IEEE Trans. Pattern Anal. Mach. Intell. 2025, 47, 8658–8673. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Sensoy, M.; Kaplan, L.; Kandemir, M. Evidential deep learning to quantify classification uncertainty. arXiv 2018, arXiv:1806.01768. [Google Scholar] [CrossRef] [Scilit]
- Dawood, T.; Chan, E.; Razavi, R.; King, A.P.; Puyol-Antón, E. Addressing deep learning model calibration using evidential neural networks and uncertainty-aware training. In 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), Cartagena, Colombia, 18–21 April 2023; IEEE: New York, NY, USA, 2023; pp. 1–5. [Google Scholar] [CrossRef] [Scilit]
- Agbelese, D.; Chaitanya, K.; Pati, P.; Parmar, C.; Mobadersany, P.; Fadnavis, S.; Surace, L.; Yarandi, S.; Ghanem, L.R.; Lucas, M.; et al. MEGAN: Mixture of Experts for Robust Uncertainty Estimation in Endoscopy Videos. In International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, Strasbourg, France, 17 October 2025; Springer Nature: Cham, Switzerland, 2025; pp. 3–13. [Google Scholar]
- Sakib, S.N.; Haque, N.; Hossain, M.Z.; Arman, S.E. PlantVillageVQA: A visual question answering dataset for benchmarking vision-language models in plant science. arXiv 2025, arXiv:2508.17117. [Google Scholar]
- Connor, R.J.; Mosimann, J.E. Concepts of independence for proportions with a generalization of the Dirichlet distribution. J. Am. Stat. Assoc. 1969, 64, 194–206. [Google Scholar] [CrossRef]
- Maaten, L.V.D.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 2008, 9, 2579–2605. [Google Scholar]
- Gal, Y.; Ghahramani, Z. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the International Conference on Machine Learning, PMLR, New York, NY, USA, 19–24 June 2016; pp. 1050–1059. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.









