Application of Deep Learning Technology in Monitoring Plant Attribute Changes

Abstract

With the advancement of remote sensing imagery and multimodal sensing technologies, monitoring plant trait dynamics has emerged as a critical area of research in modern agriculture. Traditional approaches, which rely on handcrafted features and shallow models, struggle to effectively address the complexity inherent in high-dimensional and multisource data. In contrast, deep learning, with its end-to-end feature extraction and nonlinear modeling capabilities, has substantially improved monitoring accuracy and automation. This review summarizes recent developments in the application of deep learning methods—including CNNs, RNNs, LSTMs, Transformers, GANs, and VAEs—to tasks such as growth monitoring, yield prediction, pest and disease identification, and phenotypic analysis. It further examines prominent research themes, including multimodal data fusion, transfer learning, and model interpretability. Additionally, it discusses key challenges related to data scarcity, model generalization, and real-world deployment. Finally, the review outlines prospective directions for future research, aiming to inform the integration of deep learning with phenomics and intelligent IoT systems and to advance plant monitoring toward greater intelligence and high-throughput capabilities.

1. Introduction

Monitoring plant trait dynamics involves continuously tracking morphological, physiological, biochemical, and phenotypic characteristics of plants across varying temporal and environmental conditions [1,2,3]. This process integrates multisource data from remote sensing, proximal sensing, and experimental observations to support precision agriculture, ecological management, and plant phenotyping [4,5].

Key traits—including plant height, the leaf area index (LAI), chlorophyll content, stomatal conductance, and phenological stages—serve as critical indicators of plant growth, health status, and stress responses [6,7,8,9]. These traits not only reflect genotype–phenotype–environment interactions but also guide yield forecasting and sustainable crop management [10,11,12]. Their dynamic characterization supports precision fertilization, irrigation scheduling, and ecosystem assessments under climate variability [13].

Despite their significance, conventional monitoring approaches are often labor-intensive, low in spatiotemporal resolution, and susceptible to subjectivity. Classical machine learning methods, such as SVM, LDA, KNN, and random forests, have been explored to automate trait classification and prediction tasks [14,15,16,17,18,19]. While effective on small-scale or low-dimensional datasets, these models face limitations in representing nonlinearities, integrating multimodal data, and scaling to high-throughput applications [20,21].

Recent advances in deep learning offer new opportunities to address these challenges [22,23,24,25,26]. Unlike traditional approaches requiring handcrafted features, deep learning models—particularly convolutional neural networks (CNNs), recurrent neural networks (RNNs), Transformers, and generative adversarial networks (GANs)—can automatically extract hierarchical features from raw data and generalize across diverse conditions [27,28,29,30]. These architectures have demonstrated superior performance in stress detection, yield estimation, pest and disease identification, and phenotypic prediction tasks [31,32,33,34,35,36,37,38].

For example, Wang et al. [39] developed a CNN-based deep neural network for genomic prediction that integrates omics data and employs techniques like batch normalization and dropout to enhance generalization. Similarly, transfer learning approaches using pretrained networks (e.g., VGG and ResNet) have shown strong adaptability in plant phenotyping from UAV and satellite imagery [40,41]. These methods reduce training cost and improve model robustness across spatial scales and crop types.

Given the growing need for automated, scalable, and accurate trait monitoring systems, deep learning has emerged as a transformative tool to model complex plant–environment interactions, fuse heterogeneous data sources, and improve prediction under real-world uncertainties [27].

The remainder of this paper is organized as follows: Section 2 introduces commonly used deep learning models for plant trait monitoring, including CNNs, LSTMs, RNNs, Transformers, and GANs. Section 3 provides a systematic review of specific applications of deep learning in this field, covering growth monitoring and yield prediction, pest and disease diagnosis, phenotypic and genetic trait analysis, and environmental impact assessments. Section 4 discusses the current advantages, challenges, and future research directions of deep learning techniques in plant trait monitoring. Section 5 presents the conclusions. Figure 1 illustrates the main structure of this paper.

2. Deep Learning Models for Plant Attribute Monitoring

2.1. Convolutional Neural Networks (CNNs)

Convolutional neural networks (CNNs) originated in the 1980s and were initially used for handwritten digit recognition in postal code reading. With the evolution of deep learning, CNNs have gained prominence in agriculture for their ability to autonomously extract features and deliver high performance in crop yield estimation and plant phenotypic monitoring [42,43,44].

Unlike traditional feedforward networks, CNNs employ convolutional, pooling, and fully connected layers to capture local spatial features and reduce dimensionality. Convolutional layers apply sliding filters to generate feature maps, while pooling layers enhance robustness through downsampling [45]. CNNs can process data in one, two, or three dimensions, making them suitable for integrating heterogeneous sources such as meteorological and soil data.

In trait analysis and yield prediction, CNN variants include 2D-CNNs, 3D-CNNs, and YOLO [46]. Specifically, 2D-CNNs extract spatial–spectral features via 2D kernels [47], whereas 3D-CNNs incorporate temporal dynamics for multitemporal prediction [42]. YOLO enables real-time detection by dividing images into regions and parallelizing inference [48]. For instance, a YOLOv3-based soybean yield prediction model matched ResNet101’s performance with only half the parameters, demonstrating the efficacy of lightweight CNNs [43].

CNNs have become indispensable for spatial and spatiotemporal feature learning in plant monitoring due to their generalization capability and architectural flexibility. Zhuang et al. [44] introduced an enhanced Faster R-CNN with DCNv2 and optimized anchors for maize seedling detection (Figure 2), achieving 99.53% accuracy under complex field conditions. This was the first integration of a two-stage deep learning model with an orbital high-throughput platform, enabling dynamic monitoring of maize emergence and leaf development. Field trials across 52 maize varieties confirmed the system’s robustness and generalizability (R² = 0.997 and RMSE = 43.382).

2.2. Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM)

Recurrent neural networks (RNNs) are designed to model sequential data by retaining information from previous time steps via recurrent connections, enabling effective learning of temporal dependencies in variable-length sequences [49]. They have been applied to diverse domains such as handwriting recognition, image captioning, machine translation, crop yield forecasting, and phenotypic time-series analysis [50,51,52]. However, traditional RNNs are prone to vanishing or exploding gradients during training [53], hindering long-term dependency modeling—which is particularly problematic in monitoring extended plant phenotypic changes. Moreover, the fixed-length input requirement often leads to truncation or padding, risking information loss and reduced model efficiency.

To overcome these limitations, Hochreiter and Schmidhuber [54] introduced Long Short-Term Memory (LSTM) networks, which incorporate input, forget, and output gates to control information flow, allowing the retention and selective updating of relevant temporal features [50]. Enhanced variants such as the attention-based LSTM (ALSTM) proposed by Tian et al. [55] further improve sequence modeling through attention mechanisms and regularization layers.

In plant trait dynamics monitoring, RNNs—particularly LSTMs—demonstrate superior performance in modeling crop growth stages, LAI variation, and moisture stress by capturing nonlinear temporal patterns across multisource inputs (e.g., meteorological, spectral, and soil data). These models offer robustness in handling irregular-length sequences and adaptability through transfer learning, enabling accurate estimation of yield, biomass, and phenotypic traits even under limited or imbalanced data conditions [56]. Compared to traditional statistical methods, LSTM-based models exhibit higher predictive accuracy and better generalizability across regions and crops.

Applications of RNNs and LSTMs in this field include stress response modeling and multitemporal climate adaptation analysis [57,58]. For instance, Ali et al. [59] developed a multimodal LSTM framework integrating molecular and phenotypic features to predict drought stress (Figure 3), achieving 97% accuracy—outperforming the RNN (94%), Gradient Boosting (96%), and SVM (82%). This study, covering 101 plant genera, underscores LSTM’s effectiveness in early stress detection and crop resilience management, offering valuable insights into precision agriculture and food security enhancement.

2.3. Transformer

The Transformer, which was proposed by Vaswani et al. [60], is a deep learning architecture that replaces traditional RNN and CNN structures with self-attention mechanisms to model global dependencies. Its encoder–decoder structure allows for efficient sequence modeling and has achieved significant success in natural language processing, computer vision, and time-series forecasting. In agriculture, its application is expanding to areas such as remote sensing and plant phenotyping [61].

In crop monitoring and trait analysis, Transformer variants—including the Vision Transformer and the Swin Transformer—offer notable advantages. They effectively capture long-range temporal and spatial dependencies in multitemporal imagery and growth sequences, making them well suited for UAV/satellite-based plant monitoring. Compared to CNNs, their multi-head self-attention mechanism provides greater flexibility in handling complex field conditions like occlusion and variable lighting while also facilitating multimodal data fusion (e.g., hyperspectral, RGB, and meteorological inputs) [62]. Moreover, their strong pretraining and transfer learning capabilities reduce reliance on large labeled datasets and enhance generalizability.

Transformers have been applied to weed detection, pest and disease recognition [63], and yield prediction. Rahman et al. [64] pioneered the use of Transformers in drought phenotyping of blueberries by applying them to plant spectral reflectance data (Figure 4). Their model introduced a patch-based dimensionality reduction module to handle high-dimensional spectral inputs and employed a Transformer encoder with multi-head self-attention and an MLP regressor to estimate leaf water content (LWC). The model achieved an R² of 0.81 in cross-cultivar prediction—surpassing traditional and deep learning baselines—demonstrating its robustness and adaptability. Emerging Transformer-like models such as Mamba have also shown promise in agricultural domains and merit further exploration [65,66].

2.4. Generative Models

Data scarcity and limited diversity are major challenges in applying deep learning to plant trait dynamics monitoring, particularly due to high variability in environmental conditions, crop types, and disease or pest manifestations. Generative models, especially generative adversarial networks (GANs) and variational autoencoders (VAEs), offer powerful solutions by synthesizing realistic and diverse training data to improve model robustness and generalization [67].

GANs comprise a generator and a discriminator trained adversarially, enabling the creation of synthetic images that closely mimic real-world data [68]. In plant disease and pest detection, GANs enhance datasets by simulating varied lesion patterns, pest morphologies, lighting conditions, and complex backgrounds. In contrast, VAEs learn latent data distributions to generate structurally coherent and continuously interpolated samples [69], making them particularly suitable for modeling subtle phenotypic variations and addressing class imbalance [70].

Integrating GANs and VAEs into data augmentation pipelines reduces reliance on large annotated datasets, captures fine-grained crop-specific traits, and enhances scalability across diverse geographic and climatic contexts [71,72]. Although agricultural applications are still emerging, successes in biomedical imaging and natural image generation validate their effectiveness in improving data quality and model performance [73,74,75].

For instance, Debnath et al. [76] proposed a GAN–U-Net hybrid framework that integrates multisource vegetation indices and deep residual regression for high-precision crop area and biomass segmentation. The approach incorporates Taylor expansion and the Coot optimization algorithm to accelerate convergence and stabilize segmentation performance. Tested on the DeepWheat dataset, this model significantly outperformed conventional deep learning baselines in emergence and biomass estimation, demonstrating the potential of combining generative models with swarm intelligence optimization for phenotypic monitoring.

Table 1. Deep learning models applied to plant trait dynamics monitoring.

Model Type	Main Characteristics	Advantages	Disadvantages	Algorithms and Applications	Potential Scenarios
CNN	Multi-layer convolution and pooling structures that automatically extract local spatial features	Excellent performance in image classification and object detection; strong feature extraction capacity	Limited in capturing long-range dependencies; less effective for temporal modeling	ResNet, DenseNet, and VGG; maize emergence rate and leaf emergence speed monitoring [44]	Plant pest and disease identification, leaf phenotyping, and crop distribution classification
RNN	Processes sequential data through recurrent connections	Strong sequence modeling capabilities; suited for temporal dependencies	Prone to vanishing/exploding gradients; limited in capturing long-term dependencies	Basic RNN; accurate regression of lettuce plant height from single-view sparse 3D point clouds [77]	Crop growth modeling and yield prediction
LSTM	Adds gating mechanisms to RNNs to enhance long-term memory	Effectively learns complex dynamics in long sequences	Computationally intensive; sensitive to parameters	Stacked LSTM; Poplar seedling variety and drought stress classification [57]	Multitemporal yield prediction, phenological monitoring, and stress analysis
Transformer	Self-attention mechanism capturing global dependencies	Highly efficient parallel computation; supports multimodal data fusion	Large model size; requires substantial training data	Vision Transformer (ViT); Hyperspectral imagery for blueberry drought phenotyping [64]	Hyperspectral feature extraction and multimodal phenotypic prediction
GAN	Generates high-quality synthetic data via adversarial training	Mitigates sample scarcity; enhances model robustness	Training instability; difficult generator–discriminator balance	Pix2Pix GAN; Novel leaf disease augmentation model for plant disease diagnosis [78]	Pest and disease sample expansion, data augmentation, and rare phenotype simulation
VAE	Probabilistic generative model learning latent representations	Generates diverse samples; interpretable latent variables	Lower sample clarity compared to GANs	$\beta$-VAE; dimensionality reduction and generation of hyperspectral plant phenotyping data [79]	Phenotypic representation, anomaly detection, and phenomics data generation
GRU	Simplified LSTM variant	Fewer parameters; efficient training	Slightly lower expressiveness than LSTM	GRU network; optimized diagnosis of yellow vein mosaic virus [80]	Pest and disease monitoring and continuous growth dynamics prediction
Ensemble DL	Integrates multiple models to improve robustness	High accuracy; good generalization	High computational resource consumption	Ensemble CNN + LSTM; automated genotype classification and dynamic phenotyping recognition during plant growth [81]	Cross-scale yield prediction and stress adaptation modeling

summarizes the main characteristics, strengths and weaknesses, representative algorithms and case studies, and potential application scenarios of deep learning models used in plant attribute monitoring.

3. Applications of Deep Learning in Monitoring Plant Attributes

3.1. Growth Monitoring and Yield Prediction

Crop yield forecasting aims to quantitatively estimate crop quantity or quality over a defined future period by integrating multisource data on climate, soil, crop varieties, and management practices [82]. It plays a vital role in guiding agricultural production, policy formulation, and market regulation. Amid rising global food demand—which is projected to increase by 60% by 2050—and mounting challenges from climate change and geopolitical instability, accurate yield prediction is essential for ensuring food security and production resilience [83,84].

Traditional forecasting models struggle to handle the nonlinear interactions, high-dimensional features, and incomplete data inherent in complex agroecosystems [85]. Deep learning, with its multilayer architectures and end-to-end modeling capabilities, offers powerful solutions to these challenges. It has been successfully applied to major crops such as maize [86,87], soybeans [86,88], rice [89,90], wheat [56,91,92], tomatoes [93,94], and lettuce [95,96].

Chu et al. [89] proposed a hybrid model integrating dual multilayer perceptrons and an RNN to extract spatial–temporal features for rice yield prediction, achieving low MAE and RMSE across both growing seasons. The model demonstrated rapid convergence and strong generalization over 81 counties. Khan et al. [86] compared DNN, 1D-CNN, and GRU networks for maize and soybean yield estimation using GPP data, with the 1D-CNN achieving R² = 0.864 (maize) and 0.750 (soybean). Their transfer learning strategy enabled cross-crop and cross-region adaptation, reaching R² = 0.903 and 0.931 for crop-type and climate zone transfers, respectively—highlighting the feasibility of remote sensing-based yield prediction under data-scarce conditions (see Figure 5).

Chakraborty et al. [97] developed a U-Net-based segmentation model for quantifying almond flower density from UAV RGB imagery, enabling early yield prediction. The model outperformed the Enhanced Bloom Index (EBI) baseline (R² = 0.78 vs. 0.14) and maintained robustness across three years of multi-location data. By validating against diverse meteorological conditions and using five-fold cross-validation, the study provided a scalable method for floral monitoring and laid the foundation for cross-crop model transfer.

3.2. Disease and Pest Diagnosis

Plant diseases and pests—which are caused by fungi, bacteria, viruses, nematodes, and insects—pose serious biotic stress throughout the crop life cycle, leading to physiological disorders and yield losses [98,99,100]. Their complexity, diversity, and spatial variability present major challenges for accurate identification, particularly under complex field conditions [101]. To facilitate automation, deep learning-based diagnosis tasks are typically decomposed into classification (“what”), detection (“where”), and segmentation (“how”) subtasks [102,103], as illustrated in Figure 6.

Traditional methods (e.g., manual inspection and rule-based image processing) suffer from limited feature representation, sensitivity to environmental variation, and poor adaptability [104,105]. In contrast, deep learning approaches—especially CNNs, GANs, and Transformers—offer superior feature learning, cross-task transferability, and robustness under limited data through techniques such as data augmentation, pretraining, and multimodal fusion [106,107].

At present, a variety of deep learning strategies have been developed to address different monitoring scenarios and data characteristics.

Close-range imagery with deep learning has achieved high classification and segmentation accuracy. CNN-based models (e.g., VGG and ResNet) exceed 95% accuracy in leaf disease classification [108], while segmentation networks such as U-Net and Mask R-CNN enable lesion quantification. Xiong et al. [109] proposed AISA (with MobileNet + improved GrabCut + the excess green index) to achieve automated leaf segmentation, reaching 98% top-three accuracy. Wang et al. [110] developed DUNet (DeepLabV3+ + U-Net) for cucumber disease severity grading, achieving 92.85% accuracy and high robustness under complex field conditions (Figure 7). Recent advances have demonstrated that integrating UAV-captured imagery with deep learning models—such as dual-branch CNN-attention frameworks—can significantly improve the accuracy and robustness of rice lodging detection under complex field conditions [111].

Field-scale monitoring using UAV imagery enables spatial mapping and dynamic tracking of diseases and pests in major crops (e.g., rice, wheat, and maize) [112,113,114]. Deep models—such as 1D-/3D-CNNs, RNNs, and Transformers—extract spatial–spectral–temporal features from RGB, multispectral, and hyperspectral UAV data for severity grading, lesion extraction, and risk prediction [115].

IoT-based multimodal monitoring further enables real-time risk forecasting by integrating environmental sensor data (e.g., temperature, humidity, and CO₂) with deep learning models [116]. For example, Sharma et al. [117] deployed a wireless sensor network combined with fuzzy logic to identify pest/disease outbreaks in rice and millet fields. Cai et al. [118] integrated IoT sensing with neural networks for vegetable pest warning, achieving 98% accuracy and an F1-score of 0.97 on 761 samples.

Generative and transfer learning techniques help mitigate small or imbalanced datasets. GANs and VAEs synthesize lesion images, while transfer learning fine-tunes pretrained models [119]. Mallick et al. [120] built a real-time MobileNetV3-based SoC platform for soybean/mustard pest diagnosis, achieving >93% accuracy with 24× inference speedup and 19% energy savings. Wang et al. [121] introduced SCM-BDC, a causal feature extraction–based zero-shot model for pest/disease classification, improving H-scores by 5.2% and 1.4% on APTV99 and ADTV68, respectively (Figure 8).

Lightweight deployment and mobile monitoring further enable real-time detection on edge devices. Model compression (e.g., quantization and pruning) enhances performance on smartphones and UAVs. Mallick et al. [120] developed an Android app linked to an SoC inference platform for high-efficiency mobile detection. Chen et al. [122] introduced an AR glasses–based rice pest monitoring system using an RDP detector (YOLOv5X + CLIP), achieving 87.4% mAP and 86.5% recall—14.6% higher precision than vision-only models—highlighting the potential of multimodal fusion and wearable AI in smart agriculture.

3.3. Phenotyping and Genetic Trait Analysis

A plant phenotype refers to the set of observable traits resulting from the interaction between the genotype and the environment, encompassing morphological, physiological, biochemical, and molecular characteristics [4]. Plant phenomics integrates multimodal imaging—including RGB, thermal infrared, multispectral, hyperspectral, X-ray, and LiDAR—with deep learning and genetic approaches to enable high-throughput, non-destructive, dynamic monitoring of phenotypes across multiple scales, accelerating the understanding of genotype–phenotype interactions and advancing crop improvement [2].

In digital plant phenomics, deep learning has become a key technology for addressing the high dimensionality, complexity, and heterogeneity of phenotypic data. CNNs, RNNs, and their variants support automatic extraction, classification, and regression prediction of phenotypic traits and are widely applied in stress detection, trait quantification, and gene function discovery [123].

3.3.1. Automatic Extraction of Phenotypic Traits from Images

Traditional phenotyping relies on manual measurements, which are time-consuming and highly subjective. High-throughput imaging has enabled the acquisition of RGB, multispectral, thermal infrared, and LiDAR data; however, extracting features from large-scale unstructured imagery remains a major bottleneck [79]. Figure 9 illustrates common computer vision tasks in plant phenotyping research. Leveraging end-to-end feature learning, CNNs have shown strong performance in tasks such as leaf area measurement, fruit counting, and canopy density estimation. Among them, VGGNet is particularly well suited for extracting high-resolution leaf phenotypes [124]; ResNet utilizes residual connections to capture fine-grained morphological variations [125]; and Inception applies parallel multiscale convolutions, facilitating the analysis of complex structures.

Integrating transfer learning with Inception–ResNet architectures can enhance the accuracy of multimodal image-based phenotypic recognition [79]. Mask R-CNN has been applied to single-plant segmentation, fruit counting, and 3D point cloud reconstruction. For example, Bheemanahalli et al. [126] used Mask R-CNN to automatically extract sorghum stomatal phenotypic traits and combined the results with genome-wide association studies to reveal their genetic basis. Atkins et al. [127] employed Cascade Mask R-CNN for Arabidopsis silique detection and segmentation, supporting QTL analysis.

Given the temporal dependency of phenotypes, combining CNNs with RNNs (LSTM/GRU) or Transformers enables modeling of growth dynamics. For instance, Namin et al. [81] developed a CNN-LSTM framework to extract dynamic growth features, improving genotype classification accuracy from 76.8% to 93.0%, significantly outperforming traditional SVM and CNN-CRF methods.

Multimodal image fusion further improves phenotypic recognition precision. Shuo Zhou et al. [128] developed Maize-IAS, a Mask R-CNN-based software for maize phenotyping that enables pixel-level leaf segmentation and trait measurement. Li et al. [129] created the Angle_Measurement software, which integrates YOLOv5, Canny edge detection, and the Hough transform to automatically measure cotton fruit branch angles (as shown in Figure 10). The detection accuracy was highly consistent with AutoCAD (maximum correlation 0.95) and was 500 times faster than manual measurements. Yu et al. [130] proposed the AUNet segmentation model for dynamic lettuce phenotyping, achieving an mIoU of 0.9561, outperforming multiple classical models.

3.3.2. Applications of Deep Learning in Genotype–Phenotype Association

Traditional GWAS and genomic selection approaches primarily utilize linear models, which are limited in capturing nonlinear interactions among polygenic loci, epigenetic factors, and environmental variables [131]. Deep learning models, by contrast, can autonomously extract latent features from high-dimensional omics data without explicit feature engineering [79]. CNNs, RNNs, Transformers, and graph neural networks (GNNs) have been applied to predict regulatory elements, identify cis-acting sites, and perform phenotypic regression, thereby enhancing predictive accuracy and generalizability [132].

Early models such as DeepBind and DeepSEA employed CNNs to predict transcription factor binding and chromatin accessibility, while Basset learned epigenetic features such as DNA methylation patterns [133]. Champigny et al. [133] further demonstrated the feasibility of multitrait prediction using whole-genome DNA methylation data. In parallel, integrative multi-omics learning has gained momentum, with deep learning architectures—such as multichannel CNNs and Transformer variants—capable of fusing SNPs, transcriptomics, and epigenetics for multitask learning.

For instance, Wang et al. [134] developed WheatNet, a CNN-based field phenotyping and prediction system (Figure 11), combining deep phenotypic prediction with GWAS. This approach identified key loci (e.g., Ppd-D1, Ppd-B1, and chromosome 1B) associated with wheat flowering time and revealed epistatic interactions. The study also proposed the “spike emergence rate” as a novel developmental trait, providing new insights into the genetic control of flowering phenology.

Interpretability methods such as SHAP and DeepLIFT, combined with deep networks, have enabled in silico functional variant analysis. Peleke et al. [135] built a CNN model to decode cis-regulatory sequences across four species (Figure 12), achieving 80–87% accuracy and AUC values of 0.85–0.94. Their approach—combining DeepLIFT with TF-MoDISco—extracted conserved regulatory motifs across tissues and species, confirming cross-species transferability and validating results using a multi-genotype tomato dataset.

In genomic selection, deep regression models directly map SNPs to traits, facilitating multitask breeding applications. Rairdin et al. [136] used RetinaNet to identify over 40 novel SNPs associated with SDS resistance. Wang et al. [137] proposed Cropformer, a Transformer-based model integrating SHAP interpretability, which improved trait prediction by up to 7.5% and boosted accuracy by 12.2–19.1% through multimodal fusion.

Overall, the integration of deep learning with multi-omics data is reshaping functional genomics and precision breeding by revealing complex causal links between genotype and phenotype.

3.4. Monitoring the Environmental Impact on Plants

Plant stress—triggered when environmental conditions deviate from optimal conditions—induces physiological responses that hinder growth and productivity [138]. Stressors are broadly categorized into biotic (e.g., pathogens and pests) and abiotic (e.g., drought, salinity, heat, cold, nutrient deficiency, and heavy metals) factors (Figure 13) [139,140]. Among them, abiotic stress is a major cause of yield reduction, adversely affecting morphology, physiology, and molecular processes such as photosynthesis inhibition, oxidative stress, and hormonal disruption [139,141,142,143].

With intensifying climate change, land degradation, and resource depletion, the frequency and severity of abiotic stresses are escalating, threatening sustainable agriculture and food security. Breeding stress-tolerant crops is critical yet challenging. Thus, real-time, dynamic, and accurate monitoring technologies are essential for early detection and timely mitigation [38,144,145].

Traditional methods (e.g., manual inspection and lab assays) lack scalability, consistency, and precision. Emerging technologies—such as UAVs, satellites, and ground sensors—generate high-dimensional and nonlinear data that conventional statistical and machine learning models struggle to analyze effectively. Deep learning overcomes these limitations by automatically extracting latent, nonlinear features from complex datasets, enabling early detection, severity assessments, and spatial localization of stress signals with high accuracy [38].

Recent studies have applied CNNs, RNNs, LSTMs, and GANs to classify various abiotic stress types—including drought, heat, salinity, and nutrient deficiency—using multimodal data sources such as RGB, fluorescence, thermal, multispectral, hyperspectral, and in situ sensor data [2]. These deep models enable end-to-end learning, reduce reliance on manual feature engineering, and demonstrate strong generalization across stress types and environmental conditions [123]. As a result, deep learning has become a core component of intelligent crop stress monitoring systems, supporting precision agriculture through robust, automated, and scalable solutions.

3.4.1. Water Stress Assessment

An accurate assessment of crop water stress is critical for understanding plant–environment interactions and forms the basis for precision irrigation. Traditional methods commonly rely on direct in situ measurements of soil moisture, leaf water potential, and stomatal conductance [146]. While informative, these approaches are labor-intensive, temporally and spatially limited, and often unsuitable for large-scale dynamic monitoring.

Recent advances in deep learning offer new avenues for efficient and scalable water stress assessments. With end-to-end modeling capabilities, deep neural networks can autonomously extract relevant phenotypic and environmental features from multimodal inputs, thereby improving both accuracy and generalizability. Models such as CNNs, LSTM networks, and temporal convolutional networks (TCNs) have shown promising results in soil moisture estimation, evapotranspiration prediction, water status evaluation, and stress-level classification [147]. In particular, deep learning approaches leveraging RGB, thermal infrared, multispectral, and hyperspectral imagery have demonstrated superior performance over traditional machine learning and empirical models in crops such as maize, rice, sugarcane, and chickpeas. These methods report improved classification accuracy, a reduced root mean square error (RMSE), and a higher coefficient of determination ($R^2$) across diverse growing conditions.

For example, Yu et al. [148] developed a predictive framework integrating a high-throughput phenotyping platform with deep learning, combining an Inception–residual–attention CNN with multiple recurrent neural networks (Figure 14). This approach achieved the highest Rp² (up to 0.9583) for predicting SSC, NO²⁻, pH, and Ca²⁺; reached a water stress classification accuracy of 98.86%; and, for the first time, enabled pixel-level visualization of pH distribution based on hyperspectral data, providing technical support for precision irrigation and quality management.

3.4.2. Nutrient Deficiency

Crop nutrient deficiency refers to the insufficient supply of essential elements during plant growth and development, leading to a series of physiological and morphological changes that ultimately impact yield and quality [149,150]. A nutrient stress assessment relies not only on absolute measurements but also on relative indicators to comprehensively reflect the nutritional status of both soil and plants.

Traditional nutrient monitoring methods depend on in situ chemical measurements and visual inspection, which are labor-intensive, time-consuming, and difficult to scale for high-throughput, dynamic assessments in large agricultural settings [151]. Deep learning, with its end-to-end modeling and automatic feature extraction capabilities, has significantly improved the accuracy and efficiency of nutrient deficiency detection [151]. Its applications in plant nutrient deficiency monitoring span key technical approaches such as multi-level image analysis, feature learning strategies, ensemble methods, and multimodal data fusion.

CNNs have been extensively applied to visual symptom-based crop nutrient deficiency diagnosis using RGB, near-infrared, and hyperspectral imagery. Custom-designed CNNs and ResNet-based transfer learning models have achieved over 98% accuracy in identifying nitrogen, iron, manganese, and calcium deficiencies in crops such as sorghum and rice. Ensemble learning and feature fusion have further improved generalization and robustness. Deep learning integrated with multispectral and hyperspectral data has proven effective in capturing spectral–spatial features for diagnosing nutrient stress in quinoa, cowpea, and cauliflower. Moreover, combining deep models with physiological indicators—such as chlorophyll content and photosynthetic efficiency—enhances both interpretability and practical value. For instance, Chandel et al. [152] developed an AlexNet-based nitrogen stress classification and variable-rate fertilization system, achieving an F1 score of 0.973 for three-category classification and outperforming GoogLeNet. As shown in Figure 15a, field validation demonstrated that the system reduced nitrogen fertilizer use by 37.5% while maintaining stable yields, underscoring the potential of deep learning for precision nutrient management in sustainable agriculture.

3.4.3. Salinity and Heavy Metal Stress

Soil salinity and heavy metal stress are major factors limiting crop yield in global agriculture. However, the physiological responses of plants to salinity are often highly complex and variable, posing challenges for experimental design and interpretation [142,154]. Conventional detection methods—such as ion analysis, electrical conductivity measurements, and physiological index assessments—are accurate but typically time-consuming, costly, and demanding in terms of field operations, making large-scale, real-time monitoring difficult [155]. Deep learning models, with their superior feature extraction and nonlinear modeling capabilities, can directly mine complex phenotypic information from high-dimensional multimodal data, significantly improving the accuracy and efficiency of stress assessments. They have shown remarkable advantages in salinity and heavy metal stress detection, driving the rapid development of noninvasive, automated, and high-throughput monitoring technologies [28].

In salinity stress research, CNNs, LSTMs, and hybrid models have been widely applied to electrical signal time-series analysis, spectral interpretation, and phenotypic classification. Yao et al. [156] developed the SLSTM-TCNN model, which effectively predicts wheat leaf signal dynamics under varying NaCl concentrations. Combined with a NaCl stress discrimination model and a salt tolerance classification model, this approach enabled high-precision identification of salt-tolerant lines. Feng et al. [157] integrated hyperspectral imaging with deep learning for precise segmentation and quantitative analysis of okra phenotypes, revealing strong correlations between spectral data and chlorophyll content, photosynthesis, and ion accumulation under stress conditions (r > 0.7). Additionally, Khotimah et al. [158] proposed the SC-CAN network, which incorporates spectral convolution and channel attention mechanisms to enable early diagnosis of wheat salt stress, achieving an average F1 score of 83.03.

In heavy metal stress detection, deep learning has also demonstrated superior performance. For example, Deng et al. [153] addressed the severe impact of salinity stress on soybean seedling growth and photosynthetic function by proposing a noninvasive detection method that combines a deep CNN network with chlorophyll fluorescence imaging (ChlF) (Figure 15b). The study collected image data of three key chlorophyll fluorescence parameters (Fv/Fm, Y(NO), and Inh) in soybean seedlings subjected to different salinity levels and systematically evaluated the performance of six classical CNN models (AlexNet, GoogLeNet, ResNet50, ShuffleNet, SqueezeNet, and MobileNetv2) in salinity stress recognition. Furthermore, a feature fusion strategy was implemented to enhance recognition accuracy by integrating multisource fluorescence image information. The experimental results showed that fusing the three ChlF image features extracted from the average pooling layer of ResNet50 and classifying them using an SVM classifier increased the recognition accuracy to 98.61%. Compared with traditional visual assessments and SPAD measurements, the deep learning approach demonstrated higher resolution and consistency across five salinity stress levels. The study also employed UMAP visualization to intuitively illustrate the discriminative power of high-dimensional deep features under different salinity conditions, supporting interpretability in deep learning models.

Overall, deep learning, through end-to-end learning, feature selection, and multisource data integration, provides high-accuracy and scalable solutions for the early detection, phenotypic quantification, and grading of salinity and heavy metal stress. In the future, the deep integration of deep neural networks with hyperspectral imaging, fluorescence imaging, and UAV-based remote sensing is expected to further advance crop stress monitoring toward greater intelligence and high-throughput capacity.

3.4.4. Temperature Stress

Temperature fluctuations directly impact chemical reaction rates, gas solubility, and water uptake, significantly affecting plant metabolic activities [150]. To cope with temperature stress, plants establish complex regulatory mechanisms at the cellular and metabolic levels, including hormone signaling, light signaling, circadian rhythm regulation, and reactive oxygen species homeostasis [159,160,161]. Leveraging digital technologies to accurately detect and mitigate yield losses caused by heat and cold stress is essential for safeguarding food security.

Temperature stress induces complex responses across metabolic, physiological, and epigenetic layers. Traditional assessment approaches relying on manual observation or single-sensor measurements often suffer from subjectivity and limited timeliness [141,149]. In contrast, deep learning has emerged as a key technology in this field, offering powerful feature extraction and nonlinear modeling capabilities.

In image-based applications, combining RGB, multispectral, and thermal infrared remote sensing data with deep convolutional neural networks enables precise identification of temperature stress. For cold stress, Yang et al. [162] employed a 10-layer CNN to interpret spectral reflectance of maize seedlings in the visible–near-infrared range, achieving a correlation of 0.829 between model outputs and chemical measurements and demonstrating the suitability of deep convolutional architectures for quantifying cold damage phenotypes.

For heat stress, Ryu et al. [163] compared the LS-SVM, RP-CNN, DNN, and PLS-DA algorithms in garlic, showing that LS-SVM achieved the best performance for heat stress detection using multispectral snapshot imagery. Xing et al. [164] developed the CNN-LSTM-att model by incorporating recurrent networks and attention mechanisms to improve classification of heat–drought stress in Chinese fir seedlings, reaching an accuracy of 96.91%. In cotton anther dehiscence detection, Tan et al. [165] combined YOLOv5 and Faster R-CNN models, significantly improving detection precision and adaptability, with suitability for mobile deployment.

Additionally, research has shown that multimodal data fusion is critical for improving the accuracy of temperature stress diagnosis. Integrating RGB, thermal infrared, and hyperspectral channels not only strengthens the identification of temperature anomalies but also provides a more comprehensive basis for quantifying stress across crops and growth stages. Deep learning models can fully exploit complex spectral, spatial, and temporal features, addressing the limitations of traditional machine learning in handling nonlinear relationships.

Overall, deep learning, through end-to-end feature learning and model integration, has markedly enhanced the efficiency and accuracy of temperature stress phenotyping. Looking ahead, integrating deep networks with multisource sensing and edge computing to build high-throughput, real-time intelligent monitoring systems is key to advancing precision agriculture and climate-resilient crop management. For example, Mao et al. [166] addressed the limitations of conventional freeze damage assessment methods—manual operations, time consumption, and subjectivity—by proposing and validating an automated monitoring strategy combining UAV-based multisource remote sensing with deep learning (as shown in Figure 16). The prediction accuracy reached Rp² = 0.862, and the CNN-GRU model significantly outperformed baseline methods, marking the first demonstration of the value of background information in freeze damage estimation of dense-canopy crops. Overall, the integration of deep learning with multisource sensing and edge computing will drive crop stress monitoring toward higher throughput, better real-time capabilities, and greater intelligence.

4. Discussion

4.1. Advantages

In the field of plant trait dynamics monitoring, deep learning technologies have demonstrated clear advantages over traditional machine learning and manual feature engineering approaches, owing to their distinctive algorithmic strengths and powerful feature learning capabilities. First, deep learning models possess end-to-end automatic feature extraction capacity. Through multilayer nonlinear network structures, they can autonomously discover the most discriminative feature representations from raw multisource data, substantially reducing dependence on domain expertise and manual intervention and overcoming the limitations of conventional algorithms in feature design and selection. Compared with shallow models reliant on prior knowledge, architectures such as CNNs, RNNs, and their derivatives—like LSTMs and U-Nets—can fully capture higher-order spatiotemporal correlations embedded in images, spectra, and time-series data, thereby enhancing the precision in characterizing plant physiological status, phenotypic dynamics, and stress responses.

Second, deep learning exhibits significant advantages in handling high-dimensional, large-scale, and multimodal datasets. In recent years, with the rapid proliferation of remote sensing imagery, UAV multispectral data, and high-throughput phenomics datasets, deep learning technologies have leveraged deep network architectures to integrate multichannel information, enabling high-resolution monitoring and prediction of continuous plant trait variations. Studies have shown that deep learning models substantially improve prediction accuracy and robustness in typical plant phenotyping tasks such as leaf area index estimation, canopy structure segmentation, pest and disease recognition, and drought stress detection. In many cases, their classification accuracy and detection performance have consistently exceeded 95%, far surpassing comparable traditional machine learning models.

Third, deep learning models possess strong nonlinear modeling capabilities and cross-domain transfer learning capacity, enabling them to effectively accommodate complex phenotypic variations across different crop types, observation scales, and environmental conditions. By incorporating transfer learning and pretrained models such as ResNet, VGGNet, and InceptionNet, researchers can rapidly develop high-performance models even under limited labeled sample conditions, thereby mitigating data scarcity challenges while enhancing model generalizability and applicability.

Furthermore, deep learning demonstrates a high degree of automation and intelligence in plant trait monitoring. Leveraging deep convolutional neural networks and derivative architectures for semantic segmentation and instance segmentation—such as U-Net and Mask R-CNN—these approaches achieve precise segmentation and feature extraction of leaves, fruits, and canopy structures, enabling continuous tracking and quantitative analysis of growth status, nutrient levels, and stress responses.

In summary, the primary strengths of deep learning in monitoring plant trait dynamics lie in its end-to-end feature learning, capacity to model complex nonlinear relationships, large-scale data integration, enhanced generalization through transfer learning, and automated, intelligent feature extraction. Collectively, these advantages have accelerated high-throughput phenotyping, precision agriculture decision support, and crop breeding research, providing essential technological support for efficient, accurate, and sustainable plant monitoring.

4.2. Challenges

The high predictive accuracy of deep learning models in plant trait monitoring fundamentally depends on large-scale, high-quality annotated datasets. However, unlike industrial computer vision tasks, data acquisition in agricultural settings faces considerable challenges [167]. On the one hand, the uncertainty of field environments, dynamic crop growth cycles, and the resolution heterogeneity of multimodal sensors make obtaining high-quality labeled samples extremely costly. Most existing studies rely on phenotype datasets collected over limited areas and time periods, which are insufficient to meet the training demands of deep models.

To address data scarcity, researchers have proposed several strategies: first, using data augmentation techniques such as rotation, translation, scaling, and color perturbation to enhance sample diversity and improve generalization; second, introducing transfer learning, in which networks are pretrained on large public datasets such as ImageNet and then fine-tuned for agricultural phenotyping tasks to improve performance under limited sample conditions; third, generating synthetic data with GANs to expand training datasets with simulated imagery. Although these methods partially alleviate data limitations, deep models remain highly dependent on data quality and sample diversity, especially in cross-regional and cross-growth-stage scenarios.

Deploying deep learning models in diverse agricultural environments faces severe generalization challenges. Different crop types, climatic conditions, field management practices, and sensor configurations lead to significant distributional discrepancies in input data, creating large performance gaps between training and application domains. For example, in tasks such as leaf phenotype analysis, pest and disease detection, and drought stress monitoring, convolutional and temporal models often overfit to specific acquisition conditions, making it difficult to maintain stable predictive performance across fields or growth stages. Furthermore, the “black box” nature of deep neural networks limits the interpretability of predictions in agricultural contexts, undermining the credibility of resulting decisions.

To improve model transparency and interpretability, researchers have proposed several methods: first, using feature attribution techniques such as integrated gradients, SHAP values, and DeepLIFT to quantify the contributions of input variables to model outputs; second, combining feature visualization to map intermediate layer outputs as intuitive heatmaps or feature channels, facilitating the identification of key phenotypic regions; third, exploring attention mechanisms and rule-based hybrid models to enhance interpretability of decision logic. Although these techniques improve transparency to some extent, balancing predictive accuracy with interpretability in large-scale agricultural applications remains an urgent research challenge.

The integration of deep learning with IoT and robotics is driving plant trait monitoring toward a new era of real-time, high-throughput, and automated assessments. By deploying multimodal sensors—including RGB cameras, thermal imagers, multispectral and hyperspectral cameras, and LiDAR—combined with UAV imagery and ground-based mobile platforms, researchers can continuously collect morphological and physiological crop data to dynamically sense growth status, pest and disease incidence, and stress responses. However, this integration still faces multiple challenges. First, the heterogeneous data generated by IoT devices differ substantially in temporal and spatial resolution, spectral ranges, and acquisition conditions, requiring deep models to achieve cross-modal feature alignment and consistent representation; otherwise, predictive performance can be unstable. Second, real-time monitoring and decision-making depend on efficient inference in edge computing environments, but conventional deep networks are computationally heavy and parameter-intensive, limiting deployment on low-power UAVs and robotic platforms.

To address these issues, several technical pathways have emerged: first, the development of lightweight network architectures such as MobileNet, EfficientNet, and depthwise separable convolution, which significantly reduce model size and inference latency to enable real-time operation on embedded platforms; second, the use of compression techniques such as pruning, quantization, and distillation to shrink parameter counts while retaining key representational capacity and improving deployment flexibility; third, the proposal of multimodal data fusion frameworks that leverage attention mechanisms, cross-modal alignment, and multisource feature interactions to achieve robust representation and perception in complex field scenarios. For example, some studies have combined convolutional neural networks with Transformer architectures to jointly process imagery and temporal sensor data, using attention modules to focus on critical physiological indicators.

Although these methods have demonstrated feasibility in experimental environments, their universality and stability in large-scale agricultural production remain limited. First, while lightweight models improve inference speed, they often sacrifice some phenotypic detail extraction capabilities, making it difficult to balance high predictive accuracy and real-time performance. Second, insufficient sensor calibration and environmental adaptability still cause data noise and observation bias to significantly impact model performance. Additionally, autonomous navigation and task scheduling for robots remain underdeveloped in complex field environments, imposing higher demands on path planning algorithms and deep perception networks. In the future, combining edge–cloud collaborative computing, ultra-lightweight networks, and self-supervised pretraining on multisource data holds promise to further enhance the deployment efficiency and application value of deep learning models in automated precision agriculture.

4.3. Future Perspectives

With the rapid advancement of deep learning algorithms, sensor technologies, and agricultural digitalization, plant attribute monitoring is accelerating toward a new stage characterized by high-throughput, automation, and intelligence. However, translating research innovations into real-world agricultural settings remains a significant challenge. Future research and applications are expected to achieve breakthroughs in several key areas:

Deep integration of multimodal data: Although deep learning has already demonstrated excellent performance on single-image or single-sensor data, future efforts must further fuse hyperspectral, thermal infrared, LiDAR, 3D point cloud, and multi-omics data to build a unified multimodal feature space, enabling holistic perception of plant morphology, physiology, and molecular characteristics [168]. Such integration can enhance robustness under complex field conditions.

Dynamic monitoring across spatiotemporal scales: For precision agriculture and ecological monitoring, it is essential to establish large-scale, cross-temporal monitoring systems covering scales from leaves to canopies and from individual fields to regions. Integrating deep neural networks with temporal modeling approaches such as Transformers and LSTM will facilitate continuous prediction of plant growth, stress responses, and yield dynamics. Furthermore, addressing temporal model drift—where prediction accuracy degrades over time due to changing field conditions—will be critical to maintaining long-term deployment reliability.

Interpretability and decision transparency: As deep models are increasingly adopted in agricultural production, their “black box” nature has become more prominent. Future research should prioritize improving model interpretability and transparency by employing feature attribution, attention visualization, and causal inference to enhance the comprehensibility of model outputs and strengthen user trust and adoption. In particular, agricultural end-users such as farmers, breeders, and technicians often lack data science backgrounds, making model interpretability essential for real-world adoption. Future studies should explore how interpretable outputs (e.g., visual heatmaps, rule-based summaries, and confidence scores) can be effectively translated into intuitive decision support tools. Integrating these tools into user-friendly platforms or interfaces will help bridge the gap between complex model predictions and practical field use, thereby increasing trust, usability, and decision confidence among non-expert users.

Few-shot and zero-shot learning: Given the high costs and difficulties associated with agricultural data collection and annotation, intelligent data generation and transfer learning based on GANs, VAEs, and ZSLs will become critical approaches to improving model robustness. Moreover, these techniques can mitigate data scarcity under new tasks or rare crop varieties.

Lightweight deployment and edge computing: To enable real-time inference on field-deployed UAVs, robots, and mobile devices, lightweight deep networks (e.g., MobileNet and EfficientNet), model compression, and edge–cloud collaborative computing will become core research directions, supporting the large-scale implementation of these technologies [169]. In addition, field conditions such as strong light variation, occlusion by leaves or soil, sensor disturbance (e.g., dust, vibration), and unstable network transmission must be considered during deployment. Solutions such as adaptive image enhancement, real-time calibration, multi-view fusion, and fault-tolerant system design should be explored to improve operational stability.

Bridging the gap between research and deployment: Future work should explicitly address real-world challenges such as domain shift, unseen environmental noise, hardware limitations, and long-term model drift. Building large-scale benchmark datasets under realistic field scenarios and promoting open-source deployment pipelines will help accelerate the transition of deep learning technologies from research laboratories to farms and agricultural enterprises [170,171].

5. Conclusions

Deep learning technologies, with their end-to-end feature learning capabilities, advantages in modeling nonlinear relationships, and powerful data fusion capacity, are reshaping the research paradigm of plant attribute monitoring. This review systematically summarized recent advances and state-of-the-art achievements in applying deep learning models—including CNNs, RNNs/LSTMs, Transformers, and GANs—to key applications such as plant growth monitoring, yield prediction, pest and disease identification, abiotic stress detection, and phenotypic analysis. Existing studies demonstrate that deep learning consistently outperforms traditional machine learning approaches under complex environmental conditions. Its ability to extract nonlinear features from high-dimensional multimodal datasets provides precise and reliable technical support for plant monitoring. However, challenges related to data scarcity, cross-domain generalization, model interpretability, and edge deployment remain critical constraints to large-scale adoption. In the future, integrating multimodal remote sensing imagery, high-throughput phenomics, and agricultural IoT, together with advanced techniques such as transfer learning, generative models, and causal inference, is expected to further enhance the applicability and accessibility of deep learning in plant attribute monitoring. Through interdisciplinary innovation, deep learning holds great promise for providing more robust technological support to drive the digitalization, intelligence, and greening of agricultural production, thereby promoting yield improvement, efficient resource utilization, and sustainable agricultural development.