Perspectives and Challenges of Machine Learning for Applications in Agriculture and Vegetation Using Remote Sensing

Abstract

Recent advances in Earth observation and machine learning have significantly enhanced the capacity to monitor agricultural systems and terrestrial vegetation across spatial and temporal scales. This editorial synthesizes the contributions of eleven studies published in the Special Issue ‘Machine Learning for Applications in Agriculture and Vegetation Using Remote Sensing’. These contributions highlight emerging methodological trends, as well as persistent challenges in remote sensing for agriculture and vegetation measurement and monitoring, and reflect the growing dominance of deep learning in high-resolution mapping and segmentation. An increasing importance of multi-sensor data fusion, integrating multi- and hyperspectral satellites, UAV, and environmental data, is found. Deep learning is emerging as an effective approach for retrieving key biophysical parameters such as biomass, crop height, and yield. The collected studies also emphasize the critical role of sensor characteristics and scale, particularly the trade-offs between spectral, spatial, and temporal resolution in vegetation analysis. Despite notable progress, several limitations remain. Model transferability across regions and sensors is still constrained and multi-source data integration often lacks standardized frameworks. Empirical approaches still dominate the retrieval of biophysical variables, limiting robustness and physical interpretability. The contributions also reveal a persistent gap between high-resolution, small-scale analyses and their scalability to regional or global applications. Therefore, this editorial argues for a transition towards hybrid modeling approaches that combine physical knowledge with data-driven machine learning methods, the adoption of formal data assimilation frameworks for multi-source integration, and the development of scalable and uncertainty-aware workflows. The broader scientific context of the contributions is given by providing a critical perspective on the current state of the field and outlining the key research directions necessary to advance remote sensing in agriculture and ecosystem monitoring.

1. Introduction

Remote sensing Earth observation (EO) provides an essential and established source of temporally nearly continuous and spatially explicit acquisition of information for monitoring agricultural systems and terrestrial vegetation dynamics across different scales. Advances in sensor design, computational capacities, and multiple satellite missions, including ESA’s Copernicus program or new hyperspectral missions such as PRISMA and EnMAP, combined with the emergence of ultra-high-resolution unmanned aerial vehicles (UAV) imaging, have transformed the retrieval and analysis of multi-dimensional datasets. They enabled the dense mapping of biophysical and biochemical vegetation properties across scales with unprecedented detail [1,2,3]. These developments have already enhanced the capacity to retrieve key variables such as canopy structure, leaf area index, chlorophyll content, biomass, and plant stress indicators, which are critical for sustainable agricultural management and ecosystem monitoring, especially given increasing pressures on global food systems driven by population growth, climate change, and resource constraints [4,5,6].

In parallel, artificial intelligence (AI), particularly machine learning (ML) and its domain of deep learning (DL), have emerged as dominant analytical and methodological approaches in remote sensing EO and climate change research, as they enable the extraction of complex, non-linear relationships in, for example, high-dimensional big EO data [7,8,9]. Recent advances in data-driven approaches, including transformer-based architectures for satellite image time series and multi-modal learning frameworks, have further enhanced the ability to model vegetation dynamics and environmental processes [10,11]. Another major recent trend is the development of foundation models for agriculture and ecosystem monitoring. By leveraging large-scale pretraining, these models learn transferable representations that can support a broad spectrum of downstream applications. Their task-agnostic nature facilitates adaption to new tasks through fine-tuning or zero-shot transfer. The approach is especially relevant in remote sensing, where state-of-the-art geospatial foundation models (e.g., AlphaEarth [12], TESSERA [13], and Prithvi-EO [14]) can exploit the heterogeneous and multimodal characteristics of EO data [15]. ML methods are therefore widely applied to several problems such as crop classification, biomass, and yield estimations, drought stress detection, and disease monitoring, often outperforming traditional empirical or physically based approaches [6,16,17].

However, despite these advances, purely data-driven methods often lack interpretability and physical consistency of the governing vegetation dynamics, limiting their robustness and transferability under changing environmental conditions and constraining their operational applicability especially in agriculture [8,18].

To address these limitations, increasing attention has been directed towards integrating data-driven ML models and approaches with process-based simulations—so-called hybrid frameworks. These may include radiative transfer model (RTM) inversion, physics-informed neural networks (PINNs), and digital twin concepts, all aiming to combine the predictive power of AI with the explanatory strength of physical models using remote sensing data [8,19,20,21]. The integration of remote sensing observations into process-based models has a long tradition in the form of data assimilation [22]. Methods such as the Ensemble Kalman Filter (EnKF) are widely employed to assimilate leaf area index (LAI) observations into crop and ecosystem models. Recent advances in AI are further extending these approaches through hybrid frameworks that combine physically based modeling with data-driven learning [23]. Such integration is essential for ensuring that EO-derived information remains not only accurate but also physically meaningful and transferable across spatial and temporal domains, reflecting a broader transition in Earth system science toward theory-guided data driven modeling [24,25].

Thus, the central challenge is no longer the availability of EO data for analytical tools, but the development of models that are physically consistent, transferable across domains, and operationally reliable.

Within this context, this Special Issue on ‘Machine Learning for Applications in Agriculture and Vegetation Using Remote Sensing’ was established to advance methodological development and application-driven research at the intersection of AI and remote sensing. A total of 32 manuscripts were submitted, of which 11 were accepted and published after a peer-review process and editorial quality check. The accepted contributions comprise 63 unique authors from a wide range of institutions worldwide, including China (16), Canada (12), USA (11), Australia (10), Italy (8), Japan (4), Germany (1), and Ireland (1), reflecting the global and interdisciplinary nature of this research domain. The published studies address diverse topics, including vegetation and land-use mapping, crop monitoring, multi-source data fusion, and DL modeling approaches, thereby providing a comprehensive overview of current developments and emerging trends towards integrated, data-driven frameworks. Simultaneously the contributions reveal persistent challenges and limitations in generalization, scalability, and treatment of uncertainty.

2. Synthesis of Contributions and Emerging Scientific Directions

The contributions in this Special Issue collectively demonstrate that ML has become the dominant paradigm for analyzing EO data in agriculture and vegetation applications, reflecting a shift from isolated methodological developments to more integrated and scalable data-driven frameworks. Across all of the studies, a wide range of ML approaches—including Random Forest, U-Nets, transformer models, and generative adversarial networks (GAN)—are employed to estimate vegetation properties, classify land cover, or detect environmental stressors. This trend reflects a broader disciplinary shift, where data-driven methods increasingly outperform traditional statistical and physical based approaches in predictive tasks, particularly in high-dimensional remote sensing data contexts [6,7,8,26].

However, this transition remains methodologically incomplete: while ML models achieve high predictive skills and accuracy, they are typically optimized within specific datasets for a specific domain and rarely incorporate the physical processes governing vegetation dynamics. As a result, they only capture empirical relationships rather than causal mechanisms, limiting both interpretability and generalization [24]. This disconnect between predictive performance and physical realism has been widely recognized as a key limitation in AI-driven EO [8,16,17,24].

2.1. Deep Learning for High-Resolution Mapping and Semantic Segmentation

DL has become the dominant paradigm for high-resolution vegetation mapping at different scales, enabling the extraction of more complex spatial patterns from heterogeneous landscapes. The Special Issue emphasizes the high accuracy of encoder–decoder architectures and transformer-based models in both sparse and structurally complex vegetation systems.

The mapping of sparse poplar forests using Gaofen-2 data shows that tailored DL architectures incorporating features of multiple scales significantly improve segmentation performance under conditions of low canopy density [contribution 1].

Also at local scales, UAV-based segmentation of photosynthetic and non-photosynthetic vegetation (NPV) highlights the capacity of attention mechanisms of SegFormer architectures to resolve fine-scale structural heterogeneity [contribution 2].

At broader continental scales, DL enables the production of consistent national agricultural land-use maps from multi-temporal SAR and optical data, demonstrating its scalability when sufficient training data are available, as well as the suitability of DL for data integration [contribution 3].

These findings are consistent with the broader literature, which shows that DL neural networks can outperform conventional classifiers in complex classification tasks due to their ability to learn hierarchical and non-linear feature representations [6,7,27]. Nevertheless, these performance gains must always be interpreted critically, as model accuracy is often evaluated under conditions of spatial autocorrelation, which inflates metrics and obscures generalization limits [28]. The dependence on large, labeled datasets also constrains the applicability in regions where reference data are scarce or unreliable.

A fundamental limitation of DL lies in the general absence of physical constraints in most current DL workflows. Purely data-driven models lack mechanistic interpretability and are very sensitive to domain shifts, which undermines their robustness in more operational contexts. Recent work demonstrates that integrating RTMs or process-based constraints into ML architectures can improve generalization and stability [29,30]. The contributions of this Special Issue implicitly highlight the need for such hybrid approaches, as current DL models remain predominantly empirical and site-specific.

2.2. Multi-Source Data Fusion and Integrated Modeling

The integration of heterogeneous data sources is essential to profit from the complementary nature of spectral, structural, and environmental information. The contributions demonstrate that the combination of multi-source data substantially improves the characterization of vegetation processes, particularly under conditions of environmental stress.

An analysis of maize drought with a conditional generative adversarial network (CGAN) model integrates meteorological variables, soil moisture, evapotranspiration, and solar-induced chlorophyll fluorescence (SIF), revealing that the dominant drivers of drought stress shift dynamically across phenological stages. This emphasizes the importance of incorporating physiological indicators such as SIF, which provide direct information on photosynthetic activity [contribution 4].

Similarly, the integration of UAV, satellite, and weather data for crop disease detection demonstrates that multi-source fusion enables early detection and spatially explicit severity mapping [contribution 5].

At national scales, the combination of survey-based reference data with satellite observations for grazing land estimation highlights the importance of linking EO data with ground-based in situ measurements to improve statistical validity [contribution 6].

These approaches follow recent advances emphasizing the values of data fusion for improving predictive performance and robustness [31]. Most of the current implementations remain largely empirical, and the differences in spatial–temporal resolution and measurement uncertainty introduce inconsistencies that are rarely addressed explicitly. The absence of standardized fusion strategies limits comparability across studies and constrains reproducibility.

A critical gap remains the limited adoption of data assimilation frameworks, which provide a formal mechanism for integrating observations with process-based models while accounting for uncertainty [23]. Such approaches are already standard in atmospheric sciences but remain underutilized in agricultural remote sensing. The studies in this Special Issue demonstrate the potential of multi-source data integration but also highlight the need for transition toward physically consistent and uncertainty-aware fusion methods.

2.3. Machine Learning for Biophysical Parameter Retrieval and Crop Monitoring

Vegetation biophysical parameter retrieval remains a central application of remote sensing and the contributions to this Special Issue demonstrate the continued relevance of ML in this domain.

Estimations of alfalfa height using Sentinel-2 show that ensemble learning approaches outperform traditional regression models, particularly when spectral indices are complemented by full-band reflectance information [contribution 7].

Similarly, pasture biomass estimation benefits from the integration of spectral data with weather variables and a temporal interpolation, which enhances both predictive accuracy and temporal consistency [contribution 8].

High-resolution phenotyping UAV images can deliver early-season canopy traits and demonstrate good predictors of yield and competitive ability [contribution 9].

Established literature shows that ML improves the retrieval of vegetation properties by capturing non-linear relationships between spectral signals and biophysical variables [32]. However, the contributions also expose some persistent limitations. Vegetation indices such as NDVI exhibit saturation effects in dense canopies, reducing sensitivity to biomass variations and limiting their applicability in high-productivity systems [33]. Moreover, empirical models often lack transferability, as spectral–biophysical relationships vary across environmental conditions, crop types, and sensor characteristics.

Another critical issue is the insufficient validation of models across independent sites and seasons. Many studies rely on random cross-validation, which does not adequately capture spatial and temporal variability, leading to overestimated performance [28]. Addressing these limitations requires integrating spectral, structural, and environmental variables within unified modeling frameworks, as well as incorporating process-based constraints to improve physical consistency via hybrid model approaches such as PINNs. The contributions in this Special Issue demonstrate the methodological progress but also highlight the need for more rigorous validation and model generalization strategies.

2.4. Sensor Characteristics and Scale Effects

Sensor characteristics as well as scales fundamentally influence the retrieval of vegetation properties. Two contributions to this Special Issue address the trade-offs between spectral, spatial, and temporal resolution.

A comparison between hyperspectral PRISMA data and multispectral Sentinel-2 images shows that higher spectral resolution enhances classification of tree typologies, while multispectral data remain competitive in heterogeneous landscapes due to their higher spatial resolution and greater temporal availability [contribution 10].

The review of polar vegetation monitoring further emphasizes the importance of combining optical, SAR, and UAV data to overcome environmental constraints such as low vegetation cover and frequent cloud cover [contribution 11].

These findings reflect a fundamental principle in remote sensing: no single sensor can capture the full complexity of vegetation systems. Trade-offs between spectral detail, spatial resolution, and revisit time necessitate multi-sensor approaches [34]. The integration of observations across different scales still remains a major challenge. Differences in sensor characteristics, viewing geometry, and atmospheric effects complicate cross-scale model transfer, while the lack of standardized benchmarking datasets limits a systematic evaluation.

More recent research also highlights the importance of multi-resolution data fusion and cross-scale modeling frameworks in addressing these challenges [35]. The included contributions again show the potential of combining sensors but also reveal the need of systematic approaches to integrate and evaluate multi-scale observations.

3. Outlook and Future Research Directions

All studies in this Special Issue collectively demonstrate that remote sensing for agriculture and vegetation monitoring is shifting toward integrated, hybrid data-driven systems, while still revealing several structural challenges which must be addressed to achieve robust and operational solutions for food security and ecosystem monitoring. Model transferability remains a critical limitation, as data-driven approaches often fail under changing environmental conditions or when applied to new sensors. The approaches are very sensitive to domain shifts, requiring the integration of domain adaptation techniques or physics-informed hybrid modeling to improve robustness.

The integration of multi-source data must move beyond empirical approaches towards formal and standardized frameworks that explicitly account for scale mismatches and uncertainty. Data assimilation and hybrid spatial–temporal modeling provide promising pathways, enabling the consistent integration of observations and models. Scalability also remains a major challenge, particularly in bridging high-resolution UAV observations with regional and global satellite data. Transfer learning and multi-resolution modeling can offer potential solutions but also require further development and validation.

Uncertainty quantification is essential for operational applications, yet remains largely absent from most workflows, especially for DL frameworks. Uncertainty quantification and standardized validation protocols are necessary to ensure the validity and reliability of remote sensing products in decision-support systems. Finally, the transition from research to operational tools requires the development of real-time processing pipelines and user-oriented products that can support agricultural and ecosystem management and policy.

4. Conclusions

This Special Issue highlights the ongoing fundamental transformations in remote sensing for agriculture and vegetation monitoring, which are mainly driven by current advances in sensor technology and data availability, as well as methodological innovations. DL, multi-source data fusion, and high-resolution observations have substantially improved our capacity to monitor global dynamic vegetation systems. But the field remains constrained by limitations in model generalization and transferability, physical consistency, data integration, and uncertainty treatment. Addressing these challenges requires a shift towards physically informed and uncertainty-aware robust hybrid modeling frameworks that combine interpretable AI systems with physical knowledge. Standardized data integration methodologies need to be developed to ensure reproducibility and comparability. The integration of multi-scale observations and translation into operational decision-support systems also represent key priorities for future research. Progress will determine the extent to which remote sensing can further reveal its potential as a reliable and scalable tool for supporting sustainable and resilient agricultural and ecosystem management in a rapidly changing environment.