Deep Feature Fusion with Vegetation Indices for Wheat Lodging Monitoring Using UAV Multi-Spectral Imagery

Highlights

What are the main findings?

• Spectral indices (VARI, EXG, and MCARI) showed strong capability for monitoring wheat lodging.
• Deep features, particularly YOLO12 combined with BP, significantly improved monitoring accuracy, achieving the best performance.

What are the implications of the main findings?

• Deep features substantially enhance the discriminative power of UAV-based crop monitoring beyond traditional spectral methods.
• Integrating spectral and deep features provides a scalable framework for precision agriculture and intelligent crop monitoring.

Abstract

Lodging is a major agricultural hazard that can substantially reduce crop yields. Timely and accurate monitoring of winter wheat lodging is important for assessing potential yield losses, guiding field management, and mitigating further lodging damage. Recent advances in unmanned aerial vehicle (UAV) remote sensing and artificial intelligence have provided new opportunities for lodging assessment. In this study, a novel monitoring framework was proposed by integrating deep features extracted from UAV multi-spectral images with machine learning algorithms. Sensitivity analysis was conducted to identify vegetation indices (VIs), which are highly correlated with lodging. These sensitive VIs were combined with original multi-spectral bands, and YOLOv8, YOLO12, SAM1, and SAM2 were used for feature extraction. The SHAP method was applied to analyze feature importance and model interpretability. The results indicated that VARI, EXG, and MCARI were the most effective VIs for lodging monitoring. Furthermore, three feature representations, including a spectral feature set, deep features, and fused features, were evaluated. The highest accuracy was achieved using YOLO12 deep features combined with a BP classifier, reaching an accuracy of 98.20%, a precision of 98.38%, a recall of 98.56%, and an F1-score of 98.56%. Overall, incorporating deep features significantly improved monitoring performance. The proposed framework provides an accurate and effective approach for crop lodging monitoring using UAV multi-spectral imagery.

1. Introduction

Wheat (Triticum aestivum L.) is one of the world’s three major staple crops and serves as an important source of dietary energy for the global population, playing a pivotal role in global food security [1,2]. In China, wheat constitutes a cornerstone of the national grain production system [3]. The North China Plain (NCP) is one of the main wheat-producing regions in China, and the stability of its wheat production is of great significance for ensuring regional food security [4]. In recent years, climate change has intensified the frequency and severity of extreme weather events, including strong winds, heavy rainfall, and typhoons, thereby elevating the risk of winter wheat lodging during critical growth stages from jointing to grain filling [5]. Lodging, typically classified into stem lodging and root lodging, severely disrupts water and nutrient transport within plants, diminishes photosynthetic efficiency, and impedes grain filling [6]. These physiological perturbations ultimately culminate in substantial yield reductions [7]. Consequently, timely and accurate lodging monitoring is imperative for disaster assessment, yield prediction, agricultural insurance indemnification, and evidence-based governmental decision-making pertaining to disaster mitigation and relief [8,9].

Traditional wheat lodging monitoring primarily relied on manual field surveys, which were time-consuming, labor-intensive, and associated with high economic costs [10]. In contrast, remote sensing techniques facilitated rapid, efficient, and large-scale monitoring of lodging distribution and provided a macroscopic perspective that was unattainable through conventional ground-based observations [11,12]. In particular, Satellite remote sensing offers long-term and wide-area image time series, making it well-suited for continuous monitoring of crop lodging over extended temporal scales [13]. Guan et al. (2022) proposed a gridded lodging percentage estimation method based on Sentinel-2 imagery and analyzed the spectral responses corresponding to different lodging levels [14]. The approach achieved an R² of 0.64 and an RMSE of 25.24 for lodging percentage estimation on the test dataset [14]. Tang et al. (2022) developed a PTCNet deep learning semantic segmentation model using high-resolution GF-2 imagery [15]. By integrating multi-scale features with vegetation indices and edge information, large-scale wheat lodging areas were extracted, achieving an F1-score of 85.31% and an intersection-over-union (IoU) of 74.38% [15]. Although satellite remote sensing offers clear advantages for large-scale crop lodging monitoring, it is highly susceptible to cloud cover and precipitation, which frequently lead to data gaps [16].

The unmanned aerial vehicles (UAVs), as highly flexible remote sensing platforms, can acquire centimeter-level high-resolution imagery, enabling near real-time observations at the field scale [17,18]. In particular, UAVs equipped with multi-spectral sensors can capture richer spectral information than conventional RGB cameras. Numerous studies have investigated crop lodging monitoring by extracting spectral features, vegetation indices, color features, and texture features from UAV multi-spectral imagery [19,20]. Other studies have directly used multi-spectral imagery or multi-spectral derived vegetation indices as inputs for deep learning models. For example, Yang et al. (2022) employed a ResNet-50 model to classify maize lodging severity using UAV multi-spectral imagery, achieving an overall accuracy of 96.32% and a Kappa coefficient of 0.95 [21]. Similarly, Zhao et al. (2025) combined 22 multi-spectral vegetation indices with a Swin-T model to identify maize lodging type, severity, and direction, obtaining an overall accuracy of 96.02% and a Kappa coefficient of 0.95 [22]. Despite these advances, vegetation indices, color features, and texture features are generally regarded as shallow features and have limited capacity to fully exploit the information content of multi-spectral imagery. As a result, lodging monitoring performance based solely on these features may be suboptimal under complex field conditions [23].

Deep features are abstract representations that are automatically learned by deep learning models through nonlinear transformations within hierarchical network structures and are used to characterize the semantic information of input data [24]. As the network depth increases, deep features become increasingly abstract, exhibiting stronger representational capacity and improved generalization performance. Deep features have been widely applied in crop disease and pest identification tasks. Sethy et al. (2020) extracted deep features of rice leaf diseases using 11 convolutional neural networks, including AlexNet, VGG series, GoogLeNet, and ResNet series, and combined these features with an SVM for classification [25]. The results showed that the combination of ResNet50-derived deep features and SVM achieved the best performance, with an F1-score of 98.38% [25]. Similarly, Dash et al. (2023) extracted deep features of maize leaf diseases using DenseNet201 and performed classification using a Bayesian-optimized SVM [26]. Their approach achieved a classification accuracy of 94.6%, outperforming an SVM model without deep feature integration. These studies indicated that a two-stage strategy, in which deep features are first extracted using deep learning models and then modeled with machine learning algorithms, could yield higher performance comparable to that of end-to-end deep learning approaches. Moreover, compared with the “black-box” nature of deep learning models, this strategy exploited the strengths of machine learning in feature contribution analysis, thereby enhancing model interpretability. By applying interpretability methods such as SHapley Additive exPlanations (SHAP), which are based on Shapley values, the contribution of each input feature to the model output can be quantitatively assessed [27]. Han et al. (2022) extracted multi-source features from RGB and multi-spectral UAV imagery to develop a SMOTE-ENN-XGBoost model for maize lodging monitoring and employed SHAP for feature contribution analysis, achieving a high F1-score of 0.93 under imbalanced data conditions [28].

Deep learning models used for feature extraction have traditionally been based on early convolutional neural network (CNN) architectures, such as AlexNet, VGG, GoogLeNet, and ResNet [29]. These models were generally characterized by relatively shallow network depths and simple structural designs, which limited their feature representation capacity and restricted their ability to capture global contextual relationships. With the rapid advancement of deep learning, architectures with stronger feature mining capabilities have been continuously proposed, among which the Transformer architecture has attracted particular attention [30]. Transformers are capable of effectively modeling long-range dependencies and capturing global contextual information, thereby substantially enhancing the recognition of complex patterns in imagery. In recent years, the YOLO series and the Segment Anything Model (SAM) series have emerged as representative advanced frameworks for object detection and semantic segmentation, owing to their efficient network designs and superior performance compared with early CNN-based models [31]. The YOLO series adopted CNN-based architecture and achieved efficient real-time detection by jointly optimizing feature extraction and object localization in an end-to-end framework, with YOLOv8 and the latest YOLO12 being among the most representative models. The SAM framework was a large-scale, prompt-driven image segmentation model based on the Transformer architecture, exhibiting strong zero-shot segmentation capabilities that enabled automatic or interactive segmentation of arbitrary objects without retraining. Given the high computational cost and practical difficulty of end-to-end retraining, SAM was typically adapted using lightweight fine-tuning strategies [32]. In this context, extracting deep features from SAM has emerged as a low-cost and efficient approach for exploiting the representational power of large-scale segmentation models. Following the release of SAM1, the more advanced SAM2 further enhanced segmentation performance and generalization capability. These advanced architectures provide new opportunities for extracting more discriminative and robust deep features for crop lodging monitoring. However, their potential, particularly that of YOLO and SAM, remains insufficiently explored in UAV multi-spectral imagery-based winter wheat lodging monitoring.

In this study, we proposed a winter wheat lodging monitoring method that integrates deep features from UAV multi-spectral imagery with machine learning algorithms to improve monitoring accuracy. The main contributions of this study are summarized as follows: (1) A Spearman rank correlation coefficient (Spearman’s ρ) was employed to select optimal vegetation indices from 14 candidate indices to extract spectral feature for monitoring winter wheat lodging; (2) deep feature extractors based on YOLOv8, YOLO12, SAM1, and SAM2 were designed to extract deep features from the spectral feature set; (3) winter wheat lodging monitoring models based on Random Forest (RF), Support Vector Machine (SVM), Backpropagation Neural Network (BP), and Linear Discriminant Analysis (LDA) were constructed using the spectral feature set, deep features, and fused features as inputs, respectively, with SHAP employed for feature contribution analysis and model interpretability.

2. Materials and Methods

2.1. Study Area

Multi-spectral imagery of lodged winter wheat was acquired at the Nanpi Eco-Agricultural Experimental Station (NEES), located in Nanpi County, Hebei Province, China (116°40′E, 38°02′N) (Figure 1a). The winter wheat at the experimental station was planted in November of the previous year and harvested in early July of the following year. The wheat experiments at the station included many projects, such as determining optimal fertilization management practices and testing different wheat varieties. Shortly before harvest, strong wind weather caused severe lodging over large areas of mature winter wheat. As illustrated in Figure 1b, field photographs of the lodged winter wheat were collected by on-site personnel to document lodging conditions.

2.2. Data Collection

2.2.1. UAV Multi-Spectral Image Acquisition

Multi-spectral imagery of lodged winter wheat was acquired using a DJI Phantom 4 Multi-spectral UAV (DJI, Shenzhen, China, https://ag.dji.com/p4-multispectral (accessed on 2 June 2026)). The onboard imaging system comprises one RGB sensor for visible-light imaging and 5 monochrome sensors for multi-spectral data acquisition. The multi-spectral bands include blue (B: 450 nm ± 16 nm), green (G: 560 nm ± 16 nm), red (R: 650 nm ± 16 nm), red-edge (RE: 730 nm ± 16 nm), and near-infrared (NIR: 840 nm ± 26 nm). The inclusion of the RE and NIR bands provides enhanced sensitivity to variations in the canopy structure associated with winter wheat lodging. The UAV followed pre-planned flight routes at a flight height of 60 m above ground level, with forward and side overlaps of 85% and 80%, respectively. All multi-spectral images were collected between 11:00 and 12:00 to minimize illumination variability, reduce shadow effects, and ensure relatively stable solar radiation conditions around noon.

2.2.2. Image Preprocess

The Agisoft Metashape Software Version 1.0.0.1 (https://www.agisoft.com/) was used to generate 5-band multi-spectral orthomosaic images of lodging winter wheat, with a spatial resolution of 3.13 cm/pixel. To distinguish lodged winter wheat from healthy wheat, manual pixel-level annotations were performed. The region of interest (ROI) tool in ENVI 5.6 (https://www.nv5geospatialsoftware.com/Products/ENVI (accessed on 2 June 2026)) was employed to delineate lodging areas and generate corresponding label maps for model development (Figure 1c). The orthomosaic images and their corresponding label maps were subsequently cropped into 256 × 256 image patches. Patches located at image boundaries, containing no lodged winter wheat or exhibiting severely imbalanced ratios between lodged and non-lodged winter wheat, were excluded. After filtering, a total of 163 image patches were retained for subsequent analysis (Figure 1d).

2.3. Methods

The whole data processing and data analysis contained 4 stages that were shown in the workflow (Figure 2). The first stage was the UAV multi-spectral image acquisition and data processing (Section 2.2). The third and fourth stages focused on feature extraction and model training and validation, respectively.

2.3.1. Vegetation Indices Extraction

Lodged and non-lodging winter wheat exhibit differences in canopy structure, leaf health, and spectral reflectance, and vegetation indices can effectively enhance these differences [33]. The 14 vegetation indices listed in

Table 1. Vegetation indices derived from multi-spectral images.

Vegetation Index	Formula	Reference
Visible Atmospherically Resistant Index (VARI)	(G − R)/(G + R − B)	[34]
Excess Green Index (EXG)	2G − R − B	[35]
Modified Chlorophyll Absorption Ratio Index (MCARI)	((REG − R) − 0.2(REG − G)) (REG/R)	[36]
Normalized Red-Green Difference Vegetation Index (NDIg)	(R − G)/(R + G + 0.01)	[37]
Triangular Greenness Index (TGI)	G − 0.39R − 0.61B	[38]
(Normalized Green-Red Difference Index) NGRDI	(G − R)/(G + R)	[39]
Green-Blue Vegetation Index (GB)	G/B	[40]
Normalized Red Index (NRI)	NIR/R	[41]
Difference Vegetation Index (DVI)	NIR − R	[42]
Normalized Difference Red Edge Index (NDRE)	(NIR − REG)/(NIR + REG)	[43]
Renormalized Difference Vegetation Index (RDVI)	(NIR − R)/(NIR + R) ^ 0.5	[44]
Leaf Chlorophyll Index (LCI)	(NIR − REG)/(NIR + R)	[45]
Atmospherically Resistant Vegetation Index (ARVI)	(NIR − R + γ(R − B))/(NIR + R − γ(R − B))	[41]
Normalized Difference Vegetation Index (NDVI)	(NIR − R)/(NIR + R)	[46]

Note: B, G, R, REG, and NIR represent the blue band, green band, red band, red-edge band, and near-infrared band, respectively.

were widely used to characterize vegetation health and, therefore, have the potential to discriminate between lodged and non-lodging winter wheat. To quantify the relationship between vegetation indices and lodging area, Spearman’s ρ was employed to rank the correlations between vegetation indices and lodging. The top 3 vegetation indices were selected from the 14 candidate vegetation indices and combined with the 5 original multi-spectral bands, including blue, green, red, red-edge, and NIR, to construct the spectral feature set. Spearman’s ρ is a nonparametric statistical measure used to assess the strength and direction of a monotonic relationship between two variables. It is computed based on the ranks of the original data rather than their absolute values, with correlation coefficients ranging from −1 to 1. Values closer to −1 or 1 indicate stronger negative or positive correlations, respectively, whereas values close to 0 indicate a weak or no correlation. The formulation of Spearman’s ρ is given as follows:

$$\rho =1-{\displaystyle \frac{6{\sum }_{i=1}^n{d}_i^2}{n\left(n^2-1\right)}}$$

(1)

where n is the number of samples, and d_i represents the difference between the ranks of the i-th observation for the two variables.

2.3.2. Deep Feature Extraction

To comprehensively evaluate the effectiveness of different deep feature representations for winter wheat lodging monitoring, 4 deep feature extractors were designed based on the YOLOv8 and YOLO12 architectures and the SAM1 and SAM2 architectures. Overall, we employed 2 convolution-based and 2 ViT-based deep feature extractors to extract deep features from two types of 3-channel inputs in the spectral feature set: the RGB bands and the 3 selected vegetation indices.

The YOLOv8 and YOLO12 deep feature extractors both consisted of a backbone and a neck, while the segmentation head commonly used in YOLO architectures was removed. Only the earliest high-level semantic features in the neck were retained as deep feature representations. Although these features had the lowest spatial resolution, they encode high-level semantic information, making them well-suited for representing large-scale lodging patterns. The YOLOv8 deep feature extractor (Figure 3a) consisted of two key modules: C2f (CSP Bottleneck with two convolutions) and SPPF (Spatial Pyramid Pooling Fast). The C2f module fused residual block features with the original features through convolution operations, enhancing feature integration while reducing computational cost. SPPF expanded the receptive field through multiple max-pooling operations, capturing multi-scale features while maintaining computational efficiency. The YOLO12 deep feature extractor (Figure 3b) removed SPPF and replaced part of the C2f modules with C3k2 and A2C2f. A2C2f built upon C2f by incorporating Area Attention, retaining cross-stage feature fusion and residual connections, and enhancing the detection of large objects. C3k2 used smaller convolutional kernels and residual structures, while retaining cross-stage branches and concatenation from C2f. In this study, the nano-scale variants, namely YOLOv8n and YOLO12n, were adopted to initialize the YOLO-based deep feature extractors using their official pretrained weights and default configurations. The model architecture, initialization settings, and pretrained parameters were kept unchanged before feature extraction, including the default backbone, neck, prediction head, convolutional layers, activation functions, and normalization settings. Given a three-channel input image of size 256 × 256 pixels, the deep feature extractor produced a feature map with 64 channels at a spatial resolution of 32 × 32. The extracted deep feature maps were then upsampled to 256 × 256 using bilinear interpolation, corresponding to an upsampling factor of 8.

SAM1 and SAM2 were both based on Vision Transformer and were designed to achieve a zero-shot, general-purpose segmentation model. SAM1 generated over 1 billion labeled masks from 11 million images to achieve zero-shot transfer of the model. Furthermore, SAM2 employed a more advanced Hiera image encoder, achieving notable improvements in both accuracy and inference speed over SAM1. SAM1 deep feature extractors (Figure 3c) used a Vision Transformer (ViT) structure pretrained with MAE. Multi-head self-attention and feedforward networks were used to capture long-range dependencies of the patch embeddings. SAM2 deep feature extractors (Figure 3d) adopted a stacked ViT module architecture and replaced part of the Global Attention with Mask Unit Attention to reduce the computational cost associated with extensively stacked Global Attention. This design was based on the observation that Global Attention in the first two stages of the image encoder has a limited impact. Therefore, local attention was applied in the first 2 stages, while Global Attention was used in the remaining stages. In this study, the official SAM1 ViT-B and SAM2 base-plus pretrained models were adopted to initialize the corresponding SAM1- and SAM2-based deep feature extractors using their default configurations. The model architecture, initialization settings, and pretrained parameters were kept unchanged before feature extraction, including the default image encoder, prompt encoder, and mask decoder configurations. The pretrained parameters were not fine-tuned on the winter wheat lodging dataset, and the models were used only as deep feature extractors. Given a 3-channel input image with a spatial resolution of 256 × 256, the SAM-based feature extractors generated deep feature maps with 256 channels at a spatial resolution of 64 × 64, which were subsequently upsampled to 256 × 256 using bilinear interpolation with a factor of 4. Compared with the YOLO-based feature extractors, the SAM-based extractors produced feature maps with 4 times more channels and 4 times larger spatial resolution.

2.3.3. Model Establishment

RF, XGBoost, BP Neural Network, and LDA Algorithm were employed to construct winter wheat lodging monitoring models. RF is an ensemble method that integrates multiple decision trees through bootstrap sampling and random feature selection. XGBoost is a gradient boosting framework that builds additive decision trees using gradient information and regularization to improve decision accuracy. The BP Neural Network, a feedforward neural model trained via backpropagation, is capable of modeling complex nonlinear relationships between deep features and lodging labels. LDA served as a classical linear classifier that identifies discriminative projection directions by maximizing between-class variance while minimizing within-class variance. Model hyperparameters were optimized using a grid search strategy, which exhaustively evaluated parameter combinations within the predefined search space shown in

Table 2. Optimal hyperparameters of the machine learning model.

Model	Hyperparameter	Values Range	Optimal Value	Explanation
RF	n_estimators	50~500	50	The number of decision trees
	max_depth	None, 10~50	20	None means trees grow until all leaves are pure
XGBoost	n_estimators	50~500	200	Number of boosting rounds
	max_depth	3~10	6	Depth of each tree
	learning_rate	0.01~0.2	0.1	Step size shrinkage
	subsample	0.5~1.0	0.8	Fraction of samples used in each tree
BP	hidden_layer_sizes	(64,1), (128,64), (256,128)	(128,64)	Controls network depth and width
	activation	relu, tanh, logistic	relu	ReLU improves training speed
	solver	Adam, sgd	Adam	Adaptive optimizer suitable for small datasets
	max_iter	20~200	50	Maximum training iterations
LDA	solver	svd, lsqr, eigen	svd	SVD solver does not require a covariance matrix
	shrinkage	None, 0~1	None	Shrinkage stabilizes covariance estimation

and selected the optimal configuration based on performance.

2.3.4. SHapley Additive exPlanations

SHAP is a recently developed interpretable artificial intelligence approach designed to elucidate model decision-making processes. It quantifies the contribution of individual input variables to model outputs, thereby explicitly evaluating the importance of each feature in the prediction results. SHAP analysis was applied to investigate the contributions of spectral features and deep features to the decision-making of winter wheat lodging monitoring models under different machine learning algorithms. The aim is to provide an interpretable analytical basis for this two-step winter wheat lodging monitoring strategy. The SHAP value is calculated as follows:

$${\mathsf{\Phi }}_i\left(f\right)=\sum _{S\subseteq N\setminus i}{\displaystyle \frac{\left|S\right|!\hspace{0.17em}\left(\left|N\right|-\left|S\right|-1\right)!}{\left|N\right|!}}\left[f\left(S\cup i\right)-f\left(S\right)\right]$$

(2)

where $N$ denotes the complete set of input features, and $S$ represents a subset of features excluding feature $i$. ${\mathsf{\Phi }}_i\left(f\right)$ corresponds to the SHAP value associated with feature $i$, and $f\left(S\right)$ indicates the expected model output conditioned on the feature subset $S$. All SHAP analyses in this study were implemented using Python 3.10 with SHAP version 0.48.0.

2.4. Experimental Design and Model Evaluation

To comprehensively evaluate the contribution of different feature types, three experimental configurations were designed: (1) winter wheat lodging monitoring based on a spectral feature set; (2) winter wheat lodging monitoring based on deep features; and (3) winter wheat lodging monitoring based on fused features, where the spectral feature set in (1) and the deep features in (2) were concatenated along the channel dimension to construct the model inputs. The dataset was divided into training, validation, and test sets in a ratio of 7:2:1. All feature maps were standardized to a spatial size of 256 × 256 × n, where n denotes the feature dimension. Each feature map was then reshaped into a vector of size 65536 × 1 × n to ensure spatial consistency and enable subsequent machine learning model construction. The corresponding lodging label maps were reshaped in the same manner; therefore, each pixel was treated as an individual sample with its associated feature vector and class label. The machine learning models were, therefore, trained to classify each pixel as lodged or non-lodged based on the spectral, deep, or fused feature representations.

For each training run, all machine learning algorithms were trained using the same set of hyperparameters. Model performance was systematically evaluated using accuracy, F1-score, recall, and precision, which are defined as follows:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{TP+TN}{TP+FN+FP+TN}}$$

(3)

$$\mathrm{F}1\text{-}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}={\displaystyle \frac{2\times TP}{2\times TP+FN+FP}}$$

(4)

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{TP}{TP+FP}}$$

(5)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{TP}{TP+FN}}$$

(6)

where TP (True Positive) represents winter wheat that is actually lodged and correctly predicted as lodging, FP (False Positive) refers to background pixels incorrectly predicted as lodging, FN (False Negative) denotes winter wheat that is actually lodged but incorrectly predicted as background, and TN (True Negative) represents background pixels correctly predicted as background.

3. Results

3.1. Analysis of Extracted Image Features

3.1.1. Optimized Vegetation Index

To fully exploit the spectral information contained in multi-spectral data, a total of 14 vegetation indices were calculated. Spearman’s ρ was then applied to identify three indices that were most strongly associated with winter wheat lodging. Figure 4a presents the Spearman correlation coefficients between the 14 vegetation indices and winter wheat lodging. VARI (ρ = 0.5981), EXG (ρ = 0.5977), and MCARI (ρ = 0.5967) exhibited the highest correlation values, all showing positive relationships with lodged areas. Moreover, TGI (ρ = 0.5641), GMR (ρ = 0.5570), and GB (ρ = 0.4800) were positively correlated, whereas NDIg (ρ = −0.5954), NRI (ρ = −0.4681), NDRE (ρ = −0.3088), and LCI (ρ = −0.2759) showed negative correlations.

Figure 4b shows the RGB image used as a visual reference. Figure 4c–e illustrates pronounced differences among the three selected vegetation index maps. VARI exhibited relatively low overall brightness while effectively suppressing non-lodged areas, resulting in strong regional segmentation performance. EXG could also reliably delineate lodged regions. However, its high sensitivity to the green spectral component caused saturation under varying lodging severities, which reduced its discriminative capability compared with VARI. In contrast, MCARI displayed substantially higher brightness and an extremely high sensitivity to winter wheat lodging. Any heterogeneous disturbance originating from the background or canopy structure was significantly amplified, leading to misclassification of non-lodged areas as lodged in some regions and thus reducing the robustness of its discrimination performance. Overall, all three vegetation indices effectively distinguished lodged from non-lodged winter wheat areas and were suitable as input features for subsequent deep feature extraction.

3.1.2. Deep Features Analysis

Deep feature extraction was performed using YOLOv8, YOLO12, SAM1, and SAM2 as feature extractors, with two types of three-channel inputs from the spectral feature set: the RGB channels and the selected vegetation indices, namely VARI, ExG, and MCARI. The extracted deep features were subsequently integrated by upsampling and channel-wise stacking to construct unified deep feature representations. Figure 5 illustrates the results of eight representative single-channel deep feature maps. Most feature channels exhibited clear responses to winter wheat lodging. Figure 5a–d shows the deep feature maps extracted by YOLOv8 and YOLO12. Due to their lower spatial resolution and higher level of semantic abstraction, these features captured lodging-related characteristics at more abstract semantic levels while requiring less storage space, making them suitable for efficient computation. Figure 5e–h presents the deep feature maps extracted by SAM1 and SAM2. Compared with the YOLO-based features, the SAM-based feature maps had larger spatial dimensions and a lower degree of semantic abstraction. They preserved more fine-grained structural information and enabled clearer discrimination between lodged and non-lodged winter wheat. However, the SAM-based features contained four times as many channels as those derived from YOLOv8 and YOLO12, resulting in substantially higher storage requirements and computational costs.

3.2. Results of Winter Wheat Lodging Monitoring

3.2.1. Winter Wheat Lodging Monitoring Based on Spectral Feature Set

In this study, the spectral feature set was used to build the models, serving as the baseline for winter wheat lodging monitoring.

Table 3. Lodging monitoring accuracy based on the spectral feature set.

Model	Accuracy	F1-Score	Precision	Recall
RF	84.83%	87.11%	87.35%	86.87%
BP	84.93%	87.44%	86.06%	88.86%
XGBoost	84.29%	86.61%	87.12%	86.10%
LDA	83.12%	85.77%	85.32%	86.22%

summarizes the performance of the winter wheat lodging monitoring models developed on RF, BP, XGBoost, and LDA. Among the four models, the BP model achieved the best overall performance, with an accuracy of 84.93%, a recall of 88.86%, a precision of 86.06%, and the highest F1-score of 87.44%. The RF and XGBoost models showed similar performance to the BP model, with accuracies of 84.83% and 84.2%, respectively. In contrast, the LDA model yielded the lowest performance, with an accuracy of 83.12%, an F1-score of 85.77%, a precision of 85.32%, and a recall of 86.22%. Figure 6 presents the winter wheat lodging monitoring results produced by the four models. All models successfully identified the lodged winter wheat areas. Overall, while differences in performance among the four spectral feature-based models were relatively small, the results indicated substantial potential for further improvement in monitoring accuracy.

3.2.2. Winter Wheat Lodging Monitoring Based on Deep Features

Winter wheat lodging monitoring models were constructed using RF, BP, XGBoost, and LDA based on deep features extracted by YOLOv8, YOLO12, SAM1, and SAM2. Figure 7 displays the performance of 16 models across accuracy, precision, recall, and F1-score using a heatmap representation. Overall, the models based on RF and BP consistently outperformed those based on XGBoost and LDA across different deep feature types. YOLOv8-BP and YOLO12-BP achieved the highest accuracies of 98.19% and 98.20%, respectively, with precision, recall, and F1-score also exceeding 98%. RF showed slightly lower but still robust performance, with all evaluation metrics exceeding 97%. In contrast, XGBoost and LDA exhibited markedly lower performance. In particular, YOLOv8-LDA achieved only 77.45% accuracy, lower than the spectral feature-based baseline, indicating that LDA was less effective in using complex deep features. For SAM-based deep features, performance differences among models were relatively smaller. The RF model achieved the best performance, with SAM2-RF outperforming SAM1-RF, yielding an accuracy of 97.82% and 96.64%, respectively. XGBoost and LDA demonstrated improved performance on SAM-based deep features compared with YOLO-based features. The accuracies of SAM1-XGBoost and SAM2-XGBoost reached 91.77% and 93.98%, respectively, while those of SAM1-LDA and SAM2-LDA were 88.63% and 89.63%, all exceeding the baseline performance based on the spectral feature set. Overall, the YOLO12-BP model achieved the best overall performance, attaining the highest values across all evaluation metrics except precision, with an accuracy of 98.20%, an F1-score of 98.56%, a precision of 98.38%, and a recall of 98.56%.

Figure 8 shows the lodging prediction maps of the 16 models. The RF and BP models under various deep features were highly consistent with the labels, showing excellent overall performance. In contrast, XGBoost showed fragmentation when using YOLO-based features but improved prediction quality with SAM-based features. The LDA model exhibited similar behavior to XGBoost, showing severe fragmentation and misclassification on YOLO-based features.

3.2.3. Winter Wheat Lodging Monitoring Based on Fused Features

To comprehensively evaluate the contribution of fused features to winter wheat lodging monitoring, the spectral feature set was concatenated along the channel dimension with deep features extracted by YOLOv8, YOLO12, SAM1, and SAM2 to construct four fused feature inputs. Lodging monitoring models were then developed using RF, BP, XGBoost, and LDA algorithms. Figure 9 summarizes the performance of the 16 model configurations using a heatmap of evaluation metrics. Among them, the SAM2-RF model performed the best, with an accuracy of 97.84%, an F1-score reaching 98.34%, a precision of 98.34%, and a recall of 98.30%. The performance of YOLOv8-RF, YOLO12-RF, and SAM1-RF was comparable, with accuracies of 97.11%, 97.50%, and 97.84%, respectively. In contrast, models based on BP, XGBoost, and LDA exhibited clear performance gaps relative to RF. Notably, BP-based models showed substantial degradation: YOLOv8-BP, YOLO12-BP, SAM1-BP, and SAM2-BP performed close to or worse than the baseline results, and SAM2-BP achieved the highest BP accuracy of only 86.07%.

The winter wheat lodging monitoring results based on fused features are presented in Figure 10. The RF model achieved winter wheat lodging range monitoring that was almost consistent with the labels, with clear boundaries and strong regional integrity. XGBoost ranked second, although it correctly identified the overall lodging extent; minor fragmentation and small holes, however, were observed in patch-level regions. By comparison, BP produced severely fragmented predictions, resulting in markedly inferior performance to RF and XGBoost. In addition, LDA performed better when fused features were constructed with SAM-based deep features than with YOLO-based deep features.

3.3. SHAP-Based Feature Importance Analysis

To quantify the contributions of the multi-spectral bands and the selected vegetation indices within the spectral feature set to winter wheat lodging monitoring, SHAP analysis was performed on models constructed using the spectral feature set and deep features. For the spectral feature set-based models, the RF, BP, XGBoost, and LDA results showed consistent feature importance rankings (Figure 11). Across all four models, the NIR band exhibited the highest mean absolute SHAP value, indicating that it is the most influential spectral predictor for lodging monitoring. The R band ranked second in the XGBoost and LDA models, consistent with its sensitivity to lodging-induced changes in canopy structure. In contrast, the three vegetation indices, ExG, VARI, and MCARI, consistently showed low mean absolute SHAP values across all models, suggesting that they contributed less predictive information than the raw spectral bands.

To further examine the response patterns of deep feature channels, SHAP analysis was also conducted for models developed using deep features (Figure 12). For YOLOv8-based models (YOLOv8-RF, YOLOv8-BP, YOLOv8-XGBoost, and YOLOv8-LDA), channels 54, 31, and 60 consistently exhibited high mean absolute SHAP values. Similarly, for YOLO12-based models (YOLO12-RF, YOLO12-BP, YOLO12-XGBoost, and YOLO12-LDA), channels 49, 51, and 91 ranked among the most important across all four algorithms. These results suggested that key deep features extracted by the YOLO architecture were robustly identified and exploited by different machine learning classifiers. By contrast, feature importance rankings for SAM-based deep features varied substantially across models. For SAM1-based models (SAM1-RF, SAM1-BP, SAM1-XGBoost, and SAM1-LDA), the overlap among channels in the top five mean absolute SHAP values was limited, indicating that different classifiers prioritized different channels. For SAM2 features, channels 241, 40, and 254 ranked highly in the RF and XGBoost models, whereas their importance was markedly reduced in the BP and LDA models.

Overall, the SHAP results provided an interpretable basis for the proposed two-step strategy by showing that model decisions were mainly governed by key spectral bands and a limited number of deep feature channels, although channel-level importance varies across extractor architectures and classifiers.

4. Discussion

4.1. Feature Importance Evaluation of Deep Features

A comprehensive comparison was conducted across models constructed using the spectral feature set, deep features, and fused features. In the baseline experiments utilizing the spectral feature set, the four winter wheat lodging monitoring models (RF, BP, XGBoost, and LDA) exhibited minimal performance disparities. When using low-dimensional spectral features alone, the four algorithms achieved broadly comparable performance, suggesting that the overall accuracy was primarily constrained by the limited representational capacity of the input features rather than the classifiers. Conversely, models constructed using deep features achieved substantial performance improvements. The best configuration, YOLO12-BP, reached an accuracy of 98.20% and an F1-score of 98.56%, representing increases of 13.27% and 11.12%, respectively, relative to the baseline BP model. The RF and XGBoost models also improved markedly, with overall gains of approximately 5%~11%. Although LDA benefited from certain deep feature inputs, its performance remained less stable than that of the nonlinear models. These results indicated that deep features provided richer spatial structural representations and higher-level semantics, which enhanced discrimination between lodged and non-lodged areas and mitigated the limitations of conventional spectral bands and vegetation indices in capturing complex lodging patterns.

After fusing the spectral feature set with deep features, model performance did not continue to increase. Compared with the deep feature stage, RF-based models maintained stable performance, with accuracy and the F1-score remaining within the 97%~98% range. XGBoost showed modest improvements (approximately 2%~4%) when fused features were used. By contrast, BP performance decreased substantially: its results were comparable to, or even worse than, the baseline models, and far below its best performance achieved with deep features alone. This decline may be attributable to the limited adaptability of BP to the direct concatenation of a low-dimensional spectral feature set with high-dimensional deep features. LDA consistently produced the lowest performance among the four algorithms. In the baseline experiment, the accuracy of LDA was 83.12%. However, with deep features, the SAM1-LDA and SAM2-LDA models achieved a certain degree of improvement, with a maximum increase of approximately 6.51%. Overall, the YOLO12-BP model based on deep features achieved optimal performance. It attained an accuracy of 98.20%, an F1-score of 98.56%, a precision of 98.38%, and a recall of 98.56%. The overall performance based on fused features exceeded that of the baseline experiment but remained inferior to the performance achieved using deep features alone.

4.2. The Potential of Combining Deep Features and ML for Winter Wheat Lodging Monitoring

The results indicate that combining deep features with machine learning algorithms is a feasible strategy for winter wheat lodging monitoring. Lodging changes the canopy texture, structural arrangement, and visual appearance of wheat plants. Deep features extracted from general-purpose segmentation models can provide more discriminative representations of lodged and non-lodged regions. This suggests that the rich semantic information learned by foundation models can be transferred to crop lodging monitoring tasks.

Typically, deep learning models are trained or fine-tuned for a specific downstream task to optimize performance in a given domain and scenario. Long et al. trained a MobileNetV3-M model to identify wheat lodging under different flight heights, achieving average accuracies of 86.8% [47]. Liu et al. developed an Enhanced Wheat Lodging Index and combined it with a two-peak search dynamic thresholding algorithm to extract lodging regions from UAV images, achieving an overall accuracy of 96% [48]. These studies demonstrate that UAV imagery can provide effective information for wheat lodging monitoring, but task-specific deep learning models and index-based thresholding methods usually rely on dedicated training datasets, predefined lodging features, or stable image conditions. In addition, Liu et al., Pan et al., and Zhu et al. proposed Lodge-Unet, an improved Mask-RT-DETR, and WLUSNet, respectively, to address different requirements in wheat lodging monitoring, including boundary refinement, real-time detection, and lightweight segmentation [49,50,51]. These studies achieved strong performance by designing specialized networks for specific tasks and application scenarios. In contrast, our framework does not require the development of separate task-specific deep networks. Instead, it extracts deep features from general-purpose segmentation models and combines them with machine learning classifiers to distinguish lodged and non-lodged areas. This strategy provides a more unified and flexible solution for wheat lodging monitoring while reducing dependence on dedicated network design and task-specific deep model training.

In recent years, general-purpose segmentation foundation models have become an increasingly influential paradigm. Meta has released the Segment Anything Model family, including SAM1 and SAM2. These models are trained on large-scale and diverse datasets, which substantially improves generalization and zero-shot segmentation capability. However, effectively deploying general-purpose segmentation models in agricultural remote sensing and crop monitoring remains challenging. Madadum et al. combined SAM-based mask extraction with photometric and geometric augmentation to improve YOLO-based watermelon leaf disease detection [52]. Wang et al. used SAM and its variants for automatic pear fruit annotation to reduce dataset construction costs [53]. These studies demonstrate the potential of SAM-based models in intelligent agricultural applications, but they also indicate that practical deployment often still requires task-specific data processing, downstream model training, or manual correction. To address these limitations, an alternative is to extract deep features from the backbones of foundation models and use them as high-level semantic inputs to machine learning classifiers or lightweight models. This feature-based strategy leverages the rich representations learned from massive datasets while avoiding the computational overhead and instability associated with end-to-end fine-tuning. In this study, we adopted this two-step strategy to develop a winter wheat lodging segmentation framework that integrates deep features with traditional machine learning algorithms. Specifically, deep features extracted from general segmentation models were used as input representations, and multiple machine learning classifiers were trained to discriminate lodged and non-lodged regions. The experimental results indicate that this framework effectively exploits the representational strength of foundation model features while maintaining practical computational efficiency, providing a useful reference for applying large foundation models to agricultural remote sensing and crop monitoring tasks.

4.3. Limitations and Prospects

This study employed a two-step strategy that combines deep features with machine learning algorithms to achieve accurate winter wheat lodging monitoring using UAV multi-spectral imagery. Nevertheless, several limitations should be addressed in future work. First, fine-tuning general vision foundation models remains challenging due to high computational demands, training instability, and sensitivity to hyperparameter settings. Consequently, this study did not conduct a systematic comparison with fine-tuning-based approaches in terms of predictive performance, computational cost, and inference efficiency, which limits a comprehensive evaluation of the relative advantages of the proposed strategy. Deep features extracted from foundation models typically have substantially higher dimensionality than conventional multi-spectral bands and vegetation index features. In this study, no feature selection or dimensionality reduction was applied. This design increases the complexity of feature utilization and may partially explain the suboptimal performance observed for some model configurations. In addition, the present experiment was conducted in a specific winter wheat production region. The applicability of the proposed method to other regions with different climatic conditions, soil backgrounds, wheat varieties, planting densities, and field management practices remains to be further evaluated.

Future research will focus on the following aspects. We will conduct a systematic comparison between the proposed feature-fusion strategy and model fine-tuning methods. This evaluation will assess the trade-offs between the two approaches across multiple dimensions, including classification accuracy, training cost, computational consumption, inference efficiency, model flexibility, and interpretability. Additionally, the dataset labeling system will be further refined. Categories such as field ridges will be introduced to verify the discriminative capability of this strategy in complex agricultural scenarios. Simultaneously, this methodology will be extended to lodging monitoring for other crops, such as maize, rice, and cotton. It will also be applied to related tasks like pest identification and Leaf Area Index estimation to comprehensively evaluate its adaptability.

5. Conclusions

This study proposed a novel method for monitoring winter wheat lodging by integrating deep features derived from UAV multi-spectral imagery with machine learning algorithms. Four deep feature extractors were employed to extract high-level semantic features from a spectral feature set. Subsequently, multiple lodging monitoring models based on RF, SVM, BP, and LDA were developed using a spectral feature set, deep features, and fused features as respective inputs. The experimental results demonstrate that the inclusion of deep features significantly enhanced the performance of the lodging monitoring models. Specifically, the YOLOv8-BP model, which integrates YOLOv8 deep features with the BP algorithm, achieved the highest accuracy of 98.20%. Moreover, SHAP analysis was applied to quantify feature contributions and to interpret how deep feature channels influenced model decisions, providing insights into channel-level response patterns. In conclusion, the strategy of combining deep features with machine learning algorithms demonstrated superior performance and feasibility for winter wheat lodging monitoring using UAV multi-spectral imagery. This research provided a valuable reference for the application of deep features from general segmentation models within agricultural remote sensing applications.