Combined Use of Spectral and Structural Features for Improved Early Detection of Pine Shoot Beetle Attacks in Yunnan Pines

Abstract

The long-lasting outbreak of the pine shoot beetle (PSB, Tomicus spp.) threatens forest ecological security. Effective monitoring is urgently needed for the Integrated Pest Management (IPM) of this pest. UAV-based hyperspectral remote sensing (HRS) offers opportunities for the early and accurate detection of PSB attacks. However, the insufficient exploration of spectral and structural information from early-attacked crowns and the lack of suitable detection models limit UAV applications. This study developed a UAV-based framework for detecting early-stage PSB attacks by integrating hyperspectral images (HSIs), LiDAR point clouds, and structure from motion (SfM) photogrammetry data. Individual tree segmentation algorithms were utilized to extract both spectral and structural variables of damaged tree crowns. Random forest (RF) was employed to determine the optimal detection model as well as to clarify the contributions of the candidate variables. The results are as follows: (1) Point cloud segmentation using the Canopy Height Model (CHM) yielded the highest crown segmentation accuracy (F-score: 87.80%). (2) Near-infrared reflectance exhibited the greatest decrease for early-attacked crowns, while the structural variable intensity percentile (int_P50-int_P95) showed significant differences (p < 0.05). (3) In the RF model, spectral variables were predominant, with LiDAR structural variables serving as a supplement. The anthocyanin reflectance index and int_kurtosis were identified as the best indicators for early detection. (4) Combining HSI with LiDAR data obtained the best RF model accuracy (classification accuracy: 87.31%; Kappa: 0.8275; SDR estimation accuracy: R² = 0.8485; RMSE_cv = 3.728%). RF integrating HSI and SfM data exhibited similar performance. In conclusion, this study identified optimal spectral and structural variables for UAV monitoring and improved HRS model accuracy and thereby provided technical support for the IPM of PSB outbreaks.

1. Introduction

Driven by global climate change, forest ecosystems are increasingly facing disturbances from both native and non-indigenous insect pests and diseases, affecting approximately 65.3% of the forest area in Asia and Europe [1,2]. In China, the occurrence of pests has been characterized by a persistent high-incidence rate and frequent outbreaks, leading to an annual impacted area of 12 million ha and an economic loss amounting to CNY 110 billion. The pine shoot beetle (PSB, Tomicus spp.), widely recognized as one of the most aggressive native pests in southwestern China, has triggered multiple outbreaks since the 1980s [3]. These persistent outbreaks, spanning over 40 years and affecting more than 1.5 million ha of Yunnan pine (Pinus yunnanensis Franch.) forests, constitute one of the most severe and protracted ecological disturbances ever recorded in regional forest ecosystems. Consequently, it is of significance to develop an effective pest-monitoring and disaster-warning system to accurately detect and then minimize PSB damage in its very early stage.

While Carl Linnaeus first documented the PSB in Systema Naturae, describing its European populations as inhabiting “younger pine shoots, hollowing and drying them out, thus functioning as nature’s gardener” [4], its ecological impact in Southwest China’s Yunnan pine forests diverges sharply. European PSB populations, constrained by co-evolved tree resistance and predators, cause minimal mortality. In Yunnan’s drought-stressed monocultures, however, PSB’s post-emergence maturation feeding on healthy shoots [5] drives large-scale die-offs, shifting its role from an ecosystem balancer to a catalyst of biodiversity loss and forest degradation. The life cycle of the PSB can be divided into two distinct phases: (1) the shoot-feeding phase and (2) the trunk-colonization phase (Figure 1). During the shoot-feeding phase (from May to November), the pioneering adult beetles feed in healthy shoots and achieve sexual maturity. There are two reasons why this phase is considered as the early attack stage. Firstly, the symptoms of newly damaged shoots are almost invisible to the human eye. Secondly, shoot-to-shoot transferring in the crown leads to significant declines in tree growth vigor and resistance [6,7,8] and thereby make the host tree more susceptible to subsequent trunk-colonization attacks (shoot damage ratio, SDR > 60%) [5,9,10]. During the trunk-colonization phase (from December to April of the next year), mature beetles construct maternal galleries within the inner bark of the trunk for mating and reproduction, which leads to visible red crowns [11,12] (Figure 1).

From the perspective of Integrated Pest Management (IPM), only cutting down the trunk-colonized trees cannot efficiently control the PSB population [13]. By the time those red (or grey) attack trees are located through field investigations, the PSB would have long left the trunks and began new infestations, spreading to other areas. Moreover, due to the large area covered by Yunnan pine forests and the frequent occurrence of PSB disturbance, the approach of using aerial spraying with chemical and bacterial insecticides is neither environmentally friendly nor effective. These above-mentioned facts suggest that the early and accurate detection (individual-tree scale) of PSB attacks is the critical step in its IPM. Though this step could be performed by field surveys, which is the most accurate approach, these surveys are both labor-intensive and time-consuming and, thereby, normally restricted to a limited number of plots. In particular, the mountainous terrain in Yunnan Province poses a significant challenge for conducting field surveys.

Given that anomalies in crown health are more readily visible from a bird’s eye view, remote sensing (RS) technology emerges as an exceptionally powerful tool for assessing PSB infestations. Detailed information on the spatial distribution and intensity of pest disturbances would empower local forest managers to effectively plan and implement targeted countermeasures [14]. However, Yunnan Province experiences a prolonged rainy season (from May to October), which largely coincides with the PSB shoot-feeding phase. The dense clouds and rainy weather conditions often make it difficult for satellite optical sensors to obtain stable, high-quality images [15,16]. In such circumstances, Unmanned Aerial Vehicle (UAV) RS stands out with its short data collection cycle and flexible flight modes. These features enable UAVs to swiftly capture high-resolution images, overcoming the limitations of traditional satellite RS in Yunnan. In particular, the advent of Hyperspectral Remote Sensing (HRS) technology has offered an excellent chance to monitor early plant stress caused by forest pests [17,18,19].

For PSB monitoring, several studies have been conducted across different scales (leaf, crown, and plot level) to comprehensively document the impact of PSB attacks on tree physiology and spectral characteristics. Lin et al. (2018) [20] investigated the spectral data of Yunnan pine needles under different vigor conditions, calibrated the chlorophyll and water absorption coefficients of the needle radiation transfer model Liberty, and proposed a novel Yellow Index (YI) for evaluating PSB attacks at the leaf level. Wang et al. (2018) [21] extracted spectral features of damaged shoots and established multiple linear models and decision rules for a shoot-level damage classification. Based on this, Liu et al. (2020) [22] further studied the correlation between photosynthetic activity parameters and spectral data of the damaged needles and, thereafter, introduced simulation equations to predict the attack stage at the shoot level. Wang et al. (2019, 2022) [23,24] analyzed the relationship between needle temperature and the physiological status of damaged shoots and thus discovered the potential of using thermal infrared images for PSB monitoring. Liu et al. (2021) [25] explored the spectral and physiological signatures of PSB-induced crowns via a manipulated insect infestation experiment and, thereafter, proposed suitable spectral variables and timing for PSB early detection at the crown level. Yu et al. (2018) [26] integrated Landsat TM, ETM+, and OLI data and proposed a threshold-based damage classification method at the plot level. They demonstrated that combining the Moisture Stress Index (MSI) with multi-temporal imagery could generate continuous forest damage level maps and could determine the outbreak timing and pest population spread dynamics.

Despite these valuable efforts, several research gaps persist, which continue to pose challenges for RS approaches in accurately identifying early-stage PSB attacks. Firstly, symptoms of shoot-feeding attacks are difficult to identify, because trees with slightly damaged crowns only exhibit a few withered or reddish shoots, thus displaying highly similar spectral variations with healthy crowns [27]. Therefore, an extremely high spectral and spatial resolution is required for the RS sensor. Secondly, shadows cast due to varying camera angles and solar elevation angles can result in the loss of canopy spectral information or can distort the overall reflectance curve. Thus, it is essential to incorporate additional data sources to mitigate these effects. Thirdly, in Yunnan pine stands, healthy crowns and withered branches often overlap and obscure each other. Using only spectral data is unable to accurately depict such overlapping structures (e.g., in the vertical direction), leading to the underestimation of damage by RS models [27,28].

These gaps indicate that a combination of spectral and structural features extracted from early-attacked crowns might help improve the HRS detection accuracy. On one hand, Light Detection and Ranging (LiDAR) technology, which can generate detailed three-dimensional point clouds of tree canopies through high-density laser pulses, is highly effective in characterizing forest pest damage. These point clouds facilitate individual tree segmentation and depict the spatial distribution of branches within the crown. Additionally, point cloud intensity can help reflect spectral characteristics in the near-infrared (NIR) region [29,30], providing a new perspective for the early detection of PSB. On the other hand, the technology of UAV photogrammetry has been developing rapidly in recent years. Digital Surface Models (DSMs) and Canopy Height Models (CHMs) produced through the structure from motion (SfM) technique can be used for precise individual tree crown segmentation [31]. Some studies also found that the CHM, based on SfM and LiDAR point clouds, has a good correlation [32]. Näsi et al. (2015) [33] combined SfM technology with Hyperspectral Images (HSIs) to detect trees attacked by the spruce bark beetle (Ips typographus), achieving a classification accuracy of 76%. In subsequent research, they concluded that this method of integrating 3D structural and spectral data, due to its cost-effectiveness, has the potential to replace traditional field surveys [32]. Sankey et al. (2017) [34] employed SfM technology, LiDAR point clouds, and HSIs simultaneously to monitor forests in the southwestern United States, demonstrating that the integration of multiple RS technologies significantly improved the accuracy of tree species classification. Therefore, it is reasonable to speculate that, following precise crown segmentation and the integration of spectral and structural data sources, the accuracy of PSB early detection could be further improved.

Moreover, no study has yet fully defined the early attacks caused by the PSB. Many studies have divided the PSB attack periods at individual shoot levels (early, middle, and late), while the others have divided the damage degree of the canopy (light, moderate, and severe) [20,21,22,23,35]. However, these studies did not incorporate the insect’s life history to define the scope of early monitoring. Such a definition is instructive for implementing appropriate countermeasures when the PSB causes early-stage damage.

The primary objective of this study was to integrate UAV-based HSI, LiDAR, and SfM data to explore the potential of these RS technologies in the early detection of PSB attacks. The experiments were conducted through the following steps: (1) compare the performance of three individual tree crown segmentation algorithms (object-oriented, CHM-based, and seed point-based) for extracting spectral/structural features from UAV collected data; (2) classify Yunnan pines into five damage levels and elucidate spectral/structural characteristics of early-attacked crowns; (3) employ machine learning models to select a set of optimal spectral/structural variables for identifying early-attacked crowns; (4) predict the crown damage level (or continuous crown SDR) using different data combinations and evaluate the model performance.

2. Materials and Methods

2.1. Study Site and Data Collection

We performed the fieldwork in the Tianfeng Mountain, Dali Bai Autonomous Prefecture, Yunnan Province, China (25°18′N, 100°54′E, 2000 m a.s.l.; Figure 2a). The plantation is dominated by Yunnan Pines (P. yunnanensis), which cover approximately 5000 ha. As documented by the local forestry administration, two species of the PSB (T. yunnanensis and T. minor) have been responsible for the widespread mortality of pine trees since the outbreaks in 2013. In August 2018, two standard field plots, each measuring 50 m×50 m, were established in the study area (Figure 2b). The boundary coordinates of each plot and the location of individual trees were determined using a Real-Time Kinematic (RTK) GPS device HI-Target A8 GNSS (Guangzhou Hi-target Navigation Tech Co., Ltd., Guangzhou, China), which offers an accuracy of 2.5 mm. For individual trees in the plots, measurements were taken of their Diameter at Breast Height (DBH), tree height, and crown size, with the latter being characterized by the crown width and the total number of shoots. Meanwhile, the individual crown shoot damage ratio (SDR) was collected using ground visual estimations by two experienced observers. SDR is a widely used metric for quantifying the severity of PSB-induced damage, as defined by the Standard of Forest Pest Occurrence and Disaster (LYT1681-2006) [36]. Specifically, SDR is calculated as the proportion of damaged shoots to the total number of shoots (via a detailed count) in each tree crown. Based on this metric, all trees in the field plots were categorized into five damage levels: healthy trees (H; SDR < 10%); lightly damaged trees (L; 11% ≤ SDR < 20%); moderately damaged trees (M; 21% ≤ SDR < 50%); severely damaged trees (S; SDR ≥ 51%); and withered trees (W; trees with no needles), which represent a vigor condition of the PSB post-attack.

2.2. UAV Data Acquisition

We collected the UAV data at the same time as the field survey. The data acquisition settings during the UAV flights are presented in

Table 1. Data acquisition settings during the UAV flights.

Camera	Nano Hyperspec	Pandar 40	Zenmuse X5S
Spatial resolution	0.2 m/pix	200–1800 points/m2	0.05 m/pix
Spectral resolution (band)	270	1	3
Camera setting	8° (FOV)	10° (off-nadir viewing angle)	72° (FOV)/45° (camera angle)
Flight height (m)	70	70	55
Flight speed (m/s)	8	8	5
Along/across-track overlap (%)	80/60	80/60	80/70
Solar zenith (°)/azimuth (°)	25/175
Illumination condition	Sunny (clear sky with less than 30% cloud cover)
Number of ground control points	10

. Detailed information on data processing and model establishment is illustrated as a workflow in Figure 3.

2.2.1. HSI Data Acquisition

In September 2018, HSIs were captured using a DJI Matrice 600 drone equipped with a push-broom hyperspectral camera, the Nano Hyperspec (Headwell Photonics Inc., Nuremberg, Germany) (Figure 2c). The camera consisted of 270 channels, covering the visible to near-infrared (VNIR) region from 400 to 1000 nm. To facilitate subsequent radiometric calibrations and reflectance corrections of the HSIs, a 1.5 m × 1.5 m gray–white cloth, with a rough surface and near-Lambertian characteristics, was laid out in the plot. A linear interpolation method was employed to resample and expand the spectral channels to 600 so that vegetation indices (VIs) could be calculated. A total of 10 red cloths of a 1 m × 1 m size were uniformly laid out in the plots, with their corners serving as ground control points (GCPs) (Figure 2c). These GCPs were located by the RTK device and were used for geometric corrections.

2.2.2. LiDAR Data Acquisition

Concurrently with HSI acquisition, LiDAR point cloud data were collected using a LiAir 200 LiDAR UAV (Green Valley Inc., Beijing, China) equipped with a 40-line laser sensor, the Pandar 40 (Figure 2c). This sensor emits laser pulses at a frequency of 720 kHz and a wavelength of 905 nm (NIR region), with the point density ranging from 200 to 1800 points/m² (Table 1). The point cloud data were processed using the LiDAR360 5.0 software (Green Valley Inc., China) for GCP registration, noise filtering, and subsequent tree crown segmentation. Firstly, the ground points were separated using an Improved Progressive TIN Densification (IPTD) algorithm [37]. Subsequently, all the point clouds were normalized with respect to the ground points. Irregular Triangular Network (TIN) interpolation was then applied to generate the Digital Elevation Model (DEM) and the Digital Surface Model (DSM), both with a grid resolution of 0.1 m × 0.1 m. Finally, the Canopy Height Model (CHM) was derived by subtracting the DEM from the DSM.

2.2.3. Photogrammetry Data Acquisition

Photogrammetry data (collected using the SfM technique) were captured using a DJI Inspire 2 drone (Dajiang Innovation Technology Co., Ltd., Shenzhen, China) equipped with a Zenmuse X5S camera (Dajiang Innovation Technology Co., Ltd., Shenzhen, China). The flight route was carefully designed using the Pix4D capture 4.5.6 software (Pix4D Inc., Prilly, Switzerland), with the flight altitude set at 55 m and the camera angle at 45°. The flight mission was conducted with an 80% along-track overlap and a 70% across-track overlap, employing a double-grid pattern for acquiring 3D-modeled oblique photography images of the pine forest canopy. Subsequently, the Agisoft PhotoScan Professional 1.4.3 software (Agisoft Inc., St. Petersburg, Russia) was utilized to align the photographs and generate a dense point cloud product, with a point density ranging from 100 to 500 points/m².

2.3. Individual Tree Crown Segmentation

By reasonably employing various algorithms of individual tree crown segmentation, it is possible to automatically and batch process spatial positioning and feature extraction [38]. Here, we examined three methods. For HSI, an object-oriented segmentation method was used. Initially, the nearest neighbor method in the eCognition 10.4 software (Definiens Imaging Inc., Munich, Germany) was employed in conjunction with image-texture and spectral characteristics to separate tree crowns from shadows and soil backgrounds (Figure 4a). After that, a binary watershed analysis and the Euclidean distance were employed to separate overlapping tree crowns [39]. Finally, the shadowed areas that could not be separated and the overlapping tree crowns were excluded. Only the remaining sunlit tree crowns were selected for further analysis. For the LiDAR data, a watershed segmentation algorithm based on CHM was employed [40]. After extensive testing, the algorithm achieved the highest accuracy when the CHM resolution was set to 0.5 m × 0.5 m. This configuration included a buffer size of 50 pixels and a canopy starting height of 0.8 m. Additionally, Gaussian smoothing was applied to effectively remove noise. Meanwhile, the point clouds of overlapping tree crowns were merged and treated as a single tree with combined coverage. For the SfM data, a seed point-based point cloud segmentation (PCS) algorithm was employed [41]. This algorithm starts from a seed point (i.e., a global maximum) and uses a distance threshold and minimum distance rule to estimate lower points, developing various sub-points into individual tree clusters. Since the segmentation accuracy is highest when the threshold is close to the tree’s actual crown radius, the SfM data were input with the distance threshold and height-above-ground points both set to 2 m before segmentation.

There are many methods for evaluating segmentation accuracy [42,43,44]. To reflect the effectiveness of tree crown detection through both HSI and point cloud data, this study employed precision (p) (Equation (1)), recall (re) (Equation (2)), and the F-score (F) (Equation (3)) to assess the segmentation accuracy of individual trees.

$$\mathrm{p}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}}\times 100\%$$

(1)

$$\mathrm{r}\mathrm{e}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}}\times 100\%$$

(2)

$$\mathrm{F}=2\times {\displaystyle \frac{\mathrm{p}\times \mathrm{r}\mathrm{e}}{\mathrm{p}+\mathrm{r}\mathrm{e}}}\times 100\%$$

(3)

Here, TP (true positive) refers to the number of trees correctly detected, FP (false positive) refers to the number of trees falsely detected, and FN (false negative) refers to the number of trees that were missed. The detection rate of trees is indicated by re (recall); p (precision) indicates the accuracy of tree segmentation; and F (F-score) represents the overall accuracy, taking both FP and FN into account.

2.4. Spectral and Structural Feature Extraction

Following the segmentation of individual tree crowns, both the spectral and structural features of the tree crowns were extracted. Numerous studies have employed vegetation indices (VIs) derived from various combinations of wavebands captured by HRS platforms. This approach effectively transforms high-dimensional spectral data into meaningful and interpretable spectral variables. In a previous study, a manipulated PSB infestation experiment was employed to reveal suitable spectral indicators for monitoring PSB early attacks [24]. For the current analysis, we selected 20 of these indicators that demonstrated high variable importance (

Table 2. Candidate spectral variables.

Spectral Variables	Formula and Description	Reference
Red-edge position linear interpolation (REP_LiA)	700 + 40 × ((R670 + R780)/2 − R700)/(R740 − R700)	[45]
dRE (AMP)	Max 1st derivative in red edge region	[46]
Sum_Dr2A	Sum of 1st derivative reflectance between R680 and R780
Simple ratio index (SR)	R800/R680	[47]
Simple ratio index 2 (SR2)	R750/R700	[48]
Normalized difference vegetation index (NDVI)	(R800 − R670)/(R800 + R670)	[49]
Green normalized difference vegetation index (GNDVI)	(R800 − R550)/(R800 + R550)	[50]
Optimized soil-adjusted vegetation index (OSAVI)	(1 + 0.16) × (R800 − R670)/(R800 + R670 + 0.16)	[51]
Enhanced vegetation index (EVI)	2.5 × [(R800 − R670)/(R800 + 6 × R670 − 7.5 × R475 + 1)]	[52]
Simple ratio pigment index (SRPI)	R430/R680	[53]
Photochemical reflectance index (PRI)	(R570 − R531)/(R531 + R570)	[54]
Plant senescing reflectance index (PSRI)	(R680-R500)/R750	[55]
Carter index (CI)	R760/R695	[56]
Structure insensitive pigment index (SIPI)	(R800 − R445)/(R800 − R680)	[57]
Ratio vegetation stress index (RVSI)	(R718 + R748)/2 − R733	[58]
Vogelmann red edge index 1 (VOG1)	Sum [R734:R747]/Sum [R715:R726]	[59]
Vogelmann red edge index 2 (VOG2)	(R734 − R747)/(R715 + R726)
Green peak height (GH)	1 − [R500 + 0.35 × (R670 − R500)]/R560	[21]
Red valley depth (RD)	1 − R670/[R560 + 0.55 × (R760 − R560)]
Water index (WI)	R900/R970	[60]
Disease water stress index 4 (DSWI-4)	R550/R680	[61]
Anthocyanin reflectance index (ARI)	1/R550 − 1/R700	[62]
Carotenoids reflectance index (CRI)	1/R510 − 1/R550	[63]

R: reflectance at the specific wavelength (nm), indicated by the subscript.

).

Partial Least Squares Discriminant Analysis (PLS-DA) is a classification technique based on Partial Least Squares Regression (PLSR). It transforms the regression results of PLS into a set of latent variables, which can be used to predict the dependent variable (i.e., category). In this study, the separability of tree samples at different damage levels was assessed using the principal component score plot in PLS-DA, with the maximum distance method employed to enhance the separation between classes.

Correctly segmented LiDAR and SfM point cloud data were utilized to calculate structural variables. Specifically, LiDAR variables can be categorized into two types: intensity metrics (e.g., intensity percentiles) and geometric metrics (e.g., canopy height percentiles) (

Table 3. Candidate structural variables.

Categories	Structural Variables	Descriptions
Intensity metrics	int_P25, int_P50, int_P75, and int_P99	25th, 50th, 75th, and 99th percentile of return intensity
	int_C25, int_C50, and int_C75, int_C99	25th, 50th, 75th, and 99th cumulative percentile of return intensity
	int_IQ	Interquartile range of return intensity
	int_mean	Mean value of return intensity
	int_cv	Coefficient of variance of return intensity
	int_kurtosis	Kurtosis of return intensity
	int_skewness	Skewness of return intensity
Geometric metrics	elev_P25/TH, elev_P50/TH, elev_P75/TH, and elev_P99/TH	25th, 50th, 75th, and 99th percentile of heights (normalized by tree height)
	elev_IQ	Interquartile range of height percentile
	Canopy relief ratio (CRR)	$\mathrm{C}\mathrm{R}\mathrm{R}={\displaystyle \frac{mean-min}{max-min}}$ mean/min/max is the average/min/max height of all points within the statistical unit.
	Canopy cover (CC)	$\mathrm{C}\mathrm{C}={\displaystyle \frac{n_{vegfirst}}{n_{first}}}$ nvegfirst represents the number of vegetation points from the first return, while nfirst represents the total number of points from the first return.
	Leaf area index (LAI)	$\mathrm{L}\mathrm{A}\mathrm{I}=-{\displaystyle \frac{\mathrm{c}\mathrm{o}\mathrm{s}\left(ang\right)\times ln\left(\mathrm{G}\mathrm{F}\right)}{k}};ang={\displaystyle \frac{{\sum }_{i=1}^n{angle}_i}{n}}$ ang represents the average scanning angle. GF represents the gap fraction. k is the extinction coefficient, ln is the natural logarithm, and n is the total number of points.
	Gap fraction (GF)	$\mathrm{G}\mathrm{F}={\displaystyle \frac{n_{ground}}{n}}$ nground represents the number of ground points extracted with Z-values below the height threshold, while n represents the total number of points.

). The intensity percentile refers to the process of normalizing all LiDAR points within a statistical unit by intensity, sorting them, and then calculating the intensity of the top n% of points within each statistical unit. The Pandar 40 sensor utilizes a constant pulse energy emission system with automatic gain control (AGC) optimized for UAV operations. Unlike LiDAR, which is an active remote sensing technology capable of penetrating the canopy to some extent, SfM primarily captures the external contour of the canopy and cannot fully penetrate to obtain internal structural features. Therefore, SfM variables only include geometric metrics.

2.5. Model Establishment and Performance Assessment

Several non-parametric machine learning algorithms that incorporate internal cross-validation, such as Artificial Neural Networks (ANNs), Support Vector Machines (SVM), and random forests (RFs), have been used for selecting spectral variables and developing prediction models. Among these methods, RFs have been shown to be particularly effective. This is because RFs are less likely to overfit, can effectively deal with nonlinear data, possess strong resistance to noise, and have a relatively straightforward implementation process [63,64]. In this study, we employed RFs in combination with spectral and structural variables for crown-damage-level classification and SDR prediction. In order to assess the capability of the HSI, LiDAR, and SfM data in identifying PSB early attacks, five combinations of different data sources were established: (1) HSIs only, (2) LiDAR only, (3) SfM only, (4) HSIs combined with LiDAR, and (5) HSIs combined with SfM. Additionally, the Mean Decrease in Gini (MDG) index was taken to rank the full spectral dataset and the spectral/structural variables based on their importance in the RF model.

Given the advantages of Leave-One-Out Cross-Validation (LOOCV) for an unbiased estimation with small samples and for the ease of detecting outliers, this study employed LOOCV to examine the accuracy of the RF model in predicting SDR. The coefficient of determination (R²) and Root-Mean-Square Error (RMSE_cv) were utilized to quantify model accuracy. In addition, the confusion matrix is an effective method for describing both the global and individual class accuracy of a model [65]; hence, we used the confusion matrix to examine the RF model’s ability of classifying crown damage levels. Overall accuracy (OA), Producer Accuracy (PA), User Accuracy (UA), and Kappa coefficient were calculated through the confusion matrix.

The “randomForest” package in the R 12.0 software was used for modeling, as well as ranking, the importance of variables. GraphPad Prism 8.3 (GraphPad Software Inc, La Jolla, CA, USA) and R Studio 4.2 (R studio Inc., Boston, MA, USA) were used for data visualization and plotting.

3. Results

3.1. Individual Tree Crown Detection

The accuracy of the segmentation algorithms was evaluated using ground survey data and visual interpretations of UAV imagery (

Table 4. Comparison of accuracy for tree crown segmentation algorithms.

Accuracy	Object-Oriented Segmentation	Seed Point-Based PCS	CHM-Based PCS
True positive (TP)	173	235	259
False positive (FP)	36	95	61
False negative (FN)	97	35	11
Recall (re, %)	82.78	71.21	80.94
Precision (p, %)	64.07	87.04	95.93
F-score (F, %)	72.23	78.33	87.80

). Overall, the PCS based on the CHM and watershed algorithm achieved the highest F-score (87.80%), followed by the seed point-based PCS (78.33%). It can be stated that LiDAR and SfM point cloud data could correctly segment the majority of tree crowns (87.04–95.93%). In comparison, the object-oriented segmentation algorithm had the lowest accuracy (72.23%). It successfully separated vegetation from bare soil and shadows but struggled to distinguish understory vegetation and confused deadwood with bare soil, resulting in a high number of FNs (97 trees) and the lowest precision (64.07%). The seed point-based PCS had a higher number of FPs (95 trees). This may be attributed to the relatively sparse density of point clouds collected by SfM, which resulted in incomplete crown contours. Consequently, some single trees were identified as multiple trees due to interconnections and occlusions within the standard plot. Additionally, some low-lying tree crowns (35 trees) were not recognized. The CHM-based PCS had a slightly lower number of FPs (61 trees), possibly due to noise in the LiDAR point cloud being misidentified as individual trees or some trees having larger crowns, which led to the identification of a single tree as multiple trees. Overall, this individual tree segmentation algorithm performed the best, with the lowest number of FPs (only 11 trees) and 95.93% of the tree crowns correctly segmented.

3.2. Spectral Signature

The results indicate that the crown spectral curves at different damage levels exhibited variations in the green peak, red valley, and near-infrared platform regions (Figure 5a). Specifically, multiple t-tests revealed significant differences (p < 0.05) between the early-attacked crowns and healthy ones in the wavelength regions of 523–584 nm, 645–679 nm, and 706–1000 nm. These regions, with the largest differences, were found near the green light at 557 nm (absolute difference, −0.007; relative change, −10%); the red edge at 678 nm (absolute difference, +0.006; relative change, +17%); and the near-infrared region at 775 nm (absolute difference, −0.047; relative change, −17%) (Figure 5b,c). In terms of absolute differences, the near-infrared platform showed the greatest decrease in reflectance.

3.3. Structural Signature

The intensity percentile results of the point cloud can describe the distribution of internal laser echoes within the tree crowns at different damage levels (Figure 6a). The results indicate that visually from the curves, the point cloud intensity of crowns influenced by shoot-feeding damage was generally less than that of healthy ones, with the gap widening progressively between the p30 and p90 percentiles. This finding is consistent with the canopy reflectance characteristics in the near-infrared region (Figure 5a) and aligns with the spectral analysis results (Figure 5). The point cloud intensities of the top 50% (p50–p95) for early-attacked crowns were significantly lower than those of healthy ones (p < 0.05) (Figure 6b,c). The reason for this distribution may be that as Yunnan pine trees progress further into senescence, the needles on the damaged shoots start to contract and cluster together, and some shoots broke off and fell. This change leads to a reduction in crown coverage, as evidenced by a lower leaf area index (LAI). The point cloud intensity curves for moderately and severely damaged trees exhibited almost the same shapes, possibly due to the low needle density in these damaged levels. The maximum difference in intensity percentiles occurred at the p90 percentile (absolute difference, −3.47; relative change, −12%), which may serve as a diagnostic signature of PSB early damage. Notably, the intensity distributions for healthy and withered trees were similar (Figure 6a and Figure 7i,k). It is possible that the LiDAR system’s high point density enabled penetration through sparse withered canopies, capturing signals from persistent understory vegetation. Also, structural degradation in withered trees could alter pulse scattering dynamics, i.e., less scattering of the LiDAR pulses, because targets were geometrically more uniform.

3.4. Feature Variables

A total of 12 representative spectral and structural variables were compared (Figure 7). The results indicate that spectral variables effectively captured the differences in the spectral curve among all the damage levels. For instance, the dRE (AMP), Green peak height (GH), anthocyanin reflectance index (ARI), disease water stress index 4 (DSWI-4), ratio vegetation stress index (RVSI), and red valley depth (RD) varied progressively with increasing canopy damage severity (p < 0.0001) (Figure 7c–h). In contrast, structural variables exhibited larger standard deviations in the boxplots, with most failing to reflect changes (p > 0.1). Variables such as the Crown Cover (CC), elev_IQ, and gap fraction (GF) showed no significant differences across damage levels (p > 0.1) (Figure 7l–n). Only the int_mean, int_skewnss, and int_P75 demonstrated a better discriminative ability (Figure 7i–k). Overall, compared to structural variables, spectral variables were more sensitive to signature changes in early-attacked crowns.

The results of the PLS-DA analysis indicate that spectral variables exhibited superior performance in differentiating standing withered trees (Figure 7a). Specifically, the healthy trees and those at lightly damaged levels were almost indistinguishable from each other. However, there was a certain degree of separability between these two groups and moderately damaged trees. Moreover, healthy trees could be completely separated from severely damaged trees and withered trees. In contrast, when tree crowns were analyzed based on structural variables, the PLS-DA score plot showed extensive intermingling and overlapping among different damage levels, rendering it impossible to effectively differentiate the damage levels (Figure 7b).

3.5. Integration of Different Data Sources for Modeling

The RF model was employed to classify the tree damage levels using five different combinations of data sources, while also ranking those variables’ importance (Figure 8). In general, spectral variables predominantly contributed to the importance ranking in the RF model, supplemented by LiDAR and SfM structural variables. Specifically, the spectral variables generally exhibited higher MDG values than the structural variables, with the structural variables extracted from LiDAR data showing slightly greater importance than those from the SfM data. Notably, the anthocyanin reflectance index (ARI) had a notably higher importance compared to other indices (MDG: 9–10), indicating that changes in the color of coniferous needles are a key factor influencing model performance. The importance of the AMP ranked second. Structural variables, such as int_kurtosis and int_skewness, demonstrated higher importance when the LiDAR data were included in the modeling, reflecting the differences in the distribution of point cloud intensity across the damage levels. Furthermore, when the SfM data were included in the modeling, variables such as ele_P25/TH and the ele_IQ showed a certain level of importance. This indicates that there were discernible differences in point cloud height intervals across the damage levels, and the SfM data effectively captured these variations, contributing to the overall classification accuracy of the model. Last but not the least, the spectral reflectance around 720 nm emerged as a key predictor when incorporating the full spectral dataset in the RF model (Figure 8).

The overall accuracy (OA) of combining three data sources for RF modeling ranged from 79.10% to 87.31% (Figure 9). The use of HSI data alone achieved an OA of 79.10% and a Kappa coefficient of 0.7177. In contrast, the OAs and Kappa coefficients for the LiDAR and SfM data alone were lower (OAs: 58.96–65.67%; Kappa: 0.4111–0.5265). However, it is noteworthy that the UAs for identifying the withered level using LiDAR and SfM data were relatively high (77.78–82.35%), demonstrating that structural information can enhance the model’s ability to identify dead trees (with UA reaching up to 100%). The fusion of HSI and LiDAR data resulted in a substantial improvement in the detection model for PSB early attacks (OA increased from 79.10% with the HSI dataset alone to 87.31% with the fused dataset) (Figure 8). Similarly, the fusion of HSI and SfM data achieved a comparable OA of 84.33%. Additionally, structural information played a very positive role in improving the UAs for lightly damaged crowns and moderately damaged crowns (with increases of 27.78% and 30.00%, respectively).

When using RF to predict crown SDR, results similar to those obtained from RF classification were observed (Figure 10). Notably, when the SDR was relatively high, the predictive capability of the HSI model was diminished, with the predicted values deviating from the ground-measured values (R² = 0.7759 and RMSE_cv = 4.789%). This situation was improved by incorporating LiDAR and SfM data into the model. The combination of HSIs and LiDAR yielded the highest coefficient of determination (R² = 0.8485) and the lowest RMSE_cv (3.728%). This performance was superior to that of the ground-based hyperspectral model (R² = 0.71; RMSE_cv = 8.92%) reported by Liu et al. (2021) [25]. A reason for this may be that the SDR composition in their manipulated PSB-infestation experiment was different from this study.

In summary, for the application of PSB early detection, an RF model that integrates HSI and LiDAR data is recommended to be employed, as it achieved an OA of 87.31% and an EA of 84% (Figure 11a). When LiDAR data are unavailable, an RF model that combines HSI with SfM data is also expected to achieve comparable accuracy, with an OA of 84.33% and an EA of 82%.

4. Discussion

4.1. Definition of Pine Shoot Beetle Early Attacks

Currently, there is a diverse range of definitions for the early detection of forest wood borers. For instance, in North America and Europe, many studies reached an agreement on the initial infestation stage of the mountain pine beetle (MPB, Dendroctonus ponderosae) and the spruce bark beetle (Ips typographus). During the initial stage, the crown needle color is not visibly changed to the human naked eye and remains green. Thus, this stage is also referred to as the green attack stage or the early attack stage [66,67]. Netherer et al. (2015) [68] defined the green attack stage as when the tree trunk has already been successfully invaded by the bark beetle, and the canopy’s water content has undergone corresponding changes, but the needle color is still indistinguishable to the human naked eye. In Asia, for pine wilt disease (PWD) caused by pine wilt nematode (PWN, Bursaphelenchus xylophilus), Yu et al. (2021) [69] define the early infection stage as the moment when the pine needles begin to change color, which can be observed through remote sensing. Other studies directly inoculate healthy pines with PWN, considering the manipulated inoculated pines to be in the early infection stage [70].

However, whether it is MPB or PWD, the canopy damage they cause is characterized by systemic wilting, with crown needles discoloring uniformly. This is markedly different from the heterogeneous discoloration process caused by early PSB attacks, which involve scattered reddish dead shoots interspersed among mostly green shoots within the crown. Therefore, the common definition of an early attack cannot be directly applied to the situation in this study. So, what does an early-attacked Yunnan pine forest look like? Liu et al. (2019) [71] discovered that during the shoot-feeding phase, T. yunnanensis and T. minor both widely and uniformly damage the canopies of Yunnan pines in the forest stand (ecological niche breadth B_i > 0.8). At this point, the PSB acts as the primary pest, causing a continuous SDR increase and gradually shifting the tree vigor from healthy to weakened (Figure 1).

The “technical regulations for cleaning damaged trees by Yunnan pine shoot beetles” (LY/T 2352–2014) [72] defines the initial attack stage of the PSB as Yunnan pines with green or light green needles, branches beginning to turn yellow, and a small number of boring holes and resin on the main and lateral branches. However, this regulation does not provide a quantitative SDR range for this stage. During the shoot-feeding phase, when the crown SDR reaches 11–50%, symptoms of partial dead shoots surface [71]. After “shoot-to-trunk” transferring, no new invasion holes of the PSB are found in the trunks of healthy, lightly damaged, or moderately damaged Yunnan pines. However, in severely damaged (SDR > 50%) Yunnan pines, new invasion holes are found, and adult beetles are discovered under the bark, with an infestation rate of 41.8% [73]. This indicates that the PSB preferentially chooses severely damaged trees for trunk colonization, as the resin resistance in high-SDR Yunnan pines is significantly lower than that in low ones. Lieutier (2003) [5] also reported that when the SDR exceeds 60%, the PSB is always likely to colonize the trunk, leading to tree death. Geostatistical results show that the spatial distribution pattern of several PSB species in Yunnan pine forests is aggregated, which facilitates adult trunk colonization and reproduction [74]. Based on this phenomenon, Wu (2019) [75] concluded that cleaning Yunnan pines with a high SDR value (>50%) and red crown trees can effectively reduce the PSB populations in the stand and can achieve the goal of IPM.

A field survey revealed that the SDR of the attacked Yunnan pines in Tianfeng Mountain was mainly distributed between 11–60% (561 trees), with very few trees falling into the range of 61–100% (only 7 trees were found). In contrast, a considerable number of trees exhibited complete crown withering, with an SDR of 100% (47 trees) (Figure 11b). In other words, in most cases, PSB early attacks only result in a partial crown die-back with an SDR value below 60%. It can be inferred that in the early-attacked forest stands, the population of PSBs is not predominantly concentrated on the very few severely damaged red crown trees. Instead, most trees are widely affected by shoot-feeding attacks. Given this scenario, it is likely that a large number of early-attacked crowns would be overlooked, based on the conclusions of previous studies and existing regulations.

In summary, the potential targets for monitoring PSB early attacks could be Yunnan pines with an SDR value between 11% and 50%. During this stage, the trees are only affected by shoot-feeding attacks, and the PSB population is not yet transferred to the trunk. An effective prevention and control approach (i.e., UAV targeted spraying, biological control, pheromone trapping, and odor interference technology) can serve as a strong supplement to the traditional cleaning of colonized wood. Moreover, the PSB has a prolonged shoot-feeding phase, extending from May to October. Previous research has confirmed that after 4 weeks of shoot-feeding damage, the crown can reach a “switch point” in color change that can be identified by HRS [25]. Therefore, early detection should be most effective when conducted in the early-to-mid-shoot-feeding phase, when the SDR value is between 11% and 50% and is identifiable through the HRS approach. This study proposes two detection models: one combining HSIs with LiDAR and another combining HSIs with SfM. Both models achieve high OAs of 87.31% and 84.33%, respectively. These models can be effectively used for the early detection of PSB infestations.

4.2. Contribution of Structural Features

Recent advancements in UAV sensor fusion have shown promising results in forest pest detection. He-Ya et al. (2024) [76] integrated multispectral VIs (NDGI) and LiDAR structural features (25% height percentile) with an SVM model, achieving exceptional accuracy (OA: 95.8%) in identifying larch caterpillar (Dendrolimus superans) infestation severity. Zhou et al. (2023) [77] used UAV–hyperspectral and LiDAR data, analyzed via PLS–SVM, to distinguish between Asian longhorned beetle (ALB) infestation and drought stress in poplars in China’s Three-North Shelterbelt (OA: 78.79%). They found that the LiDAR metric (elev_sqrt_mean_sq) was key for detecting droughts, while the band R970 was crucial for identifying ALB damage. In this study, the results from variable importance ranking (Figure 8) and confusion matrices (Figure 9) demonstrate that integrating spectral variables from HSIs with structural variables from LiDAR or SfM significantly enhances classification accuracy. This improvement is primarily attributed to the contribution of point cloud intensity variables, with studies suggesting that echo intensity information can be used to estimate reflectance characteristics in the near-infrared region. In this study, the intensity of the latter 50% of point clouds (int_p50-int_p95) from early-attacked crowns was significantly lower than that of healthy ones (p < 0.05). Additionally, structural variables are highly beneficial for detecting withered trees (UA reached 100%), and the precise detection and mapping of withered trees is valuable for understanding the spreading direction and dynamics of PSB populations [78]. Furthermore, combining LiDAR and SfM point clouds with their respective individual tree segmentation algorithms can fully utilize elevation information to assist the extraction of spectral variables from HSI.

Defoliation is a key symptom of most stress factors affecting trees. As tree vigor further declines, the needles begin to draw together, and the needle length shortens [79]. This leads to a decrease in LAI and canopy closure. In the case of MPB, Shrestha et al. (2024) [80] classified insect damage (MPB and Douglas-fir beetle) by assigning multi-spectral reflectance to vertical point clouds, achieving high accuracy (OA: 93.5%) in separating healthy and top-kill damaged trees. However, in this study, structural variables, such as LAI and CC, did not show regular changes across different damage levels (Figure 6). This may be attributed to the fact the reddish shoots damaged by the shoot feeding remained erect for a long time without falling off, which is unlike PWD, which can cause systematic wilting of the tree crown and rapid death (within 90 days). As a result, shoots were difficult to identify by LiDAR structural variables. Also, solely utilizing structural variables extracted from LiDAR or SfM point clouds was insufficient for early detection. The primary reason for this is that point cloud data cannot accurately invert the physiological and biochemical characteristics of damaged tree crowns [35]. Moreover, in terms of information quantity, the geospatial information contained in this study’s LiDAR point clouds is only a subset of the full waveform LiDAR data. In the future, using the waveform part of the laser signal with full waveform LiDAR data can potentially extract richer structural information [81].

Notably, the SfM point cloud data demonstrated good integrability with the HSI data in both individual tree crown segmentation and early-attack detection, meaning that the HSI and SfM fusion scheme can achieve an OA value similar to that of the HSI and LiDAR fusion scheme. Currently, the cost of using LiDAR technology is relatively high. In contrast, SfM can be performed with lost-cost UAV cameras, showing great potential to replace LiDAR data in the early detection of PSB. Moreover, most PCS algorithms are based on the elevation and contour features of individual trees. The density range of LiDAR point clouds obtained in this study was from 200 to 1800 points/m², while the density range of SfM was only from 100 to 500/m². It is evident that increasing the density of SfM point clouds could yield better structural features of individual trees and stands. Therefore, in future studies, diverse UAV-based SfM photography designs—such as circular flight paths and high–low combined flight paths can be implemented to achieve improved detection performance. Moreover, UAV technology, though developing rapidly and though ideal for detailed local data collection, is limited by its flight range and battery life. To enhance large-scale forest pest monitoring, future research should explore applying UAV-derived spectral/structural variables with up-scaling strategies to broader airborne or spaceborne imagery. Also, establishing protocols for how often and where to deploy UAVs within a larger monitoring network could help bridge the gap between scales.

5. Conclusions

This study advanced the early detection of PSB attacks by establishing a novel UAV-based framework that synergizes spectral and structural RS data. Three key improvements emerged from our findings: First, the integration of HSI-derived VIs with LiDAR-derived metrics enabled a reliable detection (OA: 87.31%) of PSB early attacks, overcoming the limitations of single-sensor approaches. Second, the comparable performance of a cost-effective HSI-SfM fusion highlighted the potential for democratizing precise forest pest monitoring in resource-limited regions. Third, our life-cycle-informed definition of PSB early attacks provides an operational threshold for timely intervention, addressing a critical gap in PSB IPM protocols.

Future research should prioritize the following research directions: First, the framework should be scaled to landscape-level monitoring through UAV swarm deployments and edge computing implementations. Second, meteorological stress indicators (e.g., drought indices) and host–pest phenological models should be incorporated to enhance predictive capabilities. Third, the convergence of lightweight sensors and AI-driven analytics promises to transform our proposed framework into an autonomous early-warning system, ultimately contributing to the resilience of China’s critical pine ecosystems in the face of escalating biotic threats.