Integrating Multi-Scale Remote-Sensing Data to Monitor Severe Forest Infestation in Response to Pine Wilt Disease

Abstract

Pine wilt disease (PWD) is one of the most destructive forest diseases that has led to rapid wilting and mortality in susceptible host pine trees. Spatially explicit detection of pine wood nematode (PWN)-induced infestation is important for forest management, policy making, and practices. Previous studies have mapped forest disturbances in response to various forest diseases and/or insects over large areas using remote-sensing techniques, but these efforts were often constrained by the limited availability of ground truth information needed for the calibration and validation of moderate-resolution satellite algorithms in the process of linking plot-scale measurements to satellite data. In this study, we proposed a two-level up-sampling strategy by integrating unmanned aerial vehicle (UAV) surveys and high-resolution Radarsat-2 satellite imagery for expanding the number of training samples at the 30-m resampled Sentinel-1 resolution. Random forest algorithms were separately used in the prediction of the Radarsat-2 and Sentinel-1 infestation map induced by PWN. After data acquisition in Muping District during August and September 2021, we first verified the ability of a deep-learning-based object detection algorithm (i.e., YOLOv5 model) in the detection of infested trees from coregistered UAV-based RGB images (Average Precision (AP) of larger than 70% and R² of 0.94). A random forest algorithm trained using the up-sampling UAV infestation map reference and corresponding Radarsat-2 pixel values was then used to produce the Radarsat-2 infestation map, resulting in an overall accuracy of 72.57%. Another random forest algorithm trained using the Radarsat-2 infestation pixels with moderate and high severity (i.e., an infestation severity of larger than 0.25, where the value was empirically set based on a trade-off between classification accuracy and infection detectability) and corresponding Sentinel-1 pixel values was subsequently used to predict the Sentinel-1 infestation map, resulting in an overall accuracy of 87.63%, where the validation data are Radarsat-2 references rather than UAV references. The Sentinel-1 map was also validated by independent UAV surveys, with an overall accuracy of 76.30% and a Kappa coefficient of 0.45. We found that the expanded training samples by the integration of UAV and Radarsat-2 strengthened the medium-resolution Sentinel-1-based prediction model of PWD. This study demonstrates that the proposed method enables effective PWN infestation mapping over multiple scales.

1. Introduction

The most harmful pine wilt disease (PWD), caused by the pine wood nematode (PWN, Bursaphelenchus xylophilus), has led to fast wilting and mortality in susceptible host trees (usually Pinus species such as Pinus thunbergii, Pinus densiflora, and Pinus massoniana) [1]. The original biological equilibrium of the forest system was badly disturbed by PWD, which also caused massive economic losses. The exploration into remote-sensing techniques and algorithms for reconnaissance and mapping of the PWN infestation has been spurred by the severity of the present PWD outbreak in North China, the accompanying economic repercussions, and the necessity for forest management and sustainable development [2]. Therefore, in order to prevent and mitigate the different negative effects as well as to enhance decision making, it is necessary to develop precise and effective tools for monitoring forest disturbances in response to PWD [3].

For extensive forest disease surveillance, microwave and optical satellite remote-sensing techniques have been developed in the past decades [4,5,6]. The slow loss of leaf water and chlorophyll content during the infection phase is the essential tenet of remotely sensed monitoring of PWD using optical images. However, the availability of optical remote sensing of forest disturbances was constrained, since the optical satellite data were frequently missing in gloomy and foggy regions. Additionally, the optical signal is affected by the illumination and view geometries, which hinder the retrieval of forest disease situations [7,8]. Microwave remote sensing offers a valuable supplement and/or alternative to optical remote sensing for monitoring forest disease, because it uses active sensors (such as synthetic aperture radar (SAR)) to produce incident radiation and penetrate clouds, and it can thus acquire satellite images regardless of the weather and illumination conditions [9,10]. PWN disasters are characterized by the water stress that these disasters subject tree parts (such as foliage, stems, and trunk) to over the course of two to three months, significantly altering the dielectric constant of SAR [11]. Due to the drop in water content during the infestation period, the dielectric constant changes the SAR back-scattering characteristics, giving the potential to identify PWD using SAR.

Previous studies proposed a variety of SAR-based methods for monitoring forest diseases, from mining sensitive bands to using machine learning methods to establishing physically based retrieval models. Though the extremely complex imaging mechanism of the SAR signal restricts the physically based modeling and interpretation of forest water stress induced by insects like PWN, microwave backscattering modeling has been one of the fascinating topics in the remote sensing community over the past few decades [12,13]. The investigation of backscattering change in forest areas has proven challenging due to multiple factors (e.g., stand architecture, environmental conditions affecting the vegetation water content, and soil surface moisture). Different SAR vegetation indices, such as the radar change ratios (RCR) [14], have been suggested as an alternative to better depict the state of the forest by reducing the stochastic component of the signal. Machine-learning techniques in the SAR domain have the advantage of not requiring the intricate physically based modeling and, when properly trained, can produce relatively high classification accuracy [15,16,17]. Through polarization decomposition and other operations [18,19], it is possible to extract hundreds of polarimetric characteristics and vegetation indicators that are sensitive to forest conditions using full polarization SAR data. This considerably enhances the feature dimensionality in machine-learning models. The accuracy and precision of machine-learning models may typically be increased by increasing feature dimensionality.

However, there are difficulties in accurately monitoring the PWN infestation conditions across wide areas due to the restricted availability of ground surveys that provide training samples for machine-learning algorithms [8,20,21]. In general, ground-based field measurements cost a lot to employ and are difficult to access. For instance, earlier studies used a very little amount of ground reference to develop and test retrieval models, which inevitably had a negative impact on the model’s resilience and generality [6,8]. Comparably, unmanned aerial vehicles (UAV) are an efficient and low-cost alternative technique to collect reference data of forest diseases that were typically obtained by labor-intensive field campaigns [22,23]. Through combining machine-learning techniques and UAV-observed imagery, the close-range monitoring of species’ invasion and tree disease has been greatly improved [24,25,26]. On the contrary to the low effectiveness in extracting low-, mid-, and high-level feature representations in conventional machine-learning techniques, deep learning methods offer significant potential as the most efficient option [23]. Multiple types of sensors, RGB cameras, and both multi and hyperspectral cameras and LiDAR have been used in this context. Many studies have shown promising results for UAV-based data with tree detection and species classification with various CNN-based frameworks. Some scholars demonstrated the outperformance of a 3D-CNN to classify tree species (i.e., pine, spruce, and birch) in a boreal forest, with a combination of RGB and hyperspectral data [27]. Similarly, a previous study incorporated ResNet-18 in the DeepLabv3+ architecture to detect three types of palm trees in the Amazonian forest [24]. Another example applied a deep convolutional generative adversarial network (DCGAN) to discriminate between healthy and diseased pinus trees in a heavily dense forested park area [28]. Although recent developments in UAV technology offer prospects for accurately identifying infested trees at the forest stand level, the training samples for images with dozens of meters of resolution are still very few [20]. This study aims to fill this gap by creating a novel strategy for expanding training samples by the incorporation of UAV and high-resolution satellite imagery.

Linking the UAV surveys and high-resolution satellite imagery is a possible way to expand the training samples, because high-resolution imagery generally covers tens of kilometers [29]. In addition, the strategy of integrating UAV data and high-resolution satellite imagery to calibrate moderate-resolution models is one of the classical synergy modes of UAV and satellite imagery for remote-sensing applications [30]. The combination of UAV and satellite remote sensing has a significant potential to overcome current constraints of earth observations, as previous studies have reported [20,29]. Therefore, this study aims to answer: (1) how to integrate UAV and SAR satellite imagery into a two-level up-sampling framework for expanding training samples; and (2) whether the ability of mapping forest disturbance response to PWD can be improved by using the two-level up-sampling framework.

2. Data

2.1. Study Area

The study area was mainly composed of conifer forests located in both coastal flat terrain and complex mountains (Figure 1). The forests are predominantly even-aged stands of black pine (Pinus thunbergii) and some red pine (Pinus densiflora) forest stands, which are mainly located at elevations ranging from 0 to 480 m. The region characterizes a warm temperate monsoon continental climate with obvious changes across seasons. The pioneer pine wood nematodes (PWN) in the study area were found in the Autumn of 2016 and have become widespread since then, particularly resulting in the rapid mortality of a large area of coastal defense plantations and mountain forests in the east of the study area. The PWN-induced black pine mortality reached its peak between 2019 and 2020 in this region. The PWD outbreak and expansion are possibly caused by the propagation of PWN hosted in Monochamus alternatus from the neighboring provinces. Many efforts have been undertaken to monitor the infested forest areas by field surveys and to protect the healthy forest areas by cutting infested trees.

2.2. UAV Field Campaign and Dataset

In order to produce the ultrahigh-resolution PWN infestation map of local regions, we acquired a UAV dataset in a field campaign conducted from 15–20 August 2021. According to infestation levels and access options, we distributed 17 forest sample plots around the study area on both flat and mountainous terrain, with an approximate 100 × 100 m² size (Figure 1). These plots were positioned according to the trade-off between accessibility and the random sampling rule. The canopy coverage of these plots ranges from about 0.2 to about 0.9. The color of the PWN-infested tree ranges from red to gray.

Aerial surveys of the 17 selected infested forest plots were conducted (

Table 1. Overview of multi-scale data used in this study.

Data Source	Wave Length	Acquisition Time	Resolution	Data Coverage
UAV	RGB	15–20 August 2021	~5 cm	17 plots
Radarsat-2	C-band	5 September 2021	~8 m	Part of Muping district
Sentinel-1	C-band	15 September 2021	30 m (Resampled)	Entire Muping district

). The DJI Phantom 4 Multispectral quadcopter (DJI Technology Co., Ltd., Shenzhen, China) equipped with a real-time kinematic (RTK) Global Navigation Satellite System (GNSS) module, barometer, compass, and inertial measurement unit (IMU) was used to collect remotely sensed images. The integrated multispectral imaging system was composed of an RGB camera and five individual narrow-band cameras (i.e., blue (450 nm ± 16 nm), green (560 nm ± 16 nm), red (650 nm ± 16 nm), red-edge (730 nm ± 16 nm), and near-infrared (840 nm ± 26 nm). Each camera provides an image size of around two million pixels with a 16-bit radiometric resolution. The focal length was fixed at 5.74 mm, resulting in a field of view (FOV) of 46° × 38°. The UAV-based images were acquired at a flight altitude of 100 m above ground level, thereby resulting in a ground sample distance of approximately 5 cm. The flight paths were designed to produce a minimum photographic side overlap of 70% and a forward overlap of 80%. The UAV also acquired multispectral images of a Lambertian panel with a reflectance of 0.5 placed on an open space next to the plot before the UAV left the home point and after returning to the home point. After data acquisition, RGB geo-rectified orthomosaics were generated using Agisoft PhotoScan Pro (Agisoft LLC, St. Petersburg, Russia) in places in which the geolocation accuracy was less than 5 cm. The detailed technical workflow of orthomosaic production can be found in [22].

2.3. Radarsat-2 Imagery

In order to expand the training samples based on UAV data, we introduced Radarsat-2 imagery to broaden the spatial extent of PWN infestation surveys (Table 1). All of the forest plots in the study area were covered by one single quad-polarization Radarsat-2 image that was taken on 5 September 2021, with an 8 m resolution and a 25 km by 25 km swath. The C-band SAR on board the Radarsat-2 satellite collects data in any combination of horizontal and vertical polarizations (i.e., HH, HV, VV, VH). The SNAP version 8.0 software was employed to obtain the backscattering coefficient and the polarimetric values. We first performed radiometric calibration to transform raw pixel values into radar backscatter coefficients. Next, we applied a Refined Lee speckle filter with a 5 × 5 window size to reduce the speckle and maintain the valuable information. We then extracted the polarimetric parameters such as span, pedestal height (PH), radar vegetation index (RVI), radar forest degradation index (RFDI), canopy scattering index (CSI), and volume scattering index (VSI) [31]. We used model-based decomposition methods that decompose the scattering matrix into different scattering mechanisms (i.e., direct, double-bounce, and volume scattering mechanisms), such as the Freeman–Durden, Yamaguchi, Cloude, Touzi and Van Zyl methods [19]. Additionally, several other polarimetric decomposition methods, such as Sinclair decomposition and Pauli decomposition, were also applied [32]. All the decomposed images were finally geocoded using the Range-Doppler terrain correction with a Shuttle Radar Topography Mission (SRTM) 30-m-resolution digital elevation model (DEM). Finally, the conifer forest pixels within the decomposed images were extracted for modeling the infestation severity using the 30 m resolution land cover-type map.

2.4. Sentinel-1 Imagery

In order to map the PWN infestation situation over the study area, we applied the proposed algorithm onto the Sentinel-1 imagery (Table 1). The Sentinel-1 image captured on 15 September 2021 covering the entire study area was extracted and downloaded from the Google Earth Engine (GEE). The 30 m resolution SRTM DEM was also downloaded from GEE. The Sentinel-1 mission is a constellation comprised of two satellites in a sun-synchronous orbit with a 180° difference in their orbital phasing. These satellites are equipped with C-band synthetic aperture radar equipment that collect data in all weather conditions. The primary operational mode over land is the Interferometric Wide (IW) swath mode, which provides data in either single (HH or VV) or double (HH + HV or VV + VH) polarization with a ground resolution of 5 m × 20 m. The level-1 Ground Range-Detected (GRD) product image that GEE downloaded was routinely resampled into a 30-m spatial resolution. The pre-processing steps including terrain correction and radiometric calibration as well as speckle filtering were executed before the download. The image consisted of two layers of the backscattering coefficient in double polarizations VV and VH that were converted to decibels (dB).

3. Methodology

The general workflow of this study is shown in Figure 2, which mainly consists of three steps: (1) detection of PWN-infested trees from UAV images and production of Radarsat-scale reference infestation map; (2) Radarsat-2-based modeling of PWN infestation and Sentinel-scale reference infestation map production; (3) Sentinel-1-based modeling of PWN infestation and regional infestation map production.

In the presented study, we proposed a two-level up-sampling scheme for PWN infestation mapping over large areas. The UAV-based infestation map was firstly scaled up to high-resolution Radarsat-2 scale to calibrate the PWN infestation mapping model from Radarsat-2 data. Subsequently, the Radarsat-2 infestation map was used as reference data to train a machine-learning model to map Sentinel-1-based infestation at 30 m resolution. In this process, we used an intermediate Radarsat-2 image to scale up from UAV to Sentinel-1 imagery because the intermediate layer could expand our training dataset beyond the circa 100 pixels that would have been produced from only using UAV images. The produced Sentinel-1 infestation map was validated by independent Radarsat-2 data and UAV data that were not used in the up-sampling and/or training process. The following sub-sections provide details of the steps.

3.1. UAV-Based PWN Infestation Modeling, Predicting, and Up-Sampling

3.1.1. Labeling of Infested Trees on UAV Orthomosaic

On the UAV-based orthomosaic, all the infested trees with red tones were annotated through rectangle-shaped bounding boxes. Correspondingly, each labeling item is composed of a bounding box and its associated class (i.e., infested tree). The orthomosaic was divided into numerous image tiles with a 448 by 448-pixel image size in order to meet the requirements of a particular deep-learning model that is described in the following section. To augment the labeling dataset, each orthomosaic was divided into numerous overlapping image tiles with the number of overlapping rows for two successive images along the column direction fixed to 350 pixels.

3.1.2. Deep-Learning Modeling and Detection of Infested Trees

In this study, we opted for a YOLO-type model [33], as these have been reported to attain similar precision yet higher prediction speed compared with other alternative object-detection CNN models (e.g., faster R-CNN). Within the YOLO family, we employed the latest YOLOv5 implemented in Pytorch (https://github.com/ultralytics/yolov5 (accessed on 26 September 2022)), i.e., YOLOv5 [34], in the modeling and detection of infested trees due to the increased accuracy and speed over previous YOLO versions. YOLOv5 is composed of three components, which are the backbone, neck, and head, respectively. The backbone module uses Cross-Stage Partial Networks to efficiently extract image features from an input image. The neck module uses a Path Aggregation Network (PANet) to generate feature pyramids to improve the generality on object scaling. The head module of YOLOv5 is the same as the previous versions (e.g., YOLOv3), which includes applying anchor boxes on features, generating output vectors with the associated class confidence, objectness score, and bounding boxes. Among the available YOLOv5 architectures, we opted for the YOLOv5x model due to its outperformance over smaller architectures. In our study, we replaced the PANet by a Bi-directional Feature Pyramid Network (BiFPN).

We used standard data augmentation tricks including random crops, rotations, hue, saturation, and exposure shifts before the model training. The labeling dataset was randomly divided into a training dataset (60%), validation dataset (20%), and testing dataset (20%) on an item-by-item basis. The maximum epoch, batch size, and learning rate were set as 100, 8, and 0.0032, respectively. The default patience threshold of 100 was set to interrupt the model training process after 100 consecutive epoches without improvements. The model depth multiple and layer channel multiple of YOLOv5 were set as 1.33 and 1.25, respectively. The YOLOv5 model was run on an Intel(R) Core(TM) i9-10900K 3.70 GHz × 16 processer (CPU) with 32 GB of RAM under the WINDOWS 10 operating system (64-bit). We used accuracy metrics, i.e., precision, recall, accuracy and average precision (AP), to measure the performance of the trained YOLOv5 model based on the testing dataset. The AP corresponding to intersection over union (IoU) thresholds of 0.5 ([email protected]), 0.75 ([email protected]). and 0.95 ([email protected]) were provided. Meanwhile, the APs over different IoU thresholds ranging from 0.5 to 0.95 (mAP@ (0.5:0.95)) were averaged to use to be an additional metric. Here, according to the values of precision and recall, the precision–recall curve can be drawn with the precision as the y-axis and the recall as the x-axis, and the average precision (AP) is the area under the precision–recall curve. Therefore, the integral of the precision–recall curve is the AP. The following formulas show the calculation of these accuracy metrics where TP, FP, TN, and FN refer to true positive, false positive, true negative, and false negative, respectively:

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(1)

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(2)

$$\mathrm{Accuracy}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$$

(3)

$$\mathrm{AP}={{\displaystyle \int }}_0^1\mathrm{p}\left(\mathrm{r}\right)\mathrm{dr}$$

(4)

3.1.3. UAV-Based PWN Infestation Map and Up-Sampling

The trained YOLOv5 model was applied onto the orthomosaic of each forest plot to detect infested trees. Every detected tree was marked by a bounding box that records the pixel coordinates of the box vertices. As a proxy for infestation severity, we measured the canopy coverage of infected trees. Given that the bounding boxes usually contain forest floor pixels, we extracted a central region from a bounding box with the empirical ratio of 2/3 to improve the accuracy of the infested tree crown cover estimation. The pixel value of all the extracted regions was then assigned as 1, and the remaining pixel values were 0 in the orthomosaic, hence producing the georeferenced infestation mask image.

To generate the labeling data of Radarsat-2 image pixels, we resampled the UAV-based georeferenced infestation mask image according to the actual pixel resolution of Radarsat-2 imagery. As the Radarsat-2 image offered the geographical position of the center of each pixel, we derived the geolocations of four vertices of each pixel and projected them onto the UAV infestation mask image to obtain the image tile corresponding to the satellite pixel. The infestation severity, defined as the percentage of infested tree-crown cover in the unit horizontal surface area, was calculated by averaging the pixel values within the image tile. Finally, the PWN infestation of the many Radarsat-2 pixels was obtained.

3.2. Radarsat-2-Based PWN Infestation Modeling, Prediction, and Up-Sampling

We used a random forest (RF) machine-learning algorithm to model the relationship between Radarsat-2-derived parameters and PWN infestation situations, as it has been widely used in the classification and regression problems in the remote-sensing domain due to its high accuracy and robustness [2,35,36]. RF is a non-parametric ensemble learning algorithm composed of many independent decision trees in which each tree contributes with a single vote for the assignation of the most frequent class to the input vector [37]. The vote determination at each node is usually based on the Gini criterion. The ensemble approach in RF is known as bootstrap aggregation or bagging because trees are built using bootstrap sampling, and the results of all the trees are then combined. RF can efficiently process high-dimensional data and show less proneness to overfitting and is easy to tune [38]. RF can also measure the relative importance of the different input features or variables during the classification process. Technically, RF switches one of the input random variables while keeping the rest constant and then measures the accuracy decrease which occurred using the Gini Index decrease.

The input Radarsat-2 variables of an RF model contained the various polarimetric parameters derived by multiple decomposition methods mentioned in Section 2.3 and the vegetation indices as shown in

Table 2. Summary of the input variables of the random forest (RF) model used in the modeling of PWN infestation from Radarsat-2 image. The variables are comprised of backscattering-derived parameters and polarimetric decomposition parameters.

Methods	Input Variables	Reference
Backscattering-derived parameters
Indices	Span, PH, RVI, RFDI, CSI, VSI	[19]
Polarimetric decomposition parameters
Freeman–Durden	Freeman_dbl 1, Freeman_surf 2, Freeman_vol 3	[39]
Yamaguchi	Yamaguchi_dbl, Yamaguchi_surf, Yamaguchi_vol, Yamaguchi_hlx	[40]
Cloude	Cloude_dbl, Cloude_surf, Cloude_vol	[41]
Touzi	Touzi_alpha, Touzi_phi, Touzi_psi, Touzi_tau	[42]
Van Zyl	VanZyl_dbl, VanZyl_surf, VanZyl_vol_g	[43]
H/α/A	alpha, anisotropy, entropy	[41]
Sinclair	Sinclair_1, Sinclair_2, Sinclair_3	[44]
Pauli	Pauli_1, Pauli_2, Pauli_3	[41]

Note: ¹ dbl refers to double-bounce scattering, ² surf refers to direct surface scattering, ³ vol refers to volume scattering.

. In addition, the SRTM DEM data resampled into Radarsat-scale resolution using the Kriging spatial interpolation method were also involved in the input variables. RF needs to tune two parameters in model creation and training, i.e., the number of trees and the number of input variables, when splitting at each tree node. In this study, the random forest package within the statistical software R was used. We modified both parameters across a wide range of values until reaching convergence, i.e., the optimal model parameters. We trained the RF model using a five-fold cross-validation with five repetitions. The number of predictors randomly picked as candidates at each split, known as mtry, is the most crucial parameter to be tweaked in the RF model. In our case, the optimal RF model consisted of 1000 trees with eight explanatory variables randomly selected at each tree node.

The trained RF model was used to predict the PWN infestation from the Radarsat-2 image. We assumed that the predicted PWN infestation could provide reference data while modeling the PWN infestation of the Sentinel-1 image. Therefore, the predicted PWN infestation map was resampled into a 30-m resolution, remaining consistent with the resampled Sentinel-1 image.

3.3. Sentinel-1-Based PWN Infestation Modeling and Predicting

The RF model was also used in the modeling relationship between the PWN infestation and the Sentinel-1 image pixel values. The input variables of the RF model consisted of backscattering coefficients and vegetation indices, as shown in

Table 3. Overview of the input variables of the random forest (RF) model used in the modeling of PWN infestation from Sentinel-1 image.

Input Variables	Formula
Backscattering coefficient	${\sigma }_{VH}$, ${\sigma }_{VV}$
Normalized difference polarimetric ratio (NDPR)	$\frac{{\sigma }_{VV}-{\sigma }_{VH}}{{\sigma }_{VV}+{\sigma }_{VH}}$
Polarimetric Ratio (PR)	${\sigma }_{VV}-{\sigma }_{VH}$
Radar vegetation index (RVI)	$4/\left({10}^{\frac{{\sigma }_{VV}-{\sigma }_{VH}}{10}+1}\right)$

. As described in the above section, the RF model parameters were also iteratively tuned to determine the optimal number of trees and the number of splitting variables. In our case, the optimal RF model of Sentinel-1 included 1000 trees with five explanatory variables randomly selected at each tree node. The trained RF model was then applied to predict the PWN infestation from the Sentinel-1 image, hence producing the regional PWN infestation map.

4. Results

4.1. UAV-Level PWN Infestation Evaluation

The trained object-detection CNN model was applied for the UAV RGB orthomosaic of each plot. As shown by an example plot in Figure 3a, almost all PWN-infested trees were effectively detected, although less healthy trees and soil patches were misclassified.

The accuracy evaluation based on the test dataset concluded that all accuracy metrics (i.e., recall, precision, accuracy, and [email protected]) were beyond 70% (Figure 4a), demonstrating that the deep-learning algorithm was capable of accurately detecting infested trees in a variety of environmental conditions. The [email protected], [email protected], and [email protected] were 0.74, 0.26, and 0.0002, respectively. The AP@[0.5:0.95] was 0.34, which is comparable with previous related studies, e.g., [45]. The accuracy of the CNN model was further evaluated based on visual detection on each UAV orthomosaic (a total of 8071 infested trees were visually determinated) (Figure 4b), which showed good fitting with a 1:1 line (y = 1.08x + 7.60) and an R² of 0.94. In order to produce a reference infestation severity map, we therefore assumed that the deep-learning-based infested tree detection was 100% correct.

As described in Section 3.1.3, the two-thirds area in the center of each detected bounding box was considered as pure pixels of infested trees. According to the pixel footprint of the Radarsat-2 image, the infested tree pixels were resampled into Radarsat-2 resolution (Figure 3b). It should be noted that we assumed both the UAV orthomosaic and Radarsat-2 images were with highly accurate geographical positioning. In the resample process, the infestation severity was calculated based on the infested area in the resampled pixels (Figure 3c). To minimize uncertainty in the labeling dataset, the pixels with an infestation severity of larger than 0.25 were extracted to produce the Radarsat-2 labeling dataset (Figure 3d).

4.2. Mapping PWN Infestation Map from Radarsat-2

We used the trained random forest algorithm to predict the PWN infestation situations from the Radarsat-2 image with input variables including backscattering coefficient -related indices and polarimetric decomposition parameters. The confusion matrix is shown in

Table 4. Confusion matrices and accuracy indices of random forest algorithms corresponding to Radarsat-2 and Sentinel-1 data.

Random Forest	Radarsat-2		Sentinel-1
	Infested	Healthy	Infested	Healthy
Infested	80.67%	37.50%	83.07%	12.37%
Healthy	19.33%	62.50%	16.93%	87.63%
Overall accuracy	72.57%		85.04%
Kappa coefficient	0.44		0.70

. The overall accuracy and Kappa coefficient of the PWN infestation classification were 72.57% and 0.44, respectively.

As shown in Figure 5, the infested pixels were highlighted by orange color. The PWN infestation follows an increasing trend along the transition flat region and mountain region and tends to be concentrated in complex, rugged terrain areas. Compared to UAV images, the infestation map of Radarsat-2 largely expanded the training data for Sentinel-1 images.

The importance ranking of the top 20 variables of the random forest algorithm using mean decrease Gini is illustrated in Figure 6.

The Span shows the most contribution to the infestation classification with the following variables’ polarimetric decomposition parameters. There was no apparent difference of importance among the polarimetric parameters found.

4.3. Sentinel-Level PWN Infestation Mapping

As shown in Figure 7, the PWN infestation map from the Sentinel-1 image was predicted using a trained random forest algorithm described in Section 3.3. The overall accuracy and Kappa coefficient of the PWN infestation classification based on the Radarsat-2 infestation map were 85.04% and 0.70 (Table 4), respectively. For each pixel, the corresponding backscattering coefficient and indice data were extracted from Sentinel-1 data as input to the random forest algorithm. Apparently, the PWN infestation areas were concentrated in the continuous pine forest region and tend to be severe in rugged terrain. The spatial distribution of the PWN infestation followed the cluster-like mode. The lesser infestation along the coast was noticed due to the felling of infested trees by the local authority. Because of the access difficulty, the infestation situations in mountain regions were not destroyed. This suggested that effective forest management strategies should be implemented to minimize further PWN spreading. The importance ranking of the top three variables of the random forest algorithm using mean decrease Gini is illustrated in Figure 8. The backscattering coefficient of VH polarization was found to be most important, followed by VV polarization.

We used independent UAV data that were not used in the up-sampling and/or training process to validate the Sentinel-1-derived PWN infestation map. As the direct co-registration between the UAV image and the resampled Sentinel-1 image is quite challenging, we adjusted the corresponding relationship between the resampled 30-m pixels and UAV image footprint to obtain the optimal validation accuracy of the Sentinel-1 infestation map. Figure 9 shows the optimal overall accuracy and Kappa coefficients under different UAV-based infestation severity thresholds wherein the threshold divides the infestation severity into infested pixels and non-infested pixels. As Section 3.1.3 and Figure 3 mention, the satellite-level infestation map was produced by using the threshold. With the increase in threshold, both the overall accuracy and the Kappa coefficient gradually increased except the case of the threshold on 0.3. This can be explained by the fact that a higher threshold improved the quality of the training dataset and decreased the uncertainty in the training dataset. In particular, the threshold of 0.3 led to a decrease in the Kappa coefficient due to the lesser amount of the training dataset. Figure 9 demonstrates that the threshold of infestation severity of 0.25 is most suitable. Therefore, the overall accuracy and Kappa coefficient of the Sentinel-1 PWN infestation based on the UAV-based infestation reference were 76.30% and 0.45.

5. Discussion

As an active remote-sensing technique, the multi-scale SAR data offers opportunities to monitor the dynamics and spreading of a PWN infestation in less-limited weather conditions. However, accurately monitoring the PWN infestation situations over large areas has been challenging due to the limited availability of ground surveys that offer training samples for machine-learning algorithms. To address this problem, we integrated unmanned aerial vehicle (UAV) surveys and Radarsat-2 satellite imagery in an up-sampling approach for expanding the number of training samples at the 30-m resampled Sentinel-1 resolution. In other words, we determined a machine-learning model between UAV data and Radarsat-2 data in a small area and then applied the model onto Radarsat-2 data to increase the number of reference data for model training and validation in a Sentinel-1-scale machine-learning model of a PWN infestation. Although a similar idea has been applied in previous studies, e.g., [20,29,46], this study offers a crucial improvement, as the PWN infestation and active microwave satellite were specially considered as well as the UAV as an alternative of ground surveys was explored.

Our results first verified that the UAV-produced infestation map provided a rapid and reliable alternative for traditional ground-based field sampling (Figure 3 and Figure 4), supporting the training sample generation. Next, the integration of UAV and high-resolution Radarsat-2 images largely boosted the number of training samples of the Sentinel-1 (2283 effective samples) compared to the direct scaling up of the UAV surveys to Sentinel-1 pixels (109 effective samples). Here, the effective samples mean the samples with an infestation severity level of larger than 0.25. This implies that surveys in very limited spatial extents would be rather infeasible for training machine-learning models. This finding was consistent with several previous studies [20,29]. We demonstrated that the predicted Sentinel-1 infestation map was reliable, with an overall accuracy of larger than 75% (Table 4 and Figure 9), no matter the independent UAV surveys or Radarsat-2 data used in the validation process. The accuracy of our predictions is consistent with some previous studies [5,19,47]. For example, Meng et al. (2022) monitored the severity of a southern pine beetle infestation severity and its change in a temperate mixed forest using 30 m plot measurements and Landsat-8 imagery, resulting in an overall accuracy rate ranging from 68% to 91% for different input spectral vegetation indices [3].

The reader may notice that the overall accuracy of the Sentinel-1 PWN infestation map validated by the Radarsat-2 reference was higher than that validated by the UAV reference. This can be understood in terms of some errors that existed in the generation of the Radarsat-2 reference based on the UAV reference, which were propagated to the production of the Sentinel-1 map. In addition, as Figure 9 indicates, the empirical threshold also influences the model’s accuracy. Theoretically, the threshold should be zero, as a threshold larger than 0 will certainly result in the misdiagnosis of infested regions as healthy regions. When the percentage of infected trees is extremely low, the associated reflectance signal is quite near to that of healthy pixels, leading to severe confusion between infected and healthy pixels in the classification model. We experimentally determined the threshold at 0.25 after conducting numerous tests based on a trade-off between classification accuracy and infection detectability. A large discrepancy was found between the Kappa coefficient and the overall accuracy for Radarsat-2-based predictions. This can be explained by the fact that one of the classes (i.e., the infestation class) accounts for the large majority of our data, and the class was well-described. On the confusion matrix (Table 4), we can see that a healthy class was not very reliable. This will have a large impact on our Kappa coefficient, and this explains the large difference. More recently, some scholars did not agree on the fact that the Kappa is largely considered to be more robust than the OA, because the Kappa coefficient has not provided the useful information that it is supposed to bring [48,49].

The fine quad-polarization Radarsat-2 image enables the various vegetation indices and polarimetric decomposition parameters by different physical or semi-physical decomposition algorithms, which highly enriches the feature dimensions in the machine-learning process. As Figure 6 suggests, the polarimetric decomposition parameters contribute heavily to the random forest algorithm. For the sake of model simplicity, we kept only three input variables in the random forest algorithm of Sentinel-1, as they can provide a high enough accuracy of PWN infestation classification. Notwithstanding, additional optical satellite remote-sensing data may enhance the feature of infestation signal in space because of the strong relationship between the spectral reflectance and the canopy chlorophyll content [50]. Further study would focus on the integration of active SAR and passive optical remote sensing to explore the PWN infestation severity.

Notwithstanding the advantages of the proposed method in PWN infestation mapping, several technical factors remain that challenge its accuracy. One source of discrepancy can be attributed to co-registration errors between UAV images and satellite imagery and those between two-scale satellite imagery [51]. The co-registration errors are mainly caused by the uncertainties of satellite positioning and image geometrical correction, since the UAV-equipped RTK GNSS has a positioning accuracy of 5 cm. In this study, we assumed the random forest algorithms have optimal accuracy in the case of perfect co-registration. We determined the optimal random forest algorithms by tuning the relative position between two sources of data. However, this process might be with some uncertainties induced by spatial heterogeneity, which needs further investigation in the future. Another source of discrepancy might be the UAV-based infestation severity threshold (Figure 9). It should be noted that the predicted satellite-level infestation maps only contain infestation pixels and non-infestation pixels, but do not illustrate infestation severity levels. In practice, the forest stands with low infestation severity generally characterize a weak signal in satellite pixels, largely obstructing the accurate retrieval of infestation situations. In our study, we empirically defined a threshold to determine infestation and non-infestation classes. As Figure 9 suggests, a threshold of 0.25 was selected because of a trade-off between the amount of training samples and the validation accuracy, implying that the threshold might need adjustment in the case of more UAV surveys or other study areas. Third, the soil moisture content is a critical variable affecting the backscattering coefficients [52,53], thus reducing the applicability of trained random forest algorithms over limited areas, which might be especially severe in sparse forest stands. Jointly using the off-the-shelf satellite product of soil moisture content has the potential to alleviate this negative effect. Notwithstanding, the proposed two-level up-sampling method through the integration of multi-scale remote-sensing data offers a feasible alternative to mapping the PWN infestation situations over small and large areas.

6. Conclusions

This study investigated the integration of multi-scale remote sensing to monitor the moderately and highly severe PWN infestation of pine forests over large areas. We trained a random forest algorithm to predict the PWN infestation situations from Sentinel-1 image data with selected training examples obtained by up-sampling UAV surveys and Radarsat-2 data. The UAV surveys offered reliable detection of infested trees under a range of environmental conditions with an AP of larger than 70% and R² approaching 1. The up-sampling UAV infestation map into an intermediate scale enabled the production of the Radarsat-2 infestation map with an overall accuracy of 72.57%, largely expanding the training samples of the Sentinel-1-scale machine-learning model. Note that the training samples were determined using a threshold of infestation severity of 0.25 based on the predicted Radarsat-2-based infestation map. The predicted infestation map of Sentinel-1 was independently validated by both Radarsat-2 and UAV data, yielding an overall accuracy of 87.63% and 76.30%, respectively. It was found that the PWN infestation was concentrated in the continuous pine forests in mountain regions in study area. The pixels of shelter forests along the coast were found to be less infested due to the massive felling of infested trees, being consistent with the human field surveys, which verified the reliability of the predicted infestation map. Therefore, the transferable approach has the ability to monitor the PWN infestation over multiple scales suited for different requirements and local authorities. The proposed method could be further improved to monitor the moderate and high PWN infestation severity and to use optical satellite remote-sensing images.