Detection of Pine Wilt Disease Using Time Series UAV Imagery and Deep Learning Semantic Segmentation

Abstract

The purpose of this study was to enhance the detection accuracy for pine-wilt-diseased trees (PWDT) using time series UAV imagery (TSUI) and deep learning semantic segmentation (DLSS) techniques. The detailed methods to accomplish the research objectives were as follows. Considering the atypical and highly varied ecological characteristics of PWDT, DLSS algorithms of U-Net, SegNet, and DeepLab V3+ (ResNet18 and 50) were adopted. A total of 2350 PWDT were vectorized at 9 sites, and 795 images of 2000 damaged trees were used as training data and 200 images where 350 PWDT were found, were used as the test dataset. The felled trees were tracked and the pest-controlled trees were used as to ground truth the TSUI of at least 2 years to ensure the reliability of the constructed learning data. The results demonstrated that among the evaluated algorithms, DeepLab V3+ (ResNet50) achieved the best f1-score (0.742) and also provided the best recall (0.727). SegNet did not detect any shaded PWDT, but DeepLabV3+ (ResNet50) found most of the PWDT, especially those with atypical shapes near the felled trees. All algorithms except DeepLabV3+ (ResNet50) generated false positives for browned broadleaf trees. For the trees, all algorithms did not detect PWDT that had been dead for a long time and had lost most of their leaves or had turned gray. Most of the older PWDT have been logged, but for the few that remain, the relative lack of training data may be contributing to their poor detection. For land cover, the false positives occurred mainly in bare ground, shaded areas, roads, and rooftops. This study thus verified the potential use of semantic segmentation in the detection of forest diseases such as PWD, while the detection accuracy is anticipated to increase with the acquisition of adequate quantities of learning data in future.

1. Introduction

Pine wilt disease (PWD) arises when pinewood nematodes (Bursaphelenchus xylophilus) block the water and nutrient transfer in pine trees to cause dehydration and consequent death [1]. Pinewood nematodes are transmitted through pine sawyer beetles, and the two main vectors on the Korean Peninsula are Japanese Pine Sawyer (Monochamus alternatus) and Sakhalin Pine Longicorn Beetle (Monochamus saltuarius). Monochamus alternatus is a southern species and is known to be distributed in southern Korea, all regions except Hokkaido in Japan, central and southern China, northern Vietnam, and Laos, while Monochamus saltuarius is a northern species and is distributed in central Korea, cold regions of Japan, northeast China, Russia, and Finland [2,3]. In Korea, the degree of infection and damage depends on the tree species in the pine genus; among the most susceptible are Black Pine (Pinus thunbergii Parl), Japanese Red Pine (Pinus densiflora) and Korean White Pine (Pinus koraiensis) [4].

PWD was first reported in North America [5], while it is the most severe forest disease in East Asia at present [6]. The disease has also induced serious damages to pine trees in Europe including Portugal [5]. Ecological damage has led to economic damage. In China, hundreds of millions of pine trees were damaged over a 700,000 ha area, and the economic loss was as high as CNY 30 billion [7].

As PWD disperses rapidly, the assessment of the current level of damage is difficult [8,9]. In addition, the temperature rise due to climate change has increased winter pests with a higher probability of localization, while subtropical pests have emerged [10]. The only available method to prevent the spread of PWD is early detection followed by tree felling and fumigation [11]. Nonetheless, the analysis and accurate prediction of damaged areas are limited due to low geographical accessibility and economic constraints in the case of conventional land monitoring [9,12,13].

Various attempts have been made to use UAV imagery for PWDT detection, particularly using multispectral imagery and vegetation indices [9,11], time series imagery [14,15,16], and deep learning (DL) [14,17,18]. To summarize the studies using vegetation indices and multispectral imagery, Kim et al. [8] performed a UAV forecasting mainly targeting the boundary areas of the PWD damaged area so as to resolve the difficulty in differentiating the regions with similar color values in automatic sorting of RGB images. Jung and Kim [9] determined the optimal vegetation index via a comparative analysis of the estimation accuracy of various vegetation indices including Normalized Difference Vegetation Index (NDVI), Normalized Difference Red Edge (NDRE), and satellite constellation visualization (SAVI). Kim et al. [16] used time series hyperspectral imaging to characterize the spatial distribution of PWDT adopting NDVI and Vegetation Index green (VIgreen), and the results showed the characteristics of the overall dispersion of PWDT in a large-scale area based on the data of PWDT as a function of time. These studies suggest the potential for using vegetation indices, but they use them as a means of comparing vegetation indices and studying their spatial distribution rather than their detection accuracy. In addition, in the case of time series images, the focus is on improving classification accuracy through seasonal differences, and no studies have been conducted on the accuracy of training images by tracking damaged trees.

The studies on PWD that apply DL are as follows. Yu et al. [19] obtained multispectral images to analyze and compare the target detection and machine learning algorithms. In target detection, two algorithms, Faster R-convolutional neural network (CNN) and YOLOv4, were used. In machine learning, two algorithms, namely Random Forest and Support Vector Machine, were used. The analysis indicated that the accuracy was higher for the machine learning algorithms although direct identification of images from objects could not be performed. Lim and Do [17] applied SegNet as a semantic segmentation algorithm and YOLOv2 as an object classification algorithm for automatic detection of PWDT data. The result showed approximately 57% accuracy for SegNet and 77% for YOLOv2. Hwang et al. [20] evaluated the accuracy using semantic segmentation, an object detection model, to compensate for the weakness that the object recognition model is effective for object recognition but does not reflect the location, shape, and area of the object. The models used were SegNet, FCN, U-Net, and DeepLab3, and the average classification accuracy of each model was about 86.9%, with a maximum classification accuracy of about 90%. It can be seen that the object detection model has a similar level of classification accuracy compared to the existing object recognition model.

Xia et al. [21] trained and assessed a diversity of DL models including fully convolutional networks (FCN), U-Net, DeepLabv3+, and SegNet, for the pine tree forests in the Laoshan district, Tsingtao, China. The detection accuracy of DeepLabv3+, a type of semantic segmentation, was shown to be the highest. Kim [18] applied the semantic segmentation technique to UAV and satellite images, and proposed a system to prevent the spread of PWD that can differentiate and visualize the infected, suspected, and healthy trees. Semantic segmentation has been used in the detection of objects with atypical shapes. Ronneberger et al. [22] proposed the U-Net algorithm, which showed outstanding performance even with a small amount of learning data based on data augmentation. Zhang et al. [14] composed a patch-based classifier with the ResNet18, an image segmentation algorithm, as the backbone; five-band multispectral images were formed in classification and heat map formats, and to solve the problem of insufficient learning data, geometric transformation and displacement were used to perform data augmentation.

The findings of previous studies can be summarized as follows. First, most of the existing studies utilize single-period images, which do not reflect the early, mid, and late characteristics of damaged trees over time. Second, there is a lack of research on detection accuracy in mixed forests or other land cover areas other than pine forests. In particular, it is necessary to study the detection accuracy of algorithms with regard to fall foliage or browning of broadleaf trees. Third, the atypical and highly varied ecological characteristics of PWDT are generally not reflected in the object classification. Fourth, object detection models are more suitable than object recognition models for images that contain a variety of information, such as high-resolution aerial photographs and UAV images [23]. Fifth, the need to ensure the reliability of the built training data is not clearly discussed, and most of the pilot studies are based on a small part of the research forest. Therefore, an in-depth discussion on the accuracy of detection of damaged trees by algorithms according to various ecological and natural environmental characteristics is needed.

In this study, enhancing the PWDT detection accuracy, time series UAV imagery (TSUI) was used and DLSS were adopted. To ensure the reliability of the training dataset, felled trees were tracked and pest-controlled trees were used as the ground truth for the TSUI of a minimum of 2 years. We adopted the DLSS method to improve the accuracy of detection of PWDT, and propose an optimal model through a comparative evaluation of representative models presented in previous studies.

2. Materials and Methods

2.1. Study Site

This study targeted 9 sites in Gyeongbuk, Gyeongnam, Gyeonggi, and Jeju Island regions that showed severe PWD and allowed the TSUI. Jeju is located in the south of the Korean Peninsula, Gyeongbuk and Gyeongnam in the southeast, and Gyeonggi Province in the center. A method of controlling PWD is felling with fumigation, incineration, and fragmentation, which occurs during the period between September and the following April. Thus, the TSUI at each site were obtained three to four times so as to allow the visual differentiation of PWDT. The imaging was performed three to four times at each site in May and September 2017, and May and September 2018 (Figure 1,

Table 1. Study sites and acquisition of TSUI.

Sites		Time Series UAV Imagery
		May 2017	September 2017	May 2018	September 2018
Gyeongbuk province	Bonghan-ri, Goa-eup, Gumi city	○	○	○	○
	Geoui-dong, Gumi city	○	○	○	○
	Yangwol-ri Angang-eup Gyeongju city	○	○	○	○
	Jungmyeong-ri Yeonil-eup, Nam-gu, Pohang city	○	○	○	○
Gyeonggi province	Seoha-ri Chowol-eup, Gwangju city	–	○	○	○
Gyeongnam province	Doyo-ri Sangnim-myeon, Gimhae city	○	○	○	○
	Geomam-ri Chodong-myeon, Miryang city	○	○	○	○
	Yongpyeong-dong, Miryang city	○	○	○	○
Jeju-do	Jeoji-ri Hangyeong-myeon, Jeju-do	○	○	○	○
Total		8	9	9	9

).

2.2. DLSS Data Construction

2.2.1. TSUI Acquisition

The UAV image acquisition was carried out in good weather conditions with low cloud cover, and the shooting time was from 10:00 a.m. to 3:00 p.m., taking into account the effect of mountain shading due to the altitude of the sun. The image data acquired in this study were collected using the KD-2 Mapper as a fixed-wing UAV and the RedEdge-M camera of Mica Sense (

Table 2. Specifications of KD-2 Mapper and Mica Sense RedEdge-M.

KD-2 Mapper		Mica Sense RedEdge-M
Category	Specifications	Category	Specifications
Model	Keva Drone KD-2 Mapper	Model	Mica Sense RedEdge-M
Wingspan	1.8 m	Weight	150 g
Length	1.1 m	Dimensions	12.1 cm × 6.6 cm × 4.6 cm
Weight	2.6 kg	Power	5.0 V DC, 4 W nominal
Battery	Li-ion (11.1 V, 12 Ah)	Spectral Band	Blue, Green, Red, RedEdge, NearIR
Cruise Speed	40~50 km/h	Ground Sample Distance	8.2 cm/Pixel (per band) at 120 m (400 ft.) AGL
Operation Time	60 min	Capture Speed	1 capture per second (all bands), 12-bit RAW
Operation Range	5 km
Wind Resistance	Cross wind 10 m/s

). RedEdge-M has 5 bands of blue, green, red, red-edge and near infra-red. To improve the radiometric quality of the imagery, a downwelling light sensor (DLS) is mounted on the top of the drone with the RedEdge-M. The DLS is a 5-band incident light sensor that connects directly to a MicaSense RedEdge-M camera. The DLS measures the ambient light during a flight for each of the five bands of the camera and records this information in the metadata of the TIFF images captured by the camera [24]. Before shooting, we used the Automatic Calibration Panel provided by Mica Sense to place the RedEdge sensor 1 m above the calibration panel, and then acquired metadata related to the calibration and corrected the reflection value by detecting the QR code and the panel to avoid shadows. With this information, PIX4Dmapper S/W (ver.4.7.5) [25] was used to correct for global lighting changes in the middle of a flight, such as those that can happen due to clouds covering the sun.

The flight altitude for image acquisition was set to approximately 350 m sea level [26], taking into account the surrounding topography, target site altitude above sea level, battery power, and wind impact. All of the sites analyzed are mountainous, with summits around 200~250 m above sea level. For example, the summit of Maebong Mt. in Bonghan-ri is 220 m, and Hyung Mt. in Jungmyeong-ri is 257.1 m. Therefore, we took into account the altitude of the mountainous terrain and adjusted the flight altitude to match the ground sample distance of 5.00~6.50 cm for each destination.

For overlap and sidelap, the rate was set high at 80% to prevent blind spots in forest regions. Prior to imaging, the automatic calibration panel provided by Mica Sense was used, and the RedEdge sensor was positioned 1 m above the calibration panel. Next, to prevent shading, the QR code and panel detection were used to collect the meta data regarding calibration and correct the reflectance. The acquired image data were merged using the PIX4Dmapper S/W (ver.4.7.5). Orthophotos were produced through the matching between the collected images and the ground control point determined based on the GRS80 TM coordinates obtained from the global navigation satellite system (GNSS) measurements. In addition, the accurate positional data of PWDT were obtained from the Korea Forestry Promotion Institute. The respective data were in relation to the central datum point determined based on the GRS80 TM coordinates. Next, the acquired TSUI were standardized for the positional data of PWDT with varying central points towards the central datum point using the QGIS S/W (ver.3.16.10) [27]. In addition, to minimize the coordinate errors between each TSI image, the UAV Imagery was calibrated. In this process, the coordinate reference function of the QGIS S/W (ver.3.16.10) was used to calibrate the images based on the primary TSI images. This was due to potential fine differences even among TSI images obtained in identical conditions as the wind speed could have had an influence, and as such microscopic differences on small-scale images could underlie substantial differences on larger scales of actual size.

2.2.2. GPS Point Acquisition for PWDT

Suspected infections found through resident reports, visual surveillance, and aerial surveillance are sampled with GPS points. If the primary diagnostic center confirms the infection, the secondary diagnostic center, the National Forestry Research Institute, will make the final diagnosis. The Korea Forestry Promotion Agency is in charge of the above process and has been managing GPS points of PWDT in the country since 2015. PWDT GPS points for this research were also obtained with the cooperation of the above organizations.

2.2.3. U-Net, SegNet, DeepLabv3+ Algorithms

Semantic segmentation is a deep learning algorithm that applies a segmentation label to the image in pixels [18]. The U-Net as a Semantic Segmentation algorithm is a type of supervised classification [28], which was first introduced in biomedical imaging to extract the cellular boundary data. As an artificial neural network applied in a variety of image segmentations, the U-Net algorithm allows the classification in pixels for the segmentation of objects in a given image so that it is characterized by the retention of the same resolution on the final classification output as the input, to be distinguished from other convolutional neural networks [22]. The U-Net algorithm is mainly used for image segmentation using a small amount of data [29,30]. In this study, the learning data were constructed through vectorizing to extract the boundary data for PWDT from the ground truth based on time series UAV images.

The SegNet algorithm is based on the concept of semantic segmentation, which recognizes objects on a pixel-by-pixel basis and predicts which object a pixel at that location represents. It is primarily motivated by road scene understanding applications which require the ability to model appearance (road, building) and shape (cars, pedestrians) and to understand the spatial relationship (context) between different classes such as road and sidewalk. The engine must also have the ability to delineate objects based on their shape despite their small size. Hence it is important to retain boundary information in the extracted image representation [22]. It uses convolution and maxpooling to speed up learning and has a network structure that minimizes the loss of location information through compression and upsampling [23]. Hence, it is designed to be efficient both in terms of memory and computational time during inference. It is also significantly smaller in the number of trainable parameters than other competing architectures. The key component of SegNet is the decoder network which consists of a hierarchy of decoders, one corresponding to each encoder. Of these, the appropriate decoders use the maxpooling indices received from the corresponding encoder to perform non-linear upsampling of their input feature maps. This idea was inspired from an architecture designed for unsupervised feature learning [31].

DeepLabV3+ extends DeepLabV3 by adding a simple yet effective decoder module to recover the object boundaries [32]. DeepLabV3 employs the spatial pyramid pooling module with the encoder–decoder structure. DeepLabV3+ contains rich semantic information from the encoder module, while the detailed object boundaries are recovered by the simple yet effective decoder module. The encoder module allows us to extract features at an arbitrary resolution by applying atrous convolution [33]. DeepLabV3+ uses atrous separable convolution by combining atrous convolution with separable convolution in addition to the ResNet structure proposed in DeepLabV3 to solve the performance limitations of the existing fully convolutional network (FCN) [20]. In addition, low-level features were extracted using a deep convolutional neural network (DCNN) for the input image, and to obtain high-level features in the encoder, multi-scale targets were extracted after DCNN using atrous spatial pyramid pooling (ASPP). The ASPP technique has the advantage of efficiently utilizing various features of the image through atrous convolution [23].

In this study, the learning data were constructed through vectorizing to extract the boundary data for PWDT from the ground truth based on TSUI.

2.2.4. DLSS Data Construction Based on TSUI

To ensure the accuracy of the learning data for PWDT, a spatiotemporal analysis was performed on TSUI of identical regions to extract the areas of confirmed PWDT and construct the learning data. The pine trees present in the primary TSUI images but not present in the secondary TSI images may be conjectured to have been felled in the field due to PWDT. Hence, as shown in Figure 2, the artificially felled trees were regarded as the ground truth for PWDT on TSUI in accordance with the pest control plan.

The respective concepts are summarized in Figure 3, where the TSUI are given from the primary to the quaternary in line with the processed UAV imagery in this study. Despite the variation in time, pest control is ultimately performed on PWDT and the pest-controlled trees are confirmed as PWDT on the secondary to quaternary TSUI. The trees found in identical locations through the feedback on the primary to tertiary images were registered as the ground truth for PWDT.

TSUI and GPS points of the PWDT were then used to identify the affected trees, and a MATLAB S/W (ver. R2022b)-based vectorization program was used to derive the exact tree canopy of the affected trees. The images were saved as PNG files with a size of 256 × 256 pixels. A total of 2350 PWDT were vectorized at 9 sites, and 795 images of 2000 damaged trees at 8 sites were used as training data. In addition, 200 images from Angang-eup, Gyeongju city, Gyeongbuk province, where 350 damaged trees were found, were used as the test dataset. In some cases, one image contained more than one damaged tree, so the number of datasets was smaller than the number of damaged trees found (

Table 3. Sample data construction.

	Training Samples	Test Samples	Total
Dataset	795	200	995
PWDT	2000	350	2350

).

2.2.5. Evaluation Indicators

In this study, we used precision and recall metrics to determine the performance of DL algorithms. Precision and recall can be calculated by utilizing the values of true positives (TP), false positives (FP), and false negatives (FN), which are the results of correctly detecting a tree as a tree and incorrectly detecting a non-tree as a tree [34]. The f1-score is the harmonic mean of precision and recall and is commonly used for unbalanced data sets. Since there are disproportionately more data from healthy pine trees than from PWDT in this study, we used the f1-score, which is a performance measure for unbalanced data, to verify the optimized performance of DL algorithms [35,36]. Intersection over union (IOU) is a measurement based on the Jaccard Index, a coefficient of similarity for two sets of data [37,38]. IOU is a metric that evaluates object detection accuracy and determines whether the detection of an individual object was successful, and has a value between 0 and 1 [39]. In the object detection scope, the IOU is equal to the area of the overlap (intersection) between the predicted bounding box B_p and the ground-truth bounding box B_gt divided by the area of their union [38].

The above-mentioned metrics were calculated as:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y} = \frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(1)

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n} = \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}$$

(2)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l} = \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(3)

$$\mathrm{f}1\text{-}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e} = 2 \times \frac{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n} \times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n} + \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}$$

(4)

$$J\left(B_p,B_{gt}\right)=\mathrm{I}\mathrm{O}\mathrm{U}=\frac{area\left(B_p\cap B_{gt}\right)}{area\left(B_p\cup B_{gt}\right)},$$

(5)

3. Results

In the hyperparameter setting of each algorithm model, the epoch of U-Net and DeepLabV3+ [23] was set to 100, and the epoch of DeepLabV3+ (ResNet50) and SegNet was set to 50. The MiniBatchSize setting was set to 20 for DeepLabV3+ (ResNet50) and 6 for the rest of the algorithms. Adaptive Moment Estimation (Adam) Optimizer was used for U-Net, DeepLabV3+ (ResNet18), and DeepLabV3+ (ResNet50); rmsprop Optimizer was used for SegNet; and training was performed with a constant learning rate of 0.001 (

Table 4. Setting hyperparameters for each algorithm.

Algorithm	U-Net	SegNet	DeepLab V3+ (ResNet18)	DeepLab V3+ (ResNet50)
Optimizer	Adam	rmsprop	Adam	Adam
Learning rate	0.001	0.001	0.001	0.001
Epoch	100	50	100	50
miniBatchSize	6	6	6	20
EncoderDepth	4	4	-	-

).

With the performance metrics results (

Table 5. Performance metrics results.

Algorithm	TP	FP	FN	TN
U-Net	48,767	14,111	23,739	13,016,160
SegNet	46,213	14,568	26,293	13,017,791
DeepLab V3+ (ResNet18)	48,808	18,534	23,698	13,020,583
DeepLab V3+ (ResNet50)	52,731	16,903	19,775	13,020,126

), when comparing each algorithm (

Table 6. Performance of different algorithms.

Algorithm	IOU	Accuracy	Precision	Recall	f1-Score
U-Net	0.563	0.997	0.776	0.673	0.720
SegNet	0.531	0.997	0.760	0.638	0.694
DeepLab V3+ (ResNet18)	0.536	0.996	0.725	0.672	0.698
DeepLab V3+ (ResNet50)	0.590	0.997	0.757	0.727	0.742

), DeepLabV3+ (ResNet50) achieved the best IOU and F1 score among the algorithms. DeepLabV3+ (ResNet50) had the best performance, with an f1-score of 0.742, followed by U-Net with 0.720, and the other two had similar values. For Precision, U-Net had the highest value of 0.776, while SegNet and DeepLabV3+ (ResNet50) had similar values. DeepLabV3+ (ResNet50) showed the best recall, with 0.742, while DeepLabV3+ (ResNet18) and U-Net showed similar values.

4. Discussion

Figure 4 is an example of how the algorithms used in this study segmented the PWDT. The first column is the test image for detection, the second is the ground truth image of the PWDT, and the third through sixth columns show the detection results of each algorithm. The five test images represent a variety of forest environments, including PWDT logged areas and bare ground.

Following is a summary of the findings for each image. Image 1 in Figure 4 shows a heavily logged area, and all algorithms fail to find early PWDT that are beginning to brown. SegNet did not detect any shaded PWDT, and U-Net and DeepLabV3+ barely detect the location of PWDT rather than their shape. DeepLabV3+ (ResNet50) found most of the PWDT, but false positives occurred in broadleaf trees that had begun to brown. Image 2 in Figure 4 shows the detection of PWDT near logged trees. While most algorithms are able to detect damaged trees, they are unable to detect PWDT with very small canopy widths or in shaded areas. Only DeepLabV3+ (Res-Net50) was able to detect PWDT with atypical shapes near felled trees. Image 3 in Figure 4 shows the detection accuracy in a mixed forest. All but DeepLabV3+ (ResNet50) showed false positives for browned hardwoods. According to the National Forest Statistics, the area of mixed forests in Korea is 28.1% of the total forest area, and if there is an outbreak of PWD in mixed forests, utilizing the DeepLabV3+ (ResNet50) algorithm will provide the highest detection accuracy. Alternatively, in the case of mixed forests, the detection accuracy can be improved by utilizing images from May, after the new leaves have appeared. Image 4 in Figure 4 shows late-stage dead trees that were not cut down in time. None of the algorithms is able to detect PWDT that have long since dried up and turned gray (dead PWDT). Dead PWDT are trees that have not been logged and fumigated in the current year and are the main cause of the spread of the disease. Not all affected trees are controlled due to the budget and period of control, and these dead PWDT are relatively lacking in training data for analysis because there are not many of them; this likely resulted in the failure to detect them as PWDT. Image 5 in Figure 4 shows the results for an area containing bare ground and broadleaf trees with foliage. U-Net has the highest false positive rate, followed by DeepLabV3+ (Res-Net50), with most of the false positives coming from broadleaf trees with foliage that is affected by daylight.

In detail, false positives occurred in roads, building rooftops, and daylight-affected broadleaf trees (Figure 5). Detection accuracy decreased for shaded PWDT or a grayish color PWDT that were long-dead and nearly leafless. In Images 2 and 4 of Figure 4, all but DeepLabV3+ (ResNet50) failed to detect PWDT that overlapped with surrounding pines and did not have a typical canopy shape. In Image 3 of Figure 4, all algorithms except DeepLabV3+ (ResNet50) detected the browned hardwoods as PWDT.

Overall, DeepLabV3+ (ResNet50) showed the best detection performance among the algorithms. Segnet and U-Net are more useful in detecting the exact shape and boundary, which is more useful for analyzing the area affected by PWD. In the case of DeepLabV3, the location accuracy of damaged trees is high, so it would be more desirable to quickly confirm suspected trees through field trips.

5. Conclusions

The purpose of this study was to enhance the detection accuracy for PWDT using TSUI and DLSS techniques. A total of 2350 PWDT were vectorized at 9 sites, and 795 images of 2000 damaged trees at 8 sites were used as training data. For the test dataset, 200 images from Angang-eup, Gyeongju city, Gyeongbuk province, where 350 damaged trees were found, were used.

This study is distinguished in several aspects as follows. First, the semantic segmentation models of U-net, SegNet and DeepLabV3+ were adopted and compared to determine which can best reflect the characteristics of atypical and morphologically diverse tree canopies. Second, to ensure the reliability of DL data in previous studies, felled trees were tracked and pest-controlled trees were used as the ground truth data for the TSUI for a minimum of 2 years.

The comparison of each algorithm shows that DeepLabV3+ (ResNet50) had the best performance, with an f1-score of 0.742, followed by U-Net with 0.720. For precision, U-Net had the highest value of 0.776, while SegNet and DeepLabV3+ (ResNet50) had similar values. For recall, DeepLabV3+ (ResNet50) showed the highest value (0.742), while DeepLabV3+ (ResNet50) and U-Net showed similar values. The false positives occurred mainly on bare ground, shaded areas, roads and rooftops, and were more common in broadleaf trees than conifers.

In future, a potential way to minimize overestimation is the construction of accurate learning data with an increase in datasets via data augmentation, which is predicted to further enhance the accuracy. The results of this study will contribute to the forecasting of major forest diseases such as PWD and oak wilt disease (OWD), while supporting more rapid detection and monitoring in addition to reduced manpower and cost of the conventional wilt disease detection.