Tree Species Classification in Subtropical Natural Forests Using High-Resolution UAV RGB and SuperView-1 Multispectral Imageries Based on Deep Learning Network Approaches: A Case Study within the Baima Snow Mountain National Nature Reserve, China

Abstract

Accurate information on dominant tree species and their spatial distribution in subtropical natural forests are key ecological monitoring factors for accurately characterizing forest biodiversity, depicting the tree competition mechanism and quantitatively evaluating forest ecosystem stability. In this study, the subtropical natural forest in northwest Yunnan province of China was selected as the study area. Firstly, an object-oriented multi-resolution segmentation (MRS) algorithm was used to segment individual tree crowns from the UAV RGB imagery and satellite multispectral imagery in the forests with different densities (low (547 n/ha), middle (753 n/ha) and high (1040 n/ha)), and parameters of the MRS algorithm were tested and optimized for accurately extracting the tree crown and position information of the individual tree. Secondly, the texture metrics of the UAV RGB imagery and the spectral metrics of the satellite multispectral imagery within the individual tree crown were extracted, and the random forest algorithm and three deep learning networks constructed in this study were utilized to classify the five dominant tree species. Finally, we compared and evaluated the performance of the random forest algorithm and three deep learning networks for dominant tree species classification using the field measurement data, and the influence of the number of training samples on the accuracy of dominant tree species classification using deep learning networks was investigated. The results showed that: (1) Stand density had little influence on individual tree segmentation using the object-oriented MRS algorithm. In the forests with different stand densities, the F₁ score of individual tree segmentation based on satellite multispectral imagery was 71.3–74.7%, and that based on UAV high-resolution RGB imagery was 75.4–79.2%. (2) The overall accuracy of dominant tree species classification using the light-weight network MobileNetV2 (OA = 71.11–82.22%), residual network ResNet34 (OA = 78.89–91.11%) and dense network DenseNet121 (OA = 81.11–94.44%) was higher than that of the random forest algorithm (OA = 60.00–64.44%), among which DenseNet121 had the highest overall accuracy. Texture metrics improved the overall accuracy of dominant tree species classification. (3) For the three deep learning networks, the changes in overall accuracy of dominant tree species classification influenced by the number of training samples were 2.69–4.28%.

1. Introduction

As the main part of the terrestrial ecosystem, forests play an important role in maintaining biodiversity, improving the ecological environment and mitigating climate change [1,2,3]. Natural forests have abundant biodiversity, which is essential for maintaining the stability of forest ecosystems [4,5,6]. Trees are the foundation of the forest ecosystem, and the composition of tree species is one of the key indicators to measure forest biodiversity [7]. The composition and spatial distribution of tree species are critical for forest resource management and forest biodiversity monitoring [8], which can be also used as an indicator of species richness to assess the potential of forest ecosystem services [9], explore aboveground biomass changes [10] and reveal competition among tree species [11]. The traditional method of obtaining tree species information mainly depends on the field inventory. However, this is time-consuming and labor-intensive [12,13]. Furthermore, tree species information is typically collected using sample plots, so the spatially continuous information is normally ignored [7]. Particularly in the natural forests with complex structures, it is difficult to obtain the fine-scale spatial distribution of tree species using a field inventory approach [14]. Remote sensing can provide multi-dimensional and spatially continuous information on forests, which makes it possible to access tree species composition in a timely and efficient manner [15].

Multispectral remote sensing can provide lots of spectral (or texture) and spatially continuous information about forests, which plays a key role in the identification of tree species [16,17]. However, due to the limitations of spatial and spectral resolution, traditional optical imagery is normally used to classify forest types [18]. Yu et al. [19] used SPOT-5 imagery to identify forest types in Pangu Forest Farm in the Daxing’an Mountains, and the overall accuracy of classification was 76.0%. With an improvement in spatial and spectral resolution, optical imagery can be used to identify the dominant tree species [20]. Cho et al. [21] used high-resolution WorldView-2 imagery (resolution = 0.5–2 m) to classify three dominant tree species in the native forests of South Africa, with an overall accuracy of 87.2–91.4%. Immitzer et al. [14] used 8-band WorldView-2 imagery combined with the random forest algorithm to classify 10 types of tree species in temperate forests, and the overall accuracy was 82.0%. Moreover, the combination of spectral indices and texture metrics can generally improve the accuracy of dominant tree species classification [22,23]. Deur et al. [24] used multispectral WorldView-3 satellite imagery to classify dominant tree species in deciduous mixed forests. The results showed that the overall accuracy of classification can be improved by 7.0–10.0% after adding texture metrics. Ferreira et al. [25] classified eight dominant tree species from Brazil’s semi-deciduous forests using the visible bands of WorldView-3 imagery combined with texture metrics. Compared with using the visible bands only, the average producer accuracy was improved by 22.0–34.0%.

Unmanned aerial vehicles (UAVs) can operate at a lower attitude to obtain ultra-high spatial resolution data [26]. In recent years, due to the advantages of low cost, high efficiency and fine precision, the application of UAVs in assessing forest regeneration, monitoring forest health and detecting individual trees has increased rapidly. Reis et al. [27] used UAVs to access multispectral images and classify them to evaluate the status of forest renewal, and the performance was good (OA = 94.0%). Dash et al. [28] obtained multispectral images of time series based on UAVs and simulated early stress on trees using non-parametric methods, which had a high classification accuracy (kappa = 0.694). Nuijten et al. [29] used a digital aerial photogrammetry point cloud (DAP) acquired via UAVs to segment individual tree crowns of deciduous forests in different seasons, with an accuracy of 55.0–77.1%. The high-resolution imagery of UAVs contains fine-scale forest canopy information [30], which can be used for the segmentation and detection of individual tree crowns. Marques et al. [31] used the clustering segmentation method to segment Chestnut trees from UAV high-resolution imagery with an accuracy of 97.0%. The object-oriented multi-resolution segmentation (MRS) algorithm aims to divide remote sensing imagery into objects with the same shape or spectral characteristics [32], in which the spectral variation in various ground objects could be reduced so the statistical and classification accuracy could be properly improved [33,34]. Xu et al. [35] used the MRS algorithm to segment individual trees from high-resolution RGB ortho-imagery, and the accuracy was 76.9%. However, most of the previous studies applied the MRS algorithm to plantation or the forest stands with simple structures, and there are few studies on the extraction of individual trees using the MRS algorithm in subtropical natural forests.

Deep learning is an algorithm that uses successive neural layers to reveal deeper features and more metaphysical information of images [36]. A convolutional neural network (CNN) in deep learning is commonly used to analyze the spatial features of remote sensing images, such as edges and textures, which could be suitable for tree species identification [37]. Natesan et al. [38] used a residual convolutional network to classify three types of tree species, named red pine, white pine and non-pine, in a coniferous mixed forest, with an overall classification of 80.1%. Schiefer et al. [37] used a U-Net convolutional neural network to classify nine dominant tree species in temperate forests, and the overall classification was 83.2%. Meanwhile, previous studies showed that the combination of CNN and high-resolution satellite images had a high accuracy for the identification of dominant tree species in mixed forests. Guo et al. [39] used three types of convolutional neural networks combined with WorldView-3 imagery to classify seven types of tree species in coniferous and broadleaf mixed forests. The results showed that the DenseNet model had the highest classification accuracy (OA = 75.1–78.1%). Yan et al. [40] used five types of CNN models combined with WorldView-3 imagery to classify six dominant tree species in a mixed plantation, and the results showed that the improved GoogLeNet model had the highest classification accuracy (OA = 82.7%). Compared with traditional machine learning algorithms, deep learning networks can improve the accuracy of dominant tree species classification [41]. Onishi et al. [42] used a ResNet model to classify dominant tree species in coniferous mixed forests. Compared with the support vector machine algorithm, the overall classification accuracy was improved by 5.8–13.1%. Zhang et al. [43] used ResNet50, KNN and BP neural network algorithms to classify 10 types of urban tree species, and the results showed that the overall accuracy of ResNet50 was, on average, 30% higher than KNN and BP neural network algorithms. However, there are few studies combining deep learning network algorithms with multi-source remote sensing data for dominant tree species classification, and the potential of using deep learning network algorithms to identify the dominant tree species in subtropical natural forests needs to be explored.

In view of the above research gaps, this study selected a subtropical natural forest in northwest Yunnan province of China as the study area and obtained high-resolution ortho-imagery (spatial resolution = 0.23 m) based on a fixed-wing UAV and sub-meter (spatial resolution = 0.5 m) multispectral imagery based on the SuperView-1 satellite. The random forest algorithm and three deep learning networks (lightweight network MobileNetV2, residual network ResNet34 and dense network DenseNet121) were constructed to classify the five dominant tree species. The main objectives of this paper are as follows: (1) to compare the segmentation accuracy of individual trees in forests with different stand densities (low (547 n/ha), middle (753 n/ha) and high (1040 n/ha)) from UAV high-resolution ortho-imagery and satellite multispectral imagery; (2) to compare and evaluate the overall accuracy of three deep learning networks and random forest algorithms combined with spectral and texture metrics for dominant tree species classification in natural forests; (3) to explore the variations in the overall accuracy of dominant tree species classification using three deep learning networks while reducing the number of training samples.

The workflow containing several steps of data processing is shown in Figure 1. In this study, we obtained high-resolution images from fixed-wing UAVs and multispectral imagery from the SuperView-1 satellite and then combined these with an individual tree segmentation algorithm, machine learning and multiple deep learning networks to identify the dominant tree species in the subtropical natural forests of northwest Yunnan province. Firstly, the RGB images obtained by the UAV were aligned to generate high-resolution ortho-imagery. Secondly, the object-oriented MRS algorithm was used to generate individual tree crown boundaries. Finally, the extracted individual tree crown metrics (i.e., multispectral and texture metrics) were combined with the random forest algorithm and three deep learning networks to classify the five dominant tree species.

2. Materials and Methods

2.1. Study Area

Gehuaqing (GHQ) is located in Weixi County, south of Diqing Tibetan Autonomous Prefecture, Northwest Yunnan Province of China, and the south of Baima Snow Mountain National Nature Reserve (98°57′–99°25′E, 27°24′–28°36′N), as shown in Figure 2. The elevation ranges from 2100 m to 3500 m. The study area belongs to the plateau mountain monsoon climate, which can be divided into two seasons: dry and wet. In the winter half year (November to April of the next year), the weather is sunny, and rainfall is reduced, showing the characteristics of dry season climate, while in the summer half year (May to October), the study area has more clouds and concentrated rain, forming the rainy season. The annual average temperature is 9.5 °C, the average temperature of the coldest month (January) is 1.5 °C and that of hottest month (July) is 17.3 °C. The annual average rainfall is 845.1 mm. The rate of forest coverage in the study area is 76.2%, and the dominant forest types can be divided into cold temperate conifer forests and deciduous broadleaf forests. The dominant conifer species is Yunnan pine (Pinus yunnanensis), and the dominant broadleaf species can be divided into four types: Nepal alder (Alnus nepalensis), aspen (Populus davidiana), Oriental white oak (Quercus aliena) and maple (Acer forrestii).

2.2. Field Measurements

Considering factors, such as altitude, slope and tree species types, a total of 450 trees in the study area were selected and measured. The measurement includes individual tree species and position information, diameter at breast height, tree top height and crown width. The position of the individual tree was recorded using a Trimble^® real-time kinematic (RTK) R4 GNSS with centimeter-level accuracy. The tree top height was measured via Vertex IV^® hypsometer. The tape was used to measure crown width, and the results were expressed as the average of two values measured in vertical directions of tree crown. Statistics for five dominant tree species parameters are aggregated in

Table 1. Summary statistics of the trees measured.

Scientific Name	N	DBH (cm)		Height (m)		Crown Width (m)
		Mean	SD	Mean	SD	Mean	SD
Alnus nepalensis (A.N.)	118	27.87	21.01	13.10	6.63	5.32	1.91
Quercus aliena (Q.A.)	80	55.72	19.37	23.41	6.07	6.76	2.30
Populus davidiana (P.D.)	97	30.96	23.07	11.34	5.56	5.44	2.22
Acer forrestii (A.F.)	80	37.90	18.65	14.59	4.93	6.97	2.20
Pinus yunnanensis (P.Y.)	75	38.91	17.95	22.23	8.04	5.31	1.58

Note: DBH: diameter at breast height (cm); H: height (m); N: number of trees; Alnus nepalensis = A.N.; Quercus aliena = Q.A.; Populus davidiana = P.D.; Acer forrestii = A.F.; Pinus yunnanensis = P.Y.

.

2.3. Remote Sensing Data

A fixed-wing UAV equipped with RGB camera (Sony ILCE-6000) was used to obtain RGB images from 7 November to 8 November 2018. The weather condition was clear without clouds. The UAV flew at an altitude of 800 m relative to the ground, with a speed of 20 m/s, a forward overlap ratio of more than 80% and the average ground sample distance was 0.23 m. The satellite multispectral imagery was shot by SuperView-1 satellite on 4 December 2018, with approximately 0% cloud cover and covering the whole study area. The multispectral imagery included five bands of panchromatic (450–890 nm), blue (450–520 nm), green (520–590 nm), red (630–690 nm) and near-infrared (690–890 nm), in which the spatial resolution of panchromatic band was 0.5 m and that of spectral bands was 2 m.

2.4. Data Pre-Processing

Before stitching RGB images, the RGB images with poor quality (including blurred images and images that did not reach the target height at the intersection point of the flight path) were removed. The images were matched using the GPS data and inertial measurement units (IMUs) during flight, and the matched images were stitched to obtain high-resolution RGB imagery.

The pre-processing of multispectral imagery included radiometric calibration, atmospheric correction and Rational Polynomial Coefficient (RPC) ortho-rectification. Firstly, radiometric calibration and atmospheric correction were used to convert digital number (DN) values into reflectance. Secondly, the image after ortho-rectification was fused by using the misalignment impact (RMI) fusion method [44] to obtain sub-meter-resolution multispectral imagery. Thirdly, 200 ground control points were selected from RGB and multispectral ortho-imageries to co-register the multispectral imagery to RGB imagery using spline algorithm, and the correction error was controlled within 0.01 m. Finally, the nearest neighbor method was used for resampling to obtain spatially matched RGB and multispectral ortho-imageries.

2.5. Sample Dataset of Individual Tree

The establishment of the sample dataset of individual trees was based on the pre-processed images, combined with the segmentation results and the position information of individual trees obtained from field measurement to generate the dominant tree species images, and minimum external rectangle method was used to output these images. The detailed process is shown in Figure 3. Firstly, the pre-processed image (Figure 3a) was segmented by an object-oriented MRS algorithm (Figure 3①) to generate the segmented image (Figure 3b). Secondly, combined with the position information of individual trees obtained from field measurement (Figure 3c), the dominant tree species types were labeled (Figure 3②) on the segmented image. Thirdly, the tree crown of each labeled dominant tree species type (Figure 3d) was output via minimum external rectangle (Figure 3③) method to obtain the tree crown images of individual tree dominant species (Figure 3e). Finally, data augmentation was performed based on tree crown images of dominant individual tree species to obtain a sample dataset for training.

With the help of the position of individual tree samples, the images were labeled and classified for the training of deep learning networks. The sample dataset of individual trees was divided into five types, including 118 Nepal alder (A.N.), 80 Oriental white oak (Q.A.), 97 Aspen (P.D.), 80 Maple (A.F.) and 75 Yunnan pine (P.Y.).

2.5.1. Data Augmentation

In this study, data augmentation was performed on the individual tree samples to increase the number of training samples. The data augmentation methods include rotation and brightening, such as rotating the image clockwise by 90°, counterclockwise by 90°, 180° and brightening by 50% [45,46,47,48]. After data augmentation, the number of sample datasets was increased to five times. A detailed description is shown in

Table 2. Number of samples of each dominant tree species.

Class	Training	Validation	Total
A.N.	472	118	590
Q.A.	320	80	400
P.D.	388	97	485
A.F.	320	80	400
P.Y.	300	75	375
Total	1800	450	2250

Note: Alnus nepalensis = A.N.; Quercus aliena = Q.A.; Populus davidiana = P.D.; Acer forrestii = A.F.; Pinus yunnanensis = P.Y.

.

Training an epoch in a deep learning network represents complete model training with all the data of the training samples. To avoid network overfitting, the validation data were used to evaluate the accuracy of the deep learning network after each set of training. In this paper, 80% of the sample dataset was randomly selected for model training and 20% for model validation.

2.5.2. Remote Sensing Sample Dataset of Individual Trees

To take advantage of the texture metrics of high-resolution RGB imagery, the SuperView-1 satellite multispectral imagery (SV) was resampled and stacked with the RGB imagery (RGB) to obtain the post-stacked image (SVRGB). Therefore, multispectral and post-stacked images were used for dominant tree species classification and generated two individual tree sample datasets, named “SV” and “SVRGB”. All the sample datasets included 2250 samples and five dominant tree species (i.e., Nepal alder, Oriental white oak, Aspen, Maple and Yunnan pine). A detailed description is shown in Table 2.

2.6. Spectral and Texture Metrics Calculation

After the individual tree crowns were segmented, spectral data were extracted from the multispectral imagery, and a set of vegetation indices was calculated. Then, the high-resolution RGB imagery was used to calculate seven texture metrics of each band (blue, green and red band) for a total of 21 metrics. The meanings and calculation formulas for spectral indices variables and texture metrics of individual trees are shown in

Table 3. Summary of the vegetation indices with respective equations and references.

Metrics	Equation	Reference
Difference Vegetation Index (DVI)	ρnir − ρred	[49]
Atmospherically Resistant Vegetation Index (ARVI)	(ρnir − ρrb)/(ρnir + ρrb), ρrb = ρred – γ × (ρblue − ρred), γ = 0.5	[50]
Green Normalized Difference Vegetation Index (GNDVI)	(ρnir − ρgreen)/(ρnir + ρgreen)	[51]
Modified triangular vegetation index 2 (MTVI2)	[1.5 × (1.2 × (ρnir − ρgreen) − 2.5 × (ρred −ρgreen)]/[(2 × ρnir+1)2 – (6 × ρnir – 5 × ρred0.5) – 0.5]0.5	[52]
Normalized Difference Vegetation Index (NDVI)	(ρnir – ρred)/(ρnir + ρred)	[52]
Simple Ration Vegetation Index (SR)	ρnir/ρred	[53]
Soil Adjusted Vegetation Index (SAVI)	1.5 × (ρnir − ρred)/(ρnir + ρred + 0.5)	[52]
Ratio Vegetation Index (RVI)	ρred/ρnir	[54]
Normalized Greenness (Norm G)	ρgreen/(ρred + ρgreen+ ρblue)	[55]
Normalized Green-Red Ratio (Norm GR)	(ρgreen – ρred)/(ρgreen + ρred)	[55]
Optimized Soil Adjusted Vegetation Index (OSAVI)	(ρnir – ρred)/(ρnir + ρred + 0.16)	[56]
Red Green Ratio Index (RGRI)	ρred/ρgreen	[57]

Note: the ρ_blue represents the reflectance of blue band; the ρ_green represents the reflectance of green band; the ρ_red represents the reflectance of red bands; the ρ_nir represents the reflectance of near-infrared band.

and

Table 4. Summary of the texture metrics with respective equations and references.

Metrics	Equation
Correlation (CR)	CR = ${{\displaystyle \sum }}_{i,j=0}^{N-1}i·P_{i,j}$$|\frac{\left(i-ME\right)\left(j-ME\right)}{\sqrt{VAiVAj}}|$
Contrast (CO)	CO = $|{{\displaystyle \sum }}_{i,j=0}^{N-1}{n}^2·P_{i,j}|$
Dissimilarity (DI)	DI = ${{\displaystyle \sum }}_{i,j=0}^{N-1}i·P_{i,j}|i-j|$
Entropy (EN)	EN = ${{\displaystyle \sum }}_{i,j=0}^{N-1}i·P_{i,j}$(−$\ln P_{i,j}$)
Homogeneity (HO)	HO = ${{\displaystyle \sum }}_{i,j=0}^{N-1}i·\frac{P_{i,j}}{1+{\left(i-j\right)}^2}$
Mean (ME)	ME = ${{\displaystyle \sum }}_{i,j=0}^{N-1}i·P_{i,j}$
Variance (VA)	VA = ${{\displaystyle \sum }}_{i,j=0}^{N-1}i·P_{i,j}(i-ME$)

Note: i, j represents the gray scale of image; N represents the gray level of image; P_i,j represents the gray-level co-occurrence matrix.

.

2.7. Random Forest and Deep Learning Network Classifier

2.7.1. Random Forest

Random forest algorithm creates several CART-like trees, and each tree is trained with the original training samples to vote on the most popular class of the input variables, and the output class is determined by the majority vote of the tree [58,59]. The training samples were used to train the random forest classifier, and the validation samples were used to evaluate the classification accuracy. The training and validation samples were randomly divided into 80% and 20% of the total samples for each tree species, respectively.

2.7.2. Deep Learning Networks

A deep learning network is a framework consisting of a stack of several convolutional layers, pooling layers and fully connected layers [38]. The convolutional layer is used to extract features from the input image, the pooling layer is responsible for reducing the dimension of the image and the fully connected layer classifies the image by probability. Three deep learning networks were used to classify dominant tree species in this study, namely, the lightweight network MobileNetV2 [60], the residual network ResNet34 [61] and the dense network DenseNet121 [62].

(1)

Lightweight network MobileNetV2

The core of the MobileNetV2 model is the “inverted residual structure”. In the structure, the tensor depth containing the input feature image is firstly boosted by 1 × 1 convolution; then, the output is provided to 3 × 3 depth-wise convolution, and finally, 1 × 1 convolution is used to reduce the dimension of the feature image. Each convolution is followed by batch normalization and ReLu6 activation functions. However, ReLu6 function can easily cause the loss of low-dimension feature information in the image; thus, linear activation function is selected in the dimensionality reduction stage.

The overview MobileNetV2 model is shown in Figure 4a. This framework consists of 55 training layers, including 2 regular convolutional layers, 17 inverted residual feature layers (containing 51 convolutional layers) and 2 fully connected layers. The MobileNetV2 model is composed of 7 inverted residual convolution groups (Bk1-Bk7), each of which includes a number of basic residual convolution units, and then passes the input feature image to the next layer.

(2)

Residual network ResNet34

The ResNet34 model consists of stacked blocks called “residual structure”. The main branch of this residual structure consists of two 3 × 3 convolutional layers, and the connecting line is the shortcut branch, which serves to skip one or more layers during model training. The ResNet34 model solves the problem that deep learning networks are difficult to train and introduces the idea of deep residual learning, which effectively weakens the gradient disappearance problem.

The overview ResNet34 model is shown in Figure 4b. This framework consists of 35 training layers, including 1 convolutional layer, 16 residual feature layers (containing 32 convolutional layers) and 2 fully connected layers. The ResNet34 model is composed of four residual convolution groups, each of which includes one or more basic residual convolution units. The first convolution group performs only one convolution operation and has a convolution kernel size of 7 × 7. The second to fifth convolution groups contain many consistent residual units, named Conv2_x, Conv3_x, Conv4_x and Conv5_x.

(3)

Dense network DenseNet121

Compared with ResNet34 model, DenseNet121 model proposes a more radical dense connection mechanism: all layers are connected to each other, and each layer will receive the output of all previous layers as additional input. Different from ResNet34 model’s element-level addition, DenseNet121 model uses concatenation method for image feature fusion. The specific expression is shown in Formulas (4) and (5) [62]. The dense layers in the DenseNet121 model can be directly connected to the input and residual gradients; thus, the phenomenon of gradient disappearance can be further mitigated.

$$x_l=H_l\left(x_{l-1}\right)+x_{l-1}$$

(1)

$$x_l=H_l\left(\left[x_0,x_1,\dots ,x_{l-1}\right]\right)$$

(2)

x_l represents the input at layer l, x_l−1 represents the output of layer l − 1, H_l(x_l−1) represents the nonlinear transformation for the output of layer l – 1 and [x₀, x₁, …, x_l−1] represents concatenation of outputs from layers 0 to l − 1.

The overview DenseNet121 model is shown in Figure 4c. This framework consists of 125 training layers, including 1 convolutional layer, 4 dense blocks (containing 116 convolutional layers), 3 transition layers (each transition layer has 1 convolutional layer and 1 average pooling layer) and 2 fully connected layers. The DenseNet121 model is composed of 4 dense blocks, including 6, 12, 24 and 16 dense layers, which are connected to each other.

All the code in this study was completed in PyCharm v. 2019.3 using the “torch-vision” and “tqmd” packages. The training of the three deep learning networks took 10.5 h on an NVIDIA GPU (GeForce RTX 3050, 4G). Each deep learning network was trained for 100 epochs, with a batch size of 32, an initial learning rate of 0.001 and a learning rate fall of 0.1.

2.8. Validation of Individual Tree Segmentation and Tree Species Classification

In this paper, an object-oriented multi-resolution segmentation (MRS) algorithm was used to segment individual trees from high-resolution RGB and spectral imageries in ecognition software, and the background information was removed to obtain the individual tree crowns. The MRS algorithm calculates the heterogeneity index for two adjacent objects to be segmented and compares it with the threshold value. If the heterogeneity index is smaller than the threshold value, the two objects are merged; otherwise, they will be detached [63]. The main parameters of the object-oriented MRS algorithm include scale, shape and compactness. In this study, a series of scale parameters ranging from 10 to 90 was set, and the optimal scale was found via enumeration method with step size of five. The optimal shape and compactness parameters were also calculated based on this approach.

The position information of individual trees obtained from field measurement was combined with the tree crown boundary extracted using the algorithm to evaluate the segmentation accuracy. Three measures were used to evaluate the accuracy of individual tree segmentation: recall (r, represented the rate of tree detection), precision (p, represented the detection trees precision) and F₁ score (F₁, represented the overall accuracy). The equations are as follows [64,65]:

$$r=\frac{Nt}{Nt+No}$$

(3)

$$p=\frac{Nt}{Nt+Nc}$$

(4)

$$F_1=\frac{2\left(r\times p\right)}{r+p}$$

(5)

Nt is the number of detected trees that exist in field position, No is the number of trees that are not detected by algorithm and Nc is the number of detected trees that do not exist in the field.

The overall accuracy (OA) and Kappa accuracy were used to evaluate the accuracy of the classification. The three selected typical plots contained 90 field measurement samples, and the classification accuracy was calculated by comparing the tree species types classified by the algorithm with those field measurements. The OA was calculated through the total number of correctly classified samples divided by the total number of samples [66]. Kappa accuracy is calculated using the following formula:

$$k=\frac{p_o-p_e}{1-p_e}$$

(6)

$$p_e=\frac{{{\displaystyle \sum }}_{i=1}^c{a}_i\times b_i}{n^2}$$

(7)

p_o presents the overall classification accuracy; a_i represents the number of true samples for specific class; b_i represents the number of samples predicted for specific class; n represents the number of samples.

2.9. Effects of the Number of Training Samples on the Performance of Dominant Tree Species Classification

The training of deep learning networks generally requires a large number of samples, and the performance of deep learning networks is more robust when the number of given samples is sufficient [67]. Based on the SVRGB sample dataset, 20%, 40%, 60% and 80% of the total samples of each dominant tree species were randomly selected to train the three deep learning network classifiers, we and calculated the overall accuracy of dominant tree species classification.

3. Results

3.1. Individual Tree Segmentation

As shown in Figure 5, the results showed that when the scale factor was 20, the segmented vector layer could be well matched to each tree crown from the multispectral imagery. However, when the scale factor was 10, over-segmentation occurred, with a tree crown being divided into two or more vector layers; when the scale factor was 30, part of the crowns faced under-segmentation. For UAV RGB imagery, the best segmentation performance was obtained when the scale factor was 40. However, when the scale factors were 20 or 60, the individual tree segmentation results could not be well matched to the tree canopy.

As shown in

Table 5. Accuracy assessment of individual tree segmentation in the forests with different stand densities.

Density	Nt	No	Nc	r (%)	p (%)	F1 (%)
Satellite multispectral imagery
Low	28	13	6	68.3	82.4	74.7
Middle	31	21	4	59.6	88.6	71.3
High	49	29	8	62.8	85.9	72.6
UAV high-resolution RGB imagery
Low	27	14	3	65.9	90.0	76.1
Middle	38	14	6	73.1	86.4	79.2
High	52	26	8	66.7	86.7	75.4

Note: r, represents the rate of tree detection; p, represents the detection trees precision; F₁, represents the overall accuracy taking both omission and commission in consideration.

and Figure 6, the object-oriented MRS algorithm performed well in the forests with different stand densities (low, middle and high) and had an average overall accuracy greater than 70% (Table 4). For the satellite multispectral imagery, the MRS algorithm had the best detection rate (68.3%) and F₁ score (74.7%) in the forests with low density. Under high-density forests, the detection rate of trees was 62.8% and the F₁ score was 72.6%. In the middle-density forests, the detection rate of trees was 59.6% and the F₁ score was 71.3%. For the UAV high-resolution RGB imagery, the MRS algorithm had the best detection rate (73.1%) and F₁ score (79.2%) in the forests with middle density. Under low-density forests, the detection rate of trees was 65.9% and the F₁ score was 76.1%. In high-density forests, the detection rate of trees was 66.7% and the F₁ score was 75.4%.

The performances of individual tree segmentation using multispectral imagery and high-resolution RGB imagery are shown in

Table 6. Accuracy assessment of individual tree segmentation of dominant tree species.

Data Source	Nt	No	Nc	r (%)	p (%)	F1 (%)
Satellite multispectral imagery	273	177	91	60.7	75.0	67.1
UAV high-resolution RGB imagery	318	132	111	70.7	74.1	72.4

Note: r, represents the rate of tree detection; p, represents the detection trees precision; F₁, represents the overall accuracy taking both omission and commission in consideration.

, with the average overall accuracy above 65%. For satellite multispectral imagery, the detection rate of trees was 60.7%, the precision was 75.0% and the F₁ score was 67.1%. For the UAV high-resolution RGB imagery, the detection rate of trees was 70.7%, the precision was 74.1% and the F₁ score was 72.4%.

3.2. Feature Optimization and Analysis

The importance analysis of metrics showed that normalized greenness (Norm_G), atmospherically resistant vegetation index (ARVI) and modified triangular vegetation index 2 (MTVI2) of spectral indices were most important for the dominant tree species classification (Figure 7a). With the addition of texture metrics, the contrast of the red band (red correlation), the normalized greenness (Norm_G), and the contrast of the green band (green correlation) ranked in the top three of the ten most important metrics (Figure 7b).

3.3. Training of Deep Learning Networks

In order to compare the classification performances of deep learning networks on different data sources, the satellite multispectral imagery (SV) sample dataset, the multispectral imagery and high-resolution RGB imagery post-stacked image (SVRGB) sample dataset were used to train the deep learning network classifier. For the SV sample dataset, the training processes of the lightweight network MobileNetV2, the residual network ResNet34 and the dense network DesnseNet121 are shown in Figure 8. The blue curve represented the changes in loss value and accuracy in the training samples, and the orange curve represented the changes in loss value and accuracy in the validation samples. Figure 8a shows that the loss values of the lightweight network MobileNetV2 remained stable after 80 epochs. The changes in accuracy also embodied a similar trend in Figure 8b. Figure 8c,d show that the changes in loss value and accuracy in the residual network ResNet34 tended to steady after 70 epochs. Figure 8e,f show that the changes in training loss value and accuracy in the dense network DenseNet121 showed a similar trend to that of the lightweight network MobileNetV2, which stabilized after 80 epochs of iteration.

Similar to the SV sample dataset, three deep learning networks were trained with the SVRGB sample dataset. The overall convergence of this sample dataset was faster than that of the SV sample dataset. As shown in Figure 9a,c, the loss values of the lightweight network MobileNetV2 and the residual network ResNet34 became stable after 60 epochs. In Figure 9b,d, the changes in accuracy remained steady after 60 epochs. The changes in training loss value and accuracy in the dense network DenseNet121 remained stable after 70 epochs, as shown in Figure 9e,f.

3.4. Accuracy Assessment

Table 7. Confusion matrix of five dominant tree species classifications using random forest algorithm combined with two groups of metrics.

Class	A.N.	Q.A.	A.F.	P.D.	P.Y.
Spectral metrics
A.N.	23	2	6	3	5
Q.A.	1	7	3	3	0
A.F.	2	0	6	2	0
P.D.	1	0	2	10	0
P.Y.	2	1	1	2	8
Overall Accuracy	60.00%	Kappa Accuracy	47.80%
Spectral and texture metrics
A.N.	24	1	4	2	5
Q.A.	2	7	1	1	0
A.F.	1	1	8	2	0
P.D.	1	0	3	12	1
P.Y.	1	1	2	3	7
Overall Accuracy	64.44%	Kappa Accuracy	53.61%

Note: Alnus nepalensis = A.N.; Quercus aliena = Q.A.; Acer forrestii = A.F.; Populus davidiana = P.D.; Pinus yunnanensis = P.Y.

shows the confusion matrix of five dominant tree species classification using the random forest algorithm combined with two groups of metrics. For spectral metrics, the overall accuracy of the dominant tree species classification was 60.00%, and the Kappa accuracy was 47.80%. For the combination of spectral and texture metrics, the overall accuracy of the dominant tree species classification was 64.44%, and the Kappa accuracy was 53.61%. The dominant tree species classification accuracy of using two groups of metrics improved successively, indicating that texture metrics had a positive impact on the dominant tree species classification (OA and Kappa accuracy increased by 4.44% and 5.81%, respectively). Among the five dominant tree species classifications, using the combination of spectral and texture metrics had the best performance (OA was 64.44%, Kappa accuracy was 53.61%).

Table 8. Confusion matrix of five dominant tree species classifications using three deep learning networks based on SV sample dataset.

Class	A.N.	Q.A.	A.F.	P.D.	P.Y.
Light-weight network MobileNetV2
A.N.	20	2	2	2	1
Q.A.	3	7	1	3	3
A.F.	2	0	13	0	0
P.D.	4	1	1	15	0
P.Y.	0	0	1	0	9
Over Accuracy	71.11%	Kappa Accuracy	63.01%
Residual network ResNet34
A.N.	24	1	1	2	0
Q.A.	1	8	0	2	0
A.F.	3	0	13	0	3
P.D.	0	1	0	16	0
P.Y.	1	0	4	0	10
Over Accuracy	78.89%	Kappa Accuracy	72.86%
Dense network DenseNet121
A.N.	24	2	3	1	0
Q.A.	1	7	0	2	0
A.F.	4	0	14	0	2
P.D.	0	1	0	17	0
P.Y.	0	0	1	0	11
Over Accuracy	81.11%	Kappa Accuracy	75.53%

Note: Alnus nepalensis = A.N.; Quercus aliena = Q.A.; Acer forrestii = A.F.; Populus davidiana = P.D.; Pinus yunnanensis = P.Y.

and

Table 9. Confusion matrix of five dominant tree species classifications using three deep learning networks based on SVRGB sample dataset.

Class	A.N.	Q.A.	A.F.	P.D.	P.Y.
Light-weight network MobileNetV2
A.N.	25	0	2	1	1
Q.A.	1	8	1	1	0
A.F.	0	1	12	0	1
P.D.	1	0	1	18	0
P.Y.	2	1	2	0	11
Over Accuracy	82.22%	Kappa Accuracy	77.09%
Residual network ResNet34
A.N.	27	1	0	1	0
Q.A.	0	8	0	0	0
A.F.	2	1	16	0	1
P.D.	0	0	0	19	0
P.Y.	0	0	2	0	12
Over Accuracy	91.11%	Kappa Accuracy	88.49%
Dense network DenseNet121
A.N.	28	2	0	0	0
Q.A.	1	8	0	0	0
A.F.	0	0	17	0	1
P.D.	0	0	0	20	0
P.Y.	0	0	1	0	12
Over Accuracy	94.44%	Kappa Accuracy	92.79%

Note: Alnus nepalensis = A.N.; Quercus aliena = Q.A.; Acer forrestii = A.F.; Populus davidiana = P.D.; Pinus yunnanensis = P.Y.

show the confusion matrix of five dominant tree species classifications using the three deep learning networks based on SV and SVRGB sample datasets, respectively. For the SV sample dataset, the average overall accuracy of the three deep learning networks was over than 70%. Dense network DenseNet121 had the highest overall accuracy (OA = 81.11%, Kappa accuracy = 75.53%). The overall accuracy of residual network ResNet34 was followed (OA = 78.89%, Kappa accuracy = 72.86%). Lightweight network MobileNetV2 had the lowest overall accuracy (OA = 71.11%, Kappa accuracy = 63.01%). For the SVRGB sample dataset, the average overall accuracy of the three deep learning networks was more than 80%. Among them, the overall accuracy of dense network DenseNet121 was 94.44%, and the Kappa accuracy was 92.79%, which showed the best classification performance. The overall accuracy of residual network ResNet34 was 91.11%, and the Kappa accuracy was 88.49%, whereas the overall accuracy of lightweight network MobileNetV2 was 82.22%, and the Kappa accuracy was 77.09%. Compared with the SV sample dataset, the average overall accuracy of dominant tree species classification based on the SVRGB sample dataset was improved by 11.11–13.33%.

Therefore, the overall accuracy of the dominant tree species classification of the three deep learning networks (OA = 71.11–94.44%) was higher than that of the random forest algorithm (OA = 60.00–64.44%). Among them, dense network DenseNet121 had the highest overall accuracy (OA = 81.11–94.44%).

3.5. Mapping of Five Dominant Tree Species

In this paper, the best deep learning network (dense network DenseNet121) was applied to the remote sensing images of individual trees to predict the dominant tree species. It took about 0.5 s to predict each individual tree remote sensing image and obtained the tree species type of the image. Figure 10 shows the mapping of five dominant tree species classifications in three plots (100 × 100 m). In general, Yunnan pine and Nepal alder tree species were dominant in distribution and quantity. In plot 1, the distribution of Nepal alder (A.N.) tree species was more extensive, and the distribution of Oriental white oak (Q.A.) tree species was more concentrated. In plot 2, the number of Yunnan pine (P.Y.) tree species was similar to that of Nepal alder, while aspen (P.D.) tree species were mainly distributed in the eastern of the plot. In plot 3, the number of Nepal alder tree species was the highest and widely distributed. The number of Yunnan pine tree species was second and concentrated in the southwest of the plot.

3.6. Effects of the Number of Training Samples on the Classification Performance of Three Deep Learning Networks

The number of training samples has little effect on the overall accuracy of dominant tree species classification using three deep learning networks, as shown in Figure 11. When the number of training samples varied by 20–80% of the total samples of each dominant tree species (the interval was 20%), the overall accuracy of the dominant tree species classification using three deep learning networks changed slightly (the change in OA was 2.69–4.28%). For lightweight network MobileNetV2, the overall accuracy of the dominant tree species classification was 74.09–78.37%. For the residual network ResNet34, the overall accuracy of the dominant tree species classification was 88.05–91.02%. For dense network DenseNet121, the overall accuracy of the dominant tree species classification was 90.32–93.01%.

4. Discussion

4.1. Influence of Different Data Sources on Individual Tree Segmentation

In this study, UAV RGB imagery and satellite multispectral imagery were applied for individual tree segmentation. Compared with the multispectral imagery, the individual tree segmentation accuracy of RGB imagery was higher. The reason is that RGB imagery has higher spatial resolution and clearer canopy boundaries. The lower the spatial resolution, the larger the range of ground objects contained in one pixel, thus potentially interfering with the segmentation of the target object [68]. Moreover, high-resolution RGB images have abundant texture information of tree crown edges, which can be used to accurately extract individual tree crowns [28,69]. Zhou et al. [70] segmented individual trees from high-resolution RGB images, reaching a better segmentation accuracy. However, the complex stand structure, high canopy density and high stand density in natural forests adversely affected the individual tree segmentation. In this paper, although our tree species classification accuracy was high, the individual tree segmentation accuracy was relatively low. It is relatively difficult to segment individual trees in complex stands, which has also been confirmed by previous studies. Xu et al. [35] used three methods to segment individual trees in subtropical natural forests and classified eight types of dominant tree species. The accuracy of the advanced multiresolution segmentation method was highest (F₁ score = 82.5%), and the image-based multiresolution segmentation method exhibited the lowest accuracy (F₁ score = 78.6%), which is similar to the results (F₁ score = 72.4%) obtained in this paper by using RGB images for individual tree segmentation. Zhong et al. [71] used LiDAR point cloud and hyperspectral images for individual tree segmentation in natural forests, and the total accuracy was 84.6%. Compared with our individual tree segmentation results, this accuracy was higher. However, the LiDAR point cloud provides three-dimensional information of the forest canopy structure [72], which can be better used for individual tree segmentation. The data source of this study was two-dimensional images, which do not have this advantage in the process of individual tree segmentation. After the individual tree segmentation was completed, we optimized the segmentation results using a splitting and merging method to match the tree crowns. Although the accuracy of individual tree segmentation in this study was relatively low, the classification accuracy was improved by using the optimized crown boundaries. There is a certain risk, but this risk could be acceptable considering the characteristics of natural forests, such as complex stands and extremely high canopy density. In the future, better methods and data are needed to improve the accuracy of individual tree segmentation in complex stands, so as to provide guidance for forest application.

4.2. Performance of Deep Learning Networks and Effect of Texture metrics on Tree Species Classification

Since the combination of multispectral and RGB data could provide not only spectral information but also texture metrics of the forests, the dominant tree species classification showed a higher accuracy. Previous studies have shown that the combination of many metrics generally showed higher accuracy for dominant tree species classification [73,74]. Gini et al. used texture metrics to improve classification accuracy, and they proved that the use of texture increased OA by 20% [75]. In this study, the random forest algorithm and three deep learning networks were used to classify five dominant tree species. For the random forest algorithm, the addition of texture metrics improved the overall accuracy of dominant tree species classification (the OA was increased by 4.44%). This result was consistent with the conclusion of previous studies. This result occurred because the addition of texture features makes the description of tree species more comprehensive, and the algorithm can identify more features related to the target tree species during classification, thereby improving the classification accuracy. For the deep learning networks, the combination of UAV high-resolution RGB imagery and satellite multispectral imagery enabled the models to learn the features from forest edge and texture, and, thus, the overall accuracy of dominant tree species classification was also improved (the OA was increased by 11.11–13.33%).

In the past, there have been studies using deep learning networks combined with multi-source remote sensing data to improve the overall accuracy of dominant tree species classification [67,76]. However, most previous studies applied deep learning networks to the forests with relatively simple structures. Here, we explored the potential of deep learning networks for the classification of dominant tree species in subtropical natural forests. The result showed that the overall accuracy of dominant tree species classification using the three deep learning networks was higher than that of the random forest algorithm. Previous studies have also proved this point. Onishi et al. [42] used a convolutional neural network (CNN) and SVM algorithm to classify seven classes, and the classification accuracy of CNN was 5.8–13.1% higher than SVM. Zhang et al. [43] used three convolutional neural networks (CNNs) and two machine learning (ML) algorithms to classify tree species; the classification accuracy of CNNs was higher than 70%, while that of ML algorithms was less than 70%. Among the three deep learning networks, dense network had the best performance (81.11–94.44%). The tree species dataset has more complex features, and the convolutional layers in the dense network structure are densely connected with each other, which is more suitable for extracting high-level features. In this paper, three deep learning networks were combined with multi-source remote sensing data to classify five dominant tree species in subtropical natural forests. Compared with the random forest algorithm, the deep learning networks had the potential to identify dominant tree species in subtropical natural forests. However, since the study area selected for tree species classification is only three plots, it is still necessary to explore the large-scale tree species classification.

4.3. Effect of Variation in the Number of Training Samples on Classification Accuracy

Although the variation in training samples had little effect on the overall accuracy of dominant tree species classification using the three deep learning networks, the increased number of training samples had a positive impact on the dominant tree species classification. Hartling et al. divided the total samples into several proportions (10%, 30%, 50% and 70%) to train DenseNet network and found that the overall accuracy of dominant tree species classification increased when the number of samples increased [67]. In this paper, with the increased number of training samples, the overall accuracy of the dominant tree species classification was also improved (the OA was increased by 2.69–4.28%). Furthermore, among the three deep learning networks, the dense network DenseNet121 outperformed the other two models, indicating that it was able to learn deeper information from the sample dataset and, therefore, had strong robustness to the training samples.

5. Conclusions

In this paper, UAV high-resolution RGB imagery and satellite multispectral imagery were obtained, combined with the three deep learning networks and random forest algorithm to identify five dominant tree species in the subtropical natural forest of northwest Yunnan province of China. It was preliminarily confirmed that the deep learning network can be combined with multi-source remote sensing data for the classification of dominant tree species in natural forests. In addition, individual tree segmentation was carried out in natural forests with different stand densities, indicating that the objected-oriented MRS method can be applied to individual tree extraction in natural forests. We found that the individual tree segmentation of UAV high-resolution RGB imagery was better than that of satellite multispectral imagery. A total of 12 spectral indices and 7 texture metrics were extracted from the multispectral imagery and UAV RGB imagery, and the top 10 metrics were selected via the permutation-based importance based on the RF algorithm. In general, Norm_G, ARVI, MTVI2, red correlation and green correlation are the most important metrics in the classification of dominant tree species. Additionally, we found that the overall accuracy of dominant tree species classification using three deep learning networks was higher than that of the random forest algorithm. The addition of texture metrics improved the overall accuracy of dominant tree species classification. For the three deep learning networks, the changes in overall accuracy of dominant tree species classification influenced by the number of training samples were minor, and the classification accuracy of the dense network was higher than that of the other two deep learning networks. This study demonstrated that deep learning networks have the potential to classify dominant tree species in natural forests. However, to further explore their robustness, deep learning networks need to be tested in other study areas and more tree species types.