Coastal Wetland Vegetation Classification Using Pixel-Based, Object-Based and Deep Learning Methods Based on RGB-UAV

Abstract

The advancement of deep learning (DL) technology and Unmanned Aerial Vehicles (UAV) remote sensing has made it feasible to monitor coastal wetlands efficiently and precisely. However, studies have rarely compared the performance of DL with traditional machine learning (Pixel-Based (PB) and Object-Based Image Analysis (OBIA) methods) in UAV-based coastal wetland monitoring. We constructed a dataset based on RGB-based UAV data and compared the performance of PB, OBIA, and DL methods in the classification of vegetation communities in coastal wetlands. In addition, to our knowledge, the OBIA method was used for the UAV data for the first time in this paper based on Google Earth Engine (GEE), and the ability of GEE to process UAV data was confirmed. The results showed that in comparison with the PB and OBIA methods, the DL method achieved the most promising classification results, which was capable of reflecting the realistic distribution of the vegetation. Furthermore, the paradigm shifts from PB and OBIA to the DL method in terms of feature engineering, training methods, and reference data explained the considerable results achieved by the DL method. The results suggested that a combination of UAV, DL, and cloud computing platforms can facilitate long-term, accurate monitoring of coastal wetland vegetation at the local scale.

1. Introduction

Coastal wetlands are among the most essential and valuable ecosystems on the planet and are closely connected to human welfare, playing an indispensable role in maintaining biodiversity, purifying water, resisting storm surges, carbon sequestration, microclimate regulation, and so on [1,2,3]. The threat to coastal wetlands has increased over the years due to human activities and global change, including habitat fragmentation, biological invasion, sea level rise, and water eutrophication, which make coastal wetlands one of the most endangered natural ecosystems [4,5]. Therefore, monitoring and protecting the sustainability of coastal wetland ecosystems has become an urgent need [6].

Remote sensing technology has been an indispensable tool for monitoring wetlands for decades. Remote sensing technology can overcome the drawbacks of traditional wetland surveys such as those related to wetland location, hydrological fluctuations, topography, and consumption of excessive human and material resources, and obtain large-area reflectance data of wetlands in an efficient and relatively economical way, and thus is widely used in wetland monitoring [7,8,9,10,11]. In the recent years, the rapid development of UAV technology, with its high portability, ultra-high spatial resolution, multi-sensor integration, and affordable price, has filled the gap between field surveys and traditional satellite platforms (e.g., satellite-based and airborne platforms) for global and regional large-scale vegetation monitoring, opening up new possibilities for coastal wetland vegetation monitoring at the local scale [12,13]. UAV has been applied to coastal wetland vegetation mapping [14,15,16], monitoring invasive plants [17,18], vegetation above-ground biomass retrieval [19,20], coastal wetland ecological restoration monitoring and management [21,22], and gradually become an essential tool for coastal wetland vegetation monitoring.

Wetland vegetation classification, i.e., fine delineation of wetland vegetation and accurate identification of vegetation types, is the primary analytical approach for wetland inventory, restoration, and management based on UAV [13]. Typically, remote sensing classification of wetlands is usually performed using Pixel-Based (PB) and Object-Based Image Analysis (OBIA) classification. In comparison with the PB classification, which uses only spectral information, in the OBIA classification, similar image elements are first segmented into separate objects, and then classified by a machine learning (ML) algorithm to produce the final classification results [23]. Several studies have previously demonstrated that in remote sensing monitoring of coastal wetlands, with increasing spatial resolution, the OBIA methods can produce landscape patches that are similar to wetland vegetation, thus improving the accuracy of remote sensing classification [16,24,25,26]. However, when applying the OBIA method to UAV data, the ultra-high resolution of UAV poses some challenges and uncertainties to the classification result, especially in coastal wetlands. For example, Samiappan et al. [18] reported that when using the OBIA method based on UAV for coastal wetland mapping, extensive time and effort were spent in the selection of image segmentation parameters. In addition, Martínez Prentice et al. [27] revealed that the PB method is more accurate than the OBIA method when monitoring coastal wetland vegetation based on UAV and suggested that the result of the PB method is more reflective of the distribution of vegetation in coastal wetlands with high landscape heterogeneity.

With the development of technology, DL models started to become a new trend in processing and analyzing UAV remote sensing data [28]. In remote sensing classification, DL models based on computer vision (e.g., Fully convolutional networks [29]) are able to extract hierarchical information from ultra-high spatial resolution remote sensing images and learn relevant features automatically using an end-to-end way to eventually generate fine-grained, spatially explicit classification of the class, and have shown to achieve high accuracy in UAV monitoring of ecosystems such as wetlands, forests, farmland, and grasslands [30,31,32,33,34]. Due to the excellent results achieved by DL, researchers started to compare the application of traditional ML and DL in monitoring UAV coastal wetlands. For example, Bhatnagar et al. [35] compared the classification results of several traditional ML model and DL model in UAV wetland vegetation communities and showed that the DL model (combination of ResNet and SegNet) achieved the best classification accuracy compared to the random forest algorithm (overall accuracy = 91.5%). Moreover, Gonzalez-Perez et al. [36] compared ML models (Support Vector Machine and Random Forest) and DL models (U-Net and DeepLabv3) in the classification of coastal wetland vegetation, highlighting the greater advantage of the DL model in the classification of ultra-high-resolution UAV data in coastal landscapes. Previous studies have focused on comparing the classification performance of traditional ML and DL in UAV data at the model level. However, to our knowledge, almost no studies have evaluated the classification performance of the three paradigms, PB, OBIA, and DL, in coastal wetland UAV data. Since the PB and OBIA methods are the most common paradigm for UAV data processing in coastal wetlands [15,16,27] and are mutually independent of DL methods in processing and analysis, an understanding of the paradigm shift between them would be more practical and essential for coastal wetland monitoring.

Currently, the application of UAV to monitor coastal wetlands is still in its infancy [37], in which the processing and analysis of UAV data are also relatively complex and time-consuming due to the excessive data volume of ultra-high resolution. For instance, Bhatnagar et al. [35] pointed out that the extraction of textures from UAV data of wetlands is very computationally intensive, which poses a research challenge. What is more, most studies using OBIA methods to analyze and process UAV data are usually based on commercial software solutions such as eCognition [16,38,39], whose expensive expenses also constrain the applicability of UAVs. The above issues can be solved by using cloud-based geospatial analysis platforms such as GEE [40] and Microsoft’s Planetary Computer [41]. The GEE platform, for example, allows users to upload and process their own private datasets in addition to accessing public data, and analyzing them with powerful computing power and algorithms. For example, Bennett et al. [42] classified the coral reef based on UAV data using the PB method in GEE, achieving an overall accuracy of 86%. However, there is still a dearth of research on leveraging the GEE platform to process and analyze UAV data [43]. Among them, the OBIA method, which includes texture feature extraction and image segmentation, is not common in GEE at present because it requires combining various complex functions and optimizing kinds of parameters to overcome the computational limitations of GEE [44]. In particular, no research work has so far attempted to use OBIA classification in GEE for ultra-high resolution UAV data, which limits the application of GEE in processing UAV data.

In this study, we compared the performance of PB, OBIA, and DL methods in vegetation community classification of coastal wetlands based on RGB-UAV data, where feature engineering of RGB data, including vegetation indices, extraction of texture features, and PB and OBIA methods were performed in the GEE platform. In addition, the RGB sensor is the most affordable and widely used UAV sensor, and the RGB-derived Digital Surface Models (DSM) were also used in the study. The objectives of this study are (1) to explore the feasibility of applying PB and OBIA methods for vegetation classification of coastal wetlands based on UAV data using feature extraction in GEE; (2) to investigate the relative importance of features in vegetation classification of coastal wetlands in PB and OBIA methods; (3) to compare DL, PB, and OBIA methods in UAV-based vegetation classification, and to reveal their paradigm shift and implications for coastal wetlands UAV monitoring.

2. Materials and Methods

2.1. Study Area

Chongming Island is part of the Chinese mega-city of Shanghai (31°27′31°5′ N, 121°09′–121°54′ E, Figure 1) and part of the Yangtze River estuary. It is the largest alluvial island in the world, with an area of about 1267 square kilometers. It has a typical sub-tropical monsoon climate, with an annual average temperature of about 15.3 °C and an annual precipitation of about 1022 mm. Chongming Dongtan National Nature Reserve (hereinafter referred to as CDNNR) is one of the largest migratory bird nature reserves in East Asia. CDNNR was listed in the China Report on the Protection of Wetlands in 1992, recognized as a wetland of international importance by the Ramsar Convention on Wetlands in 2001, and designated as a national migratory bird conservation nature reserve in 2005. The total area of CDNNR is about 242 km², which accounts for 20% of the area of Chongming Island, mainly distributed with vegetation communities such as Phragmites trails, Scripus mariqueter, Carex scabrifolia Steud, Imperata cylindrica, Spartina alterniflora, Solidago decurrens, etc.

2.2. Study Workflow

In this study, the specific process is shown in Figure 2. The research process can be divided into 4 important stages. (1) Data preparation. We took UAV data according to the plan, acquired the UAV RGB data and generated the corresponding DSM data, and selected several independent representative areas for manual labeling to construct the UAV remote sensing classification dataset. (2) Remote sensing classification based on traditional ML. We randomly took points from the dataset and divided it into 70% of the training set and 30% of the test set, and performed feature extraction in GEE, including spectral features and GLCM texture features, feature combination, and Simple Non-Iterative Clustering (SNIC) image segmentation, followed by classification using PB and OBIA, respectively. (3) DL-based remote sensing classification. We sliced the dataset and performed data augmentation to test the classification effects of different DL models on RGB and RGB + DSM fused datasets. (4) Model evaluation. Firstly, we analyzed the relative importance of the features for RF classification in PB and OBIA. Subsequently, we evaluated different experiments of PB, OBIA, and DL models using relevant evaluation parameters in combination with classification maps.

2.3. Data Preparation

2.3.1. UAV Data Acquisition and Preprocessing

We entered the CDNNR on 30 October 2020, and started the drone survey from 10–12 a.m., during high tide, which ranged from 308 cm to 370 cm, with a tidal datum of 214 cm below mean sea level. We used a DJI P4 RTK for the drone data collection and used DJI’s application (DJI GS Pro) to set up the drone flight plan. First, the flight area of the sample site was defined, the camera orientation was set to follow the route direction, and the photo mode was set to hover at the waypoint to ensure that the Photogrammetric of the corresponding route could be synthesized later. Meanwhile, the flight speed of the UAV was set to 10 m/s and the flight altitude to 120 m. The overlap rate of heading was set to 75% and the overlap rate of side direction was set to 60%, which could ensure a wide range of orthophoto stitching in the later stage.

Subsequently, we imported the obtained UAV orthophotos into DJI Terra software for 2D image reconstruction. The DJI Terra software has various built-in optimization algorithms, which can effectively avoid image distortion, accurately and meticulously present the target object and the survey area, and support radiation correction, and the output reflectivity map can be used for relevant scientific research. The result is an Orthomosaic map with a Ground Sampling Distance (GSD) of 4.16 cm and a DSM of the sample area, as shown in Figure 1.

2.3.2. Construction of Coastal Wetland Vegetation Dataset

Aiming to construct the classification dataset, we selected two independent areas, where one area was used as training set and validation set, and the other area as test set, with vegetation representative in low- and high-tide areas in the generated UAV Orthomosaic to perform manual pixel-level labeling for subsequent classification experiments. We selected the dataset areas based on the following principles: (1) the land cover types were diverse, including a variety of coastal wetland plants in the sample sites; (2) the dataset areas included common vegetation forms for individual vegetation classes; and (3) the number of image pixels for each type of feature sample in the area was relatively balanced. Finally, we identified five classes for the dataset: Phragmites trails, Scripus mariqueter, Carex scabrifolia Steud, Imperata cylindrica, and Flat. During the drone survey time (30 October 2020), most of the plants in our study area are still green, and the Scripus mariqueter have completely died out, becoming the main food sources of the migratory birds [45], which are also a key target for our monitoring.

Based on the field survey and with the help of salt marsh vegetation experts, we performed visual interpretation of the selected areas by combining the spectral characteristics, location, boundaries, textures, and morphology of the different classes. We used the Edit Features function of ArcGIS Desktop: Release 10.8 Redlands, CA: Environmental Systems Research Institute. to clearly outline the boundaries of different classes in the UAV Orthomosaic and then rasterized it to generate the associated pixel-level labels, where encoded pixel values correspond to different classes, and the spatial extent, projected coordinate system, and spatial resolution are consistent with the associated UAV Orthomosaic. The number and the percentages of different classes of pixels in the dataset are shown in

Table 1. Number and percentage of manually labeled pixels of each class in different training sets for ML and DL.

	Train & Validation Set in DL		Test Set in DL		ML (by Random Sampling)
Class	Labelled Pixels	Percentage	Labelled Pixels	Percentage	Labelled Pixels (Sum)	Percentage
Flat	11,799,975	10.55%	9,993,984	9.82%	21,793,959	10.20%
Imperata cylindrica	10,488,110	9.38%	10,488,110	10.31%	20,976,220	9.82%
Phragmites trails	65,549,739	58.61%	56,959,595	55.97%	122,509,334	57.35%
Carex scabrifolia Steud	14,713,632	13.16%	17,406,834	17.10%	32,120,466	15.04%
Scripus mariqueter	9,287,021	8.30%	6,916,704	6.80%	16,203,725	7.59%
Total	111,838,477	100.00%	101,765,227	100.00%	213,603,704	100.00%

, where the DL model used data for training and test separately, the ML method directly sampled randomly from the total dataset for training and validation in this study.

2.4. PB and OBIA Classification in GEE

To implement PB and OBIA methods of UAV data in GEE, first, we randomly sampled 5000 sample points in each sub-dataset of the dataset (two sub-datasets in each high- and low-tide flat, four sub-datasets in total), and categorized 70% as the training set and 30% as the validation set to participate in the subsequent model training and validation. We then performed feature engineering on the UAV data: RGB-based spectral feature extraction, texture feature extraction, and feature combination with the previously generated DSM to be added to the subsequent classification experiments. For PB classification, it can be performed directly in GEE; for OBIA classification, we used the classification method proposed by [44], combining feature extraction and SNIC image segmentation methods. The specific information is shown below.

2.4.1. RGB-Based Vegetation Index Extraction

To extract spectral features from RGB Orthomosaic, we calculated RGB-based vegetation indices commonly used for vegetation classification in RGB Orthomosaic. In coastal wetland monitoring based on UAV-RGB data, RGB-based vegetation indices also play an essential role in salt marsh mapping [46], estimation of salt marsh vegetation biomass [47], coastal wetland restoration monitoring [21], and so on. We extracted Excess of Green (ExGI), Normalized Green-Red Difference Index (NGRDI), Relative Greenness Index Green Chromatic Coordinate (GCC), Normalized Green-Blue Difference Index (NGBDI), and Visible Light Difference Vegetation Index (VDVI), and the related equations are shown in

Table 2. The RGB-based spectral indices used in this study, where R, G, B represent the red, green, and blue band of the UAV Orthomosaic, respectively.

Vegetation Indices	Formulation	References
ExGI	$2\times \mathrm{G}-\left(\mathrm{R}+\mathrm{B}\right)$	[48]
NGRDI	$\frac{\mathrm{G}-\mathrm{R}}{\mathrm{G}+\mathrm{R}}$	[49]
NGBDI	$\frac{\mathrm{G}-\mathrm{B}}{\mathrm{G}+\mathrm{B}}$	[50]
GCC	$\frac{\mathrm{G}}{\mathrm{R}+\mathrm{G}+\mathrm{B}}$	[51]
VDVI	$\frac{2\mathrm{G}-\left(\mathrm{R}+\mathrm{B}\right)}{2\mathrm{G}+\mathrm{R}+\mathrm{B}}$	[38,52]

.

2.4.2. Texture Feature Extraction

We extracted texture features based on the gray-level co-occurrence matrix (GLCM), which calculates the second-order statistics of texture features by considering the statistical distribution of intensity combinations at specific locations in the image [53]. In GEE, the algorithm uses 8-bit grayscale images as input data and is eventually able to generate 18 texture features. In this study, we used the common-used RGB grayscale conversion as shown in Equation (1) to convert the UAV RGB Orthomosaic to grayscale images for subsequent GLCM algorithm analysis.

$$\mathrm{Gray}=\left(0.3\times \mathrm{Red}\right)+\left(0.59\times \mathrm{Green}\right)+\left(0.11\times \mathrm{Blue}\right)$$

(1)

We used seven commonly used texture features [44] for subsequent classification studies based on GLCM, which are energy, contrast, autocorrelation, variance, inverse variance, mean sum, and entropy, their abbreviations and introductions are shown in

Table 3. Texture features calculated from GLCM used in this study.

Features Name	Description
GLCM_asm	Angular Second Moment; measures the number of repeated pairs
GLCM_contrast	Contrast; measures the local contrast of an image
GLCM_corr	Correlation; measures the correlation between pairs of pixels
GLCM_var	Variance; measures how spread out the distribution of gray-levels is
GLCM_idm	Inverse Difference Moment; measures the homogeneity
GLCM_savg	Sum Average; measures the mean of the gray level sum distribution of the image
GLCM_ent	Entropy; measures the degree of the disorder among pixels in the image

.

2.4.3. Image Segmentation in GEE

After feature extraction, we performed image segmentation for the UAV data, which is a vital step in the OBIA classification, and the segmentation result directly affects the final classification. The SNIC algorithm [54] is the most frequently used for implementing OBIA classification in GEE [55,56]. The user can optimize the segmentation by adjusting the parameters of SNIC, which finally generates a multi-band raster file, including clusters (segmented object) bands with IDs, and bands consisting of the average of the input bands over each cluster. The ‘seedGrid’ parameter in the SNIC algorithm, which determines the size of the segmented object, has a great impact on the segmentation results and therefore needs to be optimized [44]. We considered the landscape characteristics of UAV data through pre-experiments and finally chose the seedGrid of 36, 48, and 60, which are potentially optimal segmentation scales. For the other segmentation parameters, we empirically choose ‘compactness’ as 0, ‘connectivity’ as 4, ‘neighborhoodSize’ as 128.

It is worth mentioning that in order to alleviate the computational time limitation of GEE in a classification, we first exported the segmentation results to the asset for storage and then read it for subsequent OBIA classification experiments.

2.4.4. Random Forest Classifier and Feature Relative Importance Analysis

In this study, we chose random forest as a ML classifier for the classification experiments of PB and OBIA in GEE. Among all traditional ML algorithms, the random forest model is considered as one of the optimal remote sensing supervised classification algorithms [57], and also the most widely used ML algorithm in the GEE environment [58], because it is not only able to realize a relatively fast generation of stable and accurate classification results, but also has the ability to rank and analyze the relative importance of the individual features used for classification [59]. We selected the ‘smileRandomForest’ command in GEE for random forest classification, ‘numberOfTrees’ parameter was empirically selected to be 150, and then, we used the ‘explain’ command in GEE to rank the relative importance of the features used in the classification, which facilitated a better understanding of the key information in the classification of coastal wetland vegetation. In addition, we used geemap [58] to implement all the analysis and processing of UAV in GEE in this study.

2.5. Deep Learning

2.5.1. Data Slicing and Data Augmentation

Due to the large data volume of UAV images, directly feeding the complete UAV data into the DL network will lead to GPU memory overflow, so it is necessary to crop the UAV images to a smaller size. In this study, we used the sliding window algorithm with an overlap ratio of 0.5 to slice the UAV data into smaller datasets of 256 × 256 pixels for subsequent training [60]. Similarly, for the prediction of UAV images, we sliced the UAV data into small patches and input them into the model, and then reassembled the generated small patches with land use categories according to a certain overlap ratio to generate a classified map.

For the training of deep networks with complex structures, data slicing produces a smaller number of tiles, which can easily lead to overfitting and seriously affects the generalization ability of the model. Therefore, we need to perform dataset augmentation, i.e., increase the number of datasets to reduce the risk of model overfitting. In this study, we used (1) geometric transformations (e.g., random flip, mirroring, etc.) to simulate the shooting angles of different wetland vegetation in UAV images, and (2) non-geometric transformations (e.g., color adjustment, etc.) to simulate images of different vegetation under different lighting conditions, etc. The above operations were implemented by Python and the related open-source libraries Rasterio and Albumentations.

2.5.2. Training of DL Models

In the training process of the DL model, the difference between the predicted and labeled values is first calculated at the output by forwarding propagation of the deep neural network with a loss function. Then, by back propagation, the network parameters are biased layer by layer using the chain rule, and the weight parameters in the model are optimized by an optimizer, which reduces the loss and thus allows the network model to learn effectively [61]. In this experiment, we use three DL-based semantic segmentation models commonly used for remote sensing classification, respectively, U-Net [62], PSPNet [63], and DeepLabV3+ [64] for the classification of UAV data.

In the training process, we used cross entropy as the loss function and Adam as the optimizer. The Adam algorithm introduces the second-order momentum, and using momentum and adaptive learning rate can speed up the convergence of neural network training. We conducted our experiments in a Python environment, where we used PyTorch [65] as a DL library and set the optimization parameters using PyTorch’s recommended values, where we set the learning rate to 0.001, β₁ = 0.9, and β₂ = 0.999.

In addition, we deployed the DL semantic segmentation models via the segmentation_models_pytorch library in Python environment. The whole experiment was executed on a PC with Intel Core i9-9900K, NVIDIA Geforce RTX 2070 Super, and 64 GB memory. All experimental design of this study is shown in

Table 4. The experimental design in this study. In the PB and OBIA methods, we tested the classification performance of different combinations of bands; while in the DL method, we evaluated the influence of RGB + DSM data fusion to the classification.

Classification Paradigm	Classification Model	Combination of Features
PB and OBIA	Random Forest	RGB
PB and OBIA	Random Forest	RGB + DSM
PB and OBIA	Random Forest	RGB + VI
PB and OBIA	Random Forest	RGB + Texture
PB and OBIA	Random Forest	RGB + VI + Texture
PB and OBIA	Random Forest	RGB + VI + Texture + DSM
DL	U-Net	RGB RGB + DSM
DL	DeepLabV3+	RGB RGB + DSM
DL	PSPNet	RGB RGB + DSM

.

2.6. Accuracy Evaluation Metrics

We calculated Overall Accuracy (OA), precision, recall, and F1-Score based on the confusion matrix to evaluate the classification results of PB, OBIA, and DL classification in this study. OA can generally represent the performance of the model, but it may also produce misleading results when the number of class pixels is unbalanced. Therefore, we calculated precision, also known as user’s accuracy, which refers to the proportion of all samples whose classification prediction is truly correct. We also calculated Recall, also known as producer’s accuracy, which accurately reflects the completeness of the positive predictions in relation to all pixels that have already been accurately classified in the ground truth. Lastly, the F1-score [66], which integrates and balances the accuracy and recall metrics, was calculated to evaluate the performance of the model comprehensively.

$$\mathrm{Overall}\mathrm{Accuracy}=\frac{\left(TP+TN\right)}{\left(TP+TN+FP+FN\right)}$$

(2)

$$\mathrm{Precision}=\frac{TP}{\left(TP+FP\right)}$$

(3)

$$\mathrm{Recall}=\frac{TP}{\left(TP+FN\right)}$$

(4)

$$\mathrm{F}1-\mathrm{score}=2\times \frac{\left(Precision\times Recall\right)}{\left(Precision+Recall\right)}$$

(5)

Since PB, OBIA, and DL use completely different validation data due to the difference in classification paradigms, i.e., OBIA uses random sampling points for validation; while DL uses the complete ground truth map for validation. Their Precision, Recall, and F1-Score in the above equation cannot be directly compared. In order to compare PB, OBIA, and DL under the same evaluation system, we calculated the Intersection over Union (IoU) and the mean Intersection over Union (mIoU), both of which are common metrics for evaluating model performance in computer vision and are also gradually applied in DL-based UAV remote sensing classification [31,67]. Different from the above metric, the IoU and mIoU use the complete ground truth map rather than points, and for a class in the dataset, the IoU measure measures the degree of similarity between the predicted map and the associated ground truth map, which can better represent the performance of the model in classifying the fine boundaries of the class in high-resolution data. While mIoU is the average of IoU of each class, which is more comprehensive to reflect the classification performance of the model.

$$\mathrm{IoU}=J\left(A,B\right)=\frac{\left|A\cap B\right|}{\left|A\cup B\right|}=\frac{TP}{\left(FP+TP+FN\right)}$$

(6)

In Equation (6), A and B represent the predicted map in classification and the ground truth map, respectively. TP refers to the true positive fraction, FP refers to the false positive fraction, and FN refers to the false negative fraction, TN refers to the true negative fraction.

3. Results

3.1. Image Segmentation Results Based on SNIC Algorithm in GEE

Figure 3a–c shows the visualization of converting each SNIC raster locally to feature data, indicating the effect of SNIC segmentation of the UAV image. The seed parameter is the main parameter to be optimized in the SNIC algorithm, which can change the seedGrid of the segmented object. The results revealed that the degree of fragmentation becomes progressively lower, and the generated objects become gradually smaller as seedGrid increases.

The segmentation is too fragmented if the seed is chosen too small, which may lead to the subsequent classification to produce a mixed classification, while if the seed is chosen too large, the objects generated by the segmentation are too large, which may lead to the vegetation boundaries not being well finely segmented or even incorrectly segmented, as shown in Figure 3c. Therefore, an optimal segmentation result needs to be balanced between these two aspects to achieve the best subsequent classification results. After comparison, we believe that seedGrid of 48 has the best segmentation and Figure 3b shows that the boundaries of different vegetation are clearly distinguished, which can provide a solid foundation for the next step of classification. The above results also suggest the feasibility of using the SNIC algorithm in GEE to segment the UAV-scale images.

3.2. Results of PB and OBIA Classification under Different Feature Engineering

As shown in Figure 4, the vegetation index and texture features extracted from RGB orthophotos and RGB-derived DSM were progressively combined to evaluate the influence of feature engineering on PB and OBIA classification of coastal wetland UAV data. In general, each combination of extracted features is able to improve the performance of remote sensing classification compared to classification using RGB data only, which proves the effectiveness of feature engineering. Furthermore, for the same feature combination, both OA and F1-score can be enhanced in the OBIA classification (Figure 3b) compared to the PB (Figure 3a), which indicates the effectiveness of OBIA in the classification of UAV data. Among them, the best classification performance was achieved in the feature combination (RGB + VI + Texture + DSM) in OBIA classification, with an F1-score of 0.8517 and an OA of 87.98%. For feature combinations, the addition of DSM can greatly improve classification accuracy. In PB classification, the addition of DSM improves OA from 65.48% (RGB) to 76.38% (RGB + DSM), with a 10.9% improvement, and it is higher than the OA of 74.82% (RGB + VI + Texture); while in OBIA classification, adding DSM features improves the OA by 7.21% compared to the initial RGB combination. Specifically, the combination (RGB + VI + Texture) can achieve a considerable classification result even without adding DSM features in OBIA classification, with F1-Score and OA reaching 0.8517 and 87.98%, respectively.

3.3. Relative Feature Importance in PB and OBIA Classification

Figure 5 illustrates the relative feature importance of PB and OBIA Classification with full feature extraction (RGB + VI + Texture + DSM), where the RGB band, vegetation index, texture features, and DSM are ranked for their contribution in classification. Firstly, DSM features ranked first both in PB and OBIA classification, which proved that DSM features have important significance in discriminating coastal wetland vegetation. Subsequently, several differences in feature importance in PB and OBIA classifications were found. There is a wider variation in the importance of features, with vegetation index such as NGBDI and NGRDI ranking second and third, followed by texture features as well as other features, while ExGI and texture features such as Angular Second Moment, Entropy, ranked last and are weaker than the RGB bands in the PB classification.

Whereas in the OBIA classification, the importance of features is relatively more equal, with the texture feature Correlation ranked second, followed by vegetation indexes such as NGBDI and NGRDI, whereas vegetation indices such as GCC and VDVI are weaker than the RGB bands, which indicates that OBIA classification is better in utilizing texture features than PB classification, which is consistent with OBIA classification’s characteristic of aggregating adjacent and similar pixels by texture features and classifying them.

3.4. Results of DL Classification and the Comparison with PB and OBIA

Figure 6 shows the classification evaluation metrics for different DL models with different data combinations. As can be seen from the figure, all DL models yielded favorable classification accuracies, where F1-scores were greater than 0.89, and mIoU reached more than 80%. The non-distinction between DL models also indicates that the features of the dataset are relatively easy to learn. In particular, the DeepLabv3+ model achieves the highest classification accuracy, with an OA of 94.62% and F1-scores of 0.8188. Surprisingly, adding the fused RGB and DSM data to the DL model training does not seem to be effective in improving the model performance compared to the RGB data.

The comparison of the accuracy evaluation of PB and OBIA classification, after full feature extraction (RGB + VI + Texture + DSM), with the best classification of the DL model is shown in

Table 5. Comparison of accuracy assessment of PB, OBIA, and DL classification. The metrics in PB and OBIA classification are the average of 20 iterations under complete feature combination extraction (RGB + VI + Texture + DSM) in GEE.

Class		PB (Full Feature)		OBIA (Full Feature)			DL (RGB)
	Precision	Recall	IoU	Precision	Recall	IoU	Precision	Recall	IoU
Flat	0.7929	0.7258	0.4056	0.8401	0.8126	0.5366	0.7478	0.7670	0.6093
Imperata cylindrica	0.6938	0.5483	0.4636	0.8163	0.7293	0.6741	0.9532	0.9074	0.8687
Phragmites trails	0.8560	0.8884	0.8346	0.9065	0.9202	0.8783	0.9699	0.9832	0.9541
Carex scabrifolia Steud	0.7953	0.7663	0.5162	0.8642	0.8654	0.6500	0.9453	0.8933	0.8494
Scripusmariqueter	0.7832	0.8286	0.5460	0.8774	0.8859	0.7129	0.8874	0.9057	0.8123
Overall accuracy (%)	81.47%			87.98%			94.62%
F1-Score	0.7675			0.8517			0.8957
mIoU	0.5532			0.6904			0.8188

. Collectively, the mIoU of DL is 0.8188, which is significantly higher than results of the PB and OBIA, indicating that the overall classification performance of DL is significantly better than that of PB and OBIA. Despite the sufficient feature extraction and object segmentation of OBIA, it still shows a poor classification performance with an mIoU of 0.6904, which indicates that there are still some errors compared with the ground truth map. This result also demonstrated that there are large differences between UAV remote sensing classification paradigms. Among them, IoU reached high values for all vegetation classes in the result of DL, and the higher IoU indicated that the DL achieved a fine classification result on the vegetation boundary of coastal wetlands. Unexpectedly, the classification performance of the Flat class under the DL model is not satisfactory with an IoU of 0. 6093. The possible reason is that the high tide affects the quality of the labels produced by visual interpretation and therefore affects the effectiveness of the DL model. Nonetheless, the IoU of DL was greater than OBIA and DL in all classes, including both plant community and flat.

Moreover, detailed comparison of the results of vegetation classification on the high- and low-tidal flat using PB, OBIA, and DL methods in the study area is shown in Figure 7. For the result of PB classification, there were many pixel-like “pepper effects” and misclassification, such as the Carex class in Figure 7b and the Imperata and Scripus class in Figure 7f, which also greatly influenced the final classification results. For the result of OBIA classification, some obvious fragmentation was produced, for example, the Carex class in Figure 7c, and the Scripus class in Figure 7g. These fragmentations also affected accurate monitoring of coastal wetlands. The classification result of DL (Figure 7d,h), on the other hand, reached the classification result that best reflects the realistic distribution of the vegetation communities, which is also consistent with the high mIoU it achieved. In addition, their classification result of full RGB-UAB image is shown in Appendix A, Figure A1, Figure A2 and Figure A3.

4. Discussion

4.1. UAV Data Processing and Mapping Based on GEE

The results of PB and OBIA classification obtained in this study also highlight the feasibility of using GEE platform to process ultra-high-resolution data. Moreover, this study is the first attempt to perform OBIA classification on ultra-high-resolution UAV data in GEE, and the SNIC segmentation algorithm successfully segmented the fine patch of vegetation. It is worth pointing out that during the use of OBIA in GEE, when performing parameter optimization for image segmentation, since the SNIC algorithm generates a raster, it needs to be vectorized to evaluate the effect of image segmentation [44]. What is more, GEE can display images at the UAV scale, but it is much slower to vectorize the SNIC segmentation results due to the large size of the dataset. Therefore, we recommend using a subset of Region of Interests (ROIs) and optimize the segmentation parameters based on it. Although this step is slightly tricky, OBIA classification based on GEE is still the recommended solution for its powerful computing power and data processing capability. In addition, the processing capabilities of UAV data in GEE shown in this study can also be used in UAV-based applications apart from classification, such as reprojection, calculation of vegetation index, biomass retrieval, and so on, which can further expand the potentiality of applying GEE for UAV processing.

4.2. PB and OBIA Classification Based on RGB-UAV Data with Feature Engineering

We extracted spectral features, texture features, and RGB-derived DSM for UAV data and used the combined features for PB and OBIA classification in GEE, and the considerable results also demonstrate the effectiveness of feature engineering. Considering the limitation of spectral information in RGB-UAV data, feature engineering provides new effective features for ML classification, which consequently improved the classification results.

Our results demonstrated that DSM plays significant role in the classification of coastal wetland vegetation, which is consistent with previous studies [16,36]. Moreover, the in situ experiment also supported that elevation and micro-geomorphology have an essential effect on the distribution of coastal wetland vegetation [68]. In addition, our results showed that vegetation indices, especially NGBDI and NGRDI, have an important contribution to coastal wetland classification, ranking at the top of both PB and OBIA classification, in agreement with former research. For example, Yuan et al. [69] compared the performance of 10 RGB-based vegetation indices in UAV for extracting vegetation information and found that NGBDI had the highest accuracy and efficiency of extraction. Villoslada et al. [70] emphasized the key role of NGRDI in the classification of vegetation in coastal wetlands as it is sensitive to changes in leaf color and density of vegetation and can characterize the variation in vegetation pigment among different communities. As for texture features, our results revealed that texture features can be utilized more effectively in OBIA classification compared to PB classification, which can explain the high accuracy of OBIA classification since the image segmentation is mainly based on features such as texture features in the OBIA method [25].

This study once again proved that OBIA classification is superior to PB classification both in terms of metrics and classification maps, which is also in line with the relevant studies [71]. Furthermore, in the OBIA method, image segmentation aggregates objects into meaningful objects that are able to extract other types of information to complement the lack of spectral information of the image [72], which explains why OBIA can achieve a promising result for classification without DSM in feature combination (RGB + VI + Texture) and is superior to PB classification under complete feature combination (RGB + VI + Texture + DSM).

4.3. Paradigm Shift between PB, OBIA, and DL Classification

In this study, DL classification achieved the highest classification accuracy, which is similar to the accuracy achieved by previous applications of UAV using the DL model [30,31,73]. More particularly, the classification results of the DL method were representative of the actual wetland landscape, while the classification results of PB and OBIA were much inferior in comparison, despite the fact that they also achieved relatively high metrics of accuracy. Furthermore, the remarkable performance achieved by the DL model can be explained by its unique paradigm compared with PB and OBIA paradigms, as shown in Figure 8.

First, in terms of feature engineering, spectral features and texture features are extracted manually and combined with DSM in both PB and OBIA classification for this study. Moreover, manual feature engineering not only requires a priori knowledge, but also often requires eliminating features to avoid feature redundancy [39]. While deep neural networks can automatically extract deep abstract information from RGB data, such as leaf forms, branching patterns, and canopy shapes [30], which can explain the high accuracy achieved by DL classification only with RGB data. The sufficient features extracted by the DL model from RGB data with ultra-high resolution also explain the result that the combination of RGB data and DSM cannot improve the classification of the DL model effectively.

Second, regarding the training method. Traditional ML classification paradigms, such as the OBIA method, are trained in multiple stages, i.e., image segmentation is performed first, followed by the classification, and the segmentation process needs to be optimized for parameters. However, the details and texture features of different classes are magnified in the UAV data, and this makes the parameter optimization for fine object segmentation difficult [72], and any error in image segmentation directly affects the subsequent classification results, which also explains some fragmentation in our OBIA classification results (such as Figure 7c). In contrast, in the DL model, the model is trained in an end-to-end way, i.e., the input is the image, and the output is the classification map [29], and there is no requirement to manually tune the parameters during the process, but only to optimize the hyperparameters before training, which greatly reduces the difficulty of training and improves the accuracy of classification.

Third, concerning reference data and the methodology used in this study, the PB and OBIA methods usually use sample points obtained from field sampling for training and validation of the model. However, for instance, in the PB method, the pixel-based result of classification is difficult to reflect the real landscape of wetlands, such as vegetation patches [25]. Whereas in the OBIA method, the real landscape is generated from image segmentation, such as the SNIC algorithm used in this study. Whereas the segmentation generated solely using image processing algorithms still causes uncertainty in reflecting the reality of the wetland landscape, especially with ultra-high-resolution data. In contrast, DL models are trained directly using labeled data generated by visual interpretation in vegetation classification [74]. The ability to learn directly from fully labeled data allows DL to produce classification maps that represent the real landscape.

4.4. Coastal Wetland Monitoring Based on UAV, DL, and Cloud Computing Platform

The study demonstrates the advantages of combining DL and RGB-UAV data for vegetation classification of coastal wetlands. In this regard, UAV with RGB sensors is a commonly used consumer product that can be obtained at a more affordable price. Further, solutions based on DL and RGB-based UAV data are expected to solve problems that could only be solved by LiDAR and multispectral or hyperspectral imagery in the past, such as vegetation classification [33]. Considering that it is not available to train DL models directly based on GEE [75], because DL models must be deployed and trained by using GEE APIs outside of the GEE platform as well as using other cloud computing platforms, we therefore deployed DL models in a local environment. The training of DL models requires advanced devices such as high-performance GPUs, whereas the cloud-based platforms such as Amazon Web Service, Google Cloud Platform and combined with open-source code can alleviate this problem for better UAV-based coastal wetland monitoring. What is more, in this study, only single-temporal data were used to train the DL model, and the utilization of the DL model in multi-temporal data in UAV-based coastal wetland monitoring is recommended for further research. Considering the training of DL models usually requires a large amount of labeled data [76], how to efficiently and automatically generate deep learning labels for remote sensing classification becomes an important and pending issue in future research.

5. Conclusions

In this study, we compared the performance of PB, OBIA, and DL methods, in the classification of coastal wetland vegetation communities. In particular, we used the RGB-derived DSM, and feature engineering in the RGB data, including vegetation indices, feature texture extraction, and PB and OBIA methods were performed in the GEE platform, and random forest classifier-derived feature relative importance was also analyzed. We compared the classification maps of these three methods, as well as OA, precision, recall, and F1-score. Based on the above study we can conclude the following:

• This study showed the feasibility of using GEE to process ultra-high-resolution UAV data and successfully explored the implementation of the OBIA method for the first time to classify coastal wetland vegetation based on GEE.
• This study once again confirmed that OBIA is better than PB classification in terms of both classification metrics and classification result map, and can reduce the pepper effect.
• Our results revealed that DSM played the most important role in PB and OBIA classifications, whereas the addition of DSM seemed to have little improvement in the accuracy of DL models. Moreover, texture features of Correlation were effectively utilized in OBIA classifications and ranked second in feature contribution. In addition, vegetation indices such as NGBDI and NGBRI also ranked high in contribution to PB and OBIA classifications.
• This study demonstrated that the DL model achieves better classification than OBIA and is more capable of reflecting the realistic distribution of vegetation.
• The paradigm shifts from PB and OBIA to the DL method in terms of feature engineering, training methods, and reference data explained the considerable results achieved by the DL method.

In addition, UAV carrying RGB sensors used in this study are commonly used as consumer-grade products and can be obtained at an affordable price; and DL technology, which has been developing in the recent years, brings new possibilities for UAV applications. The utility of RGB-UAV combined with DL methods to precisely monitor coastal wetlands was demonstrated in this study. Moreover, the results also suggest that a combination of UAV, DL, and cloud computing platforms can facilitate long-term, accurate monitoring of coastal wetland vegetation at the local scale.