1. Introduction
Land cover (LC) data has great importance for different disciplines, such as biodiversity patterns [
1], natural hazards studies (i.e., landslides [
2] and wildfire [
3]), and CO
2 emissions [
4]. Additionally, there is a considerable need for current and precise information on LC and its changes for sustainable development and global warming studies [
5]. The importance of the mentioned issues and the progress of Remote Sensing (RS) technologies toward providing data with better temporal and spatial resolutions have motivated scholars and scientists to study LC mapping widely. Although the tremendous attempts exerted in LC mapping, examining the roles of balancing data, image integration, and performance of different machine learning algorithms in various landscapes has not yet received much attention from scholars.
The advent of Sentinel-1 and Sentinel-2, providing images with high spatial resolution, global coverage, and ultimately their free access, brings excellent opportunities for LC mapping. As a result, many published papers have been conducted using these images. For example, Abdi [
6] integrated these images for LC mapping complex boreal landscapes. In another study, those images were integrated for LC mapping in Colombia [
7]. Integration of radar and optic RS data can deliver complementary information to improve LC mapping accuracy, taking advantage of both data [
8]. More precisely, the geometrical characteristics of the classes are mainly examined by Sentinel-1, providing C-band. At the same time, Sentinel-2 Multi-Spectral Instrument (MSI) is sensitive to the manifest content of the LC classes [
9]. It has been reported that incorporating time-series of these images can lead to more accurate and reliable LC maps compared to using them individually [
10]. However, integrating these datasets in different landscapes for LC mapping has not yet been well documented.
To boost LC mapping accuracy, adding some supplementary information (i.e., textural information and spectral indices) into the classification procedure has been endorsed as an efficient and practical approach [
11,
12]. For example, the impact of complementary information (e.g., topographic data and spectral indices) has been investigated for LC mapping in mountainous areas [
12]. Texture information provides some continuous measure of distribution in digital numbers of a satellite image within predefined local windows [
13]. Using texture information and spectral bands can create high separation capability among different LC types, particularly in heterogeneous landscapes [
13,
14]. Moreover, it has been reported that spectral indices can also improve LC mapping accuracy [
15]. Since using a large set of features has some disadvantages, such as being time-consuming and highly computational complex [
16], selecting the most critical features in LC classification using an appropriate feature selection method can lead to a more operative and reliable LC classification procedure [
17]. In this regard, the RS community has widely employed Feature Selection (FS) methods to select the most appropriate features from a pool of available features. Among the different FS methods, Support Vector Machine-Recursive Feature Elimination (SVM-RFE) as a powerful method has been successfully applied in different RS studies to eliminate redundant features [
16].
It is generally accepted that Machine Learning (ML) algorithms can effectively improve LC classification accuracy. In this manner, although standard ML algorithms, in most cases, can obtain substantially reasonable accuracies for majority classes, they usually show poor accuracies for rare (minority) classes, mainly owing to the class imbalance problem [
18]. Since LC classes are of various distribution and extent, gaining equal samples for all LC classes is very difficult [
19], leading to the class imbalance problem and unacceptable accuracies for minority classes. To address this issue, several balancing data have been presented. However, the proposed methodologies have primarily examined specific landscapes, and their performances in different landscapes have not been investigated. For example, Naboureh et al. [
20] proposed a hybrid data balancing method for mountainous regions. In another study, Waldner et al. [
21] investigated the impact of different data balancing techniques for mapping crops. To this end, the recently proposed method, namely the Geometric Synthetic Minority Over-sampling Technique (G-SMOTE), by Douzas and Bacao (2019) [
22], has been introduced as a robust method to address the class imbalance problem. However, there is still a lack of research that can thoroughly assess the performance of G-SMOTE in different landscapes by applying different ML algorithms.
Given the importance of the issues mentioned above, the present study was an attempt to investigate the performance of G-SMOTE to handle the class imbalance problem in LC classification at six different landscapes applying three frequently used ML algorithms, including RF, SVMs, and ELM. Furthermore, the SVM-RFE method was applied for each landscape to select the most informative features and use them as classification inputs to obtain the optimal feature subset from radar and optical bands, spectral indices, and texture information. Specifically, we are going to answer the following questions in this study:
- (1)
What are the most informative features from Sentinel-1, Sentinel-2, spectral indices, and textural information for LC mapping using three well-known ML algorithms in different landscapes?
- (2)
What is the performance of the G-SMOTE algorithm in LC classification in different circumstances?
- (3)
Which ML classifier has higher accuracy on LC mapping at diverse landscapes?
4. Results
This study applied the SVM-RFE method to obtain the ranked list of the features before classification. Therefore, original features, including Sentinel-1 bands (VV, VH), Sentinel -2 bands (B2, B3, B4, B8, B5, B6, B7, B8A, B11, and B12), spectral indices (NDVI, NDWI, and NDBI), and eight texture information derived from VV band (mean, dissimilarity, homogeneity, second moment, contrast, variance, entropy, and correlation) were used.
Table 4 gives a summary of the most critical features after adopting SVM-RFE for different landscapes.
The RF, SVM, and ELM approaches using the obtained features (
Table 4) were employed for generating LC maps in six different landscapes. The LC maps were generated with two sets of reference datasets to investigate the impact of class imbalance on the LC classification accuracy; first, without balancing samples and the scorned one by adopting the G-SMOTE for balancing samples of classes. As shown in
Table 5 and
Table 6, all three ML algorithms illustrated good performance in LC mapping. For example, all generated maps obtained OA above 0.83, ranging from 0.85 to 0.93 for RF, 0.83 to 0.94 for SVM, and 0.84 to 0.92 for ELM. Our analysis also showed that adopting the G-SMOTE method to rebalance reference datasets substantially improved UA and PA accuracies of minority classes. As illustrated in
Figure 3, RF-G-SMOTE showed the best performance in four landscapes, namely coastal, cropland, desert, and semi-arid. In comparison, the SVM-G-SMOTE obtained higher accuracies for the remaining two landscapes, including plain and mountain.
Table 5.
The result of accuracy assessment methods in six different landscapes (with original GCPs). Minority classes for each landscape are highlighted.
Table 5.
The result of accuracy assessment methods in six different landscapes (with original GCPs). Minority classes for each landscape are highlighted.
Methods | | SVM | RF | ELM |
---|
Sites | Class | UA | PA | OA | UA | PA | OA | UA | PA | OA |
---|
Coastal | Barren | 0.84 | 0.67 | 0.91 | 0.55 | 0.83 | 0.92 | 0.55 | 0.83 | 0.89 |
Built-up | 0.85 | 0.75 | 0.93 | 0.65 | 0.83 | 0.75 |
Cropland | 0.7 | 0.5 | 0.43 | 0.8 | 0.5 | 0.17 |
Forest | 0.94 | 0.96 | 0.96 | 0.86 | 0.91 | 0.96 |
Water | 1 | 0.97 | 1 | 1 | 1 | 1 |
Cropland | Barren | 0.68 | 0.55 | 0.84 | 0.52 | 0.59 | 0.85 | 0.55 | 0.43 | 0.84 |
Built-up | 0.97 | 0.92 | 0.89 | 0.87 | 0.97 | 0.82 |
Cropland | 0.87 | 0.96 | 0.83 | 0.9 | 0.82 | 0.92 |
Pasture | 0.73 | 0.76 | 0.76 | 0.76 | 0.67 | 0.77 |
Water | 1 | 1 | 1 | 1 | 1 | 1 |
Desert | Barren | 1 | 1 | 0.94 | 1 | 1 | 0.93 | 1 | 1 | 0.92 |
Built-up | 0.74 | 0.61 | 0.8 | 0.58 | 0.78 | 0.58 |
Cropland | 0.9 | 0.99 | 0.8 | 1 | 0.89 | 0.98 |
Water | 1 | 1 | 1 | 1 | 0.89 | 1 |
Mountain | Barren | 0.89 | 0.96 | 0.90 | 0.83 | 0.92 | 0.91 | 0.87 | 0.74 | 0.84 |
Cropland | 0.62 | 0.73 | 0.75 | 0.5 | 0.57 | 0.65 |
Pasture | 0.96 | 0.84 | 0.91 | 0.87 | 0.81 | 0.91 |
Snow | 0.9 | 0.9 | 0.81 | 0.81 | 0.73 | 1 |
Water | 1 | 1 | 1 | 1 | 1 | 1 |
Plain | Barren | 0.92 | 1 | 0.89 | 0.89 | 0.93 | 0.90 | 0.88 | 1 | 0.89 |
Built-up | 0.95 | 0.92 | 0.92 | 1 | 0.92 | 0.96 |
Cropland | 0.88 | 0.91 | 0.89 | 0.91 | 0.88 | 0.91 |
Pasture | 0.73 | 0.65 | 0.74 | 0.74 | 0.78 | 0.56 |
Semi-Arid | Barren | 0.82 | 0.91 | 0.84 | 0.86 | 0.89 | 0.85 | 0.82 | 0.85 | 0.84 |
Built-up | 0.87 | 0.85 | 0.87 | 0.85 | 0.83 | 0.8 |
Cropland | 0.84 | 0.84 | 0.83 | 0.87 | 0.79 | 0.86 |
Pasture | 0.57 | 0.53 | 0.55 | 0.57 | 0.53 | 0.56 |
Table 6.
The result of accuracy assessment methods in six different landscapes (after adopting G-SMOTE). Minority classes for each landscape are highlighted.
Table 6.
The result of accuracy assessment methods in six different landscapes (after adopting G-SMOTE). Minority classes for each landscape are highlighted.
Methods | | SVM | RF | ELM |
---|
Sites | Class | UA | PA | OA | UA | PA | OA | UA | PA | OA |
---|
Coastal | Barren | 0.85 | 0.88 | 0.91 | 0.87 | 0.83 | 0.92 | 0.85 | 0.85 | 0.88 |
Built-up | 0.88 | 0.76 | 0.84 | 0.80 | 0.84 | 0.75 |
Cropland | 0.91 | 0.85 | 0.80 | 0.81 | 0.77 | 0.83 |
Forest | 0.93 | 0.9 | 0.90 | 0.89 | 0.90 | 0.94 |
Water | 1 | 1 | 1 | 1 | 1 | 1 |
Cropland | Barren | 0.80 | 0.77 | 0.84 | 0.75 | 0.79 | 0.85 | 0.72 | 0.77 | 0.85 |
Built-up | 0.93 | 0.90 | 0.87 | 0.85 | 0.89 | 0.80 |
Cropland | 0.88 | 0.91 | 0.84 | 0.86 | 0.83 | 0.84 |
Pasture | 0.85 | 0.87 | 0.80 | 0.82 | 0.77 | 0.79 |
Water | 1 | 1 | 1 | 1 | 1 | 1 |
Desert | Barren | 1 | 1 | 0.93 | 1 | 0.98 | 0.93.5 | 0.97 | 1 | 0.91 |
Built-up | 0.88 | 0.78 | 0.89 | 0.79 | 0.79 | 0.87 |
Cropland | 0.92 | 0.93 | 0.9 | 0.94 | 0.92 | 0.82 |
Water | 1 | 1 | 1 | 1 | 1 | 1 |
Mountain | Barren | 0.9 | 0.96 | 0.91 | 0.83 | 0.95 | 0.90 | 0.85 | 0.81 | 0.84 |
Cropland | 0.85 | 0.82 | 0.85 | 0.88 | 0.80 | 0.78 |
Pasture | 0.96 | 0.9 | 0.86 | 0.94 | 0.84 | 0.94 |
Snow | 0.92 | 1 | 0.78 | 0.80 | 0.90 | 0.89 |
Water | 1 | 1 | 1 | 1 | 1 | 1 |
Plain | Barren | 0.91 | 0.98 | 0.90 | 0.89 | 0.93 | 0.89 | 0.93 | 0.95 | 0.88 |
Built-up | 0.92 | 0.92 | 0.93 | 0.95 | 0.92 | 0.92 |
Cropland | 0.89 | 0.91 | 0.89 | 0.90 | 0.93 | 0.93 |
Pasture | 0.80 | 0.75 | 0.81 | 0.78 | 0.82 | 0.78 |
Semi-Arid | Barren | 0.86 | 0.81 | 0.83 | 0.83 | 0.87 | 0.85 | 0.80 | 0.83 | 0.845 |
Built-up | 0.86 | 0.85 | 0.89 | 0.85 | 0.82 | 0.82 |
Cropland | 0.82 | 0.83 | 0.80 | 0.85 | 0.80 | 0.83 |
Pasture | 0.79 | 0.74 | 0.75 | 0.77 | 0.77 | 0.75 |
Figure 3.
Impact of G-SMOTE on the overall accuracy of the generated LC maps.
Figure 3.
Impact of G-SMOTE on the overall accuracy of the generated LC maps.
6. Conclusions
This study analyzed the potential of RF, SVM, and ELM in LC classification at six different landscapes by integrating Sentinel-1 and Sentinel-2 images. We also used SVM-RFE to select the most informative features from Sentinel-1, Sentinel-2, spectral indices, and textural information. Furthermore, we discussed the impact of G-SMOTE on the classification accuracy of ML algorithms. The result showed that NDVI, VV, and B12 could contribute as main features to improve LC classification accuracy. Our findings also indicated that all three ML algorithms, especially RF and SVM, are robust approaches for LC classification in different landscapes.
Moreover, the results confirmed that applying G-SMOTE has a significant impact on the accuracy of minority classes. After applying G-SMOTE to ML algorithms, the differences between UA and PA metrics for minority and majority classes have decreased. However, there were significant differences among them without considering the class imbalance problem. Further study could investigate the performance of other algorithms and sample sizes in balancing data.