Siamese Detail Difference and Self-Inverse Network for Forest Cover Change Extraction Based on Landsat 8 OLI Satellite Images

Abstract

In the context of carbon neutrality, forest cover change detection has become a key topic of global environmental monitoring. As a large-scale monitoring technique, remote sensing has received obvious attention in various land cover observation applications. With the rapid development of deep learning, remote sensing change detection combined with deep neural network has achieved high accuracy. In this paper, the deep neural network is used to study forest cover change with Landsat images. The main research ideas are as follows. (1) A Siamese detail difference neural network is proposed, which uses a combination of concatenate weight sharing mode and subtract weight sharing mode to improve the accuracy of forest cover change detection. (2) The self-inverse network is introduced to detect the change of forest increase by using the sample data set of forest decrease, which realizes the transfer learning of the sample data set and improves the utilization rate of the sample data set. The experimental results on Landsat 8 images show that the proposed method outperforms several Siamese neural network methods in forest cover change extraction.

1. Introduction

Changes in forest cover affect the delivery of important ecosystem services, including biodiversity richness climate regulation, carbon storage, and water supplies [1,2]. As a considerable form of forest information extraction, forest cover change means transition between land with trees and land without trees [3]. Forest cover change mapping has important research value and economic benefits according to climate and carbon-cycle modeling [4,5], hydrological studies [6], habitat analyses [7,8,9], biological conservation [10], and land-use planning [11,12].

Remote sensing observation is a timely and accurate means to detect forest cover change on a large scale [13,14]. Compared with traditional field forestry investigation, remote sensing has an advantage of larger observation range and longer detection time span. So far, many researches on forest cover change mapping have been extensively carried out [15,16]. In the field of optical image, Kim et al. used the circa-1990 epoch of the Global Land Survey collection of Landsat images to detect forest cover change from 1990 to 2000 [17], and the results obtained 93% accuracy for forest cover and 84% for forest cover change. In the field of hyperspectral image, Huang et al. used 500 m MODIS time series images and a distance metric-based method to detect forest cover change in Pacific Northwest region of the United States and tropical forests of the Xingu River Basin in Mato Grosso, Brazil [18]. Over 80% pixels with 20% deforested area were correctly identified. In the field of synthetic aperture radar (SAR), Qin et al. [19] used the integration of the L-band Advanced Land Observation Satellite (ALOS) PALSAR Fine Beam Dual Polarization (FBD) mosaic dataset and Landsat images to map annual forests in sub-humid and semi-arid regions. The overall accuracy and Kappa coefficient of the PALSAR/Landsat forest map were nearly 88.2% and 0.75, respectively, in 2010. The accuracy of forest mapping has been improved as the development of earth observation technology and change information extraction technology.

The methods of forest cover change extraction can be divided into four categories: threshold segmentation, vegetation index segmentation, object-oriented segmentation and machine learning segmentation methods. The threshold segmentation method generates different change extraction levels using vegetable sensitive spectral bands [20]. The key point of the threshold segmentation method is to select the boundary value of the forest and other ground objects in the spectrum and index. The vegetation index segmentation method is a traditional way for detecting change information by establishing a monitoring index, such as the Enhanced Vegetation Index (EVI) [21,22,23], Global Environment Monitoring Index (GEMI) [24], Normalized Difference Fraction Index (NDFI) [25,26], Normalized Difference Moisture Index (NDMI) [27,28,29], Normalized Difference Vegetation Index (NDVI) [30,31], and Soil Adjusted Vegetation Index (SAVI) [32]. However, some vegetation indexes are particularly sensitive to observation frequency when forest cover change occurred gradually, and when change is abrupt the others are sensitive to observation frequency [33]. Different from pixel-based change detection, the object-oriented methods are usually used to extract forest cover change by reducing small spurious changes in scattered pixels [34]. The object-oriented methods require a lot of time for image segmentation and expert experience for image analysis [35,36]. The traditional machine learning segmentation method uses a wealth of training samples to train different types of classifiers for forest cover change detection, which include maximum likelihood [37], support vector machine [38,39], decision tree [40,41,42], random forest [43] and neural network (NN) [44,45]. These methods have the advantages of automatic, efficient target classification ability and require less manual labor [46]. While the machine learning segmentation method facilitates forest cover change extraction, there are still challenges in extracting complex change targets and context information [47].

Compared with above approaches, the deep neural network has the advantage of recognizing spectral and spatial features in remote sensing image synchronously. In the fields of various remote sensing image processing, deep neural network is a popular method used in target detection and image classification [48]. Along with the development of deep neural networks, diversified techniques are proposed to improve and extend the application of remote sensing [49,50,51]. As the popular architectures used in change detection, convolutional network and residual network have respective benefits and limitations. Fully convolutional network (FCN) is an architecture of convolutional neural network (CNN) and one of the most powerful algorithms for change detection of optical images [52]. However, the popular CNN-based segmentation methods, such as FCN [53], U-Net [54] and DeepLab [55], are usually developed from computer vision benchmarks composed of small-scale, high-resolution images. Timilsina, S et al. [56] trained an object-based convolutional neural network by object based image analysis thresholds and mapping tree cover changes between 2005 and 2015/16 images. This research generated tree training samples from RGB bands using only the threshold of canopy height model with manual editing. Bragagnolo, L et al. [57] compared U-Net with other state-of-the-art FCN architectures for Amazon forest cover change mapping. In this study, U-Nets achieved superior classification performance and could track forest cover changes from multi-temporal satellite imagery. Pablo Pozzobon de Bem et al. [58] used SharpMusk, U-Net, ResUnet, random forest and multilayer perception to classify the deforestation in the Brazilian Amazon with Landsat data. This research generated 844 samples through 200 × 200 pixel windows with 10-pixel overlap on image side. Compared among machine learning models, deep learning provided better classifications and have less process to remove noise in the research. The FCN architecture is more suitable for single temporal image segmentation, rather than bi-temporal image change detection. FCN architecture utilizes different weights to extract features for bi-temporal images in change detection. In fact, feature extraction process for same type bi-temporal images is identical. The feature extraction method for FCN causes more parameters of architecture and more samples to train the network. Siamese network was created for fingerprint recognition initially and promoted to detect difference of two images according to same weights. Since Siamese network is designed and developed in theory and practice, this neural network contained two sub-networks has promising potential in cover change detection of large-size bi-temporal images. Zhan et al. [59] proposed a deep Siamese convolutional network with contrastive loss weight sharing mode and achieved the better accuracy on optical aerial images, compared with no-weighted loss weight sharing mode. Nevertheless, there are still difficulties to deal with forest cover change extraction and sample generation [60,61]. In order to solve this problem, the purpose of this paper is to introduce an application of Siamese neural network (SNN) to extract forest cover change.

The rest of this paper is organized as follows. Section 2 introduces related studies that were used for inspiration or comparison during the development of this work. Section 3 describes in detail the proposed module in Siamese neural network and self-inverse network that will be tested in the proposed neural network. Section 4 contains the experiments of proposed neural network architecture and other methods for forest cover change extraction and result comparisons in the test dataset. Section 5 and Section 6 contain this work’s discussion and concluding remarks.

2. Related Work

Baldi and Chauvin [62] proposed a neural network algorithm for fingerprint recognition. Presented with a pair of fingerprint images, the algorithm calculated an estimation of the probability that an image pair came from the same finger. Bromley et al. [63] first introduced an algorithm consisting of two identical sub-networks to verify the authenticity of the signatures, which was named Siamese neural network. In the study by Bromley et al. [63], Siamese neural network utilized the distance threshold between features from two signatures to decide the authenticity.

The Siamese neural network consists of two identical neural networks that share weights during the encoding process. Different from the typical convolutional network classifying its input data, a Siamese neural network learns the similarities between the two inputs to distinguish them. When a pair of pictures are input into the Siamese neural network, the distance metric among them is calculated by the sub-network in the neural network. This is called symmetry, and the twin network measures the distance metric to measure similarity. In the experiment of change detection task, the Siamese neural network can compare each image in the test set and the training set, and then select which one is likely to be in the different category.

When the Siamese neural network was first proposed, it was used for video recognition and image matching [64,65,66,67]. Then, Siamese neural networks are enriched in change detection among various image datasets and improvement tricks [68,69,70,71,72,73,74,75,76]. There are different image experiment datasets to train and test Siamese neural network. Zhang et al. [68] proposed a spectral–spatial joint learning network for change detection task. This method integrates spatial information and spectral information in the input process, and uses the two kinds of information to explore the changing area of the multi-spectral image. Zhang et al. [69] proposed a light-weighted pseudo-Siamese convolutional neural network (PSI-CNN) for change detection between airborne laser scanning and photogrammetric data. Applied on a large building changes dataset, the proposed PSI-CNN achieved better accuracy compared with five different architectures of CNN. To address the class labels that are not available from the input images, Hedjam, R et al. [70] proposed a Siamese neural network to detect the changes occurring in a geographical area after major damage. Trained with genuine and imposter patch-pairs defined semi-supervised way, the Siamese neural network has a promising performance on four real datasets. For the change detection in aerial images, Mesquita, DB et al. [71] presented a fully convolutional Siamese autoencoder method. This method reduces the number of labeled samples and gets competitive results on two different datasets. As very-high-resolution (VHR) images are increasingly available, Chen et al. [72] proposed a deep Siamese convolutional multiple-layers recurrent neural network (SiamCRNN) for change detection in multi-temporal VHR images. Integrating the merits of both convolutional neural network and recurrent neural network, SiamCRNN has an advantage in two homogeneous datasets and one challenging heterogeneous VHR images dataset. Besides, there are a variety of adjusted skills used to improve the Siamese neural network. Jiang et al. [73] proposed a pyramid-shaped feature-based Siamese neural network to extract building change information. The research introduced a global co-attention mechanism to evaluate the correlation among the input feature pairs and utilized various attention mechanisms to improve feature dependency. Chen et al. [74] proposed a Siamese-based spatial–temporal attention neural network to capture spatial–temporal at various scales. This model uses a self-attention change detection method in the feature extraction stage and generates more distinctive training features. In the field of change detection, Siamese neural network has less parameters to extract features than fully convolutional neural network. Compared with FCN, Siamese neural network architecture utilizes same weights for bi-temporal images and saves half parameters for training process. Lee et al. [75] proposed a local similarity Siamese network (LSS-Net) to detect urban land change in remote sensing images. The research developed a change attention map-based content loss function to use content information on two sequential images and introduced a local similarity attention module to enhance the performance of the LSS-Net. Wu et al. [76] proposed an attention mechanism Siamese neural network to localize and classify damaged buildings simultaneously. This kind of network uses different backbones and attention mechanisms to obtain effective classification features and channels. R. Caye Daudt [77] proposed two Siamese extensions of fully convolutional networks. The first network contains concatenation weight sharing mode, which is named fully convolutional Siamese concatenate neural network (FCSC). The second network contains subtract weight sharing mode, which is named fully convolutional Siamese difference neural network (FCSD). The two Siamese neural networks achieved better performance than fully convolutional network, using both RGB and multispectral images. Due to the single patch stretched method, FCSD is more suited for openly change detection dataset than FCSC.

In previous papers, the focus of these studies is mainly on urban change and disaster detection, while there are little attention on forest cover change and inverse use of samples in change detection. In addition, the feature information before change and the information difference in the change the process have not been paid attention to in the previous Siamese neural network research. In view of the unique characteristics of the mutual transformation of forest cover changes and the high cost of generating forest change samples, how to improve the detection accuracy and sample utilization rate is a problem that needs to be solved. In this study, two weight sharing Siamese neural networks, fully concatenation and fully subtraction, are selected as the comparison network to compare the improvement of the network by different levels of weight sharing. Figure 1 presents the structure of the above Siamese neural network model.

3. Methodology

3.1. Overview

The main process of this study is shown in Figure 2, which mainly includes the following four parts.

(1)

Suface reflectance calculation. In order to make the images of different time comparable, the digital number (DN) value of the images used in the experiment are converted into surface reflectance. The LEDAPS and LaSRC surface reflectance algorithms released by NASA/GSFC and the University of Maryland [78,79] is used to calculate the surface reflectance in this study.

(2)

Sample generation. The deforestation and degradation samples are generated and clipped into image patches. In this process, the surface reflectance images are aligned according to coordinates and manually labeled to create sample images. Thus, the remote sensing images and sample images are clipped into patches.

(3)

Forest decrease extraction. Two new versions of the Siamese neural networks are proposed to predict forest decrease. They are Fully Convolutional Siamese Global Difference network (FCGD) and Fully Convolutional Siamese Detail Difference network (FCDD). See Section 3.2 for details. The surface reflectance variables of the six bands (blue, green, red, near-infrared and two mid-infrared bands) in the labeled pixel are input into these models. The accuracy comparison of different methods are conducted using quantitative evaluation metrics. The extraction results in the various deforestation and degradation region are analyzed visually.

(4)

Forest increase extraction. Self-inverse networks are designed for forest increase extraction. See Section 3.3 for details. The forest decrease sample dataset is reversed in time phase and imported into the network used in the forest decrease prediction. Without changing the architecture of the networks, the Siamese neural networks are trained by the same sample and tested for forest increase extraction.

3.2. The Structure of the Proposed Siamese Neural Network

In the general Siamese neural network, the image patches are encoded into vector representations through the processing of convolutional layers and pooling layers. In order to cope with the task of change detection, different techniques and improvements are added to the Siamese neural networks. In previous studies, concatenation and subtraction weight sharing modes are general choices for Siamese neural network. In the FCSC, concatenation weight sharing mode lacks the focus of change information between the former and latter phases, while subtraction weight sharing mode lacks the focus of the former phase information in the FCSD. To address the deficiency of two above networks, it’s feasible to combine the two weight sharing modes in one neural network. In this research, two new weight sharing modes are proposed to integrate the advantages of the previous conventional methods. This modification can combine the advantages of different weight sharing methods into one network. The first type of network is to apply the concatenation method to the top layer of the convolutional layer, and use the subtraction method for the remaining layers. This network is named Fully Convolutional Detail Difference network (FCDD). This modification puts more weight on the change information in different layers, while the detail image information in two phases is not discarded. The second type of network is the opposite. It applies the subtraction method to the top layer of the convolutional layer, and uses the concatenation method for the remaining layers. This network is named Fully Convolutional Global Difference network (FCGD). This modification puts more weight on the information of two phases in different layers, while the detailed change information in two phases is not discarded. Figure 3 presents the structure of the FCGD and FCDD models.

3.3. Self-Inverse Network

Self-inverse network is a network that can achieve the possibility of using only one network for bi-directional image translation [80]. The self-inverse network generates an output given an input, and vice versa, with the same neural network. For example, we call the transforming from domain X to domain Y process A, and the transforming from domain Y to domain X process B. Compared with assigning two neural networks to process A and B, respectively, a self-inverse network can be used to detect these two processes with the same network. In a self-inverse network, a function f is self-inverse, meaning

$$\mathit{f}={\mathit{f}}^{-1}$$

(1)

It guarantees a one-to-one mapping. The feature of self-inverse network is that it learns one network to perform both forward (A → B: from A to B) and backward (B → A: from B to A) translation tasks. Figure 4 shows the comparison of self-inverse network and general neural network. The decrease and increase of forest can be regarded as a pair of reverse processes to some extent. The self-reverse network is to train the same neural network with the forest decrease samples, and to predict the forest increase on the same image. With the only one Siamese neural network and the same train sample dataset, the two inverse forest change detection tasks could be completed simultaneously. As an opposite process of forest decrease, reforestation and afforestation have inverse change processes from deforestation and degradation. Reforestation stands for the establishment of a forest cover in a location where the forest has been cleared for the activities like agriculture or mining in the recent past. Afforestation is defined as an establishment of forests where there wasn’t forest before, or where forest has been missing for a period. The processes both represent a conversion of non-forest areas into new forest, which is opposite to the transformation of deforestation and degradation. The approximate self-reverse process of forest reduction and forest increase motivates us to use forest reduction samples to train the Siamese neural networks and conduct self-reverse experiments for forest increase. It’s worth noting that the varieties of forest decrease contain those not belonging to forest increase, such as forest to large roads. The difference tests the self-inverse learning ability of Siamese neural network in forest increase experiment.

3.4. Data Augmentation

Aiming to enrich the varieties of training images, the data augmentation method is used to enhance the accuracy and reduce overfitting results. Before the training dataset is given to the network architecture, the images in sample are processed by two data augmentation skills, i.e., flipping the image samples horizontally and rotating the image samples by 90 or 180 degrees. Figure 5 and Figure 6 are examples of flipped and rotated samples, respectively. Using these augmentation skills, the raw sample quantity is enlarged by three times.

4. Experiments and Results Analysis

In this section, we first present the experiment implementation details. Then, we implement quantitative analysis and qualitative analysis on the proposed method (FCGD&FCDD) and other state-of-the-art methods (FCSC&FCSD) on forest decrease extraction. Finally, the self-inverse experiment performances of the above-mentioned methods on forest increase are compared.

4.1. Implementation Details

4.1.1. Datasets

In the subtropical and tropical area, there are different types of forest cover change. In this study, two representative regions are selected as the study area, which include abundant types of features, such as cities, hills, mountains, plains, and oceans. The Landsat 8 OLI images covering the study area are downloaded from the U.S. Geological Survey, whose metadata and cloud cover are shown in

Table 1. The metadata and cloud cover of the selected Landsat 8 OLI images.

Landsat Scene ID	Path/Row	Data Acquisition	Cloud Cover	Site Center
LC81190422015270LGN01	119/42	27 September 2015	2.28%	25${}^{\circ }$ N, 118.9${}^{\circ }$ E
LC81190422018070LGN00	119/42	11 March 2018	0.03%	25${}^{\circ }$ N, 118.9${}^{\circ }$ E
LC81250442015104LGN01	125/44	14 April 2015	0.75%	23.19${}^{\circ }$ N, 108.9${}^{\circ }$ E
LC81250442018304LGN00	125/44	31 October 2018	0.02%	23.19${}^{\circ }$ N, 108.9${}^{\circ }$ E

. In addition, major noise of cloud, mountain shadows, grassland and cropland are also included in the collected dataset. Figure 7 presents the selected images in 2015 and 2018 years.

4.1.2. The Creation of Training Dataset

The training samples for each image are manually labeled. The forest cover change samples contain all deforestation and degradation types. The labeling process of these samples is based on expert experience, and we try our best to ensure the reliability and accuracy of sample according to the high-resolution images in the Google Map software. To ensure the accuracy of the samples, only identified deforestation and degradation are selected as samples. The mixed and dubious pixels in farmland boundary, grassland and shrubland area are considered as non-forest. The representative sample examples for deforestation and degradation types are shown in Figure 8. The 480 sample patches are randomly divided into training part and testing part. In the experiment, 75% of the total samples are training samples, and 25% of the total samples are testing samples. In order to ensure the consistency conditions for algorithms comparison, the training samples of forest decrease and non-decrease in FCSC, FCSD, FCGD and FCDD are the same.

4.1.3. Evaluation Metrics

Based on the error matrix, the overall accuracy (OA), the kappa coefficients (KCs), Dice Index (DI), precision, recall, F-Measure and Intersection over Union (IoU) can be calculated using the following equations:

$$OA=\frac{TP+TN}{TP+TN+FP+FN}\times 100$$

(2)

$$KC=\frac{N\sum \left(TP+TN\right)-\sum \left(TN\times FN\right)}{N^2-\sum \left(TN\times FN\right)}$$

(3)

$$precision=\frac{TP}{TP+FP}$$

(4)

$$recall=\frac{TP}{TP+FN}$$

(5)

$$F-Measure=\frac{2\times precision\times recall}{precision+recall}$$

(6)

$$IoU=\frac{TP}{TP+FP+FN}$$

(7)

where TP, TN, FN and FP represent the numbers of pixels of true forest change, true background, false background, false change, respectively.

4.2. Comparison of Forest Decrease

4.2.1. Quantitative Analysis

The overall accuracy, Kappa coefficient, precision, recall, F-Measure and IoU of each architecture are calculated to quantitatively evaluate the accuracy of forest decrease extraction, which are shown in

Table 2. Accuracy evaluation of the four methods in the test dataset.

Models	OA (%)	KC	Precision	Recall	F-Measure	IoU
FCSC 1	99.42	0.7774	0.8779	0.7023	0.7803	0.6398
FCSD 2	99.16	0.7636	0.6474	0.9431	0.7677	0.6230
FCGD 3	99.11	0.5823	0.9311	0.4277	0.5861	0.4145
FCDD 4	99.51	0.8255	0.8565	0.8013	0.8280	0.7064

¹ Siamese concatenate neural network (FCSC). ² Siamese difference neural network (FCSD). ³ Siamese global difference neural network (FCGD). ⁴ Siamese detail difference neural network (FCDD).

. Comparing the quantitative indices of the four methods in the test dataset, the kappa coefficient of the FCDD is higher than those of the FCSC, FCSD and FCGD. The kappa coefficient of the FCDD is 0.8255, while those of FCSC, FCSD and FCGD are 0.7774, 0.7636 and 0.5823, respectively. Furthermore, comparison of the kappa coefficients shows that FCDD presents the best forest decrease and non-decrease extraction result in the four methods. The precision of FCGD is highest in the four architectures, while the precisions of FCSC, FCSD, FCGD and FCDD are 0.8779, 0.6474, 0.9311 and 0.8565, respectively. The recall of FCSD is highest in the four architectures, while the recalls of FCSC, FCSD, FCGD and FCDD are 0.7023, 0.9431, 0.4277, and 0.8013, respectively. As a comprehensive evaluation standard of precision and recall, F-Measure shows an integral evaluation for the models. The F-Measures of FCSC, FCSD, FCGD and FCDD are 0.7803, 0.7677, 0.5861, 0.8280, respectively. This result means that FCDD has a better ability to detect the true forest decrease and non-decrease than the other architectures. Besides, IoUs of FCSC, FCSD, FCGD and FCDD are 0.6398, 0.6230, 0.4145, and 0.7064, respectively. This result means that considering forest decrease detection accuracy as the unique evaluation standard, FCDD shows a good ability in forest decrease detection.

4.2.2. Qualitative Analysis

To evaluate the universality of the Siamese Convolutional Networks, typical forest decrease areas, including large roads and infrastructure projects, urban expansion, commercial deforestation, are selected from the classification results. The false and true forest decrease areas of the four neural networks are contrasted by image inspection to analyze the reasons for classification difference. The classification results of the deforestation and degradation using the four models are shown in Figure 9, and the longitude and latitude information of center sites in Figure 9 are shown in

Table 3. The longitude and latitude information of site center in Figure 9.

Row	Site Center
A	22${}^{\circ }$28${}^{\prime }$30${}^{″}$N, 109${}^{\circ }$35${}^{\prime }$30${}^{″}$E
B	25${}^{\circ }$45${}^{\prime }$00${}^{″}$N, 119${}^{\circ }$34${}^{\prime }$30${}^{″}$E
C	23${}^{\circ }$10${}^{\prime }$30${}^{″}$N, 109${}^{\circ }$15${}^{\prime }$00${}^{″}$E
D	22${}^{\circ }$16${}^{\prime }$00${}^{″}$N, 109${}^{\circ }$32${}^{\prime }$30${}^{″}$E

. In Figure 9, white represents true decrease, green represents false decrease, purple represents false background and black represents true background.

The performance comparisons among forest selected logging are shown in row A and B of Figure 9. With regard to most forms of logging for timber harvesting, especially selective logging, there are various status and shapes of bareland after logging, because the forest recover growth and vegetation of diverse densities are different when the images are observed. Therefore, distinct extract results of degradation are necessary, when the four methods different phenological characters and vegetation restoration status. In row A, FCSC and FCSD have some false classification in the left upper logging area. The false classification area is the land remaining some trees, which are different from the adjacent bareland. In row B, it can be clearly seen that FCSC and FCGD have more omission forest decrease. For the extraction of sinerispeal narrow and less obvious forest decrease, FCSC and FCGD miss some detailed information in row B. According to the pictures in row A and B, FCDD and FCSD have better extraction ability in forest decrease in the selected logging area.

The performances of large roads and infrastructure projects for logging are shown in row C and D of Figure 9. Compared with selected logging, forest decrease of large roads and infrastructure projects have better obvious features of shape and spectral. In row C, there is a large road construction project at the foot of the mountain. In the prediction results of FCSC and FCGD, there are some missed extraction in the road area. FCSD and FCDD have better prediction results in the prediction of road construction and logging in row C. In row D, there are large areas of infrastructure projects and path construction for infrastructure projects. The four models extract the forest decrease of path construction well, while FCSC and FCGD miss some forest decreases in the infrastructure projects. By analysis, FCDD shows an adequate performance in extracting various degradation areas, which confirms the compatibility and robustness of the proposed algorithms.

4.3. Comparison of Forest Increase

4.3.1. Quantitative Analysis

The experiment results in

Table 4. Accuracy evaluation of the FCSD method in the forest increase dataset.

Models	OA(%)	KC	Precision	Recall	F-Measure	IoU
FCSC 1	99.33	0.6229	0.5202	0.7862	0.6261	0.4557
FCSD 2	99.57	0.6426	0.7741	0.5524	0.6447	0.4757
FCGD 3	99.17	0.5814	0.4543	0.8223	0.5852	0.4137
FCDD 4	99.76	0.8169	0.8805	0.7640	0.8181	0.6923

¹ Siamese concatenate neural network (FCSC). ² Siamese difference neural network (FCSD). ³ Siamese global difference neural network (FCGD). ⁴ Siamese detail difference neural network (FCDD).

delineate that the Siamese neural networks trained by the same forest decrease samples successfully extract the reforestation and afforestation areas. In theory, forest decrease sample dataset includes the inverse forest increase sample dataset. This provides an opportunity to train self-inverse networks using forest decrease samples. The experiment results of the self-inverse network of forest increase verify the feasibility of the above idea. Among the quantitative experiment results, FCDD attains the best score compared with other three architectures. When considering the prediction results of forest increase and non-forest increase at the same time, KC and F-Measure can be used as evaluation criterias. FCDD achieves the best result in the two evaluation criterias, followed by FCSD, FCSC and FCGD. When considering the forest increase prediction as single focus point, IoU of the four architectures show that FCSC, FCSD and FCGD (IoU of 0.4557, 0.4757 and 0.4137) are not suitable to be directly applied to self-inverse prediction, while FCDD (IoU of 0.6923) has much better performances in forest increase detection.

4.3.2. Qualitative Analysis

The prediction results of FCSC, FCSD, FCGD and FCDD are shown in Figure 10, and the longitude and latitude of site center are shown in

Table 5. The longitude and latitude information of site center in Figure 10.

Row	Site Center
A	23${}^{\circ }$36${}^{\prime }$00${}^{″}$N, 109${}^{\circ }$10${}^{\prime }$30${}^{″}$E
B	22${}^{\circ }$56${}^{\prime }$30${}^{″}$N, 108${}^{\circ }$09${}^{\prime }$00${}^{″}$E
C	24${}^{\circ }$00${}^{\prime }$00${}^{″}$N, 108${}^{\circ }$37${}^{\prime }$15${}^{″}$E
D	23${}^{\circ }$25${}^{\prime }$15${}^{″}$N, 109${}^{\circ }$13${}^{\prime }$15${}^{″}$E
E	23${}^{\circ }$01${}^{\prime }$30${}^{″}$N, 108${}^{\circ }$12${}^{\prime }$45${}^{″}$E

. In the forest increase extraction experiments, the former status is usually bareland or shrubland, which is opposite to latter status of forest increase.

According to row A and B, FCDD learned the conversion principle and extracted the forest increase in bareland and shrubland precisely. In the prediction results of FCSC and FCGD, there are some false positive area near the forest increase regions, which are mostly the transformation from shrubland to forest. Processing the forest increase in row B, FCDD draws the outline of target area with little mislassification in row B. The false positive areas in the other three experiments show that FCSC, FCSD and FCGD lack the ability of distinguishing grassland and shrubland growth from forest increase.

In the forest increase prediction results of row C, D and E, the former status is usually bareland, while the latter status is shrubland or farmland. In row C and D, the change process is difficult to classify for FCSC and FCGD. These two models classify the process from bareland to farmland as forest increase, which is due to the concatenation weight sharing modes of the two models. The subtraction weight sharing mode gives FCSD and FCDD the ability of separating farmland increase from forest increase. In row E, FCSC and FCGD have the same misclassification in the increase of shrubland. On the contrary, FCDD and FCSD have little misclassification in such area.

5. Discussion

The application of the Siamese neural network in the field of remote sensing has gradually matured, and good results have been obtained in the direction of target detection and image matching. However, research on the use of Siamese neural networks to extract forest cover changes is still lacking. In this study, there are five meaningful aspects worth to discuss, which are listed in following.

(1)

Improvement compared with traditional Siamese neural networks. In this study, the classification accuracy of forest decrease and increase in two regions is evaluated through qualitative and quantitative analysis. The results demonstrate that the classification accuracy of subtract weight mode FCDD in various forest cover change is higher than that of concatenate and subtract weight sharing mode in FCSC and FCSD. Subsequently, the performance of subtract weight sharing mode in eliminating noise of forest increase is better than that of concatenate weight sharing mode. This phenomenon is due to the fact that the subtract weight sharing mode is more able to use the different information for forest change extraction than the concatenate weight sharing mode. Due to the fully concatenation weight sharing mode, FCSC extracts two phase information and lacks focus of change information. Due to the fully subtraction weight sharing mode, FCSD is designed to focus on the change information between two phase and extracts some pseudo-change information in the same time. Combining the above weight sharing mode, FCDD has the ability of utilizing different information and pseudo-change information simultaneously.

(2)

Differences between FCDD and FCGD. As two types of Siamese neural network, FCDD and FCGD have obvious differences in the theoretical method and experiment results. In the theoretical method, FCDD has a concatenation weight sharing mode in the top layer of downsampling process, while the other layers are made by subtract weight sharing mode. In the downsampling process of FCGD, the top layer is a subtract weight sharing mode, and the other layers are a concatenate weight sharing mode. In the experiment results, FCDD has better capability of noise eliminating than FCGD, such as shrub wither and grass wilt. The subtract weight sharing mode in the more subtle convolutional layers gives FCDD better forest cover change detection ability. As the break-even point of precision and recall, F-measure shows the quantitative performance measure of predict results. FCDD has the best F-measure score in forest decrease and increase extraction experiments, following by FCSC and FCGD. This difference approves that given the same forest change image, FCDD has better ability in predicting forest cover change than FCGD.

(3)

Self-inverse network. In the field of remote sensing, it is the first time that self-inverse network is used for forest cover change detection. In such a bidirectional change field, the self-inverse network experiment not only reduces the generation cost of training data, but also extends universality of classification architecture model. According to the results in the paper, the Siamese neural network has proven its accurate detection capacity and compatible universality in forest cover translation. The types of features before and after forest decrease and increase are not completely equal, thus forest decrease and increase are not completely reversible. The feature types after the forest decrease include the feature types before the increase, which allows the sample of the forest decrease to be used for training increase. However, lacking the combination of the bi-temporal feature information and the difference information, the traditional Siamese neural networks (FCSC and FCSD) perform poorly in self-inverse prediction. On the other hand, the novel fused weight sharing strategies make the proposed Siamese neural networks FCDD more robust to be applied to a self-inverse task.

(4)

Factors affecting accuracy. In the whole process of forest cover change detection, there are several factors affecting accuracy. The first factor is image preprocessing. To ensure data consistency, most change detection maps are based on the top of atmosphere (TOA) reflectance or surface reflectance. However, due to various shooting time and geographic variation, the surface reflectance of image collection exists difference, which influences control variables of the change information extraction. Secondly, the forest decrease training dataset includes some kinds of deforestation that doesn’t exist in the reforestation. This situation leads to the problem that the training dataset of deforestation and reforestation is incompletely self-inverse. The reforestation samples are improper for self-inverse deforestation experiment. Otherwise, various mixtures between the forest change and unchange areas exist.

(5)

Training samples generation. In order to select accurate samples of forest cover changes in the complex surface, this experiment combined Landsat medium-resolution images and Google Earth high-resolution images to manually label the forest decrease and increase samples. However, this process limits the automatic processing capacity of the proposed algorithm. This problem shows that various types of typical forest cover change samples will be the main demand for future work.

Compared with the studies of forest cover change extraction in fully convolutional neural networks, the present study utilized a weight sharing mode composed of subtraction and concatenation means in the Siamese neural networks. This promotion reduced the parameters to recognize the change features in the architecture, and decreased the possibility of overfitting in the training process. In addition, the proposed self-inverse network demonstrated that Siamese neural networks have the ability of extracting forest cover increase and decrease using a series of forest cover decrease sample through the self-inverse network. This adjustment saved the cost of sample generation in forest cover change extraction and enlarged the feasibility of the Siamese neural network.

The two major contributions of this paper are listed as follows:

(1)

A novel weight sharing mode of a Siamese neural network based on U-Net for forest cover change detection is proposed, and this method obtains promising classification results.

(2)

Self-inverse network of Siamese architecture is generated. According to the self-inverse network, forest decrease sample dataset is used for change detection in forest increase, which implements transfer learning of sample dataset and improves the utilization rate of sample dataset.

On the premise of providing more numbers and types of training samples, the proposed algorithm can refine the types of changes and be used to predict large-scale forest cover detection. In addition, the model can be further improved to suitable for other types of land cover change extractions with self-inverse process.

6. Conclusions

Deep neural networks have demonstrated good capabilities in target recognition and image segmentation in the field of remote sensing. This study uses a Siamese neural network after adjusting the weight sharing method to extract deforestation, degradation, afforestation and reforestation areas in Landsat 8 OLI images. Two images of typical subtropical regions are selected, and two traditional algorithms (i.e., Siamese concatenate neural network (FCSC), Siamese difference neural network (FCSD) are included to compare with Siamese global difference neural network (FCGD) and Siamese detail difference neural network (FCDD) in their performances when extracting forest change information. Then, the performances of various forest cover change extractions and noise suppressions are comprehensively compared. The conclusions are summarized as follows:

(1)

Based on a visual comparison, the performance of the Siamese detail difference neural network extracting forest cover change is better than those of Siamese concatenate neural network, Siamese difference neural network and Siamese global difference neural network. Moreover, quantitative evaluation shows that the overall accuracy and kappa coefficients of FCDD are higher than those of the other three classifiers. The kappa coefficients of FCDD in forest decrease and increase extraction experiments are 82.55% and 81.69%, and the F-measures and IoUs of those are 0.8280 and 0.8181, 0.7064 and 0.6923.

(2)

Compared with FCSC, FCSD and FCGD, the performance of FCDD demonstrates that it can precisely extract three types of large forest decrease areas (i.e., large roads and infrastructure projects, urban expand and logging), and detailed deforestation can also be identified. Furthermore, FCDD can effectively eliminate noise, such as grassland and shrub perishment.

(3)

In the forest increase extraction, FCDD has the advantage of self-inverse function learning the principle of forest transfer to non-forest. Trained by the existed forest decrease dataset, FCDD has the capacity of detecting forest increase without the effort of amending neural network parameters.

This paper introduces a Siamese neural network for extracting forest cover change, and the results confirm that the proposed method can achieve sufficient performance. For future research, the newly released forest cover change products can be used to further enhance the automation and versatility of the proposed algorithm. Then, the proposed method can be used to map large scale forest cover change, which will help us understand the forest change information under a background of global change.