Change Detection of Deforestation in the Brazilian Amazon Using Landsat Data and Convolutional Neural Networks

Abstract

Mapping deforestation is an essential step in the process of managing tropical rainforests. It lets us understand and monitor both legal and illegal deforestation and its implications, which include the effect deforestation may have on climate change through greenhouse gas emissions. Given that there is ample room for improvements when it comes to mapping deforestation using satellite imagery, in this study, we aimed to test and evaluate the use of algorithms belonging to the growing field of deep learning (DL), particularly convolutional neural networks (CNNs), to this end. Although studies have been using DL algorithms for a variety of remote sensing tasks for the past few years, they are still relatively unexplored for deforestation mapping. We attempted to map the deforestation between images approximately one year apart, specifically between 2017 and 2018 and between 2018 and 2019. Three CNN architectures that are available in the literature—SharpMask, U-Net, and ResUnet—were used to classify the change between years and were then compared to two classic machine learning (ML) algorithms—random forest (RF) and multilayer perceptron (MLP)—as points of reference. After validation, we found that the DL models were better in most performance metrics including the Kappa index, F1 score, and mean intersection over union (mIoU) measure, while the ResUnet model achieved the best overall results with a value of 0.94 in all three measures in both time sequences. Visually, the DL models also provided classifications with better defined deforestation patches and did not need any sort of post-processing to remove noise, unlike the ML models, which needed some noise removal to improve results.

1. Introduction

Deforestation is one of the primary sources of concern regarding climate change as it is one of the largest sources of greenhouse gas emissions in the world, second only to the burning of fossil fuels [1]. Within the region of the Brazilian Amazon, studies have shown that deforestation, in conjunction with forest fires, can make up to 48% of the total emissions [2]. It also bears substantial implications regarding the conservation of ecosystems and their biodiversity in the region, and it has been linked to the loss of species [3] and general loss of ecosystem stability through fragmentation [4]. Locally, estimates also show that unchecked deforestation could lead to reductions in seasonal rainfall [5] and into the savanization of the environment [6].

Remote sensing imagery has been instrumental in the process of keeping track of deforestation in the Amazon. The Brazilian National Institute for Space Research (INPE) releases annual deforestation and land use information derived from satellite imagery data through their Program for Deforestation Monitoring (PRODES) and TerraClass projects [7,8], which have been widely used for monitoring, research, and policymaking. Carbon emission estimates from deforestation are also dependent on land use and land-use change data [1]. However, they are likely to be underestimated due to the omission of illegal logging data in official reports [9].

Change detection is one of the most common tasks within the field of remote sensing. It is defined as the process of analyzing and quantifying the state of an object or phenomenon at different times [10], and is consequently an essential tool in the processes of understanding and tackling deforestation. The changes present in the images can be semantic (of the object under analysis) or noisy (variations in lighting, shadows, among others) [11]. Therefore, the challenge in change detection is to use a method that establishes features that minimize noisy changes and emphasize the semantic changes that are intertwined. Typically, the final map of change detection techniques is a binary classification that contains unchanged and changed regions.

Several reviews and classifications of digital change detection techniques are available in the literature [10,12,13,14,15,16,17,18], evidencing a large quantity of approaches and algorithms in this research area. Tewkesbury et al. [18] provide a synthesis of change detection methods, distinctly considering the units of analysis (pixel, kernel, image-object overlay, image-object comparison, multi-temporal image-object, vector polygon, and hybrid) and the method used to identify the change (layer arithmetic, post-classification change, direct classification, transformation, change vector analysis, hybrid change detection).

Change detection methods based on machine learning (ML) algorithms typically use direct classification [18], which takes a set of stacked temporal images as input and uses complex nonlinear functions to model and determine changes. In this approach, it is not necessary to use pre-classification techniques that seek to define the best measures to detect changes (such as temporal subtraction, data transformation, and change vector analysis). In long-term time series, direct classification based on ML is predominant [19,20].

Deep learning (DL) has recently attracted increasing attention from remote sensing researchers because of its ability to automatically extract features from the image dataset, high-level semantic segmentation, nonlinear problem modeling, and mapping in complex environments [21]. DL has shown great potential in remote sensing, producing state-of-the-art results in different types of remote sensing data processing [22]: image registration [23,24,25,26], land-use and land-cover classification [27,28,29,30], object detection [31,32,33,34], image fusion [35,36,37,38], semantic segmentation [39,40,41,42], and precision evaluation [43]. DL has also been used for change detection techniques, showing superior performance with greater precision in comparison to classic ML methods [44]. The capacity for pattern recognition in the three dimensions of the image (special, spectral, and temporal) makes DL algorithms especially effective when used to change detection with common and recurring patterns [45]. DL-based change detection methods have been applied to different targets such as urban [46,47,48,49], land use/land cover [50,51,52], and landslides [53], among others. Peng et al. [54] proposed a subdivision of DL-based change detection methods that considered three units of analysis: (1) feature [55,56,57]; (2) patch [58,59,60,61]; and (3) image [62,63]. In the case of image-based DL change detection, the algorithms learn the segmentation of changes directly from bi-temporal image pairs, avoiding the negative effects caused when using pixel patches [54]. In this approach, the U-Net architecture has been successfully employed [63,64]. Among the DL algorithms, convolutional neural networks (CNN) are one of the leading types of architectures [22]. CNNs differ from traditional ML algorithms by being able to identify patterns within an n-dimensional context with multiple abstraction levels through convolutional filters and use them for inference.

The objective of this study was to investigate the use of CNNs for the detection of deforestation within the Brazilian Amazon to verify the hypotheses that DL algorithms are a viable and possibly better alternative in comparison to classic ML algorithms when it comes to mapping deforestation. Like many anthropogenic changes in the landscape, deforestation follows specific spatial patterns with support geometric or regular configurations and usually develop around official or unofficial roads, forming a dendritic or “fishbone” distribution [65]. Despite being a prime target for the application of DL algorithms, the number of studies related to deforestation is still small, given the variety of the types of algorithms available. In order to investigate the use of DL for deforestation detection, three different CNN architectures were used to classify deforested areas yearly and were then compared to two classical ML algorithms as points of reference.

2. Material and Methods

2.1. Training and Test Sites

In this study, we selected three regions within the Brazilian Amazon as study sites. These scenes encompass major deforestation centers that have developed along the “TransAmazon” (BR-230) [66,67,68] and “Cuiabá–Santarem” (BR-163) [69,70] highways (Figure 1). In the Amazon, roads are the driving forces for the spatial distribution of deforestation in the Amazon, where most deforestation occurs in the neighborhood of the main highway [71,72]. Widely discussed in the literature, the opening of roads in the Amazon forest favors the establishment of settlements, attracts migrants, facilitates the extraction of resources, increases the profitability of livestock and agriculture, and establishes access to wood [73,74,75,76,77].

The training used two scenes (Sites A and B), and validation utilized the remaining scene (Site C). We defined a bi-temporal approach for modeling and obtained Landsat 8/OLI imagery for each site for the years of 2017, 2018, and 2019, with approximately one year between each observation. Multitemporal images from similar periods of the year reduce variations in the phenology and sun-terrain-sensor geometry. The images acquired were from the dry season to minimize cloud cover and reduce noise (

Table 1. Acquisition dates for each site and corresponding Landsat scenes.

Site	Landsat Scene	Acquisition Date
		2017	2018	2019
A	227_63	July 18	July 21	July 24
B	227_65	July 18	July 21	July 24
C	230_65	June 21	June 24	July 13

). Tier 1 Landsat images were used as they offer consistent georegistration within prescribed image-to-image tolerances of less than 12-meter radial root mean square error (RMSE) and are therefore appropriate for time-series analysis [78].

2.2. Deep Learning Models

This research used three different DL architectures available in the literature: U-Net [79], SharpMask [80], and ResUnet [81]. While the U-Net and SharpMask algorithms were not developed for classification with remote sensing data in mind, studies have found that they are not only suitable, but offer state-of-the-art results [39,82]. These algorithms share similarities, being based on architectures known as autoencoders with the addition of bridges or residual connections. Autoencoders downsample the feature maps generated through convolutional filters while incrementally increasing their number to learn low-level features compactly, and then upsample them back to the original input shape for inference. This process can be further enhanced using connections bridging the downsampling and upsampling steps (Figure 2) to propagate information. These connections help speed up training and reduce the degradation of data by combining both low-level detail and high-level contextual information. Low-level spatial detail is essential for change detection and land cover classifications, and that is the main reason behind the choice of this specific type of architecture for this study.

While similar in principle and structure, the chosen architectures differ in depth and complexity.

Table 2. Total number of layers and parameters in each deep learning (DL) architecture used in this study.

Architecture	Layers	Parameters
U-Net	69	1,933,866
SharpMask	114	221,386
ResUnet	93	2,068,554

shows a summary of the number of layers and the total number of parameters in each model after adapting them for this study. Some of the inner workings of each model are also different. For example, the U-Net and SharpMask algorithms downsample the feature maps through a pooling operation, whereas the ResUnet architecture downsamples by using a stride of two between convolutional filter windows. Another example is how the models use skip connections in different ways, where both U-Net and SharpMask use exclusively long connections (linking the downsampling and upsampling sides of the architecture) while ResUnet makes use of long and short connections (between convolutional blocks).

2.3. Data Structure

The Landsat dataset consisted only of bands 1 through 7, as they share the same spatial resolution and contain most of the spectral information. Our initial training data was a bi-temporal cube stacking the base image and next year’s image, constituting 14 bands. We maintained this data structure for the RF and MLP algorithms, where each pixel is an observation, and each band is a variable. The datasets had to be restructured for the DL algorithms due to the inner workings of the CNNs and due to hardware memory constraints. To build and train the models in this study, we used the Keras [83] python library, a high-level wrapper for the well-known Tensorflow library [84]. When working with three-dimensional image data, Keras accepts inputs in the form of a four-dimensional array with shape (samples, sample rows, sample columns, channels). To convert our images to the correct format, we extracted patches through 200 × 200 pixel windows with a 10-pixel overlap on each side (Figure 3). This process generated a total of 844 samples per site per time sequence, with a total of 3376 training samples and 1688 test samples.

2.4. Ground Truth

To create our ground truth masks, we used the Brazilian Institute of Space Research’s Project for Deforestation Mapping (INPE’s PRODES) data [7] for the years of 2018 and 2019 as a visual guide and then refined it by remapping the deforestation polygons on a smaller scale. PRODES data are commonly used for deforestation reports and studies have used it before when modeling and studying deforestation dynamics [85,86]. The changes were mapped using digitizing tools from the QGIS software [87] at 1:30,000 scale and subsequently transformed into binary raster files with 0 and 1 as absence–presence codes, respectively. In this process, we mapped changes exclusively to the natural forest, regardless of the land cover type in the following year (Figure 4).

2.5. Hyperparameters

The RF model only needed two hyperparameters set, the number of trees to build (ntree) and the number of variables randomly sampled as candidates at each split (mtry). These were set to 500 trees, and three variables, respectively. The structure of the MLP algorithm consisted of a simple 3-layer network containing an input layer, a hidden layer with 256 nodes, and an output layer. The DL algorithms and MLP shared the same hyperparameters for training. Focal loss [88] was used as the loss function as it excels in classification problems with an uneven number of observations in each class, as is the case of our object of study. For gradient descent optimization, we used the adaptive moment estimation (ADAM) algorithm [89] with incorporated Nesterov Momentum (NADAM) with a learning rate of 2e-3, ${\beta }_1$ of 0.9 and ${\beta }_2$ of 0.999. The number of epochs was set to 250 and the batch size to 16 to fit the training process into memory.

2.6. Modeling Approach

Given the context of our main methodological steps described in the previous sections, a top-down view of our modeling approach is described in Figure 5. In addition to the DL algorithms, two classical ML algorithms, random forest (RF) and a simple multilayer perceptron (MLP) architecture, were used as a reference point for the assessment of the DL models. Both models have been extensively researched for land cover classification, and change detection in remote sensing data with their performance is well documented [90,91].

2.7. Accuracy Assessment

The accuracy metrics were calculated using the test site data exclusively to avoid the possibility of biased results due to overfitting. The classification results were compared to the ground truth mask for the test site in order to calculate the accuracy measures. Given that deforestation related change is typically a rare phenomenon, the change–no-change ratio is highly imbalanced [92,93]. Therefore, change detection research usually shows a predominance of invariant areas, causing a bias in some accuracy metrics. For example, overall accuracy is relatively high on most change maps [94]. The Precision and Recall measures (Equations (1) and (2)) were used to offer more insight in the distribution of errors in the classifications, along with three other measures besides accuracy: F1 score (also known as Dice coefficient), Kappa index, and mean intersection over union (mIoU) measure (Equations (3)–(5), respectively). These measures are often used to evaluate DL and ML classifications and are better suited for classifications with imbalanced datasets than overall accuracy as they equally weight class distributions.

$$Precision=\frac{True Positives}{True Positives+False Positives}$$

(1)

$$Recall=\frac{True Positives}{True Positives+False Negatives}$$

(2)

$$F1=2\times \frac{Precision\times Recall}{Precision+Recall}$$

(3)

$$Kappa=\frac{p_o-p_e}{1-p_e}$$

(4)

where $p_o$ is the rate of agreement between the ground truth and the classification, and $p_e$ is the expected rate of agreement due to chance.

$$mIoU=\frac{Io{U}_1+Io{U}_2\cdots +Io{U}_n}{n}$$

(5)

where $IoU$ is the area of intersection divided by the area of union between the classification and ground truth for a class and $n$ is the total number of classes. Finally, we used McNemar’s test [95] to evaluate the statistical significance of differences between the classifications.

3. Results

Quantitatively, the DL models showed a clear advantage over RF and MLP (

Table 3. Performance measures for the model validation results for the 2017–2018 and 2018–2019 sequences. Best results in the column in bold text.

Model	2017–2018						2017–2019
	F1	Kappa	mIoU	Precision	Recall	Overall Accuracy	F1	Kappa	mIoU	Precision	Recall	Overall Accuracy
RF	0.8014	0.8003	0.8332	0.9414	0.6976	0.9979	0.8902	0.8892	0.9000	0.8877	0.8928	0.9979
MLP	0.8926	0.8920	0.9024	0.9282	0.8597	0.9987	0.9101	0.9093	0.9167	0.9314	0.8898	0.9983
Resunet	0.9432	0.9428	0.9459	0.9252	0.9619	0.9993	0.9465	0.9460	0.9487	0.9358	0.9574	0.9990
Unet	0.9112	0.9106	0.9179	0.9223	0.9003	0.9989	0.9339	0.9332	0.9373	0.9175	0.9508	0.9987
Sharpmask	0.9223	0.9218	0.9274	0.9173	0.9274	0.9990	0.9337	0.9331	0.9372	0.9218	0.9460	0.9987

). The ResUnet model had the best results with regard to every measure with the exception of Precision in the 2017–2018 time frame. The SharpMask and U-Net models showed similar but slightly inferior results. In comparison, the RF model showed the worst results in most measures, although the performance measures still indicated a good classification. It should be noted that the RF and MLP classifications exhibited a considerable amount of impulse noise (“salt-and-pepper” type), and a majority filter was applied to reduce the noise and improve the classification both visually and quantitatively. The DL models did not require any post-processing steps as they produced classifications with virtually no noise. All models showed very high overall accuracy, but, as explained previously, this measure should be carefully considered as the ratio between the change and no-change classes is highly imbalanced and is mostly explained by the larger, no-change class. McNemar’s test results indicate that despite the seemingly similar results, the model classifications were all significantly different from each other (

Table 4. McNemar’s test p-values between model classifications. Values bellow p = 0.05 indicate the differences between classifications are statistically significant.

	2017–2018					2018–2019
	MLP	ResUnet	RF	SharpMask	U-Net	MLP	ResUnet	RF	SharpMask	U-Net
MLP
ResUnet	<0.001					<0.001
RF	<0.001	<0.001				<0.001	<0.001
SharpMask	<0.001	<0.001	<0.001			<0.001	<0.001	<0.001
U-Net	<0.001	<0.001	<0.001	<0.001		<0.001	<0.001	<0.001	<0.001

).

RF’s higher precision in the 2017–2018 frame can be explained by the low number of false positives produced. Conversely, however, it produced a very high number of false negatives within the same frame (Figure 6). The ResUnet model had the lowest number of misclassified pixels in both time sequences. It also produced the least number of false negatives out of all the models. When looking at the number of false-positive cases, the DL algorithms did not show a large difference over the ML models. With regard to false-negatives, however, they showed a clear advantage. The reduction of false-negative classifications is a considerable advantage of the DL models over the classic ML algorithms, given that underestimating the extent of deforestation is a less desirable outcome than its overestimation.

The models detected roughly the same deforestation sites at the validation site across both time sequences (Figure 7 and Figure 8). However, the DL models provided more detailed classifications within smaller scales, particularly around feature edges. Moreover, all models were able to classify “easy” deforestation patches with less complex spectral mixtures (Figure 9), but the classification of the ML algorithms degraded as the spectral signatures within the patches increased in complexity (Figure 10). RF showed a higher tendency to produce false-negatives both visually and quantitatively.

The total deforested area was slightly higher than the ground truth in the SharpMask and ResUnet predictions in both time sequences (

Table 5. Total deforested area according to the ground truth and each model’s prediction.

Reference	Deforested Area (km2)		Difference from Ground Truth (%)
	2017–2018	2018–2019	2017–2018	2018–2019
Ground Truth	152.73	233.44	---	---
Random Forest	113.17	234.79	−25.90	+0.58
MultiLayer Perceptron	141.45	223.01	−7.39	-4.47
SharpMask	154.40	239.56	+1.10	+2.62
U-Net	149.10	241.90	−2.38	+3.62
ResUnet	158.78	238.84	+3.96	+2.31

). The opposite was true for the MLP prediction, which slightly underestimated the total area in both time spans. The RF model underestimated the total deforested area by a very large portion (almost 40 km² or a 26% decrease in area) in the 2017–2018 sequence due to a large number of false negative predictions, but despite this, it came closest to the ground truth area in the 2018–2019 sequence, although, that does not necessarily mean the predicted areas were the same as the ground truth.

Processing times varied from model to model, but the MLP and DL models offered faster training and prediction times than RF, mainly due to the fact that the Tensorflow framework uses the computer’s graphical processing unit (GPU) for parallel processing instead of the central processing unit (CPU), which is traditionally used for ML. Using an NVIDIA GTX 1070 GPU and a batch size of 16, the total training time ranged from approximately 40 minutes for the simpler MLP model (around 10 seconds per epoch) to almost three hours for the more complex ResUnet model (approximately 40 seconds per epoch). Given the size of the datasets, RF took approximately six hours to train using parallel processing with an Intel Core i5-4690k processor. The difference in processing times was particularly considerable when using the models to classify the images after training. The DL models and MLP classified the test scene within seconds, whereas RF took almost an hour to complete the task.

4. Discussion

The CNN architectures used in this study showed a clear advantage to the classic ML algorithms, both quantitatively and visually, regarding deforestation mapping. Similarly, a comparative study of methods developed by [96] for wetland mapping found that deep learning methods (completely convolutional networks and patch-based deep CNN) obtained better accuracy than RF and support vector machine. The authors found that CNN may produce inferior performance when the training sample size is small, but it tends to show substantially higher accuracy than conventional classifiers using a larger training sample size. We assume that the difference in performance between the DL and traditional ML methods stems from the former’s capability to understand both the spatial and spectral context, whereas the regular ML models inherently only see the spectral information.

Although the current methodologies to detect deforestation with DL architectures vary widely, studies are in agreement that they produce excellent classification results [97,98,99]. Other analogous studies that have investigated the use of DL for single class classifications seem to corroborate with this trend, although there is a large variation between the choice of targets and architectures [34,100,101,102]. While choice and development of architectures for certain targets is a relevant topic for future research, we have found that autoencoder networks with residual connections seem to be a good starting point for classifications in remote sensing imagery as they can take advantage of spatial and spectral information in a very efficient manner.

Despite their advantages, DL algorithms are still not as accessible or as easy to use as classic ML models. Besides needing specific hardware for training, they require a relatively large quantity of samples, and developing ground truth masks for specific targets can be challenging and time-consuming in large extents as both spatial and spectral context are strictly needed, whereas the traditional ML algorithms work with simpler sampling schemes and can produce reasonably good results with a much smaller sample size. Therefore, the process of building a model for broader use (i.e., country-wide monitoring) can be complicated. However, these models have another advantage in the fact that they can be incrementally trained, meaning they could be gradually provided with new samples to update the model weights and improve their classifications with time. With that said, the “black box” nature of these networks can make them undesirable for those who might wish to know and disclose their internal workings such as public and governmental entities. Despite that, through our findings, we believe that with enough development, DL algorithms can provide a viable automatic solution for mapping deforestation in the Amazon alongside projects such as INPE’s PRODES and TerraClass.

It should be noted that while the models showed good capability for generalization within our region of study, we cannot assert that they would achieve the same results in different areas where deforestation is a common occurrence. A broader reaching model would necessarily require samples from different regions to account for the possible spatial and spectral variability from one region to another. Further research should be carried out to study the applicability of the models for similar targets in different areas. In addition, while Landsat data are enough for annual deforestation mapping between dry seasons, more frequent monitoring is virtually impossible as clouds are present above the forest canopy during most of the year, and the ground reflectance cannot reach the satellite’s optical sensors. One solution would be the use of radar data to be able to cross the cloud cover. As such, we also recommend the investigation of the use of radar data and DL algorithms to detect deforestation within a shorter time-frame.

5. Conclusions

In this study, we proposed the use of existing DL architectures to detect yearly changes in the vegetation cover in the region of the Brazilian Amazon, successfully achieving our goal. Results show that these algorithms are a viable alternative to classical ML algorithms, with the improvement of all performance measures and clear advantages such as faster prediction times and lack of noise in the classifications. The SharpMask, U-Net, and ResUnet models showed similar results. However, ResUnet achieved the best values of accuracy, Kappa, F1, and mIoU, and the least amount of errors overall. Visually, the DL algorithms also produced classification masks with well-defined deforestation patches, while the ML models showed an evident loss of quality in harder to classify patches, with a tendency to produce false-negatives and impulse noise that needed to be filtered. One of the main shortcomings of CNNs seems to be the necessity of a 1:1 ground truth regarding the extent of the area of study as the spatial context is critical. In contrast, simpler ML models can be trained on a point-by-point basis (e.g., random sampling points within the extent). Developing a whole ground truth can be an extensive process. However, we have achieved very good results with a relatively small sample size with very little augmentation in the form of overlapping sample patches. The additional bands in remote sensing data may facilitate the detection of targets with less samples, but that supposition needs further research. Furthermore, considering the models were validated by being applied to an independent dataset, the performance measures show that they have very good potential for generalization. DL is still a growing technology, particularly in the remote sensing field as not even popular libraries such as Keras and Tensorflow have built-in tools for dealing with multi-band satellite imagery, but researchers are slowly adapting and developing better architectures specific for remote sensing data. The architectures used in this study performed well in our specific task, although they were developed for entirely different targets. Therefore, these algorithms do not necessarily need to be tailored for specific cases and can even work interchangeably between fields of research.