Change Detection of Amazonian Alluvial Gold Mining Using Deep Learning and Sentinel-2 Imagery

Abstract

Monitoring changes within the land surface and open water bodies is critical for natural resource management, conservation, and environmental policy. While the use of satellite imagery for these purposes is common, fine-scale change detection can be a technical challenge. Difficulties arise from variable atmospheric conditions and the problem of assigning pixels to individual objects. We examined the degree to which two machine learning approaches can better characterize change detection in the context of a current conservation challenge, artisanal small-scale gold mining (ASGM). We obtained Sentinel-2 imagery and consulted with domain experts to construct an open-source labeled land-cover change dataset. The focus of this dataset is the Madre de Dios (MDD) region in Peru, a hotspot of ASGM activity. We also generated datasets of active ASGM areas in other countries (Venezuela, Indonesia, and Myanmar) for out-of-sample testing. With these labeled data, we utilized a supervised (E-ReCNN) and semi-supervised (SVM-STV) approach to study binary and multi-class change within mining ponds in the MDD region. Additionally, we tested how the inclusion of multiple channels, histogram matching, and La*b* color metrics improved the performance of the models and reduced the influence of atmospheric effects. Empirical results show that the supervised E-ReCNN method on 6-Channel histogram-matched images generated the most accurate detection of change not only in the focal region (Kappa: 0.92 (± 0.04), Jaccard: 0.88 (± 0.07), F1: 0.88 (± 0.05)) but also in the out-of-sample prediction regions (Kappa: 0.90 (± 0.03), Jaccard: 0.84 (± 0.04), and F1: 0.77 (± 0.04)). While semi-supervised methods did not perform as accurately on 6- or 10-channel imagery, histogram matching and the inclusion of La*b* metrics generated accurate results with low memory and resource costs. These results show that E-ReCNN is capable of accurately detecting specific and object-oriented environmental changes related to ASGM. E-ReCNN is scalable to areas outside the focal area and is a method of change detection that can be extended to other forms of land-use modification.

1. Introduction

Artisanal and small-scale gold mining (ASGM) is an emerging threat to the conservation and preservation of tropical riverine systems across the planet [1,2]. This method of mining involves the removal of aboveground biomass and the processing of alluvial soil sediments for the retrieval of minute historical deposits of gold particles. ASGM typically involves operations at a much broader spatial scale than pit mining, as the concentration of gold particles is comparatively low in alluvial fans and historical river channels [3]. As a result, ASGM is generally associated with land cover/land use (LCLU) change that can encompass large areas, including the clearing of primary tropical rainforest.

The presence of this type of mining in small pockets of the Amazon Basin is not new. However, the expansion of ASGM as a driver of land-cover change throughout Amazonia and in other tropical ecosystems has increased remarkably over the past decade. For instance, in the Peruvian department of Madre de Dios, ASGM was responsible for the removal of over 120,000 ha of primary tropical forest from 1984 to 2017 [4]. ASGM has also taken hold outside the Amazon, including in Nigeria [5], Ghana [6], Laos [7], and Indonesia [8]. The intensification of ASGM has led to profound impacts on river biogeochemistry [1], human health [9], and conserved areas [4], making it a significant driver of land-use change in tropical landscapes and riverine systems. Water is essential for the mining process, and shallow tropical water tables quickly fill any excavation. As a result, entire landscapes that were once primary forests have been converted to a mixture of ponds and bare earth, creating novel hydroscapes and greatly changing restoration potential [10] (Figure 1).

As ASGM has intensified globally, monitoring efforts to detect mining activity have been of significant interest for conservation and governance purposes. Current efforts to monitor ASGM landscapes, including the presence of mining ponds and water bodies left over from sediment extraction, generally make use of satellite-based remotely sensed imagery (e.g., [4,11]). This work often relies on indices that compare reflectance band data from these sensors to categorize the land surface into broad groups, a technique that is also used for monitoring small water bodies [12,13,14]. However, these methods generally work on a pixel-basis, and do not keep track of temporal change across time series.

Recently, developments in deep learning have led to an increased capacity for monitoring LCLU change more discretely, allowing for segmentation and labeling of individual features or objects within digital imagery. Among these methods are the use of both convolutional neural networks (CNN) and recurrent neural networks (RNN). A convolutional neural network (CNN) is a multilayer neural network that is inspired by the model of the primate visual system [15] and is utilized for learning features [16] and classification problems [17,18]. Specifically, CNN relies on two-dimensional spatial contexts within imagery data to generate edges and identify features. As a result, CNN-based deep learning is widely used for feature extraction uses such as semantic segmentation [19], landslide detection [20], object detection [21,22], and change detection [23]. RNNs have the capacity to re-apply past weights to layers in the neural network. By remembering the spatial features over time, RNNs utilize temporal contexts and functionality with time-series data. RNNs have been used for monitoring and estimating land-cover change [24,25] and crop identification [26,27]. When these two types of neural networks are combined into a singular network (ReCNN; [28]), time-series multispectral data can be analyzed in a way that detects features as well as changes in conditions of these features over time.

While it may appear that deep learning only provides a more detailed estimate of land-cover change when compared to conventional techniques, these new methods may be transformative in guiding policy and mitigation measures. For instance, in the context of ASGM, general methods using spectral indices alone describe the area of primary tropical forest biomass that has been converted [4,29] as well as the presence of new mining ponds [10]. These mining ponds, or lagoons, are generally 3–4 m deep water bodies produced as sediment is excavated for processing. The excavated areas are filled with water via hoses and pumps to hasten the erosion and liquefaction of the soil. When local mining abates, sediment concentrations in the pond water column decrease while phytoplankton and algae increase in still water [1,30]. Understanding such changes provides insight into the effectiveness of mining and conservation policy across a landscape [30]. Ultimately, deep learning methods that provide time-series data on the reflectance of individual features on a landscape may provide great utility for land-use change science and analysis.

In this work, we show how deep learning can be used to more thoroughly evaluate object-oriented LCLU change via satellite imagery. To do so, we utilize ReCNN, a combined form of CNN and RNN, into a singular network [28] to detect and categorize the changes in mining ponds created by ASGM activities. This ReCNN is compared with a semi-supervised model, support vector machines with smoothed total variation, SVM-STV. Specifically, we examine the outcomes from these models, as well as a number of labeling methods, to understand the applicability of these techniques to land-cover changes associated with ASGM. We focus on mined areas in the Peruvian department of Madre de Dios, a global hotspot of ASGM activity. We then transfer the model to other international ASGM sites to showcase its utility. Primary contributions of the study are:

• The creation of an open-source labeled dataset of water body change pertaining to ASGM that can be used for training and consistent evaluation of algorithm performance;
• An evaluation of labeling methods and approaches for use with supervised model construction;
• An assessment of supervised and semi-supervised methods in the context of detecting and characterizing mining ponds from ASGM activity;
• A test of the best-performing models at a selection of out-of-sample international ASGM sites to examine universal model utility.

2. Materials and Methods

2.1. ASGM Ponds Dataset and Change Characterization

Our main study region is located within the Peruvian department of Madre de Dios (MDD), a globally significant hotspot of ASGM activity. We selected 16 distinct smaller region samples (~70 km² each) of interest within MDD to highlight locations that had experienced mining pond surface-area increases as well as notable deforestation (Figure 2;

Table A1. The regions in Figure 2 with their sizes in pixel and km² area and their latitude and longitude of left bottom and right top.

Region Number	Size in Pixels	Area in Km2	Left Bottom	Right Top
Region-1	667 × 654	43.37	13°01′51.9″ S 69°55′29.5″ W	12°58′15.9″ S 69°51′53.5″ W
Region-2	667 × 655	43.37	13°01′21.8″ S 69°58′37.3″ W	12°57′45.8″ S 69°55′01.3″ W
Region-3	556 × 546	30.12	13°00′27.3″ S 70°00′54.0″ W	12°57′27.3″ N 69°57′54.0″ W
Region-4	556 × 545	30.12	13°00′43.6″ S 70°03′19.6″ W	12°57′43.6″ S 70°00′19.6″ W
Region-5	1109 × 548	60.25	12°59′42.4″ S 70°02′36.0″ W	12°53′42.4″ S 69°59′36.0″ W
Region-6	667 × 655	43.38	12°59′22.4″ S 70°06′47.8″ W	12°55′46.4″ S 70°03′11.8″ W
Region-7	888 × 482	42.42	12°57′16.3″ S 70°04′57.0″ W	12°52′28.3″ S 70°02′18.6″ W
Region-8	556 × 545	30.13	12°53′31.7″ S 70°03′35.8″ W	12°50′31.7″ S 70°00′35.8″ W
Region-9	556 × 546	30.13	12°55′34.6″ S 70°01′14.7″ W	12°52′34.6″ S 69°58′14.7″ W
Region-10	555 × 438	24.1	12°53′19.3″ S 69°59′47.2″ W	12°50′19.3″ S 69°57′23.2″ W
Region-11	1109 × 657	86.72	12°51′48.2″ S 69°57′52.1″ W	12°45′48.2″ S 69°54′16.1″ W
Region-12	1333 × 659	86.12	13°05′46.4″ S 70°31′00.8″ W	12°58′34.4″ S 70°27′24.8″ W
Region-13	894 × 1309	115.66	13°02′14.2″ S 70°39′33.2″ W	12°57′26.2″ S 70°32′21.2″ W
Region-14	1337 × 1311	173.56	12°56′48.5″ S 70°38′10.1″ W	12°49′36.5″ S 70°30′58.1″ W
Region-15	1560 × 1748	270.11	12°49′54.8″ S 70°35′44.9″ W	12°41′30.8″ S 70°26′08.9″ W
Region-16	668 × 655	43.38	12°57′55.2″ S 70°16′12.7″ W	12°54′19.2″ S 70°12′36.7″ W

in Appendix A). These regions were selected for two main reasons: firstly, the regions spanned a gradient of significant mining intensity, techniques, and policy enforcement over the last 15 years. Secondly, the regions were shaped so as to maximize the number of pixels undergoing change between bi-temporal images, thereby providing a more thorough test of the models. A total selection of sixteen regions allowed for a fully representative sample of sites with these two considerations in mind.

We acquired Sentinel-2 top-of-atmosphere reflectance data for these 16 regions via the Google Earth Engine platform. The Sentinel-2 satellite constellation [31] was developed for monitoring variability in LCLU conditions at frequent revisit times (5 days at the equator) and consists of 13 multispectral channels ranging from ultra-blue to shortwave infrared with pixel resolutions between 10 and 60 m GSD. Sentinel-2 data is widely used to assess LCLU change in the context of surface water [32,33]. We selected data from two different years (2019 and 2021) with very low cloud coverage to showcase periods in which significant land-use change had occurred. We removed the influence of atmospheric effects by histogram matching of corresponding images of the same region and used Sentinel-2 metadata about cloud cover to remove any residual clouds on the images.

LCLU changes due to alluvial gold mining occur across different parts of the world, so it is crucial that change-detection algorithms generalize from one geographical region to another. Thus, we included an out-of-sample testing dataset containing instances of similar alluvial gold mining in Indonesia, Myanmar, and Venezuela that are of similar intensity to that in MDD (

Table A2. Configuration of optimal model parameters for supervised (E-ReCNN) and semi-supervised (SVM-STV) models.

E-ReCNN (TensorFlow Framework)	Nesterov Adam − ${\mathsf{\beta }}_1=0.9, {\mathsf{\beta }}_2=0.999, \in =1\times {10}^{-7}$ Learning rate − 1 × 10−3 Glorot uniform initializer − uniform distribution
SVM-STV	The two parameters of ν-SVC: Nu: 0–0.2, Gamma: $1/\left(\mathrm{n}+2\right)-1/\left(\mathrm{n}-2\right)$, where n is the number of bands. The denoising parameters: Alpha1: 0–1, Alpha2: [0, 0.5, 1, 2], Mu: 5. The above five parameters are tuned on each of the 16 training regions.

in Appendix A).

For the purpose of generating meaningful labeled data, we defined three different pond states in relation to the recency of mining:

• Active state: where mining was ongoing at the time of image collection;
• Transition state: where mining was recent but not ongoing;
• Inactive state: where mining had ceased longer than 6 months prior to imaging.

Change in mining ponds was then defined as follows:

• Decrease: change from active to inactive, active to transition, or transition to inactive;
• Increase: change from inactive to active, inactive to transition, or transition to active;
• Water Existence/Absence: change from water to no-water or no-water to water;
• No Change: no state changes between time periods took place.

These basic categories can provide useful information regarding ongoing mining activities, such as intensification, cessation, and the effect of governance [1,4].

A subgroup of individuals in the research group with expertise in the characterization of alluvial gold mining manually segmented and labeled each individual pond in the dataset. Ponds were segmented by manually tracing their edges, and pond status was determined from side-by-side visual observation of the RGB and shortwave-infrared (SW) with GB composite images for each region, see Figure 3a–c. These band combinations were chosen specifically to help discriminate between active sites, in which sediment highly reflects in the red band, and inactive sites, in which photosynthetic material is present and influences the shortwave infrared reflectance (Figure 3a). For consistency, we calculated the color index C_idx = (green − red)/(green + red) distributions of pond pixels and chose thresholds of 0 and 0.15 to select ponds in a transition state.

2.2. Modeling Approaches

We considered two main approaches for modeling and quantifying change in residual ponds: a supervised deep learning method based on ReCNN [28] and a semi-supervised method involving a support vector machine and smoothed total variation regularizer [34].

2.2.1. Supervised Deep Learning Approach

We extended the ReCNN model of Mou et al. [28], originally designed to detect LCLU-type changes in urban areas using satellite imagery, for the detection of large and subtle changes relative to water bodies. First, we augmented the ReCNN model to include a second LSTM plus dropout layer between the original two LSTM layers (Figure 4) to capture subtle pond-state changes. Second, we modified the input layer to receive two temporal images separately instead of two concatenated images as is done in other studies (e.g., [35,36]). We refer to this implementation as extended ReCNN (E-ReCNN) throughout the remainder of this paper.

2.2.2. Semi-Supervised Learning Approach

Unsupervised and semi-supervised learning methods are widely applied in remote sensing applications involving small datasets and limited access to high-performance computing equipment. SVMs are powerful semi-supervised approaches that have been used to detect LCLU change utilizing spectral information of each pixel separately [37,38,39]. A recent SVM-STV approach by Chan et al. [34] also utilized spatial information contained across image regions. We modified this approach to include a lifting option for multispectral images. Lifting is a preprocessing step that can help aid with the segmentation of RGB images through the use of color spaces and additional features [40,41,42,43]. We combined the RGB and La*b* color spaces in the images as features and then performed segmentation to reduce the effects of high correlation in one color space [43].

Figure 5 illustrates the two main steps in SVM-STV. In the first step, we formed the v feature vectors from the difference in the bi-temporal images. Then we used a pixel-wise ν-support vector classifier (ν-SVC) with a radial basis function kernel to find a hyperplane maximizing the margins between each pair of classes, using a one-against-one strategy, and to assign each pixel a vector of probabilities belonging to each class [44,45]. The difference in La*b* color space between bi-temporal images is included in feature v if the lifting option is enabled. In the second step, a smoothed total variation (STV) regularizer smooths the probability vector and consequently the classification map.

2.2.3. Statistical Approaches, Training, and Operation

In order to understand the performance of the proposed approaches and the impact of spectral information and image preprocessing, we designed a number of test-train experiments across the 16 numbered regions within Madre de Dios (Figure 2). Additionally, to examine the generalizability of the approaches to ASGM sites in other locations in the tropics, we constructed a set of out-of-sample testing regions in Venezuela, Indonesia, and Myanmar.

Since the supervised and semi-supervised modeling approaches used different quantities and distributions of labels, we used slightly different training approaches for each model. For the E-ReCNN model, we used a leave-one-region-out cross-validation approach. This method is often used for classification in medical imaging (e.g., leave-one-patient-out) [18,46] to account for class imbalance and region information. Specifically, we left one of the sixteen MDD regions out for testing and used the remaining fifteen regions for training and validation. We iterated this process for each individual region, allowing each region to serve as a testing region once. For each iteration, one region’s image was selected as a test, and the remaining fifteen regions’ images were used for training (70% of all patches) and validation (30% of all patches). Because we were examining the influence of the number of channels included in the model, this process was repeated for each multispectral image in the 3, 6, and 10-Channel image sets. Nesterov Adam [47], an improved Adam optimizer [48], was used to accelerate adaptive moment estimation and the convergence of both the Adam and stochastic gradient descent (SGD). The parameters of the model producing the best average predictive results are listed in Appendix A Table A2. All testing and training using E-ReCNN were performed on the Wake Forest University DEAC HPC Cluster [49] (Appendix A 

Table A3. Calculated costs of model training.

E-ReCNN (Wake Forest University DEAC HPC cluster)	Number of Epochs: 75	Total Loss: 0.0208	Time per epoch: 548 s
SVM-STV (Local machine)	Number of Trials: 10	Training Error: Controlled by Nu	Time per trial: 3 s to 94 s

).

To train the SVM-STV semi-supervised model, we first trained the ν-SVC and then performed denoising on the probability map that ν-SVC produces. In the context of semi-supervised learning, less than 1% of labels were randomly selected for training, whereas over 99% of the labels were unknown. Thus, in the training process of each region, instead of including all the pixels into the ν-SVC, we only incorporated a subset of randomly chosen labeled points from each region. Therefore, for each of the 16 MDD regions, we first specified the number of labeled pixels per class ($N_k$) for training the model. Next, we used the preprocessed randomly selected $N_k\ast K$ labeled pixels to train the ν-SVC with five-fold cross-validation, where $K$ was the number of classes. The trained ν-SVC was then applied to predict the probability tensor. Finally, the denoising parameters were tuned based on the probability maps of each region. The training procedure for SVM-STV was computationally feasible and had a rapid training time as it only used a small portion of randomly selected labeled data (0.004–0.2%). All testing and training of the SVM-STV method were conducted in the same environment: Intel^® Core™ i7-10875H CPU @ 2.30GHz, 8 cores, 64 GB RAM, Windows 64-bit system, and MATLAB R2021a.

To examine the influence of spectral information on method performance, we constructed three sets of spectral images with varying numbers of spectral bands chosen specifically for application to water and LCLU change:

• A three-band set of images containing red, green, and blue bands (RGB);
• A six-band set of images containing red, green, blue, NIR, SWIR1, and SWIR2;
• A 10-band set of images containing red, green, blue, NIR, SWIR-1, SWIR-2, ultra-blue, and bands 5, 6, and 7, which correspond to the vegetation red edge.

To evaluate the overall performance of the methods, we used three metrics: Cohen’s Kappa coefficient [50], the Jaccard index [51], and the F-1 score [52,53]. Cohen’s Kappa coefficient provides a measure of consistency and reliability in classification tasks. The Jaccard index, also referred to as the intersection over union, measures the overlap between labels and predictions, emphasizing true positives over true negatives. The F1 measure is the harmonic mean between precision and recall and does not take true negatives into account. As a result, the changed-area accuracy is not affected by the ‘no change’ area accuracy, which is high because of the number of pixels. We did not present accuracy scores, as these can be misleadingly high (as seen in Appendix A 

Table A4. Accuracies of regions and their areas for the E-ReCNN model for histogram-matched 6-Channel images. Region 4 and Region 12 were analyzed in Discussion section and given as an example of the difference kind of ponds.

Area	Number of Regions	Accuracy	Kappa Coef.	Jaccard Coef.	F1-Score	No Change	Decrease	Increase	Water Existence
La Pampa	1	0.989	0.932	0.889	0.941	0.995	0.938	0.865	0.965
	2	0.979	0.880	0.814	0.893	0.990	0.911	0.726	0.945
	3	0.978	0.891	0.826	0.918	0.988	0.923	0.813	0.948
	4	0.986	0.945	0.916	0.954	0.993	0.969	0.896	0.956
	5	0.978	0.913	0.863	0.923	0.988	0.924	0.819	0.964
	6	0.979	0.879	0.807	0.868	0.989	0.852	0.660	0.969
	7	0.987	0.951	0.921	0.887	0.993	0.884	0.685	0.985
	8	0.996	0.984	0.974	0.904	0.998	0.774	0.854	0.991
	9	0.986	0.924	0.875	0.914	0.993	0.869	0.847	0.950
	10	0.995	0.983	0.973	0.930	0.997	0.805	0.927	0.990
	11	0.996	0.977	0.961	0.799	0.998	0.460	0.752	0.988
Huepetuhe	12	0.987	0.864	0.776	0.780	0.994	0.450	0.766	0.910
	13	0.994	0.951	0.917	0.848	0.997	0.659	0.754	0.983
Delta	14	0.989	0.924	0.884	0.855	0.995	0.686	0.772	0.970
	15	0.985	0.930	0.889	0.839	0.992	0.636	0.740	0.987
Inambari Tributary	16	0.987	0.832	0.740	0.795	0.994	0.547	0.786	0.854

in the Accuracy column) and not good indicators relative to other metrics due to severe class imbalance.

3. Results

In this section, we present the results of multi-class change detection on AGM ponds in multispectral images that were obtained from focal (MDD) and out-of-sample prediction regions. Results from change detection analyses using binary classes (change/no change) can be found in Appendix A,

Table A6. Binary Change detection results for the E-ReCNN method. Different channels results represented with background and the highest result with the minimum channel was 6 Channel results shown with bold text.

			Kappa Coef.	Jaccard Coef.	F1-Score
Original Images	3 Channel	Average	0.63	0.52	0.81
		Std Dev	0.13	0.12	0.07
	6 Channel	Average	0.71	0.63	0.84
		Std Dev	0.23	0.21	0.16
	10 Channel	Average	0.70	0.62	0.83
		Std Dev	0.25	0.23	0.17
Histogram-Matched Images	3 Channel	Average	0.53	0.42	0.76
		Std Dev	0.11	0.11	0.06
	6 Channel	Average	0.92	0.87	0.96
		Std Dev	0.04	0.07	0.02
	10 Channel	Average	0.92	0.87	0.96
		Std Dev	0.04	0.07	0.02
Histogram-Matched + Lab Lifted Images	3 Channel	Average	0.45	0.37	0.71
		Std Dev	0.16	0.14	0.10
	6 Channel	Average	0.92	0.87	0.96
		Std Dev	0.04	0.07	0.02
	10 Channel	Average	0.91	0.86	0.96
		Std Dev	0.04	0.07	0.02

and

Table A7. Binary Change detection results for the SVM-STV method. Different channels results represented with background.

			Kappa Coef.	Jaccard Coef.	F1-Score
Original Images	3 Channel	Average	0.44	0.60	0.72
		Std Dev	0.10	0.06	0.06
	6 Channel	Average	0.62	0.71	0.81
		Std Dev	0.08	0.05	0.04
	10 Channel	Average	0.67	0.74	0.84
		Std Dev	0.07	0.04	0.03
Histogram-Matched Images	3 Channel	Average	0.52	0.65	0.76
		Std Dev	0.09	0.06	0.05
	6 Channel	Average	0.64	0.72	0.82
		Std Dev	0.07	0.04	0.03
	10 Channel	Average	0.62	0.71	0.81
		Std Dev	0.09	0.05	0.05
Histogram-Matched + Lab-Lifted Images	3 Channel	Average	0.61	0.71	0.80
		Std Dev	0.07	0.04	0.04
	6 Channel	Average	0.64	0.72	0.82
		Std Dev	0.08	0.04	0.04
	10 Channel	Average	0.62	0.71	0.81
		Std Dev	0.08	0.04	0.04

.

The overall performance of the two approaches, across testing regions and using all testing sets with respect to the number of channels, ranged from 0.19 (± 0.06) to 0.92 (± 0.04). The inclusion of increased spectral information (channels) generally increased performance. Among all experimental settings, the greatest average result of multi-class change classification by E-ReCNN was a Cohen’s Kappa of 0.92 (± 0.04), a Jaccard value of 0.88 (± 0.07), and an F1 of 0.88 (± 0.05) for histogram-matched 6-Channel set images (Accuracy value 0.99 ((± 0.04), as seen in Appendix A Table A4). In contrast, the greatest average result of a multi-class change by SVM-STV was a Cohen’s Kappa value of 0.63 (± 0.07), a Jaccard value of 0.56 (± 0.06), and an F1 of 0.67 (± 0.06) for original (not preprocessed) 10-Channel set images. These results were achieved on images from the MDD region training dataset. The MDD-trained E-ReCNN approach applied to out-of-sample regions (Figure 6, right) performed similarly to the results obtained in the focal MDD region (Figure 6, left), demonstrating the generalization of E-ReCNN across different spatial regions. The SVM-STV approach performed less well on out-of-sample prediction, decreasing by 25% on average.

Overall, the E-ReCNN model performance using the 6-channel histogram-matched image sets from the 16 MDD regions resulted in outcomes with high levels of precision, recall, and F1-score (Figure 7, left). Model F1-scores for ‘no change’ and ‘water existence’ classes were 0.99 and 0.96, respectively. F1-scores for ‘increase’ and ‘decrease’ classes were slightly lower than ‘no change’ and ‘water existence’ classes, although the total quantity of labeled pixels for those two classes was notably lower. This pattern of F1-scores across classes was also seen in the out-of-sample regions (Figure 7, right). The total number of classified pixels in these regions was significantly lower, and F1 values for the ‘decrease’ and ‘increase’ classes were 0.56 and 0.57, respectively. The performance metrics not biased by smaller sample sizes, including Cohen’s Kappa and Jaccard coefficients, were higher than 0.9 for both the MDD regions as well as international out-of-sample regions. In contrast, the SVM-STV model results for ‘water existence’, ‘increase’, and ‘decrease’ classes were less accurate than the E-ReCNN model results, as shown in Figure 8 for the MDD regions (left) and out-of-sample regions (right). F1-scores for the MDD region ‘increase’ and ‘decrease’ classes were lower using this semi-supervised method than the out-of-sample regions, as modeled by E-ReCNN. Model results for the out-of-sample regions using SVM-STV were very low with respect to F1-scores, below 0.15 for the ‘increase’ and ‘decrease’ classes.

Applying the E-ReCNN and SVM-STV models on image sets with a variety of spectral channels provided inference regarding how each channel of Sentinel-2 influenced model behavior. The 3-channel RGB image resulted in roughly equivalent F-1 scores for both the E-ReCNN model and the SVM-STV model across the MDD regions (Figure 9). While the addition of near-infrared and shortwave infrared channels (1 and 2), which are often used to define water surfaces with the help of a water index (6 channel image), improved F-1 scores for both models, further including red edge channels (10 channel image), resulted in no additional improvement. Notably, E-ReCNN results appeared to be more accurate than SVM-STV for both the 6-channel and 10-channel image sets.

4. Discussion

In the context of land-use change, particularly change associated with ASGM, understanding how features across a landscape change in size and reflectance can provide critically important information for conservation and environmental policy enforcement. We show that the extension of an existing ReCNN detects multi-temporal change across landscape features when compared to an existing semi-supervised model (SVM-STV). E-ReCNN outperformed SVM-STV and unsupervised methods considerably for both the focal region in Madre de Dios as well as out-of-sample test regions with respect to F1, precision, and recall. Notably, E-ReCNN generated greater F1, precision, and recall values for the detection of water occurrence and the multi-temporal change in spectral response for each pond feature. Estimates of precision and recall for pond sediment decreased (82.8% and 86.1%) and increased (70.6% and 87.3%) within the MDD, showing that this method is capable of generating multi-temporal feature-based change maps. These results provide evidence that this method has wide applicability to the field of environmental change detection and monitoring.

One ongoing challenge in the use of satellite data for change detection related to how atmospheric conditions can cause complications when attempting to document fine-scale feature-oriented change. Although the major remote-sensing platforms, such as Landsat, Sentinel, and MODIS, are routinely processed and corrected via well-established and formalized techniques [54,55,56,57], persistent variability in surface reflectance from image to image requires careful consideration when monitoring temporal trends. We tested a number of data preprocessing approaches to understand how these challenges could be addressed and to understand how steps can be taken to improve machine learning model results. We found that histogram matching, which has primarily been used in remote sensing to denoise atmospheric effects on image mosaics [58,59] and recently in change detection [60,61,62], improved outcomes for the supervised model, E-ReCNN. In contrast, with the Lab color space variables in the semi-supervised model, SVM-STV produced the most accurate results. Some remote-sensing studies of surface water successfully identified patterns and trends without using histogram-matching (e.g., [63,64,65]). However, we note that these studies focus on large-scale changes in deep surface water extent/presence, wherein atmospheric noise plays less of a factor. We find that these preprocessing steps are necessary to achieve optimal results where we attempt to identify more subtle changes in water reflectance. The preprocessing steps should be considered in land-use change detection workflows, particularly if top-of-the-atmosphere products are utilized.

In addition to preprocessing methods, decisions regarding the inclusion of specific channels of remotely sensed data into models for analyzing LCLU change dynamics are important to ensure accurate outcomes. Critical tradeoffs between sensor spatial resolution, temporal resolution, and the availability of spectral channels can constrain the scope of land-cover change analysis. In the context of ASGM mining pond detection and classification, where patterns across years and seasons are evident, newly established commercially available satellite imagery (PlanetScope, DigitalGlobe) provide the temporal and spatial resolution necessary to detect fine-scale changes; however, these products generally are only available in a narrow set of channels. We found that the supervised E-ReCNN model generated the best outcomes in the 6-Channel and 10-Channel datasets after histogram matching, with significantly lower F1-scores in the 3-Channel dataset. When we applied lifting using L*a*b color space variables, accuracies were either unchanged (for the 10-channel dataset) or slightly decreased (for the 3- and 6-channel datasets). Consequently, we conclude that the selection of RGB images for this type of change detection may result in inferior outcomes compared to datasets with a greater number of channels in the infrared and red-edge spectrum. Commercial satellite data that lack these channels may therefore be limited in detecting important changes in aquatic systems, at least in comparison to other options.

Our modeling results showed a notable difference in accuracy between supervised and semi-supervised methods. Although the novel unsupervised learning methods presented in the literature show a great deal of potential for change detection [66,67,68], when we utilized one such unsupervised learning method [69], model performance results were substantially weaker than those provided by E-ReCNN and SVM-STV. Thus, we did not include detailed results regarding using unsupervised learning techniques for this problem. Our semi-supervised method, SVM-STV, in general, fits the data effectively by making use of a small fraction of labels, especially when only RGB data is provided. The results indicate that if a small, labeled set of a mining region in MDD is retrieved, the SVM-STV method can be trained on a desktop computer in a matter of minutes and produce reasonable results for both binary and multi-class classification. In practice, users can decide the number of expert-generated labels to acquire based on their needs, with the caveat that a fully supervised model may be more accurate and precise. In addition, if RGB images are necessary for detecting rapid change at localized scales, lifting using the La*b* color space generates enhanced results compared to data that has not been preprocessed.

Supervised model performance varied across MDD training regions (Appendix A Table A4) with respect to temporal change but was consistent across regions for detecting change/no change. Change detection F1-scores were higher in regions using water cannons relative to regions using earth-moving equipment. For example, Region 4 within La Pampa is characterized by ovular ponds with distinct edges surrounded by bare ground (Figure 10, top). This region has been heavily mined using suction pumps to displace water into mining ponds and small sluices to separate fine sediment from larger stones and pebbles. Comparatively, Region 12 in Huepetuhe (Figure 10, bottom) features the signature of the use of bulldozers and excavators to move sediment for processing; consequently, this region lacks distinct ponds with clear edges as in Region 4. We suspect that the lack of defined edges of water bodies provided an additional challenge for convolutional filters within E-ReCNN, leading to a decrease in the F1-score in Region 12. The results indicate that regions where mechanized mining is more prevalent are modeled with lower values for detecting increases and decreases in pond reflectance. Consequently, monitoring and modeling directional pond change may be more difficult in areas with differing mining typologies. However, outcomes for detecting change/no-change and water existence were excellent for both methods (Appendix A Table A6 and Table A7).

Whereas model outcomes were generally accurate across MDD regions, with slight differences between areas with different mining types, model results in the out-of-sample international regions were slightly less accurate with respect to multi-class change detection. However, the E-ReCNN out-of-sample results were still within 10% of the focal region results. This method thus retains significant performance in detecting change/no change and water occurrence in ASGM sites in different contexts and on different continents. With respect to pond increases and decreases in turbidity, both supervised and semi-supervised models generated significantly lower recall and precision for international sites compared to the MDD region results. Semi-supervised results using SVM-STV were extremely inaccurate (Figure 8, right), indicating that using this method is not advisable for accurate change detection. Supervised model results were less accurate for these out-of-sample regions, but still detected binary classes of change quite accurately overall. The construction of regional label sets may improve performance for detailed questions regarding pond status, but for general detection of AGM-associated mining ponds, the supervised model appears suitable for inference worldwide. More thorough investigations at known mined sites across the tropics would provide greater detail regarding the variability of model performance in new regions.

5. Conclusions and Future Work

In this paper, we describe the creation of a unique ASGM Residual Ponds dataset as well as a new supervised method (E-ReCNN) for detecting fine-scale changes in the environment using satellite imagery. We show how this method compares favorably to existing semi-supervised (SVM-STV) methods. We applied different preprocessing operations on three image sets with different quantities of multispectral bands to analyze their influence on the models’ results. For Sentinel-2 imagery, using a 6-band image set generated model performance higher than other band combinations, even those that included more spectral information. Pre-processing was essential to model performance, even on well-curated Sentinel-2 data, increasing model F1-scores from roughly 0.71 to 0.88 for 6-band images. For fine-scale change detection, we conclude that these images need noise reduction and calibration, such as histogram matching for E-ReCNN and the addition of La*b* color space to the SVM-STV model. Given this finding, practitioners using other change-detection methods on available satellite imagery, particularly with respect to water detection, may benefit from revisiting their results and investigating whether inaccuracies were due to preprocessing impacts.

Practitioners wishing to use the methods presented in this manuscript should consider the practical and computational demands of both change detection models. We found that classification performance is inverse to the computation demands for the two methods. Since the SVM-STV model can be trained on local machines, it is an efficient solution under the conditions of limited channels and resources. In contrast, because the E-ReCNN model consists of CNN and LSTM subnetworks, the R-ReCNN model requires lengthy training times on GPUs (Appendix A Table A3). However, it is worth noting that the E-ReCNN model, once trained a single time, appears to be capable of extension to out-of-sample regions with minimal loss in performance, and therefore once this process is completed, this method can be applied globally.

Future work may allow for an improvement of the SVM-STV model, particularly since the training size of labeled pixels used in this test was small and likely contained outliers and noisy pixels that could affect the quality of the model. Although histogram matching reduces radiometric differences in bitemporal images, the disparities among training regions can be significant and influential. Instead of randomly selecting training pixels for a generalized model, kernel density estimation may be used as an indicator that gives information about the “commonness” of each pixel [69,70]. This allows for the exclusion of outliers by only selecting pixels at high densities, generating more consistent test results. Furthermore, the SVM-STV model may be improved by including an active learning scheme, which takes into account the practical condition that there is a restricted budget for label collection. The diffusion geometry of the data can be used to push the approach even further by reducing the number of labels needed but producing greater performance [69,70,71].

Follow-up work on E-ReCNN may allow for the application of this model to other landscapes and environmental topics. While we investigated bi-temporal imagery sets in this analysis, the performance of E-ReCNN across a multitemporal image set may offer information regarding model transferability for decadal estimates of change across a landscape. Furthermore, testing E-ReCNN for use with other environmental features for the detection of change, such as fields, roads, and vegetation patches, may allow for a broad expansion of this supervised method to help monitor environmental change in other contexts and locations.