Evaluating the Accuracy of Land-Use Change Models for Predicting Vegetation Loss Across Brazilian Biomes

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The research addresses a gap in the literature by comparing the accuracy of baseline and machine learning-based models in predicting land use change, specifically natural vegetation suppression, in three Brazilian biomes. This article is well-organized and thoughtfully written, with clear arguments and supporting evidence. However, there are several areas that require further clarification and improvement.

1. Abstract: Please provide more specific information about the four baseline models and the four machine learning-based models selected.

2. Section 1: Please list common baseline models and the machine learning-based models.

3. Section 2.1: Please indicate the geographical location of the three study areas, which can be expressed in terms of latitude and longitude.

4. Section 2.1: The rationale behind choosing the grid cells with the largest absolute reduction in natural vegetation area should be elaborated. How do these areas represent broader trends in land use change?

5. Section 2.3: The reclassification process of land use maps should be described in greater detail. What were the original classes, and how were they grouped into natural vegetation and other uses?

6. Section 3: The comparisons between baseline and machine learning-based models are insightful, but why baseline models outperform machine learning models in certain cases needs further elaboration.

7. Section 5: While the limitation regarding ROC curves and overall accuracy is mentioned, it would be helpful to elaborate on why these methods are misleading.

8. Section 5: It is suggested that the authors add a paragraph discussing potential areas for future research based on the findings. This could include suggestions for improving model accuracy or incorporating additional factors into the models.

Author Response

The research addresses a gap in the literature by comparing the accuracy of baseline and machine learning-based models in predicting land use change, specifically natural vegetation suppression, in three Brazilian biomes. This article is well-organized and thoughtfully written, with clear arguments and supporting evidence. However, there are several areas that require further clarification and improvement.

Answer: Thank you very much for your thorough review and valuable suggestions. Below, each comment is addressed in detail.

 

1. Abstract: Please provide more specific information about the four baseline models and the four machine learning-based models selected.

Answer: Accepted.

 

2. Section 1: Please list common baseline models and the machine learning-based models.

Answer: Accepted.

 

3. Section 2.1: Please indicate the geographical location of the three study areas, which can be expressed in terms of latitude and longitude.

Answer: Accepted.

 

4. Section 2.1: The rationale behind choosing the grid cells with the largest absolute reduction in natural vegetation area should be elaborated. How do these areas represent broader trends in land use change?

Answer: This note has been added to the text: 

“As land use changes are typically rare occurrences when compared to persistence, the selection of grids with high rates of change aimed to ensure that all prediction periods could have change for comparison with the reference data. Furthermore, the selection of areas in three different biomes sought to represent different environmental characteristics and pressures that may affect change.”

 

5. Section 2.3: The reclassification process of land use maps should be described in greater detail. What were the original classes, and how were they grouped into natural vegetation and other uses?

Answer: The reclassification process is detailed in the supplementary materials. This warning has been added to the text: 

“Details of the reclassification process can be found in the supplementary materials.”

 

6. Section 3: The comparisons between baseline and machine learning-based models are insightful, but why baseline models outperform machine learning models in certain cases needs further elaboration.

Answer: This paragraph has been added to the text: 

“Possible reasons for the similar performance can be highlighted: (1) The amount of change during the training period was significantly different from the extrapolation period. Therefore, a large amount of change was likely allocated to areas not considered susceptible by the machine learning-based models. (2) For the Amazon and Cerrado study areas, a significant shift in the spatial patterns of change was observed between the training and extrapolation periods. Consequently, a significant portion of the change was allocated to areas that were not predicted to be susceptible according to the machine learning-based models. (3) It is known that change processes are influenced by their surroundings [4]. The vegetation suppression investigated was likely related to proximity to anthropogenic uses or to past suppressions. And (4) Predicting natural vegetation suppression is challenging. A variety of factors, including political changes, economic conditions, environmental regulations, personal motivations, mineral discoveries and wars, can influence the quantity and location of suppressions. Given the complexity of the problem, it is expected that all models will produce a significant number of false alarms and omissions. This is supported by the observation that the random model performed on par with the other’s in all scenarios.”

 

7. Section 5: While the limitation regarding ROC curves and overall accuracy is mentioned, it would be helpful to elaborate on why these methods are misleading.

Answer: This is presented in item 4. The sentence in red was added to the text. Regarding the ROC curve, it is argued that:

“The area under the ROC curve (AUC) is used in many studies as the sole approach for evaluating the accuracy of probability surfaces. Examples of this can be found in PARK et al. (2010) [24], LIN et al. (2011) [8], LIAO; WEI (2014) [25], KUCSICSA et al. (2019) [27], VOIGHT et al. (2019) [27], and COLMAN et al. (2024) [28]. This practice is problematic because the ROC curve is generally used to evaluate the susceptibility surface relative to the change data observed during the training period. Under these conditions, the evaluation only serves to indicate the fit of the training, not providing information about the ability of the susceptibility surface to differentiate changes and no changes in the prediction periods. A common bad practice is to provide only the AUC value without presenting the shape of the curve. This practice can be misleading because different curve shapes can have the same AUC value. Pontius and Parmentier (2014) presented information on the shape of the curve: “The lower left indicates the association between the high ranking index values and the Boolean feature, while the upper right indicates the association between the low ranking index values and the Boolean feature”. Therefore, analyzing the shape of the curve allows for a better understanding of the model's performance. For land use change prediction models investigating rare changes, the lower left corner should be analyzed in particular.

Additionally, this approach does not provide any information about the spatial accuracy of change allocation. The results presented in this work show that the highest AUC value does not always coincide with the best land use change prediction map. In the study area of the Cerrado biome, for example, the Random Forest model obtained the highest AUC values for all periods. However, when evaluating the land use change prediction map by figure of merit, the ANN - TerrSet model showed the best performance. This demonstrates the importance of evaluating both the susceptibility surface and the prediction map.”

 

Moreover, concerning global accuracy, it is shown that:

“Overall accuracy is a metric used to estimate the percentage of pixels correctly allocated by the prediction map. In many studies, this is the only approach used to evaluate the land use prediction map. Examples include the work of PARK et al. (2010) [24], LIN et al. (2011) [8], BALLESTORES JUNIOR; QIU. (2012) [29], CUSHMAN et al. (2017) [30], KUCSICSA et al. (2019) [26]. This method tends to be potentially misleading because it often considers the correct rejection component and excluded areas as model successes. However, the primary goal of developing a land use change model is to predict future changes. Therefore, a rigorous evaluation method should analyze the hits, misses, and false alarms components. This issue can be illustrated by the land use change prediction map derived from the Euclidean distance model for anthropogenic uses in the Amazon biome study area in 2005. The overall accuracy of the map is 85.6%, of which 96.6% is composed of correct rejections and excluded areas. When analyzing the hits, we observe that it is approximately seven times smaller than the sum of the misses and false alarms. Thus, analyzing only the global accuracy gives a misleading impression of the prediction's ability to represent future land use changes.”

 

8. Section 5: It is suggested that the authors add a paragraph discussing potential areas for future research based on the findings. This could include suggestions for improving model accuracy or incorporating additional factors into the models.

Answer: This note has been added to the text: 

“To enhance the accuracy of future models, the following recommendations could be considered: (1) Regarding the selection of predictive variables, the relevance could be tested and quantity increased. (2) The allocation of changes could be improved by considering exogenous variables (such as agricultural frontier expansion) and change constraints (such as protected lands or areas with high slopes). And (3) non-constant transition rates could be tested.”

Reviewer 2 Report

Comments and Suggestions for Authors

Comments on paper titled:

How Accurate are Land use Change Models? Assessment of Natural Vegetation Suppression Predictions in Three Brazilian Biomes

 

Round 1

First, I would like to congratulate the authors for their excellent work and interesting discussion. The paper presented a comparison of baseline and machine learning-based models for predicting land use changes in three Brazilian biomes. The study showed that, despite the complexity of machine learning models, their accuracy was similar to simpler baseline models, which are easier to interpret and use. It highlighted the limited predictive ability of both methods, particularly for distant future scenarios, with many correct predictions attributed to chance. The paper presented concerns about the use of inappropriate evaluation methods, such as the ROC curve, which can overestimate model accuracy. The study showed that baseline models are valuable for their simplicity and comparable performance, while emphasizing the need for better evaluation practices and recognition of model limitations.

The paper's topic is highly relevant and aligns well with the scope of the Land journal. It presents intriguing findings that hold significant value for land use change predictions, making it a strong candidate for publication. However, several minor revisions are required before it can be considered for acceptance. Below are the reviewer's comments:

1.      The paper title is relatively long, and the question format ("How Accurate are...") might appear vague or unconventional for a scientific audience. I recommend opting for one of the following more conventional titles:

·         "Evaluating the Accuracy of Land Use Change Models for Predicting Vegetation Loss Across Brazilian Biomes"

·         "Accuracy of Land Use Change Models in Predicting Vegetation Suppression: A Case Study of Three Brazilian Biomes"

·         "Comparative Assessment of Land Use Change Models in Predicting Natural Vegetation Suppression in Brazil"

2.      In many journals, it is common practice to include line numbers in papers; without them, reviewers may find it challenging to clearly reference specific points of concern.

3.      Overall, the quality of nearly all figures needs enhancement and improvement.

4.      P2-L25-29:The paragraph reads: “Given the formulation characteristics of the models, machine learning-based models are expected to perform considerably better than baseline models. This expectation is justified by the greater complexity of their formulation and the use of modern training methods. Despite this hypothesis, there are no detailed studies exploring this topic.…”

This paragraph may be misleading, as the expectation that machine learning (ML) models outperform conventional baseline models is not solely due to their greater complexity or the use of modern training methods. Numerous previous studies have provided evidence supporting this superiority. For instance, Gu et al. (2023) highlighted that ML methods significantly enhance land change prediction models, surpassing traditional statistical approaches by effectively capturing complex, non-linear relationships inherent in land development dynamics. Similarly, Ahmadlou et al. (2018) conducted a comparative study demonstrating that ML techniques consistently outperformed logistic regression and other statistical models in simulating land use and cover change (LUCC). Additionally, Lin et al. (2011) showed that Neural Networks outperformed logistic regression and auto-logistic

a more accurate context for the hypothesis being tested in this study.

5.       P2-L25-29:The paragraph reads: " In the central part of the figure are the processes carried out, on the left are the stages of the work, and on the right are the software used. In the central part of the figure are the processes carried out, on the left are the stages of the work, and on the right are the softwares used”

Please remove the redundant statements.

6.      P3-Comments on Figure 1:

a.      The figure contains numerous abbreviations, such as GEE, SRTM, and INDE, which may not be familiar to non-specialist readers. It is recommended to include a list of symbols and abbreviations in the appendix for clarity.

b.      Based on the sequence presented in Figure 1, what is the next step after calculating AUC with ROC curve? Kindly revised the workflow accordingly.

c.      The graphical quality of figure 1 requires enhancements

7.      P3- Study Area:

a.      “…and the 1:250,000 grid used to generate this data…” What is meant by "grid" in this context? Does it refer to a data tile or a raster pixel? What are the spatial dimensions of such grids?

b.      The provided data for the 3 selected biomass are quite limited. Kindly provide a table that presents the important characteristics of these biomasses such as: its area, annual precipitation, annual average temperature and ET, soil type, land cover, land use and range of surface elevation. Such data are essential for initial description of any case study.

8.      P4-Comments on Figure 2. This figure is used to show the locations of the 3 studied biomes.

a.      It is highly recommended to add a new sup-plot to this figure that shows the locations of these three biomass areas among the 6 biomass areas in Brazil

b.      It is also recommended to add a sub-plot that shows the location of these areas among the different states in Brazil and the location of Brazil among south America.

9.      P5-Data Used:

a.      It is noted that the land use (LU) data is sourced from the MAPBIOMAS project. It should be clarified that this project is specific to Brazil and not intended for global use. Additionally, it is mentioned that the generated LU images are derived from Landsat data with a 30m resolution. Would it have been more advantageous to use the higher resolution 10m global LU data provided by ESRI and other sources, based on Sentinel data? Kindly elaborate.

b.      It is observed that SRTM DEM 30m data was used to derive altitude and slope information for the study area. Could you elaborate on the rationale for selecting SRTM 30m over the global ALOS 30m data, despite supporting studies suggesting ALOS is generally more accurate? For instance, Abdulkareem et al. (2020) reported that ALOS DEM had a significantly lower RMSE (2.14 m) compared to SRTM DEM (3.53 m). Similarly, Apeh et al. (2019) found that ALOS W3D30 outperformed SRTM30, particularly in regions with substantial elevation variations.

10.  P5-Data Preparation: It is stated that the data preparation process was divided into three main stages: (1) reclassification of land use, (2) identification, quantification, and sampling of vegetation loss, and (3) definition of predictive variables for modelling. Kindly revise the workflow in Figure 1 to align with and use these stage titles consistently.

11.  P5-Define each variable appearing in equations 2 and 3

12.  P6 - Modelling: Please introduce the TerrSet package, noting that it has recently transitioned to a freeware license and is now renamed as LibraGIS. Additionally, introduce the Dinamica EGO package, as it may be unfamiliar to non-specialist readers. Kindly elaborate on the rationale for using different software packages for the analysis instead of relying solely on the spatial package in AecGIS or the QGIS plugins, such as LecoS or SCP.

13.  P8- Make sure to define all variables appearing in equations 4 and 5 and also define the significance of ROC curve.

14.  P8- Please defined the abbreviations of the TOC curve method and describe why the units of the axes are in km2 which totally differs than ROC curve.

15.  P9-Check the figure caption in Figure 3.

16.  P14- Check the inconsistency in the caption of figure 7a. The general caption is for Probability surfaces and the units in m.

17.  P15-P17: Improve the quality of the selected font size and style in Figures 8-10

18.  Language issues: Please conduct a thorough review of the English language and sentence structure, as there are several language issues throughout the text. For example:

a.      P2-L43-46: The term "softwares" is incorrect. "Software" is uncountable and should not be pluralized. Instead, use "software applications" or simply "software."

b.      Check repetition.

Comments for author File: Comments.pdf

Author Response

First, I would like to congratulate the authors for their excellent work and interesting discussion. The paper presented a comparison of baseline and machine learning-based models for predicting land use changes in three Brazilian biomes. The study showed that, despite the complexity of machine learning models, their accuracy was similar to simpler baseline models, which are easier to interpret and use. It highlighted the limited predictive ability of both methods, particularly for distant future scenarios, with many correct predictions attributed to chance. The paper presented concerns about the use of inappropriate evaluation methods, such as the ROC curve, which can overestimate model accuracy. The study showed that baseline models are valuable for their simplicity and comparable performance, while emphasizing the need for better evaluation practices and recognition of model limitations.

 

The paper's topic is highly relevant and aligns well with the scope of the Land journal. It presents intriguing findings that hold significant value for land use change predictions, making it a strong candidate for publication. However, several minor revisions are required before it can be considered for acceptance. Below are the reviewer's comments:

Answer: Thank you very much for your thorough review and valuable suggestions. Below, each comment is addressed in detail.

 

1.     The paper title is relatively long, and the question format ("How Accurate are...") might appear vague or unconventional for a scientific audience. I recommend opting for one of the following more conventional titles:

 

"Evaluating the Accuracy of Land Use Change Models for Predicting Vegetation Loss Across Brazilian Biomes"

 

"Accuracy of Land Use Change Models in Predicting Vegetation Suppression: A Case Study of Three Brazilian Biomes"

 

"Comparative Assessment of Land Use Change Models in Predicting Natural Vegetation Suppression in Brazil"

Answer: Accepted. The option: “Evaluating the Accuracy of Land Use Change Models for Predicting Vegetation Loss Across Brazilian Biomes” was considered the most appropriate.

 

2.     In many journals, it is common practice to include line numbers in papers; without them, reviewers may find it challenging to clearly reference specific points of concern.

Answer: Accepted.

 

3.     Overall, the quality of nearly all figures needs enhancement and improvement.

Answer: The figures have been updated to the resolution specified by the journal. The only exception was Figure 3. As this figure was extracted from another article, its resolution remains the same as in the original publication. However, this does not compromise the understanding.

 

4.     P2-L25-29:The paragraph reads: “Given the formulation characteristics of the models, machine learning-based models are expected to perform considerably better than baseline models. This expectation is justified by the greater complexity of their formulation and the use of modern training methods. Despite this hypothesis, there are no detailed studies exploring this topic.…”

 

This paragraph may be misleading, as the expectation that machine learning (ML) models outperform conventional baseline models is not solely due to their greater complexity or the use of modern training methods. Numerous previous studies have provided evidence supporting this superiority. For instance, Gu et al. (2023) highlighted that ML methods significantly enhance land change prediction models, surpassing traditional statistical approaches by effectively capturing complex, non-linear relationships inherent in land development dynamics. Similarly, Ahmadlou et al. (2018) conducted a comparative study demonstrating that ML techniques consistently outperformed logistic regression and other statistical models in simulating land use and cover change (LUCC). Additionally, Lin et al. (2011) showed that Neural Networks outperformed logistic regression and auto-logistic

 

a more accurate context for the hypothesis being tested in this study.

Answer: The term "machine learning" is employed in this section to group all those methods that utilize some kind of automated computational training to model land use change. Thus, Neural Networks, Random Forest, Logistic Regression and autologistic would be part of the same group. Baseline models do not perform any kind of training. They are, most of the time, simply a Euclidean distance surface to some object that has influence on the modeled changes. To make this idea clearer, we have changed the sentence to:

“Given the formulation characteristics of the models, machine learning-based models are expected to perform considerably better than baseline models. This expectation is justified by the greater complexity of their formulation and the use of training methods. Despite this hypothesis, there are no studies that compare the performance of machine learning-based and baseline models.”

Furthermore, some examples of models were added in previous sections of the text, helping the reader to understand the division between the two groups.

 

5.       P2-L25-29:The paragraph reads: " In the central part of the figure are the processes carried out, on the left are the stages of the work, and on the right are the software used. In the central part of the figure are the processes carried out, on the left are the stages of the work, and on the right are the softwares used”

 

Please remove the redundant statements.

Answer: Accepted.

 

6.     P3-Comments on Figure 1:

 

1.     The figure contains numerous abbreviations, such as GEE, SRTM, and INDE, which may not be familiar to non-specialist readers. It is recommended to include a list of symbols and abbreviations in the appendix for clarity.

Answer: Accepted.

 

1.     Based on the sequence presented in Figure 1, what is the next step after calculating AUC with ROC curve? Kindly revised the workflow accordingly.

Answer: Accepted. The next step is graphic and cartographic representation. 

 

1.     The graphical quality of figure 1 requires enhancements

Answer: Accepted.

 

7.     P3- Study Area:

 

1.     “…and the 1:250,000 grid used to generate this data…” What is meant by "grid" in this context? Does it refer to a data tile or a raster pixel? What are the spatial dimensions of such grids?

Answer: The term 'grid' refers to the set of cells presented in Figure 2. It is the same set of polygons used by the MapBiomas project for processing land use maps. The spatial dimension is 1.5° of longitude and 1° of latitude.

 

1.     The provided data for the 3 selected biomass are quite limited. Kindly provide a table that presents the important characteristics of these biomasses such as: its area, annual precipitation, annual average temperature and ET, soil type, land cover, land use and range of surface elevation. Such data are essential for initial description of any case study.

Answer: We believe that the reviewer made a small confusion between ‘biomes’ and ‘biomass’. The variables requested by the reviewer are relevant for biomass estimation, but our research predicts land use changes in different ‘biomes’, not biomass. A biome is a large region with a set of ecosystems that share similar characteristics, such as vegetation, climate, soil and fauna. In Brazil, the boundaries of biomes are delineated by the National Institute of Geography and Statistics - IBGE, and are available for download at https://www.ibge.gov.br/geociencias/informacoes-ambientais/vegetacao/15842-biomas.html (In Portuguese). We believe that the description of the main environmental characteristics presented in section "2.1. Study Area" is sufficient to differentiate the three biomes under study. Additionally, we included a map with the Brazilian biomes in Figure 2 to help clarify the location of the study areas.

 

8.     P4-Comments on Figure 2. This figure is used to show the locations of the 3 studied biomes.

 

1.     It is highly recommended to add a new sup-plot to this figure that shows the locations of these three biomass areas among the 6 biomass areas in Brazil

Answer: Accepted.

 

1.     It is also recommended to add a sub-plot that shows the location of these areas among the different states in Brazil and the location of Brazil among south America.

Answer: Accepted.

 

9.     P5-Data Used:

 

1.     It is noted that the land use (LU) data is sourced from the MAPBIOMAS project. It should be clarified that this project is specific to Brazil and not intended for global use. Additionally, it is mentioned that the generated LU images are derived from Landsat data with a 30m resolution. Would it have been more advantageous to use the higher resolution 10m global LU data provided by ESRI and other sources, based on Sentinel data? Kindly elaborate.

Answer: The MapBiomas project is an initiative that produces land use and land cover data specifically for Brazil. The classifications are trained with samples exclusively collected for the Brazilian territory, which confers greater representativeness of the mapped classes. At the same time, the project develops a robust assessment of the maps' accuracy, making it possible to know the agreements and disagreements for each class. Furthermore, one of the study's objectives was to test how the predictions behaved over different time horizons. As Sentinel-2 was launched in 2015, using land use maps derived from this product would limit this objective.

 

2.     It is observed that SRTM DEM 30m data was used to derive altitude and slope information for the study area. Could you elaborate on the rationale for selecting SRTM 30m over the global ALOS 30m data, despite supporting studies suggesting ALOS is generally more accurate? For instance, Abdulkareem et al. (2020) reported that ALOS DEM had a significantly lower RMSE (2.14 m) compared to SRTM DEM (3.53 m). Similarly, Apeh et al. (2019) found that ALOS W3D30 outperformed SRTM30, particularly in regions with substantial elevation variations.

Answer: The SRTM was selected because the training period used to develop the models (1995-2000) coincided with the production period of this product (2000). This choice was made to avoid potential biases that could arise from using a Digital Elevation Model produced after the training period. If we consider that processes such as mining and deforestation occurred in the study areas after 2000, these processes could potentially alter Digital Elevation Models.

 

10. P5-Data Preparation: It is stated that the data preparation process was divided into three main stages: (1) reclassification of land use, (2) identification, quantification, and sampling of vegetation loss, and (3) definition of predictive variables for modelling. Kindly revise the workflow in Figure 1 to align with and use these stage titles consistently.

Answer: Accepted.

 

11. P5-Define each variable appearing in equations 2 and 3

Answer: Accepted. The sentence was changed to:

“2.3.2. Identification of the amount of vegetation suppression projected for the future

The amount of change expected in the future was defined using the two land use maps from the training period. From these maps, it is possible to identify and quantify the areas of vegetation suppression during the training, as well as allocate samples of occurrence and non-occurrence of the phenomenon. Equation 2 exemplifies this process:

SNVTP = NVAt0 – NVAt1                                                      (2)

Where the suppression of natural vegetation during the training period (SNVTP) is equal (=) to the difference (-) between the natural vegetation area in 1995 (NVAt0) and the natural vegetation area in 2000 (NVAt1). Knowing the size of the suppressed area in the training period and assuming that in the future the rate of suppression will remain the same, it is possible to predict the expected area of suppression for any period of time. Equation 3 presents this possibility.

FSNVx =NVAt0 – NVAt1/ VAt0)* 100/ VAtx                      (3)

Where the future suppression of natural vegetation (FSNVx) is calculated (=) based on the total area of natural vegetation suppression during the training period (NVAt0 - NVAt1), divided (/) by the vegetation area in t0 (VAt0), and multiplied (*) by 1% of the vegetation area from the period preceding the extrapolation (100/VAtx). For example, if the model's training occurred between 1995 and 2000 and it was identified that vegetation suppression during this period corresponded to 1% of the existing vegetation in 1995, to estimate the changes for the year 2005, 1% of the vegetation area in 2000 was used. For 2010, 1% of the estimated vegetation area for 2005 would be applied, and so on.

This method was used to estimate the expected changes in the baseline models and in the model using Random Forest. In the other models used, this step occurs automatically.”

 

12. P6 - Modelling: Please introduce the TerrSet package, noting that it has recently transitioned to a freeware license and is now renamed as LibraGIS. Additionally, introduce the Dinamica EGO package, as it may be unfamiliar to non-specialist readers. Kindly elaborate on the rationale for using different software packages for the analysis instead of relying solely on the spatial package in AecGIS or the QGIS plugins, such as LecoS or SCP.

Answer: Accepted. The following sentence has been added to the text:

“In the model development stage, four different statistical methods were used: from the TerrSet software, the Artificial Neural Network and SimWeight methods were used; from the Dinamica EGO software, the weights of evidence method; and from the Google Earth Engine platform [19], the Random Forest method. These methods were selected due to their popularity and frequent use by modeling experts. In addition, testing the performance of different modeling methods makes it possible to identify if any performs better than another.

TerrSet is a geographic information system (GIS) developed by Clark University in 1987. Its applications encompass GIS analysis, remote sensing image processing, and spatial modeling. A new free version, TerrSet LiberaGIS V.20, was released in December 2024 [5]. 

DinamicaEGO is an environmental modeling platform developed by the Remote Sensing Center (CSR) of the Federal University of Minas Gerais (UFMG) in 2002. Its applications include land use change modeling, urban expansion modeling, wildfire modeling, and biodiversity modeling [4]. 

Google Earth Engine is a free, cloud-based platform for processing remote sensing data using JavaScript. Its major advantages are free access, a massive collection of satellite imagery, cloud-based computing, and the opportunity to collaborate with others [19].

For the TerrSet and Dinamica EGO models, the required inputs were land use maps from 1995 and 2000, and predictive variables. In these software, the creation of occurrence and nonoccurrence samples; the calculation of transition rates during the training and extrapolation periods; and the allocation of changes by cellular automata are performed automatically. In the case of the model using Google Earth Engine (GEE) and Random Forest, these steps were carried out through computer programming. Each of the four models is described in detail below.”

 

13. P8- Make sure to define all variables appearing in equations 4 and 5 and also define the significance of ROC curve.

Answer: Accepted. The sentence was changed to:

“2.6.1.1. Area Under the Curve - Receiver Operating Characteristic Curve (ROC)

The ROC curve was used to evaluate the rate of vegetation suppression samples correctly classified (true positives) and incorrectly classified (false positives) by the models for each probability level. The true positive rate can be calculated using equation 4, while the false positive rate is measured by equation 5.

TPR=TP/(TP+FN)                                     (4)

FPR =FP/(FP+TP)                                     (5)

where the true positive rate for each probability level (TPR) is obtained (=) by dividing (/) the quantity of true positives (TP) by the sum of true positives and false negatives (TP + FN). While the false positive rate for each probability level (FPR) is obtained (=) by dividing (/) false positives (FP) by the sum of false positives and true positives (FP + TP). This method is widely used to assess the accuracy of binary classifications, with applications in various fields of study [6].

The Area Under the Curve (AUC) is a way to simplify the ROC curve analysis by aggregating the value of true and false positive rates across all thresholds. In a model capable of differentiating all occurrence and nonoccurrence samples of the phenomenon, the AUC value will be 1. In contrast, for a model with performance equal to randomness, the AUC value will be 0.5.

The AUC calculation was performed on the GEE platform. For each analyzed period, 10,000 samples of vegetation suppression and 10,000 samples of non-suppression were used. The probability surfaces were divided into 100 equal parts based on their histograms. For each of these parts, the true and false positive rates were calculated. The AUC value was defined by the average performance of the models at each of the 100 thresholds.”

 

14. P8- Please defined the abbreviations of the TOC curve method and describe why the units of the axes are in km2 which totally differs than ROC curve.

Answer: Accepted. The following sentence has been added to the text:

“Unlike the ROC curve, which commonly presents hits and false alarms based on proportions of change and persistence samples, the TOC curve presents hits and false alarms in square kilometers. This occurs because the TOC curve calculation utilizes a reference map encompassing all changes and persistence within the analyzed period, enabling the generation of area-based statistics. The y-axis of the TOC curve shows the total amount of change that occurred between t0 and t1, while the x-axis represents the total area of the study area at t0 [11].”

 

15. P9-Check the figure caption in Figure 3.

Answer: Accepted. 

 

16. P14- Check the inconsistency in the caption of figure 7a. The general caption is for Probability surfaces and the units in m.

Answer: The caption has been edited to:

“Figure 7: Models with the best AUC for each study area. (A) Amazon: distance to anthropic uses from 2000. (B) Cerrado: Random Forest. (C) Pampa: Random Forest.”

 

17. P15-P17: Improve the quality of the selected font size and style in Figures 8-10

Answer: Accepted.

 

18. Language issues: Please conduct a thorough review of the English language and sentence structure, as there are several language issues throughout the text. For example:

 

1.     P2-L43-46: The term "softwares" is incorrect. "Software" is uncountable and should not be pluralized. Instead, use "software applications" or simply "software."

Answer: Accepted.

 

1.     Check repetition.

Answer: Accepted.

Reviewer 3 Report

Comments and Suggestions for Authors

This article is very interesting and makes a valuable contribution to the field. To further improve its quality, please note the following suggestions:

   -Introduce the approaches used conceptually, without going into mathematical detail. Equations can be moved to the Methods section.
   -The use of Random Forests and Neural Networks is pertinent. However, it would be useful to explain why these models have been favored over others.
   -The results show similar performance between the basic models and those based on machine learning. An analysis of the potential reasons for these similarities would be useful to better understand the results.
   -Although the figures are clear, additional maps showing areas of disagreement (for example, between predictions and reference data) would enrich the interpretation of the results.

Author Response

This article is very interesting and makes a valuable contribution to the field. To further improve its quality, please note the following suggestions:

Answer: Thank you very much for your thorough review and valuable suggestions. Below, each comment is addressed in detail.

 

   1- Introduce the approaches used conceptually, without going into mathematical detail. Equations can be moved to the Methods section.

Answer: It is well-known that equations are typically presented in the methodology section. However, Equation 1 is essential for introducing the topic and clearly explaining how the allocation models work. Given that allocation models are one of the primary sources of doubt in this type of modeling, we have decided to keep the equation in the introduction, even though it is not customary.

 

   2- The use of Random Forests and Neural Networks is pertinent. However, it would be useful to explain why these models have been favored over others.

Answer: This paragraph has been added to the text: 

“These methods were selected due to their popularity and frequent use by modeling experts. In addition, testing the performance of different modeling methods makes it possible to identify if any performs better than another..”

 

   3- The results show similar performance between the basic models and those based on machine learning. An analysis of the potential reasons for these similarities would be useful to better understand the results.

Answer: This paragraph has been added to the text: 

“Possible reasons for the similar performance can be highlighted: (1) The amount of change during the training period was significantly different from the extrapolation period. Therefore, a large amount of change was likely allocated to areas not considered susceptible by the machine learning-based models. (2) For the Amazon and Cerrado study areas, a significant shift in the spatial patterns of change was observed between the training and extrapolation periods. Consequently, a significant portion of the change was allocated to areas that were not predicted to be susceptible according to the machine learning-based models. (3) It is known that change processes are influenced by their surroundings [4]. The vegetation suppression investigated was likely related to proximity to anthropogenic uses or to past suppressions. And (4) Predicting natural vegetation suppression is challenging. A variety of factors, including political changes, economic conditions, environmental regulations, personal motivations, mineral discoveries and wars, can influence the quantity and location of suppressions. Given the complexity of the problem, it is expected that all models will produce a significant number of false alarms and omissions. This is supported by the observation that the random model performed on par with the other’s in all scenarios.”

 

   4- Although the figures are clear, additional maps showing areas of disagreement (for example, between predictions and reference data) would enrich the interpretation of the results.

Answer: Figures 11, 12 and 13 illustrate this. These figures show an evaluation using the three-map approach, where the results of the prediction models are compared with the MapBiomas reference maps. The areas shown in red represent the model's correct hits, areas where the model predicted change and the reference also indicated change. The correct rejections, in gray, are areas where the model predicted persistence, and the reference did as well. False alarms (in yellow) are areas where the model predicted change, but the reference indicated persistence. And misses (in beige) are areas where the model predicted persistence, but the reference indicated change. The analysis of misses and false alarms allows for an examination of the disagreement between predictions and the reference.

‘ In these figures, for each study area and time period analyzed, only the predictive model with the highest figure of merit value is presented. This is due to the large number of models evaluated. Considering that eight different models were evaluated in four different time periods for three different study areas, this results in a total of 96 maps. Therefore, it became impossible to present in maps the results of all models under all tested conditions. However, Figures 8, 9 and 10 show the statistical results using the three-map approach for all models.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have revised the paper according to my comments, and I would like to see it published.

Reviewer 2 Report

Comments and Suggestions for Authors

Authors have positively responded to all comments