Reusing Remote Sensing-Based Validation Data: Comparing Direct and Indirect Approaches for Afforestation Monitoring

Round 1

Reviewer 1 Report

The article "Reusing remote sensing based reference samples: comparing direct and indirect approaches for afforestation monitoring" uses both direct and indirect methods, using the GEE platform, RF model to map afforestation across Italy for the period 1985-2019. At the same time, the authors implemented a methodology to reuse samples acquired through the stratification based on remote sensing maps to optimize the effort invested in their construction. As can be seen from the article, the workload of this experiment was huge, containing 13 time series, and both methods obtained better mapping results. However, there are still some problems, as shown below:

(1)   The abstract section is too redundant and should only highly summarize the research background, purpose of the study, significance of the study, research methods, findings and conclusions of the article.

(2)   The study time period of the article is 1985-2019, and only 13 time series are mentioned in the remote sensing image dataset, and it is suggested to supplement the data list of remote sensing images used with table.

(3)   I do not understand why the number of combined class samples is different for the direct and indirect methods when selecting validation sample points. Since the two methods are to be compared, the validation points of the two methods should be consistent to be more convincing.

(4)   Section 4.3 should be moved up to the beginning of Chapter 4. This part is the intermediate result of building the classification model and it is not appropriate to put it at the end part of the result.

(5)   Figure 5 and 6 are too rough and it is recommended to redraw them.

(6)   Figure 5 shows the variables in order from largest to smallest, and it is recommended that Figure 6 be kept consistent.

(7)   Lines 347-348 refer to the results of the article on the generally accepted accuracy of the direct method over the indirect method. I think it may be caused by the different validation points chosen by the two methods.

(8)   The center of the article is a comparison of the direct and indirect methods, while the results and conclusion sections are simply a comparison of the magnitude of the accuracy of the two methods. I believe that further analysis can be done in terms of four stratification types. For example, it is obvious that the accuracy of the direct method is higher in non-afforested areas, but the overall indirect method will be higher, what is the reason here?

(9)   The hierarchical approach to selecting validation points used in the paper is interesting. However, many of the methods used in the paper are existing methods and still lack some innovation. The direct and indirect methods can be improved or combined to obtain new methods that are superior to the results of both methods.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper follows on from the author's previous paper “Estimating Afforestation Area Using Landsat Time Series and Photointerpreted Datasets” which used Landsat best available time series and Random Forest to estimate afforestation by direct mapping. This paper adds an analysis using indirect mapping and compares accuracies between the two methods. This is a good paper, but there are some issues to do with presentation (especially your tables) and insufficient detail/important information left out. Major points are listed first, minor points after.

Major Corrections

Abstract I think it is worth mentioning something like “ accuracies were high in non-afforestation classes (87%-99%) but smaller for the afforestation classes (26%- 53%)” There is quite a significant difference inbetween class accuracy and just giving overall accuracy is a little misleading

Ln 87-94 You should add a reference to your previous paper to this paragraph, since you used direct mapping there

Section 2.1 Study area This section is a bit too close to your previous paper  “Estimating Afforestation Area Using Landsat Time Series and Photointerpreted Datasets”–reword and/or give different information

Section 3.1 Methods Your account of how you used Random Forest is lacking sufficient detail. I'd like information on parameters used and your exact process. All you say is that you used RF...

All Tables – explanations of all abbreviations used should be included for every table, to save the reader having to scan up and down the report

Table 4 & 7 – you don't define a_j, and I'm uncertain what it means – area presumably? – What are the units - pixel numbers/hectares/km_2?

In Table 4 you say that “w_j is the proportion of the map in each class” (footnotes) But if I'm right in assuming that a_j is area your table doesn't seem to show that: for example “w_1=a_1/a_1+a_2”. Is this proportion of map in this class? Wouldn't proportion of map in each class be more like : “w_1=a_1/a+b+c+d”

This table confused me...

Further, Table 4 and Table 7 don't correspond: there is an extra column (“Class Accuracy”) in Table 7 that is not present in Table 4.

Ln 255 presumably afforestation and non-afforestation, though you should specify. which is A and which is B?

Ln 259 “classes C and D respectively” (I assume)

Ln 325-326 “show the predicted land cover in 1988” “land cover predictions for 2020.” I don't understand this at all – how/why are these figures predictions? They look like a straightforward 1988 and 2020 photo to me. Please clarify.

Figure 3 In caption specify source of A1-D1 and A2-D2 figures

Figure 3 A3-D3 To my eyes the Landsat pixels look green, orange and red.

Overall I think it might be clearer if you put Table 3 and 4, and corresponding tables 6 & 7, into Supplementary Info. This big block of tables is quite confusing, maybe just include direct analogues to Table 2 & 5 for indirect mapping in the Results section : I.e just the final results in this section, everything else relegated to Supplementary

Figure 4 Ln372 Do you mean “Bottom”?

Minor corrections

Ln44 “high” replace “large”

Ln 121 delete “on”

Ln 168 “three in the infrared spectrum” little bit misleading, one band is near-infrared

Ln 184 “non-forest to forest”

Ln 300 Clarify that the in/out refers to inside /outside buffer. Clarify that aff is afforestation.

Ln 336 province

Ln 464 agricultural

Ln 468 “contrary to the general opinion”

Ln 469 “Indeed, ^while^ the direct approach...”

Ln 473 “showed differences between direct and indirect methods that were not statistically significant”

Ln 474 “meaning that both are effective” this doesn't follow – just because they are getting the same accuracy does not mean they are both effective. Suggest deleting this phrase.

Ln 505 “ as desired”

Ln 511 “large” should be “high”

Ln 511 direc-->direct

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Author Response

Thank you for your response. The revisions you suggested improve the quality of the paper.

Reviewer 2 Report

In the response to Reviewer the authors have satisfactorily responded to all my concerns. I think the revised paper is now OK. However, in the revised paper the authors have used numerous different colors (dark blue, light blue, orange, purple and red) to indicate deletions and changes. In some paragraphs it is not clear what is a deletion, and what is an addition, as it is not clear what these colors mean, and consequently what the text of the paper now reads. I've listed these confusing passages below. Other than that I'm now happy with the revised paper.

I would be happier if the deletions had a line through them, and additions and changes are of a single text color. …

Some confused passages, which I think could be clarified:

 Lines 151-156 "Italy is characterized by greata large morphological diversity with, since 70% of the national area characterized is composed by hills or mountains. Moreover, the geographical position of Italy is responsible for the different climate types of climate (alpine, continental and Mediterranean) which and this contributes to the richness of vegetation and forest biodiversity." This is in purple and blue text. This whole passage seems to be an addition, so why is it in different colors? It should surely just read "Italy is characterized by great morphological diversity with 70% of the national area composed of hills or mountains. Moreover, the geographical position of Italy is responsible for the different climatic types (alpine, continental and Mediterranean), contributing to the richness of vegetation and forest biodiversity."


Ln 47-53 The herein-presented method produced returns different accuracy estimates for different map classes, which is very useful for to understanding where the map should be considered more reliable. Accordingly, the accuracy in afforestation map classes ranged between 53% ± 5.9% for the indirect map class inside the buffer – defined as a stratum
 within 120 m to the forest boundaries - and 26% ± 3.4% for the direct map outside the buffer. On the other hand, the accuracy in non-afforestation map classes was between 87% ± 1.9% for the indirect map inside the buffer and 99% ± 1.3% for the direct map outside the buffer.

Change to “The herein-presented method produced different accuracy estimates for different map classes, with afforestation accuracies ranging between 53% ± 5.9%  for the indirect map class inside the buffer – defined as a stratum within 120 m of the forest boundaries - and 26% ± 3.4% for the direct map outside the buffer. Accuracy in non-afforestation map classes was much higher, ranging from 87% ± 1.9% for the indirect map inside the buffer and 99% ± 1.3% for the direct map outside the buffer.”

Ln 255 to the GEE doc-
umentation[xx] CITATION

Ln 373-4  change to "Figures A1-D1 and A2-D2 are aerial images from 1988 and 2020 respectively, whilst images A3-D3 are the aerial images of 2020 with the predicted maps."

Author Response

Please see the attachment

Author Response File: Author Response.docx