Comparison of Lake Extraction and Classification Methods for the Tibetan Plateau Based on Topographic-Spectral Information

Round 1

Reviewer 1 Report

The paper compares the three classification algorithms (Cart, RF, and GBDT) in lake extraction in Tibetan Plateau using selected topographic-spectral features based on their importance. It is an interesting work. However, there are the paper could be improved in several areas, such as related works, providing details on the methods used, comparing with state-of-the-art methods, presentation of the results, and justifying the conclusion with quantitative results.

Here are some detailed comments which could be useful.

·      In the abstract, there are more numbers than the stated ‘overall accuracy, Kappa coefficient, user precision, and producer precision’. Not clear what are those numbers!

·      Should give a full form of an abbreviation when it occurs for the first time, e.g., RNSS, TM, etc.

·      Better to describe, at least briefly, what are TM2, …, TM5.

·      Some language and grammar errors, e.g., researches -> research.

·      Better to define the meaning of ‘water extraction’ in the context, as the readers may have different interpretations based on its literal meaning.

·      The paper is weak regarding previous related and relevant works; reference to several recent works is missing. There could be a section on ‘related works’. And results could be compared with some of the state-of-the-art methods if that is possible.

·      It’d be good to give details about the spectral bands, and which is which (blue, green, …) referred to in Section 3.2.1. It is not clear how the authors came up with the RNSS formula; any basis or mathematics behind its formulation?

·      The table caption and table should stay together (Table 2).

·      The metrics used in the evaluation (Section 3.6) should be defined and described, possibly with the reason(s) for using them. Also, should describe how these metrics are calculated along with their significance.

·      From the plots, the statement ‘It is found that the RNSS lake extraction index constructed by band combination is higher in importance order in both Cart and GBDT classification methods.’ in lines 299-301 is incorrect!

·      All the plots in Figure 6 (also in Figure 7) could be combined into one single plot with distinctive legends, for better comparative views. The results are given only in terms of overall accuracy. Why are different metrics in plots (Figure 6) and tables 4 and 5?

·      In order to make conclusive remarks based on the objectives, it’d be good to demonstrate quantitatively how the proposed method(s) reduce the influence of ice, snow, …., compared to the state-of-the-art method(s).

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Three different classification methods are presented and discussed for a use case on the Tibetan plateau, where snow and ice can be challenging to identify lake bounderies in the area. Two different dates (2021 and 2016) are analysed with some oddities in results for 2016 for the Cart method. They are apparently selected on the basis of the availability of third parties dataset for (HydroLAKES). The methods are well presented and results clearly exposed, anyway the resulting classifiers could be exposed to some overfitting. In order to estimate the generalization capability of the classifiersI would select multiple random subsets of 70/30 samples (e.g. 10 different choices) for training and test and repeat the classifications. The resulting statistics could be provided for the average classifiers. That would exclude the low Kappa for Cart in 2016 is due to a specific sampling. Even, application of the resulting classifiers to other dates could be of interest (major changes in extensions and areas could imply low generalizaton proprieties: that would be important for analysis of long time series).

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

The article deals with a very interesting topic regarding the ‘Comparison of lake extraction and classification methods in Tibetan Plateau based on topographic-spectral information’. Overall, it is a comprehensive article and the findings provided indicate that a great deal of effort was put in. Suggestions for improvements that could be performed to the manuscript prior to its publication are the following:

Abstract:

-It is suggested to avoid acronyms, parentheses and a lot of numerical results in this part of the manuscript. Try to keep the Abstract simple yet informative and focus on the research questions that the authors try to answer through the paper.

-Line 47: Is the term “artificial visual interpretation” valid? What do the authors mean? Elaborate more.

-Line 54: It is suggested to use the journal’s format regarding equations.

-Figure 1: What is the reference of this Figure?

-Line 123: “strong reality”?

-Line 132: What were the factors which were taken into account in order to select the samples?

-Figure 2: It is suggested to provide more information regarding for example:

Why the authors selected to perform Tasseled Cup Transformation among other transformations available? (lines 174-178)

Which type of SW was used to perform digital processing?

Feature selection in DEM data?

-Lines 191-214: It is unclear how spectral characteristic curves assist the study at this point.

-Line 258: What led the authors to choose these algorithms? Any pros and cons?

-Line 292: “importance scores” (Figure 4-Figure 5) What is the meaning of this term? It is suggested to elaborate more on this.

-Line 371: “absolute percentage error formula”: It is suggested to add some relevant references regarding this formula.

-Line 390: “de-cloud filtering”: It is suggested to analyze a bit more this type of filtering.

-Line 402-403: “RF classification method is the most, …..” grammatical errors. In general, English grammar and syntax need to be improved in some parts of the manuscript. Therefore, editing of the English language and style is required.

Line 436: As it has previously mentioned, it is suggested to provide information about the method in general, rather than just the command (“.explain()” method).

Lines 442-445: “In this study, artificial visual interpretation is needed for sample selection in the process of supervised classification, which could not avoid the interference of artificial subjective factors in the sampling process, and is affected by the complex natural environment of the Tibetan Plateau.”: Elaborate more. This sentence is not clear. What do the terms “artificial visual interpretation” and “artificial subjective factors” mean?

Lines 487-488: “At the same time, the method has the advantages of transferable study area and swapping of better datasets.” How did the authors come to this conclusion? What findings support this assumption?

General comments:

The authors need to describe the processing flow in a more comprehensive way and include theoretical information that could support the selected work flow. The selected machine learning classification algorithms should be analyzed more thoroughly. In addition, through the manuscript several functions are mentioned. However, the reader might not be aware of the exact SW used and the functions that are available (mode of functions, parameters to be taken into account, commands etc.). Each SW has functions with different structure or similar parts which however lead to different results. Therefore, it is suggested to rewrite those parts and explain these functions in more detail (their structure and parameters, what each function does and what are the results, what details should be taken into account, etc.).

Moreover, it would be interesting to also comment on the following: What could be some possible restrictions or challenges if the proposed methods would be implemented in a different area of interest/different case study? Are there any other factors that should be taken into account?

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 4 Report

I feel sympathy for Google Earth Engine, and find the research area interesting. Nevertheless, there are many serious questions to the research:

- Line 22

 gradient enhanced decision tree (GBDT). Did the authors mean gradient boosting decision tree (GBDT) or tree-enhanced gradient boosting decision trees or what?

- line 27 "The three classification methods can better extract the lakes"

Better than what?

* Many interesting relater works can be found on the Internet:

Icy lakes extraction and water-ice classification using Landsat 8 OLI multispectral data

Comparison of Lake Area Extraction Algorithms in Qinghai Tibet Plateau Leveraging Google Earth Engine and Landsat-9 Data

Automated Extraction of Lake Water Bodies in Complex Geographical Environments by Fusing Sentinel-1/2 Data

etc.

The author should make a review and show the differences, advantages, and novelty of their methods in the Introduction.

* Why did the authors use LANDSAT/LC08/C02/T1_TOA (calibrated top-of-atmosphere reflectance) instead of LANDSAT/LC08/C02/T1_L2?

* The authors use only Landsat 8. This is a serious limitation of the research. GEE has other products comparison of which could make the research much more interesting.

* The authors use "the mean values of various ground objects". What is the motivation for such a choice? In general median values could provide a more realistic picture and in some cases, max values can seriously improve the results.

* I understand that there is no easy way of using neural algorithms in GEE, but we are living in 2022, and comparison of standard statistical approaches with neural networks is highly desirable  

- 263 "Usually, the number of decision trees is chosen to be 10"

The hyperparameters of the models depend on the task. I don't think that "usually" is applicable here.

- line 285 

All metrics used in the research should be explained (producer precision, user precision, etc.  

* The statistical difference betwine feature importance for 2020 and 2016 is huge. Some serious explanation should be provided for that in the other case the research looks very dubious.

* Fig 7. Without ground truth, it is impossible to evaluate the output of the models

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

The revised version tried to address most of the comments and concerns from my previous review. However, there are some issues given below still remain and that could be addressed. Therefore, this reviewer thinks that the paper needs to go through a major revision again.

·      Response 1 tried to answer the first comment. However, it should be made clearer in the paper as well so that the readers understand it well. The same is true in the case of some of the other responses as well, e.g., 7.

·      I am still not convinced by Response 10. One can clearly see the magnitude of importance of RNSS is not higher compared to some other features. This needs to be precisely and clearly described.

·      This reviewer may not agree with Response 11 that the combined figure (Figure 6) is not good. Rather, the figure could help clearly see and compare different results. However, the plot legends could be better, possibly with different line types rather than just color. For example, one could use the same color for the same method but with different line types for two different years (2016 and 2021).

·      Regarding Figure 7, I do not see it of any use without the readers being able to see the differences in the results and compare them. Here also, combining (maybe year-wise) the results with different legends could help differentiate the results. Moreover, the results could be described, analyzed, and interpreted better. And, the quality/resolution of the figure could be better so that texts are readable at 100% zoom.

·      There are still figures with the figure and captions on different pages!

·      About the conclusion, if the quantitative results couldn’t be given and the conclusions are being made simply based on the visual interpretation, that needs to be discussed well with reasoning and possibly providing future perspectives.

·      And about the cover letter with the responses, it’d be better if the answers in the cover letter refer to changes made in the paper, possibly with line numbers.

Author Response

Dear reviewer,

Thank you again for your valuable comments. We have carefully revised the manuscript according to your constructive comments and suggestions. The responses are listed in the text below.

• Response 1 tried to answer the first comment. However, it should be made clearer in the paper as well so that the readers understand it well. The same is true in the case of some of the other responses as well, e.g., 7.

--[Response]: About the Point 1, we have already made clear explanation in section 4.1.2 and 4.2.1 (line380-line422). About the Point 7, we have made relevant supplements in section 3.2.1 (line232-line236).

• I am still not convinced by Response 10. One can clearly see the magnitude of importance of RNSS is not higher compared to some other features. This needs to be precisely and clearly described.

--[Response]: We don’t deny the magnitude of importance of RNSS is not higher compared to some other features. But in our experiment, the magnitude of importance of RNSS was in the top ten in the GBDT and Cart method. Here is the original data from our experiment, the top ten important are intercepted. Figures 4–5 were also constructed based on this original data.

The above data shows that the RNSS has a relatively high importance order in both Cart and GBDT classification methods compared to some other features.

• This reviewer may not agree with Response 11 that the combined figure (Figure 6) is not good. Rather, the figure could help clearly see and compare different results. However, the plot legends could be better, possibly with different line types rather than just color. For example, one could use the same color for the same method but with different line types for two different years (2016 and 2021).

--[Response]: OK, we have modified Figure 6 according to your request.

• Regarding Figure 7, I do not see it of any use without the readers being able to see the differences in the results and compare them. Here also, combining (maybe year-wise) the results with different legends could help differentiate the results. Moreover, the results could be described, analyzed, and interpreted better. And, the quality/resolution of the figure could be better so that texts are readable at 100% zoom.

--[Response]: About the Figure 7, we agree that combining (maybe year-wise) the results with different legends could help differentiate the results, moreover, the results could be described, analyzed, and interpreted better. However, the Figure 7 is just a demonstration of the overall extraction effect. In the following, we also have the specific comparison and analysis of the experimental data and key areas. And the effect of overlaying the six extraction range layers is very messy. Please believe me that it is also not good for the readers to see the differences in the results and compare them. If you don’t think it of any use, we can delete this part. Please understand this.

• There are still figures with the figure and captions on different pages!

--[Response]: Sorry, we have made corresponding modifications. Never again will you see such problems.

• About the conclusion, if the quantitative results couldn’t be given and the conclusions are being made simply based on the visual interpretation, that needs to be discussed well with reasoning and possibly providing future perspectives.

--[Response]: I’m sorry that the previous statement “We can only saw from the perspective of visual interpretation of the extracting effect image that the GBDT classification method did better than the other methods in reducing the influence of ice, snow, and so on” is not accurate enough. In our study, we used the data of extraction accuracy verification to illustrate the quality of a model, and then confirmed and clarified this conclusion by visually interpreting the extraction effects of key areas (line388-line457).

Thanks again for the constructive suggestions and comments. We hope this new version is acceptable for publication. If you have any questions and suggestions about this manuscript, please feel free to contact us. We look forward to your reply.

Best regards,

Xiaoliang Wang.

Author Response File: Author Response.docx

Reviewer 3 Report

Authors have tried to incorporate the suggestions in the new version of the manuscript. Additional comments about further improvements prior to the article publication are the following:

- Point 1: Visual interpretation as a term is sufficient excluding “manual” or “artificial”.

- Point 2: It would be more convenient to place the equation after the paragraph where it is mentioned and add a number of reference (e.g. (1) as a format) on the right side, when referring to an equation (as suggested by the journal).

-Point 3: Add this piece of information in short, on the image caption.

-Point 4: “strong currency” is still not correct. It is suggested to revise again the manuscript and pay attention to English grammar and syntax.

- Point 7: It is suggested to add this piece of information in the manuscript (as described by the authors), making clear the reason that this curves analysis helps the study.

-Point 8: It is suggested to add this piece of information in the manuscript (as described by the authors)

-Point 13: There is still room for improvement regarding the explanation of the commands. It is suggested to pay more attention to the methods used in general and describe the concepts behind those methods that were used.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 4 Report

Comments for author File: Comments.docx

Author Response

Please see the attachment.

Author Response File: Author Response.docx