Construction of High Spatial-Temporal Water Body Dataset in China Based on Sentinel-1 Archives and GEE

Round 1

Reviewer 1 Report

The manuscript has been very much improved and the methodology and the results are better explained. I would suggest that this manuscript is published.

Author Response

Thank you for your encouragement.

Reviewer 2 Report

Finally good job :-)

Author Response

We greatly appreicate your encouragement.

Reviewer 3 Report

Missing reference: ASTER Global Water Bodies Database: for description see: https://lpdaac.usgs.gov/products/astwbdv001/

Line 53: GSWE is called Global. Is it? What are the source data? I do see some of the info in a later table. A few words here would help.

Figure 9: it would be a better comparison if 9b only had the same 4 classes as 9a

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 4 Report

The authors develop and evaluate Sentinel-1 SAR-based methods to map open water and monitor its dynamics using a study area of China in its entirety. A considerable amount of work is reflected in the analysis described. And an improved estimation of surface water extent in China is provided by the product. This is important work.

However, the manuscript requires significant revision before being acceptable as a peer-reviewed journal publication.

As I read through the manuscript the first time, I found myself wanting to insert many of the content of section 5.4 “Uncertainty of this study’ UPFRONT – to properly bound the problem being addressed: mapping OPEN WATER DYNAMICS in China. I needed to see caveats to the approach explicitly stated: any water with significant vegetation canopy over its surface is NOT being addressed, let alone detected through this approach!

I wanted to comment about speckle – how is that being dealt with if at all? I left my comments regarding this in text below as I wrote it before getting to section 5.4.

A big concern with the manuscript then is the transformation of focus from water bodies to wetlands as it progresses. The title focuses on ‘water bodies’. The conclusions mention only “wetlands’. Perhaps this is an attempt to tailor the paper to better match the ‘wetland special edition’. But the preliminary extraction of water body section makes no mention of wetlands even with its further classification of water bodies. No wetlands are used for backscatter coefficients analysis and/or threshold determination. The effect of vegetation on wetlands on backscatter is well known and is not indicated in the method shown. Instead, it is simply described as an “uncertainty’ in the last section prior to the conclusions in which the value of the approach to map wetlands is asserted!

There is a vast body of literature on remote sensing of wetlands using SAR. It does NOT include the possibility of accurately mapping ‘wetlands’ over a region as broad as China using a single threshold of SAR intensity. You state this in section 5.4. It should be stated in section 1!

Given the methods used and results provided, the broad use of ‘wetlands’ as a category should be deemphasized in this manuscript. The focus is on inundation in places or during periods when surface vegetation is flooded such that it is effectively eliminated.

Here are some additional, specific comments about the methods:

How large (either in square area or dimensionally) are the training sample locations? How exactly where they chosen (e.g., random point sampling within the land cover classification – therefore, stratified random sampling)? You should be this specific.

Did ANY of the coefficient sample points or the accuracy assessment points fall on edges of water bodies and other features?

A slope of 3% for masking can easily eliminate edges of large water features that will expand and contract in size within that 3% slope area – especially as calculated from SRTM, which often depicts intra-canopy heights as “ground”.

Does speckle have any impact and if so, how was this addressed? It seems the images are averaged within the month to create the monthly intensity value. That ‘multi-look’ would decrease speckle and if used, should be explicitly described.

Was the -23 db value chosen because it was used by other authors cited or because it is the mean value generated from the ~13,000 point (or were they area?) samples?

If I read the methods correctly, the “annual cloudless image” is some composite of images throughout the year and was used to establish an NDWI-based water mask and a snow mask. Was this done on a pixel basis? Was the first clear pixel chosen as the value? Was the mean clear pixel value chosen? The description of this dataset is very inadequate. What about areas in China where the snow gives way to open ground AND WATER for some portion of the year? Is that portion that was captured by the annual cloudless image? Is snow generally absent during the rainy/cloudy season?  Conversely, aren’t there water bodies on China that might be absent during cloudless periods?

Also, is the dilation of identified potential water areas by 1 pixel adequate to allow for annual dynamics of these water bodies?

Visual inspection of ~15K points is a great deal of effort that is much appreciated. But the validation section requires revision to clarify the procedure. Please describe exactly how the GSWE 2018 is used for random sampling. Are only “wet areas” as declared in GSWE sampled? Are ‘wet’ vs ‘dry’ determined from GSWE and then checked with high-resolution imagery? Why use the GSWE at all as part of validation processing?

Line 190. These points are marked with the source date of the high-resolution images from Google earth, correct? And only images from 2018 were used. Does this result in any bias given when these images are collected? That is, I strongly suspect they are from cloud free periods - which also tend to be drier. How might/does this bias your results?

Is the overall accuracy is the average of the monthly accuracies? If so, show the error matrices (and sample sizes) for each month. If instead it is based on all points ‘lumped together’, show the error matrix for the year and explicitly state this. Unfortunately, this latter approach does little to answer the criticism that the accuracy assessment is biased toward relatively dry periods.

I have never read an omission error rate of 0. You must have inserted an incorrect number in the manuscript – especially when figure 8 shows areas of ‘under-extracted water surface’. I will note that it would be far more likely to achieve a 0 omission rate if the accuracy assessment was biased toward open water features that are relatively permanent and that are also being assessed using ‘truth’ data from imagery collected more commonly during dry periods. Unfortunately I think your approach meets all these criteria. A more detailed analysis that is more completely reported would be needed to prove me wrong.

How many water bodies smaller than 6.25 ha were sampled by your validation sampling scheme? *Note that I’m not suggesting you redo the visual inspection of 6,554 points! Rather, I’m trying to make the point that the validation as performed underrepresents edges and small water features both by the spatial and temporal sampling conducted. I strongly expect that in reality, the method has omission errors higher than whatever value is calculate through your approach and correctly inserted in the manuscript. This would at least be recognized in the discussion/section on uncertainty of the analysis.

The comparison of a single (very large) lake area with altimetry is very interesting. Figure 5 is wonderful. However, this was only done for Dongting Lake, correct? If so, I think ‘validation’ too strong a word for this particular analysis.  

Section 5.1 – comparison with GSWE is entirely valid and illustrated nicely. Well done.

Section 5.2 – Change the title of this section to read “Open water wetland classification based on HSWDC

Line 136 – 2 studies is not ‘a lot’.

Other comments about figures and the table:

Figure 1 The caption should read ‘number of images per month’ rather than ‘number of monthly images’ [It’s not possible to have 17 monthly images].

Figure 4 Scale bars are needed in figures labelled as “a” and these lables should be in white font so as to be visible.

Table 1 – increase spacing between row entries.

Data distribution

As per the policy of the Journal, you need to provide everything necessary for someone else to replicate the work. A more complete description of methods may be adequate. An alternative would be to make your accuracy assessment points available online as this is the only dataset not openly available to others.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 4 Report

The authors reconciliation of review comments are appreciated. The explanation of methods is much improved. There are some sentences for which additional editing/grammatical improvements are required. 

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

The subject of the research presented in this manuscript is the construction of a water body dataset covering China. It is a very interesting and important topic as it addresses the difficulty of optical sensors in water bodies mapping.

However, there are several points in the methodology section that need to be better explained and more developed.

Also, English needs extensive editing throughout the paper. There are a lot of parts where the meaning of the text is not clearly understood (i.e. lines 32, 80, 106, 115, 119, 120, 145, 147, 224, 228, 235, 239, 248, 253, 256, 272, 276, 302, 317) and there are a lot spelling and other minor mistakes in the text such as mistaken use of capital letters, or not correct form of numbers (i.e. lines 27, 28, 33, 54, 56, 60, 76, 79, 97, 99, 101, 102, 123, 127, 136, 166, 171, 182, 188, 195, 217, 222, 231, 253, 259, 314). Generally, the text should be better edited.

Furthermore, I would suggest that the following points should be addressed.

Lines 21, 29: In the abstract there are web-site links that should be included in the text as references.

Line 86: There is inconsistency in reference in line 86 and in the references of Table 1.

Lines 100-102: Six land cover types are mentioned in the text, not seven

Methodology section: There is inconsistency in the tense of the verbs. In the beginning it is in present tense, then past and then present tense again.

Line 110: Add space after Figure

Line 123: Add the letters (a) and (b) in the diagrams since they are mentioned in the legend

Line 135: Add reference for Water Mask-slope or explain better your methodology.

Line 138: A reference should be added and the NDWI should be more clearly described

Line 140: Replace um with μm

Line 140: Explain if the mean composites OLI images were downloaded or composed.

Lines 141-143: Explain how the threshold was defined. In the text above bare land is analysed not snow

Lines 152-158: Explain how the two regions were defined

Line 184: Mention the full name the first time you include rms

Lines 186-189: We can see in Figure 6 that CHSWD overestimates water level compared to GSWE and water level altimetry, especially during winter time. This is not mentioned or explained in the results

Lines 244-245: Explain what is depicted in Figures 9a and 9b in the legend

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors — 01

Remarks:

Line 65: “the same time, sentinel-1 provides data on 10m emissivity”.

Reviewer’s remark 1: It should be “Sentinel-1” and “10m resolution”…

Lines 68-71:

Therefore, in this study, our work includes (1) establishing a large-scale water classification method based on time series sentinel-1 data, (2) extraction of China waters on a monthly scale from 2016 to 2018 based on the GEE platform and (3) comparison with existing water products and evaluation of its accuracy.

Reviewers’ remark 2:  In my opinion the research sequence should be as follows:

1. Clear definition of the goal / aim / purpose / of the research. What is the main scientific aim? The product or the methodology?
2. Establishing a large-scale water classification method based on time series Sentinel-1 data.
3. Extraction of China waters on a monthly scale from 2016 to 2018 based on the GEE platform,
4. Evaluation of the accuracy and validation of the results
5. Comparison with existing water products.

Lines 136 – 140:  NDWI index

Reviewers’ remark 3:  I kindly ask the authors to explain in details and to justify their choice of the NDWI index. Please pay attention that there are 3 different indices “hidden” under the same name. NDWI according to: 1) McFeeters, 2) Gao and 3) Rogers/Kearney and they are calculated using: 1) green/NIR, 2) NIR/SWIR, and 3) Red/SWIR bands respectively… Please justify why do you use this one defined by McFeeters? Why have you omitted SWIR bands?

  Fig.4    

Reviewers’ remark 4:  Please explain the graphs in this figure in more details. Why the sigma naught values variability shows an almost regular pattern for forest, snow and buildings, especially at VV  polarization? What is the reason of the alternate high and low values for the consecutive acquisitions?

Lines 152-153: “The confusion of the bare land and waters mainly occurs in the west region of China. so, we divide the whole China into two parts: the east (including Beijing, Tianjin, Hebei, Liaoning, Shanghai, Zhejiang, Fujian, Shandong, Guangdong, Taiwan, Henan, Jiangsu, Anhui, Hubei, Hunan, Jiangxi and Hainan) and the west (the other provinces) (Figure 1).”

Reviewers’ remark 5: This is the weakest point of the methodology. How other researchers can repeat the approach and the methodology on other test sites? What does it mean “mant ainly occurs”? In my opinion it is a methodology failure and a lack of the scientific reliability. In this context please consider to test other NDW Indices (remark no. 3).

Fig.12    

Reviewers’ remark 6:  I don’t see any significant differences between the pictures.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

I thank the authors for addressing most of comments adequately.

Minor English language editing is required in my opinion.

Reviewer 2 Report

Second revision:

1. In my opinion the paper represents just a case study and the methodology can neither be considered as robust and universal nor repeatable by other researchers on other geographic locations (even geographically similar).
2. What is the “threshold method”? It is a method of separation of two groups of pixels based on the assumption that ALWAYS two objects (clusters) have the same threshold/level of signal which can split them correctly. The second assumption is that at any moment these two clusters have the same physical properties. E.g. surface water is a smooth surface (this is the idea of thresholding during separation of “water”/”non water” pixels). But this assumption is not always true. During windy days the surface of water is rough then the backscatter increases. Thus, in a consequence, this image has to be eliminated because it introduces two types of errors:
   1. “ Negative alarms”  - it shows the pixels of “water” as “non-water”  (due to temporarily higher backscattering)
   2. “Positive alarms” - it shows smooth surfaces having very low backscatter as “water”. Smooth surfaces are not always water (e.g. meadows, sand dunes, flat grasslands, etc.

So, introducing a  common level of backscattering for all SAR images in order to establish an universal threshold always “valid” for separating water / non-water surfaces is not justified. I understand the reasoning of the authors proposing the values -15dB for VV and -23dB for VH polarization as the values the most frequent statistically, but besides the values themselves they have to take into account the time-series, i.e. temporal variability of the backscattering coefficient at the same point and to try to reduce the number of positive and negative alarms…

3. I don’t clearly understand the category of the land cover: “bare land”. For me “bare land” is a terrain without vegetation cover, often called “bare soil”. Let’s assume that the “bare land” is indeed the “bare soil”. In that case the backscattering would be as low as for surface of calm water only if the terrain were permanently flat and smooth. But it is not possible all year round. The bare soils are changing their roughness and in consequence their backscatter a few times a year following different agricultural practices. I ask the explanation: What is the “bare land”?
4. The next question: How it is possible to have similar values of the backscattering coefficient for different types of agricultural crops in the “farmland” class during the whole season?. Agricultural crops mapping is based on their variability in time and operationally used for example for agricultural statistics…
5. I’m not convinced that the methodology and the results will have a great scientific soundness.
6. English needs some polishing.