Sub-Pixel Waterline Extraction: Characterising Accuracy and Sensitivity to Indices and Spectra

Round 1

Reviewer 1 Report

This paper presents and innovative method to increase resolution of waterline extraction from pixel to subpixel based on a lad/water indexing method. The methodology is interesting. The following comments are given to authors and must be addressed before publication:

The abstract is too long. Please reduce and focus on your innovation. Please follow the same reference style throughout the paper. In some lines you use the style (Author, year), and in others you use numbering [1]. Unify. When referencing, don't use "e.g.", just use the reference, and avoid using brackets if you are commenting the refernce. For example, in line 72, the correct reference format would be "(e.g. Landsat imagery available since 1972, [27])". Referencesshould be ordered numerically (if you are citing more than one, they should appear as [1,7,10,24], not [1, 24, 10, 7]. Line 122: say which high resolution imagery you will use. Equations should be numbered. Line 143: BoA data? Line 146: enumerate environments used to make them easier to find. Figure 1: include vertical axes label. Line 228: explain OTSU thresholding. A graphical map of the methodology should be included. At the moment there is too much text and it is difficult to follow the thinking process. A sketch would improve the quality of the paper very much. Line 242: provide formulation for Euclidean distance. Same applies for all the paper: you say you are using certain formaulations, but you don't give many equations. Please include them. Same for figures, there is a general lack of figures in the paper. Include more example figures, please, and reduce the text where possible. Line 252: include image example of WorlView-2 images used. Lines 299 to 316: I believe this should be included in the asbtract, if not completely extracted from this section and exchanged by the abstract. Table 1: Include units for STD and RMSE. Highlight relevant values in table. Figure 2 (and similar figures): not so clear in B/W, please change colour scheme. Line 441: You said that ebfore, please shorten this section. This applied to the whole paper, quantity in text does not mean quality. Table 2: include units in RMSE and STD. Figure 6.a: scale is missing. Figure 6: can you increase the quality/contrast of the image?

 

Author Response

We thank the reviewer for their helpful and considered comments on the first draft of this manuscript. We have addressed all comments below, and feel the changes have helped produce a far more accessible and more streamlined study.

NOTE: Responses to reviewer comments are provided in blue. New or edit text is provided in bold.

 

The abstract is too long. Please reduce and focus on your innovation. 

Lines 299 to 316: I believe this should be included in the abstract, if not completely extracted from this section and exchanged by the abstract. 

We have rewritten the abstract to simplify its introduction, and lead more directly into the innovation of the paper. The new abstract now focuses 75% of the text on the innovation of the study, compared to ~50% of the abstract in the original manuscript. The abstract is now below the word limit for Remote Sensing (290 words). We have also moved the content previously at Lines 299-316 to the abstract, and removed this from the Results and Discussion.

Please follow the same reference style throughout the paper. In some lines you use the style (Author, year), and in others you use numbering [1]. Unify. When referencing, don't use "e.g.", just use the reference, and avoid using brackets if you are commenting the refernce. For example, in line 72, the correct reference format would be "(e.g. Landsat imagery available since 1972, [27])". References should be ordered numerically (if you are citing more than one, they should appear as [1,7,10,24], not [1, 24, 10, 7]. 

We have converted all author-year references in the manuscript to numbering style (e.g. lines 63-64 in the original manuscript), removed “e.g.” from in text references, and avoided using non-reference text within reference brackets. We have also renumbered all references to appear in order.

Line 122: say which high resolution imagery you will use. 

We have updated line 120-121 to read:

“We test the method using high resolution Worldview-2 imagery”

 

Equations should be numbered. 

We have numbered all equations

 

Line 143: BoA data? 

We have clarified that the surface reflectance data is atmospherically corrected at Lines 150-151 :

“This data is available as atmospherically and terrain corrected ‘analysis-ready’ data processed to surface reflectance”

Line 146: enumerate environments used to make them easier to find. 

We have rearranged the following sentence at Lines 152-155 and enumerated the environments to match with Figure 1:

We focused on five commonly studied environments to explore the influence of contrasting spectral properties on waterline extraction performance: a) sandy beaches [30,35,36,29,1,6,40], b) artificial shorelines [28,48], c) rocky shorelines [48,25,10], d) wetland vegetation [45–47], and e) tidal mudflats [43,44,7,10].

Figure 1: include vertical axes label. 

We have revised this figure to include a vertical ‘Surface reflectance’ label

Line 228: explain OTSU thresholding. 

We have moved the text previously located in the Discussion to Lines 238-239:

“This image processing approach identifies a threshold value that best separates a histogram of water index values into two distinct classes [e.g. water and land; 58].”

A graphical map of the methodology should be included. At the moment there is too much text and it is difficult to follow the thinking process. A sketch would improve the quality of the paper very much.

Same for figures, there is a general lack of figures in the paper. Include more example figures, please, and reduce the text where possible. 

Line 252: include image example of WorldView-2 images used. 

We have added two new figures to address the reviewer’s comments above: a new flowchart outlining the overall methodology (Figure 1), and a new image example of the WorldView-2 image analysed in the Real world application section of the paper (Figure 3).  This brings the total number of figures in the paper up to 10 including the graphical abstract

Line 242: provide formulation for Euclidean distance. Same applies for all the paper: you say you are using certain formaulations, but you don't give many equations. Please include them. 

In this case, we have opted to include a reference to the software used to calculate the distances in this study rather than the equation. We believe including a direct reference to the software provides the highest form of reproducibility by allowing future uses to replicate our approach exactly, especially for the large number of distance measurements involved in this study. Accordingly, we have updated Lines 252-253 to read:

“At 1.0 m intervals along the reference waterline, we calculated Euclidean distance to the nearest point on both the sub-pixel and whole-pixel waterlines using the distance method from the shapely Python package [57].”

 

Table 1: Include units for STD and RMSE. Highlight relevant values in table. 

We have included units for both these measures, and updated the Table 1 caption to read:

“Root mean squared error (RMSE in metre units) and standard deviation (in metre units)”

For both RMSE and Standard Deviation, we have also highlighted the best performance (i.e. sub-pixel vs. whole pixel) for each scenario in both Table 1 and 2:

“Bold cells for the RMSE and standard deviation columns indicate the best modelling performance for each scenario (i.e. sub-pixel or whole-pixel waterlines).”

Figure 2 (and similar figures): not so clear in B/W, please change colour scheme. 

We thank their author for this observation, which is also important for allowing colour-blind readers to better interpret our findings. Accordly, we have darkened the blue colours on all relevant figures, and lightened the orange sections. These two colours are now distinguishable in black and white:

Line 441: You said that before, please shorten this section. This applied to the whole paper, quantity in text does not mean quality. 

We have simplified this section at Lines 454-456 to read:

“By progressively spatially degrading high resolution (2 m) WorldView-2 remote sensing imagery, it was possible to simulate lower resolution satellite imagery and assess the impact of spatial resolution on waterline extraction performance in a more complex real-world scenario.”

Table 2: include units in RMSE and STD. 

We have included units for both these measures, and updated the Table 2 caption to read:

“Root mean squared error (RMSE in metre units) and standard deviation (in metre units)”

Figure 6.a: scale is missing. 

Please refer to the upper right of the now Figure 8 for the scale

Figure 6: can you increase the quality/contrast of the image?

In the original version of the figure, the background of the image was based on a semi-transparent greyscale version of the RGB Worldview-2 image as a compromise between showing the underlying data, and not creating an overly ‘busy’ figure that would distract from the waterline comparison. However, to make this figure more clear, we have replaced the greyscale image with a semi-transparent RGB version (as also recommended by Reviewer 2)

Reviewer 2 Report

The paper tilted “Sub-pixel waterline extraction: characterising accuracy and sensitivity to indices and spectra” by Bishop-Taylor et al. explored the accuracy of subpixel contour-based approach to delineate waterline in various beach environments. To simulate a “truth” high resolution images that have realistic reflectance properties representative of various beach environment, the authors extracted pairs of reflectance values from Landsat 8 images over Australia, and applied that to a synthetic shoreline. The authors tested how choosing different water index and various threshold values can affect the accuracy of the resulting shoreline location, assessed by the RMSE and standard deviation of the distance between the reference shoreline and the shoreline extracted from coarser images.

Assessed based on both synthetic landscape and a real world example. The manuscript presents a much-needed assessment of the contour-based approach in deriving subpixel waterline based on relatively coarse satellite images. And the conclusion of the paper and the discussion of the caveats could be useful extending to extracting subpixel boundaries between other types of features. The logic of the paper is clear and figures convey the points clearly. Overall, the paper is well written and the research well presented. The paper will be suitable for publication after the authors respond to the following comments.

Table 1 is informative but hard to digest? I would try to see if well-designed barplot(s) could do a better job at conveying the information.

In Figure 1 and lines 152–156: it seems that the boxed contains multiple pixels; what then were the values used in the synthetic landscape? Is it the mean for each band? It is unclear to me. 

Lines 241–242: Please clarify how the difference was calculated. The authors mentioned it was calculated at one meter interval, but how do you define “matching”?

In Fig. 2 and Fig.5 how was the positive and negative error defined? Was the negative or positive error biased landward?

In Fig. 2: Why there were two cases where there were no valid waterline?

Lines 375–382: although smaller, there are similar undulating patterns along the subpixel coastline in the case of the tidal mudflat in Fig. 3. If the undulating patterns in the case of sandy beach were caused by nonlinear response of the water index to land/water area ratio, then what could be the cause here? Also it is not entirely clear how the average spectral was calculated and visualized in Figure 4. Would be useful to have that in the methods section.

Table 2 is really useful for readers to quickly assess the potential error range when using the method proposed on images of various resolutions, and in a realistic setting, one would not know the truth to derive the “optimal” threshold. Thus, I would recommend using the “OSTU” based thresholds instead of using the “optimal” thresholds that replied on prior knowledge.

Lines 512–515: would it be worth experimenting on a Sentinel 2 image, via scaling up to a gradually coarser resolution and test whether the undulating pattern appears if AWEI is used?

In Figure 6: which band was shown in the background? Would be helpful to plot the color image or make the background image higher contrast. Right now it is hard to distinguish the waterline from the image.

Line 561: I could not find the code for this paper at the link provided by the author. It will be helpful to provide a specific address for this project.

Author Response

We thank the reviewer for their helpful and considered comments on the first draft of this manuscript. We have addressed all comments below, and feel the changes have helped improve a range of areas that were previously confusing or poorly explained in the first draft.

NOTE: Responses to reviewer comments are provided in blue. New or edit text is provided in bold.

 

 The paper tilted “Sub-pixel waterline extraction: characterising accuracy and sensitivity to indices and spectra” by Bishop-Taylor et al. explored the accuracy of subpixel contour-based approach to delineate waterline in various beach environments. To simulate a “truth” high resolution images that have realistic reflectance properties representative of various beach environment, the authors extracted pairs of reflectance values from Landsat 8 images over Australia, and applied that to a synthetic shoreline. The authors tested how choosing different water index and various threshold values can affect the accuracy of the resulting shoreline location, assessed by the RMSE and standard deviation of the distance between the reference shoreline and the shoreline extracted from coarser images.

 Assessed based on both synthetic landscape and a real world example. The manuscript presents a much-needed assessment of the contour-based approach in deriving subpixel waterline based on relatively coarse satellite images. And the conclusion of the paper and the discussion of the caveats could be useful extending to extracting subpixel boundaries between other types of features. The logic of the paper is clear and figures convey the points clearly. Overall, the paper is well written and the research well presented. The paper will be suitable for publication after the authors respond to the following comments.

 

Table 1 is informative but hard to digest? I would try to see if well-designed barplot(s) could do a better job at conveying the information.

We thank the reviewer for their comment, and agree that the table is difficult to digest. Table 1 is intended to serve as a counterpart to Figure 4 (previously Figure 2), which provides a more visual presentation of the same results. The intention behind including such a large table is so that we can directly reference exact numbers in the Results and Discussion section, while these would be difficult to present precisely in Figure 4.  In order to not introduce additional duplication of the results already presented visually in Figure 4, we have at this stage elected to not add an additional bar-plot figure to the manuscript.  We have however attempted to aid interpretability by bolding the best RMSE/Standard deviation for each corresponding sub-/whole pixel comparison, and included this in the Table 1 and Table 2 captions:

“Bold cells for the RMSE and standard deviation columns indicate the best modelling performance for each scenario (i.e. sub-pixel or whole-pixel waterlines).”

In Figure 1 and lines 152–156: it seems that the boxed contains multiple pixels; what then were the values used in the synthetic landscape? Is it the mean for each band? It is unclear to me.

To clarify this, we have updated the Figure 2 (previously Figure 1) caption, and the text at Lines 156-57:

“Figure 2 Paired samples of land and neighbouring water surface reflectance spectra from five contrasting environments along the Australian coastline (a-e). Spectra were extracted from cloud-free Landsat 8 OLI imagery from the Digital Earth Australia archive (Dhu et al., 2017; Lewis et al., 2017) by taking the mean value within the sample regions above for each band.”

“Paired samples of land and neighbouring water spectra were extracted from cloud free imagery along the Australian coastline by taking the mean value for each satellite band within each sample region (Figure 2). Imagery acquired at low tide was selected in the case of the tidal mudflat environment.”

 

Lines 241–242: Please clarify how the difference was calculated. The authors mentioned it was calculated at one meter interval, but how do you define “matching”?

We agree with the reviewer that this phrasing was confusing, and have revised the paragraph at Lines 249-253 to read:

“We evaluated the ability of our two extracted waterlines to reproduce the true waterline position by computing distances (errors) between the 1.0 m high-resolution reference waterline and each set of sub-pixel and whole-pixel waterlines. At 1.0 m intervals along the reference waterline, we calculated Euclidean distance to the nearest point on both the sub-pixel and whole-pixel waterlines using the distance method from the shapely Python package [57]”

 

In Fig. 2 and Fig.5 how was the positive and negative error defined? Was the negative or positive error biased landward?

We thank the reviewer for catching this omission from the methods. We have included an extra line at Line 253-256 which reads:

“Distances were assigned a direction (water- or land-ward offset from the reference waterline) based on whether the comparison point fell in a ‘land’ or ‘water’ pixel in the synthetic landscape (e.g. if the comparison point fell in a ‘land’ pixel, this distance was assigned a negative value to infer a land-ward offset)”
 

In Fig. 2: Why there were two cases where there were no valid waterline?

These cases occurred under 0 threshold scenarios for tidal mudflats environments for the MNDWI and AWEI water indexes. These indices both failed to separate water and land using a 0 threshold due to the similarity in surface reflectance values between turbid muddy water and wet intertidal mud-flat. These two classes could only be separated using either optimal or OSTU-derived thresholds. To clarify this, we have added the following to the Figure 4 (previously Figure 2) caption:

“Two tidal mudflat scenarios failed to extract waterlines due to the inability of a 0 MNDWI or AWEIns water index to separate turbid water from wet intertidal mud.”

 

This issue is also discussed at Line 414-416:

“In an extreme example, a ‘zero’ threshold for tidal mudflat environments completely failed to differentiate between land and water for both the MNDWI and AWEIns indices due to the spectral similarity of wet substrate with turbid water in the short-wave infrared satellite bands that were utilised by both indices ([19,20], Figure 4).”

 

Lines 375–382: although smaller, there are similar undulating patterns along the subpixel coastline in the case of the tidal mudflat in Fig. 3. If the undulating patterns in the case of sandy beach were caused by nonlinear response of the water index to land/water area ratio, then what could be the cause here? 

All results based on Normalised Difference water indices were characterised by non-linear responses to increasing proportions of land/water to different extents. This result was much more extreme when spectral contrast was high (e.g. sandy beach), but can still be seen in the slightly non-linear line in the low contrast wetland vegetation example in Figure 6. It is therefore not unexpected that at least some undulating patterns exist in the tidal mudflat lines, but these artefacts should be significantly reduced compared to higher contrast environments. We have adjusted the Figure 5 caption to better characterise the result:

“Sub-pixel waterlines for the AWEIns index and all water indices within low spectral contrast tidal flat environments more closely followed the reference shoreline shape with greatly reduced undulating artefacts.”

Also it is not entirely clear how the average spectral was calculated and visualized in Figure 4. Would be useful to have that in the methods section.

We have clarified that we used a weighted average to simulate the spectra of a pixel as the proportion of land was progressively increased (Lines 391-393):

“To visualise this, we can calculate an NDWI value based on the weighted average spectra for each satellite band as the percentage of land within the pixel increases from 0 to 100%.”

 

Table 2 is really useful for readers to quickly assess the potential error range when using the method proposed on images of various resolutions, and in a realistic setting, one would not know the truth to derive the “optimal” threshold. Thus, I would recommend using the “OSTU” based thresholds instead of using the “optimal” thresholds that replied on prior knowledge.

The primary purpose of the results presented in Table 2 is to show the absolute theoretical limits of the waterline extraction technique (even if these are unlikely to be achieved without knowing optimal thresholds ahead of time). Because of this, we will that ‘optimal’ thresholds are still of value to users, particularly because ‘OTSU’-based thresholds are likely to vary significantly between study areas. In order to not over-complicate Table 2 and to provide consistency with Table 1, we have opted to additionally include both ‘OTSU’ and ‘zero’ thresholds in a new Appendix A1 table. This allows users who wish to refer to ‘OTSU’ or ‘zero’ threshold results to obtain that information, while not creating another overly complex table in the body of the manuscript.

Lines 512–515: would it be worth experimenting on a Sentinel 2 image, via scaling up to a gradually coarser resolution and test whether the undulating pattern appears if AWEI is used?

We thank the reviewer for their suggestion - although this is out of scope for this current paper, we agree that this is a natural next step for this research and hope to continue our assessment of the accuracy and limitations of the technique in future work.

 

In Figure 6: which band was shown in the background? Would be helpful to plot the color image or make the background image higher contrast. Right now it is hard to distinguish the waterline from the image.

In the current version of the figure, the background of the image was based on a semi-transparent greyscale version of the RGB Worldview-2 image as a compromise between showing the underlying data, and not creating an overly ‘busy’ figure that would distract from the waterline comparison. However, to make this figure more clear, we have replaced the greyscale image with a semi-transparent RGB version.

 

Line 561: I could not find the code for this paper at the link provided by the author. It will be helpful to provide a specific address for this project.

We have updated the link to point directly to a page that directs readers to both the raw code, and several example Jupyter notebooks that demonstrate how the functionality can be used: https://github.com/GeoscienceAustralia/dea-notebooks/tree/subpixel_waterlines

 

Reviewer 3 Report

Sub-pixel waterline extraction: characterizing 3 accuracy and sensitivity to indices and spectra

by Bioshop-Taylor et al.

 

Accurately extracting waterlines are especially important to understand the processes of sea level rise in the coastal regions, as this study attempted using multiple satellite measurements. However, I would like to recommend this study to be published in Remote Sensing after improving three issues below.

Three major problems in this study:

In order to map the precise locations of waterline, authors SHOULD understand mean sea level.   Due to tidal changes, the waterline extracted based on satellite imagery are different at different tidal stage. Instantons waterline extraction would be fine, but waterline at the mean sea level should be desirable and more scientific. So, authors need to sort out the waterlines at the mean sea level and then estimate RMSE. Regarding RMSE compared to reference waterline, I could not follow how authors did. The reference waterline, again, should be at the mean sea level, and thus one can obtain real RMSE. Provide more information about reference waterline at mean sea level. According to different beach styles, one can have different results for waterlines. I think rocky beach is clear to identify waterline. However, wet tidal flat conditions are not straightforward because of wet send/mud hold water that making many uncertainties in spectral signals. Authors also need to discuss about this issue.

Author Response

We thank the reviewer for their time and valuable feedback on the first draft of this manuscript. We have addressed all their comments below.

Note: responses are provided in blue, with changes to the text highlighted in bold.

 

Accurately extracting waterlines are especially important to understand the processes of sea level rise in the coastal regions, as this study attempted using multiple satellite measurements. However, I would like to recommend this study to be published in Remote Sensing after improving three issues below.

Three major problems in this study:

In order to map the precise locations of waterline, authors SHOULD understand mean sea level.  Due to tidal changes, the waterline extracted based on satellite imagery are different at different tidal stage. Instantons waterline extraction would be fine, but waterline at the mean sea level should be desirable and more scientific. So, authors need to sort out the waterlines at the mean sea level and then estimate RMSE.

Regarding RMSE compared to reference waterline, I could not follow how authors did. The reference waterline, again, should be at the mean sea level, and thus one can obtain real RMSE. Provide more information about reference waterline at mean sea level. 

We thank the reviewer for their comments, and strongly agree that understanding mean sea level and tidal influences is critical to accurately modelling consistent waterlines through time and space. We see the satellite-based waterline extraction process broken into two primary aspects:

1. The ability of the selected waterline extraction algorithm itself to accurately and precisely reproduce true waterline positions under controlled conditions
2. Developing an experimental design where the above waterlines can be consistently extracted from satellite data by accounting for environmental and sensor-related factors such as tides, sensor noise, white water, cloud cover etc We see this study as addressing aspect 1 above: assessing the absolute performance of contour-based waterline extraction in both a synthetic and real-world experiment where tides and other environmental/sensor factors were kept constant. This allowed us to separate the performance of the algorithm from the influences of other important variables (like tide): In the synthetic synthetic landscape experiment, only underlying spectral characteristics of the landscapes were modified so that tides did not need to be accounted for in the RMSE results In more complex real world application experiment based on high resolution imagery, a single image was used to extract a reference waterline, which was then compared against waterlines extracted from the same image after it was degraded to lower spatial resolutions. This controlled for tide and enabled a like-for-like comparison between reference and lower resolution waterlines as all waterlines were derived from the same image. We hope this clarifies why waterlines in either experiment were not standardised to Mean Sea Level prior to calculating RMSE. For future work aimed at using the sub-pixel waterline method to analyse changing shorelines through time based on satellite data, we agree with the reviewer and strongly recommend that extracted waterlines should be normalised to a consistent datum (e.g. MSL or MHW) prior to calculating accuracies. We have added the following line to the start of the Methods section at Line 130-133 to clarify this to future readers:

“We conducted a synthetic landscape experiment to evaluate how accurately and precisely extracted waterlines could reproduce true waterline positions under controlled conditions, without the influence of environmental or sensor-related factors (e.g. tides, sensor noise, white water, cloud cover).”

 

This is also clarified for the real-world applications section in Line 287-290:

“To eliminate the confounding influence of tidal processes and coastal change between image acquisitions, we simulated lower resolution satellite imagery using spatially degraded versions of the WV-2 image itself rather than obtain co-incident imagery from other satellite platforms (e.g. Landsat or Sentinel 2).”

 

According to different beach styles, one can have different results for waterlines. I think rocky beach is clear to identify waterline. However, wet tidal flat conditions are not straightforward because of wet send/mud hold water that making many uncertainties in spectral signals. Authors also need to discuss about this issue.

We have added additional discussion around this point at Line 414-432 and the Figure 4 caption:

“In an extreme example, a ‘zero’ threshold for tidal mudflat environments completely failed to differentiate between land and water for both the MNDWI and AWEIns indices due to the spectral similarity of wet substrate with turbid water in the short-wave infrared satellite bands that were utilised by both indices ([19,20], Figure 4).”

“Two tidal mudflat scenarios failed to extract waterlines due to the inability of a 0 MNDWI or AWEIns water index to separate turbid water from wet intertidal mud.”

 

Round 2

Reviewer 1 Report

Comments have been addressed apropriately, and paper is now suitable for publication.

Reviewer 3 Report

Although the additional explantions for my first comment (regarding different waterlines due to diffeerent

tidal level) are not quite what the waterline extraction should be, this paper can be a good research

resource for some others to understand how the waterlines can be classified with different spectral

and spatial resolutions.