An Evaluation of Citizen Science Smartphone Apps for Inland Water Quality Assessment
Round 1
Reviewer 1 Report
This is an excellent manuscript which is well written and the topic is timely contribution. However, it needs some minor revision before being considered for publication. Below is my comment:
A location map (showing sampling points) is needed.
Water chemistry of the study area is needed in section 2.1.
A description of land use and land cover and nutrient export based on existing literature in the study area is needed section 2.1.
“rs” need to be subscripted in equation 1
Figure 3 title is not clear. Please clearly state what 3b represents.
Figure 4 title “best matches the photo of the water” : I think there is a good possibility the interpretation of best-match color can be biased by user’s instantaneous judgment. I wonder how many persons are there during the best-match color selection ? It will be great to have the opinion of multiple users.
Figure 13 needs more discussion in section 4. Based on supplemental Table 1, the performance of Hydrocolor is excellent in some sites while it struggles in others. Please label the points of figure 13 with an abbreviated site name (e.g. “st_clair”). In the cases of significant discrepancies, please provide more explanation where the method of Hydrocolor fails and how it can be improved.
Figure 14: Please use the same range of values in x and y axis.
Figure 14 needs more explanation in the text on why the hue angle is higher using Hydrocolor than Satlantic.
Author Response
Please see the attached document in which we respond to the comments from all three reviewers. We thank them for their valuable reviews.
Dear Editor,
We thank all reviewers for their time and efforts to improve the manuscript. All comments are addressed in detail, below.
Reviewer 1:
This is an excellent manuscript which is well written and the topic is timely contribution. However, it needs some minor revision before being considered for publication. Below is my comment:
We like to thank this reviewer for the kind words and suggestions to improve the manuscript. Below we address all points raised.
A location map (showing sampling points) is needed.
This map has been added and inserted as Figure 2.
Water chemistry of the study area is needed in section 2.1.
This is not relevant to the present study which is largely related to optical water quality parameters at the time of measurement and the testing of two smartphone apps. All optical water quality parameters are reported.
A description of land use and land cover and nutrient export based on existing literature in the study area is needed section 2.1.
This is added into Section 2.1.
“rs” need to be subscripted in equation 1
We made an error in the symbol for remote sensing reflectance, 4 lines above eq. 1. This is corrected and now have consistently used Rrs without the subscript throughout the paper.
Figure 3 title is not clear. Please clearly state what 3b represents.
Addressed.
Figure 4 title “best matches the photo of the water” : I think there is a good possibility the interpretation of best-match color can be biased by user’s instantaneous judgment. I wonder how many persons are there during the best-match color selection ? It will be great to have the opinion of multiple users.
Yes, we agree that further research could assess variability between the subjectiveness of different users. The results of the paper show that at least for two experienced persons the results of colour selection in the field is quite consistent. Remarks to this effect have been made throughout the paper.
Figure 13 needs more discussion in section 4. Based on supplemental Table 1, the performance of Hydrocolor is excellent in some sites while it struggles in others. Please label the points of figure 13 with an abbreviated site name (e.g. “st_clair”). In the cases of significant discrepancies, please provide more explanation where the method of Hydrocolor fails and how it can be improved.
We have completely revised the graph and revised the text in Section 3.5.
Figure 14: Please use the same range of values in x and y axis.
Addressed.
Figure 14 needs more explanation in the text on why the hue angle is higher using Hydrocolor than Satlantic.
Figure 14 (now figure 15 in the revised manuscript) shows that Hydrolor is higher than Satlantic hue angle only in approximately half of the meaurements; in the other half it is lower. The key issue here is why the hue angle would range over a greater range of angles in Hydrocolor than it does for Satlantic. The reason for this needs further thorough investigation and we have reflected this in the text.
Reviewer 2 Report
In recent year, citizen science for water quality monitoring has attracted much attention. Based on field data collected in Eastern Australia, this study evaluated two Apps for monitoring inland water quality. This is an interesting work for readers to know more about the applicability of smartphone Apps. However, the results are not so reliable and the discussions are not deep enough.
As indicated by the water-quality parameters in the supplementary file, these study areas are turbid、 shallow waters with much high concentrations of CHL, SPM, as well as CDOM.
In such turbid waters, I think it maybe not appropriate to measure the remote sensing reflectance with the method shown in Section 2.3.
I’m curious about the accuracy to derive the Lw (0+,λ) directly from underwater radiance Lw (0-, λ) which was measured by immersing the tip pf Satlantic spectrometer in the water. As we all know, in such turbid waters, light could be attenuated very quickly by water components and water itself, even passing through a thin layer of water. The attenuation coefficient also shows much spectral variability which will affect the spectral distribution of Lw (0+,λ), and Rrs. How to manage the detector to be just beneath the surface(0-)?
The discussions are not deep enough for us to know exactly the reason for these differences.
Generally, I think more description about the method to measure Rrs spectra are expected. This is the major point.
Author Response
Reviewer 2:
In recent year, citizen science for water quality monitoring has attracted much attention. Based on field data collected in Eastern Australia, this study evaluated two Apps for monitoring inland water quality. This is an interesting work for readers to know more about the applicability of smartphone Apps. However, the results are not so reliable and the discussions are not deep enough.
As indicated by the water-quality parameters in the supplementary file, these study areas are turbid, shallow waters with much high concentrations of CHL, SPM, as well as CDOM.
In such turbid waters, I think it maybe not appropriate to measure the remote sensing reflectance with the method shown in Section 2.3. I’m curious about the accuracy to derive the Lw (0+,λ) directly from underwater radiance Lw (0-, λ) which was measured by immersing the tip pf Satlantic spectrometer in the water. As we all know, in such turbid waters, light could be attenuated very quickly by water components and water itself, even passing through a thin layer of water. The attenuation coefficient also shows much spectral variability which will affect the spectral distribution of Lw (0+,λ), and Rrs. How to manage the detector to be just beneath the surface(0-)? The discussions are not deep enough for us to know exactly the reason for these differences. Generally, I think more description about the method to measure Rrs spectra are expected. This is the major point.
We thank this reviewer for carefully reading the manuscript and asking about the quality of the Satlantic reference measurements. Both at the design phase and execution of the campaign, great care was taken to have a measurement protocol in place. Quality control during the processing of the spectra, including removing outliers in a large (N= 70 to 90) set of repeated measurements at the same location, has been carried out. We did not include all of this information in order to keep the focus on the App performance.
Not all of the waterbodies chosen for sampling were turbid. The choice of waterbodies was deliberately from very low turbidity (e.g. Lake St Clair) through to very high turbidity (e.g. Brisbane River). The aim being here to cover as wide a range of optical water qualities as possible within a small group of waterbodies.
The main purpose of the spectroradiometric sampling was to establish an independent measure of the high spectral resolution reflectance to compare against the RGB reflectances measured by the Smartphone apps. In the paper, the principal basis of the comparison has been to extract hue angle from both spectra and the apps. Hue angle is largely driven by shape so the absolute magnitude of the reflectance is less relevant for these comparisons. In hindsight, we would have improved the spectroradiometric sampling, but at the same time we stand by the quality of the spectra obtained, as displayed in the updated version of Figure 6. We found the Lw measurements to be more stable than those made above the water surface. The derivation of Lw (0+,λ) from Lw (0-, λ) correcting for refraction effects is a relatively common method applied related studies.
There is no ‘best’ protocol with which to make these measurements. This is supported by recent work of Ruddick et al. 2019*, who reviewed methods in this field for irradiance and remote sensing reflectance, our methods largely follow the recommendations made in this paper and those of the earlier reviews cited in our manuscript.
Our method was adapted to the conditions of the field work, i.e. operating from a small boat in remote locations with minimal equipment to carry (mentioned in Section 2.3 already). Luckily, in nearly all situations we were able to make measurements under calm conditions. Therefore, the sensor could be suspended just underneath the surface of the water as close as possible to the surface. The measurements were repeated (nominally 70-90 measurements which were averaged, following exclusion of any obvious outlying spectra). These ensured we overcame some of the variation that occurs when making these measurements.
*Ruddick et al. 2019. A Review of Protocols for Fiducial Reference Measurements of Downwelling Irradiance for the Validation of Satellite Remote Sensing Data
over Water. Remote Sensing, 1742.
*Ruddick et al. 2019. A Review of Protocols for Fiducial Reference Measurements of Water-leaving radiance for the Validation of Satellite Remote Sensing Data
over Water. Remote Sensing, 2198. Remote Sens. 2019, 11, 2198; doi:10.3390/rs11192198
Reviewer 3 Report
Dear Authors,
I would like to thank you for the interesting work. Please find my comments and suggestions below.
General comments:
The work is presenting novel approaches to complement in situ water monitoring systems for Australian inland waters. It has high relevance, and clearly demonstrates emerging technological solutions.
Abstract
Line 15-17:
Can you please state here already the parameters that can be obtained by using the Apps?
EOW App: FU scale an indication to the visual appearance of the water surface
HC App: SPM
Line 28-30:
1) Is this consistent with the conclusions in Sec. 5 Line 539-541 and 543-548?
2) This study did not conduct comparison between in situ and remotely sensed reflectance.
Introduction
Line 103-104:
Is CHL, SPM and turbidity retrieval with the Apps thoroughly discussed? Line 408-418: Can you report the results in figures and elaborate further on them?
Materials and Methods
A map on the 13 sites would be nice to see. I suggest including the supplementary Table 1 in the article.
Is the footprint of the radiometric measurements discussed (20 cm above water)? What are the footprints of the Apps?
Line 119-122: Ok, but can you include a representative map indicating the location of the lakes?
Line 128: 500 ml
Line 131: -80 oC
Line 133: 80 ml
Line 141: 3 Hz
Line 186-188:
HC App: SPM concentration and turbidity:
Can you include the algorithms in addition to the references?
Line 219-220: EOW App
“…the eye of the observer to compare the actual colour of water with a set of pre-defined colours for water”: Can you say something about how this database of pre-defined colours of water was built up? Where the database comes from? What kind of water bodies are included…etc.?
Results
Figure 5: Can you please include legends (Site 1-13) and units for reflectance on y-axis.
Line 284-286: Please include a map and Table 1 and refer to that.
Line 289-291: How do you correct for these variations in height?
Line 360: Rrs (RGB)
Figure 9, 10, 12 and 13:
Can you include the 1:1 line?
Line 424: Rrs (RGB)
Discussion
Line 478-479:
This paper is not discussing the retrieval of CHL, CDOM and other water constituents from the obtained in situ measurements.
Line 501: Rrs (RGB)
Author Response
Reviewer 3:
Dear Authors,
I would like to thank you for the interesting work. Please find my comments and suggestions below. General comments: The work is presenting novel approaches to complement in situ water monitoring systems for Australian inland waters. It has high relevance, and clearly demonstrates emerging technological solutions.
We thank the reviewer for the attention and kind words and have addressed all suggestions to improve our manuscript below.
Abstract
Line 15-17:
Can you please state here already the parameters that can be obtained by using the Apps?
à EOW App: FU scale an indication to the visual appearance of the water surface
à HC App: SPM
These have been added.
Line 28-30:
- Is this consistent with the conclusions in Sec. 5 Line 539-541 and 543-548?
The text has been slightly adapted to make these comments fully consistent - please refer to the relevant sections in the paper.
Line 28-30 - we conclude that: 1) colour is a reliable monitoring parameter on its own, and; 2) tailored algorithms to convert remotely sensed reflectance and colour to composition should be developed for lakes individually.
Line 539-541 - This analysis has shown that water colour is very complex and that this App should not be used as surrogate for other water quality variables of interest which is not the intention of the FU scale.
Line 543-548 - The results for the HC App were less conclusive than expected. In these difficult circumstances with sun almost near zenith and a wide range of waters, from highly reflective (Brisbane River) to highly absorptive (Lake St Clair), the results were prone to errors. Before an assessment can be made of the applicability of the SPM algorithm in the App, it is essential to first reduce the errors in the retrieved Rrs. It remains important to take multiple measurements especially under cloudy conditions.
2) This study did not conduct comparison between in situ and remotely sensed reflectance. A question of pedantics, I guess. Deleted reference to ‘remotely sensed’.
Introduction
Line 103-104:
Is CHL, SPM and turbidity retrieval with the Apps thoroughly discussed? à Line 408-418: Can you report the results in figures and elaborate further on them?
Good point – this is toned down.
Materials and Methods
A map on the 13 sites would be nice to see. I suggest including the supplementary Table 1 in the article.
A map with the sites was also requested by another reviewer – this has now been added as Figure 2 in the manuscript. We prefer to keep the table as supplementary material; all readers can see and download the material.
Is the footprint of the radiometric measurements discussed (20 cm above water)?
this is done ~2.8cm
What are the footprints of the Apps?
Smartphone cameras all have a large Field Of View; the width in order of 60 degrees. The footprint depends on the number of pixels and height of the measurement. If we take 2048 pixels width and a height of 1-2 meters, each pixel covers on average over the image 0.1-0.2 cm at the water surface.
Line 119-122: Ok, but can you include a representative map indicating the location of the lakes?
A map with the sites was also requested by another reviewer – this has now been added as Figure 2 in the manuscript.
Line 128: 500 ml - addressed
Line 131: -80 oC - addressed
Line 133: 80 ml - addressed
Line 141: 3 Hz - addressed
Line 186-188:
HC App: SPM concentration and turbidity:
Can you include the algorithms in addition to the references?
We think the reference to the HC app (Leeuw and Boss, No. 32) are entirely sufficient should someone wish to access the algorithms.
Line 219-220: EOW App
“…the eye of the observer to compare the actual colour of water with a set of pre-defined colours for water”: Can you say something about how this database of pre-defined colours of water was built up? Where the database comes from? What kind of water bodies are included…etc.?
The brief derivation/history of the Forel-Ule index was covered in lines 71-76 of the Introduction. However, we have clarified here that we are referring to this index with respect to the EyeOnWater app. Further history is outlined in the given reference [40].
Results
Figure 5: Can you please include legends (Site 1-13) and units for reflectance on y-axis.
Because the reflectance is normalized to the Rrs at 550 nm, the parameter is a unitless measure, hence no units are required. A legend is now included.
Line 284-286: Please include a map and Table 1 and refer to that.
Map included as Figure 2.
Line 289-291: How do you correct for these variations in height? No change made. As with the measurement of spectral reflectance the height at which the measurements are made does not need to be taken into account using the assumption of a well-mixed surface water. Figures 6a-c confirm that the water colour is uniform save for the interfering features noted in the paper and evident in the photos.
Line 360: Rrs (RGB)
Addressed
Figure 9, 10, 12 and 13: Can you include the 1:1 line?
Added to all figures.
Line 424: Rrs (RGB)
Addressed
Discussion
Line 478-479:
This paper is not discussing the retrieval of CHL, CDOM and other water constituents from the obtained in situ measurements.
We agree, but we are simply here referring to the fact that CHL, SPM and CDOM are the parameters responsible for driving water colour measured by the apps.
Line 501: Rrs (RGB)
Addressed
Round 2
Reviewer 2 Report
After reading these two reviews by Ruddick et al. (2019) about the measurements of Lw and Ed, I insist that the sampling method of Rrs in this manuscript may not be appropriate, especially in turbid waters. Ruddick et al. (2019) showed us four broad families of methods to measure Lw, and all of them are different from the way used in this paper. Since my research is about ocean optics, I want to recommend authors to do a comparison to identify that these measurements would not bring much bias to the whole result.
This work is very interesting. Robust evaluation results are expected for readers to know the applicability of those Apps. By the way, I think the references about the methods to measure Rrs could be updated.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
