A Citizen Science Approach for Documenting Mass Coral Bleaching in the Western Indian Ocean

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

See attached Word file with comments.

Comments for author File: Comments.pdf

Author Response

Thank you for taking the time to carefully review my article, and I apologize for the confusion. It was written for coral scientists and so includes a lot of methods details and data to show that the AI is working properly. I realize this could be confusing to someone from another field. Same for the DHW data; it's really only something coral scientists use though in that case, it is fairly easily interpretable: the higher the DHWs the worse the bleaching (normally). I have also taken the vast majority of your other suggestions, including adding a map and removing a severely bleached reef image, which evidently did not render properly upon article upload. Also note that the methodology upon which this article is based in now "in press." Nevertheless, I have re-written out most of the details so that the reader doesn't have to constantly go back and forth between two articles. In closing, thank you again for your constructive comments and hopefully it is now somewhat clearer what I set out to achieve.  

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This manuscript presents a way to measure coral bleaching along Kenya’s coast using an AI image recognition model trained on an impressive number of images and annotated points taken during citizen science dive surveys. The model estimated that overall bleaching was over 90% with an overall accuracy of 84%. The false positive rate of 24%. The study also investigates the relationship between observed bleaching and temperature stress metrics that are publicly available, confirming that this positive correlation holds for Kenya, though it only explained approximately 1/4 to 1/3 of the total variation driving percent of corals bleached. The results are useful for documenting bleaching magnitude over time with a data collection method that can be deployed opportunistically over a wide area.

This is a worthwhile study, though the manuscript organization needs improvement. I spent a lot of time flipping between sections trying to find critical details about the analyses. I have outlined major issues below:

The Methods section needs additional details on the model used for correlating bleaching prevalence and degree heating weeks. There is also no information in the methods section on the metaanalysis or how the historical bleaching data was used to assess trends in coral cover over time. This is a critical oversight.

Knowing how much recently dead coral cover is at a site seems extremely useful for management. The manuscript reports on percent live coral cover without reporting on dead coral cover. Was this because of the low number of CoralNet tags for dead coral? If there is a sufficient number of tags, analysis should include regression between DHWs and percent dead coral cover as another metric of the impacts of thermal stress.

The model has done a great job of minimizing false negatives. However, the false positive rate, while not out of line with image recognition studies of natural phenomena, is high enough to warrant adjustments of the estimates of bleaching. I am wondering how much this stems from an imbalanced number of labels across categories. The labeled data shows that there are ~2.5x more bleached coral tags than healthy coral tags, so by chance alone the model would be more accurate in assigning a bleached rather than healthy label to any given point. One possible solutions is raising the threshold of confidence for assigning a bleached label compared to a healthy label.

While I am familiar with the opportunistic nature of citizen science and its limitations for standardizing dates of sampling, the author does not address the effects of time and accumulation of degree heating weeks in their comparison of bleaching across sites and regions. Because the sampling was conducted as DHWs were rising, it is unclear whether differences between sites are true spatial differences or differences due to these other variables. The author could add time or DHWs as a term in their comparisons to account for this.

The author suggests that tourist sites have different levels of bleaching compared to non-tourist sites, but stops short of offering evidence even though there are enough data to compare them. The first time this point is encountered is near the end of the paper. The author notes that this has substantial potential to bias the bleaching estimates, and since the data exist to investigate this, the author should. A comparison of false positive rates between tourist and non-tourist sites would improve confidence in the CoralNet classification and elucidate whether this source of bias impacts the reliability of the algorithm’s labels. A comparison of bleaching percentages between tourist and non-tourist sites would also clarify whether there is potential for bias in bleaching estimates based on site type.

Additionally, there are a lot of details in the figure captions that should be in the main text, including statistical results. I found it difficult to follow the results when key information was in the figure caption placed several pages later.

 

Minor comments

Throughout: “Note that” is overused and often can be dropped without changing the meaning of the sentence.

L13. “Since this bleaching event took my Kenyan colleagues off-guard, I sought to aid them…” I can appreciate that this study was a spontaneous response to an emerging event, but the language here feels colloquial.

L14-16. The description of the study area could be made more concise here, e.g. “the majority of the Kenyan coastline” or “over 90% of the Kenyan coastline”

L23. The final sentence of the introduction does not offer much insight. 1-2 sentences on the broader implications of this study for management or the potential of this citizen science monitoring approach would be more appropriate. Does CoralNet have the potential to allow for widespread monitoring of Kenyan coral reefs? How could the results from this study inform reef management in Kenya?

L45-48. Reword to make divemasters and scientists the subject of this sentence.

L60-61. This is the first time that the methods of citizen science and AI are mentioned. Some background on how it is already being used to address reef bleaching in the introduction would provide useful context.

L64. A brief paragraph on the study setting is needed. What are key details about Kenya’s reefs that the reader should know?

L79. Change “<25 m” to “at <25 m depth”

L82. Mention the resolution of the NOAA Coral Reef Watch data. Was this the 5km raster data or was it data from a virtual station?

L96. In the image tagging process, how were dead corals (which presumably died from bleaching) distinguished from bleach corals that were still alive? This is an important distinction with major implications for the results.

L109. Delete “perhaps”

L121-124. There is no need to mention the sources that were not used if there were satisfactory results from the author’s own image tagging efforts.

L134. What was considered single replicate here? Site or image?

L138. More detail is needed to describe this model. A stepwise regression is mentioned in the results without being described here first. Additionally, please include how the different regression models were compared and what metric informed the final choice of terms to include (e.g. continuing to add or drop terms until R² was maximized or BIC was minimized/equivalent).

L139. More detail on spatial and temporal aspects of the DHW data are needed. Were the DHW data based on the pixel closest to the site, an average of pixels covering only dive sites, all pixels on the Kenyan coast from northernmost to southernmost site, or data from a virtual station? Likewise, what times do the DHWs represent? The date of survey, an average across all survey dates, or something else? Are these daily data or weekly/monthly composites?

L151. Table 1 Caption. For sites with multiple survey dates, do the temperature data represent the mean of all dives?

L163 Table 2. This is an impressive number of features scored.

L165-171. Figure 2 caption. “Although not evident…” This text can be moved to the methods text and reference the figure to describe the circumstances under which a point was labeled “unknown.”

L174. Remove “only” to avoid editorializing when first presenting the results. In a combined Results and Discussion section, commentary like this can be presented later in the paragraph as part of the interpretation.

L185. Is the 24% false positive rate in line with other image classification studies?

L194. Combine section 3.5 with this one for better continuity.

L202. Add standard deviation here. Were the DHWs consistently lower or was the difference variable?

L203. Although DHW temperatures frequently differ from in situ measurements in many places worldwide, the DHWs are still often linked to bleaching outcomes, so even if the number of degrees experienced is not the same as at depth, sea surface DHWs are not necessarily an underestimate of thermal stress, just one with a different intercept.

L205. Was this the highest value in the study area or globally?

L206. Was does “Temperature” refer to in situ temperature or satellite estimated?

L213. The manuscript was using “I” and switched to “we” here. I think either are acceptable for a single-author publication given the nature of citizen science work, but please be consistent.

L219. Use “and” instead of “&” .

L224. I am concerned that temporal variability and thermal stress are not accounted for when testing for effects of site. Temporal effects are not discussed in the corresponding part of the Methods section either. Since DHWs are increasing as the surveys progress, can DHWs and/or day of year be added as independent variables?

L235. “Projections” These aren’t projections. They are reference levels based on historical climatology.

L236. Missing parentheses around (b).

L238-240. “Surveys ranged from… recorded for Kenya.” This can be moved to the results.

Figure 3. What is “HS” in the right-hand vertical axis label? Define in the caption.

L247-248. It is unclear what calculations are being referred to here.

L249. The meta-analysis needs to be described in the methods.

L249-284. These paragraphs contain a lot of detail but lack unifying topic sentences or final sentences to communicate the overall message. It is easy for the reader to get lost in the amount of detail without knowing what is the important takeaway here.

L252. “the results shown in Figure 7…” The results of the regression over coral cover over time are not discussed in this paragraph. This section discusses the increases and decreases

L271-275. This sentence is long and hard to follow. It switches between % coral cover and % loss, and it is not always clear which the author is referring to.

L280. Capitalize “although”.

Section 3.5. This whole section could be folded in with section 3.1.

L286. “Recall that” is not necessary.

L301. The influence of tourist sites on false positive rate should be explored further. Since half the sites were not tourist sites, there appears to be sufficient sample size to analyze whether the false positive rate is significantly different. This also has implications for bias of future citizen science surveys. If citizen scientists go to more popular or healthier dive sites because they provide a better experience, then the overall picture of the reef health will be biased. This is worth discussing in terms of broader implications and utility of citizen science data collection.

Figure 4. This figure could go in Appendix A with the temperature data. What do the red lines represent? In the axes labels, change “actual” to “in situ” to be consistent with the caption.

L324. I would not use the % health coral cover assessed against the entire benthos because each site is going to have its own baseline of total coral cover, confounding spatial effects and DHW effects. Instead, present the slope of the regression between % bleaching and DHWs here. That is stronger evidence and is controlled for overall coral cover.

L338. This is the first time depth bins are mentioned. Please describe the rationale for binning them this way in the methods and describe how they are incorporated into the regression.

L339. To round out the discussion section, a paragraph on the pros and cons of this approach and more generally a citizen science plus AI approach to coral bleaching monitoring would help situate the results in the broader context of how we respond to coral beaching. How does the accuracy of the approach here compare to other image-based monitoring programs? Is the CoralNet approach good enough to provide data for specific management or monitoring objectives in the region?

L345-346. Figure 5 caption. While it’s fine to include the p and t values in the figure itself, these details belong in the text of the results section.

Figure 5:

• Add X-axis label: “Site (in order of survey date)”
• The inset font is quite small. Perhaps adjusting the right hand axis of the main graph to start at 0 would allow more room for the inset to enlarge.
• Are the DHWs site-specific or means across all site on the printed date?
• Make the DHWs a line graph for each date of survey instead of having floating text.

L349. Figure 6 caption. Specify that the units for healthy coral cover are also percent.

L350. “in panel (a)” is missing parentheses.

L353-354. Save presentation of the means for results text or draw lines in the figures to signify the mean if it is important to have the information present in the figure itself.

L355-356. The content in parentheses is better placed in the text of the results, not the figure caption.

L356. Remove “have also been used to”. The meaning of the sentence is the same without it.

Figure 6a. Legend titles could be made more consistent. Use the same notation for percent healthy coverall cover and percent bleached coral.

L369-370. Figure 7 caption. This information goes in the results.

Figure 7. The declines look like it is largely driven by 2024’s surveys. Does this result still hold if that year is taken out? The text does not discuss the significance of this trend and instead focuses on the fold-difference between the 1990’s and now, so I am wondering if this figure is necessary, especially since the past bleaching data is already presented in Table 3.

L374-377. Figure 7 caption. “Obura estimated … described herein” could be moved to Results/Discussion.

L385-389. Comparison of false positive rates (and false negative rates) for tourist vs non-tourist sites would provide more evidence to support or refute this speculation.

L426-428. “Although not statistically… at greater depths.” Move to the results text.

Author Response

Thank you so much for your detailed, insightful, and helpful review of my article. Not having any co-authors in this case means peer review is perhaps even more important than ever. I believe I have addressed nearly all of your concerns, many of the more significant ones having been shared by reviewer#1 (e.g., failure to really justify the method, failure to leave the reader with take-home messages, and the lengthy, detailed nature of the figures and tables). In all honestly, I am not Mr. "Citizen Science will be the panacea coral reefs need," so I had taken a very measured, science-based approach; did this project yield valuable data? Yes? Is it as good as a 20-person surveys by PhD scientists? Probably not. Is the AI perfect and ready for widespread use? Probably if you do not need species-level resolution. In other words, I am still trying to sell myself on the method and was actually more interested in the severity of the bleaching. I have never seen anything so shocking, and I have been studying bleaching for 20 years. Of course, I don't want personal sentiments to get in the way, but really the take-home message is "oh my god, that was a catastrophic bleaching event" more so than it is my telling the coral world that we should all do citizen science. 

In any event, your comments and questions have surely improved this article, and I appreciate your efforts in helping me out once again. Please see the attached document for my detailed responses. I think the only one I didn't take for now is combining the sources of error into a single section because I now talk about non-AI related error in that section. However, there may still be a way to break it up into an AI error section and a survey/surveyor error-related section that comes later, so I will be keen to see what you think. 

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The manuscript deals with the combination of AI tools with citizen-science (massive imagery taken from non-scientist divers) with the overall aim of proposing, and applying, a method for contributing to coral reefs monitoring with reference to the effects of marine heatwaves. The idea is interesting and could provide progress in the topic and I thank Authors for the effort. Despite that, the present form of the paper does not clearly define the main goal of the manuscript; the novelty as well as the contribution to scientific progress are also lacking. Moreover, the manuscript has, formally, the scientific structure but it does not explain the fundamental details that are needed for the reader to replicate and apply the method. The main purpose is also confused: is the manuscript intended to focus on ecology results (then a deeper analysis and discussion of the impact for coral colonies could be expanded) or on methods (then a more detailed technical description of the approach is required to allow reproducing it)? Results need to be deeply discussed with reference to the potentiality of the citizen-science AI-based approach proposed whereas more concern is given to listing the results in terms of bleaching evolution. These are of course important but, if the focus is on ecological results, major work is required to contribute to scientific progress on this topic, which could be a discussion on the impact and consequences of these modifications.

I have further specific suggestions but I would like to focus on three main concerns: 1) if whatever tourist divers shared their unstructured image sets, how is it possible that all divers have the same camera model? There was a sort of light instruction kit to those interested in contributing to science analysis? 2) here there is the proposal and application of a method based on citizen science approach + AI tool which can be interesting and potentially useful but there is no validation given nor attempt in doing it. Maybe references [11] is able to give an answer in this side but it is still under review thus not useful for the reader. Lines 305-310: this comment is fine in a massive and 'global' way. An experiment/research that proposes an innovative approach based on technologies must instead be validated by proposers, at least in a test area (with eventual limitations that is sufficient to list for completeness and replication). This comment shows the weakness that the work has, in the present form, from a scientific point of view. 3) Lines 286-292: is such a ‘big’ error acceptable? what is the purpose of doing analyses of this type with such big errors? would it not be more appropriate to check with ‘gold standard’ first? for me the whole methodology is not assessable until validated (maybe until [11] reference is published that is declared to validate the approach). At the same time, in my opinion to publish the same approach (furthermore unvalidated), just consisting in applying an existing image processing tool, in different sites (as intended by titles of [11] and [13]) is scientifically questionable and not scientific significant from the method side. If ecological site-specific results has a special meaning this should be discussed as the main goal of the paper (and declared at the beginning) with still a contribution to the progress of the science (for instance combined to predictive models of ecosystem evolution or I do not know what could be the case).

Finally, I suggest to Authors to focus more on the scientific significance of the work in order to actively contribute to the progress of the topic rather than increasing the number of publications (references [11] and [13]) with the same or similar approach (I imagine) applied in different test sites (Remote Indian Ocean, Red Sea, Western Indian Ocean). I would suggest at least waiting for the publication of reference [11], if it provides validation of the approach and later propose publications with different site-specific ecological results. With this idea, the present form of the manuscript could be revised and resubmitted.

Author Response

Thank you for taking the time to carefully review my article, and I have done my best to attempt to address your concerns. Alongside the comments and suggestions of the other reviewers, I hope you will now see a superior-quality article. Briefly, I was trying to fill a critical knowledge gap in bleaching in the Western Indian Ocean. This is the 3^rd article on this approach, so the novelty is not on the method, but on the bleaching data. For this reason, I have provided better justification in the Introduction for the approach, and I have written more about temporal dynamics of bleaching in the region by comparison to other articles. I did not explain this meta-analysis goal in the prior methods, so now have included a new section on this. My goal is to have about half of the article on the use of citizen science and AI, so I have greatly expanded both the Introduction and the Materials and Methods. I then dedicate the other half to gauging changes in coral cover over space and time at the survey sites by comparison to other studies in the area (i.e., a meta-analysis). 

Please see the attached document for my detailed responses or, more importantly, checked out the heavily revised article. 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for allowing me to review the resubmitted manuscript: A citizen science approach for documenting mass coral bleaching in the Western Indian Ocean for the journal Environments.

The author has made substantial improvements to this manuscript and I believe it just needs some minor revisions for publishing. See comments below:

• An inset site map has still not been provided showing the surveyed region within the map of Africa. I believe it would provide better spatial context. Scale bar would be helpful as well.
• Accuracy of the AI still needs clarification as to the 84% accuracy, but then up to 24.2% false result. They do not add up for my understand, so please clarify this.
• Figure 5 – still need to further explain the lower/upper case letters in figure b stacked column
• Use of “fold” change – this would be much clear as a percentage change, especially for decreases which makes no sense as a fold change.
• Table 1 – put units in top row as needed
• Where is supplemental figure S1 in Appendix B (will need to explain that figure if used)
• Line 110 – says maximum of 40m but table 1 only has down to 26m maximum depth
• Line 122 – says 25 m , should be 26m according to table 1
• Figure 1 header after the image, other figure headers are before the image
• Table 2 into the supplementary information
• Clarify a little earlier in the methods that these analyses only cover scleractinian corals
• Abbreviations – check the list, missing some... NOAA, WIO, NR, GCRMN, CORDIO and others...

Author Response

Thank you for taking the time to review my article again, and specifically for making further suggestions to strengthen it. I have modified Figure 1, clarified a few things as requested, and made the remainder of the suggested changes. Please see the attached document, as well as the revised manuscript itself, for details. The only change I did not make is the moving of Table 2 into the supplement because it will be important for showcasing the strengths (and weaknesses+limitations) of the AI. However, if the editors and other reviewers feel like it is distracting, I can consider to make this change. Of note, it shows that the overall accuracy (84%) and false-positive rates are different; the former is for all 17 labels. The latter is just for bleaching vs. healthy corals specifically. 

Thank you again for the time you took to review this article twice. It is certainly more readable and better presented from having taken your suggestions. 

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

The paper is much improved and reads better. I appreciate the author's incorporation of the prior suggestions. The additional details on why depth binning choices were made and the additional analysis comparing tourist to non-tourist sites help to bolster the paper. A few minor issues remain, and once these are addressed, then I would recommend this paper for publication. I commend the author for creating and testing this useful system for citizen science monitoring of coral bleaching.

L84-86. Wording of this sentence is a little awkward. I suggest making it more concise.

L97. What does “framework” refer to here?

L182. A description of the statistical tests for comparison of tourist and non-tourist sites is currently absent from the Methods and is needed in this section. (It can be brief.)

L198. “lone virtual station…” Cite the specific virtual station page here.

L438-440 and 443-444. This is text belongs in the Methods section. Limit the details about this in the Results/Discussion section to describing and interpreting the results of this test.

L445. “Wilcon test” Do you mean Wilcoxon?

Figure 4. This is a good improvement to the main graph. It’s now clearer how the DHWs are correlating with the bleaching percentages. I think the vertical axis on the inset graph may have been cut off. The top number is missing and the points are at the very top of the frame.

L533-536. This is still a possibility if the sites surveyed are not representative of the typical range of conditions at tourist and non-tourist sites, but the additional analysis of the two site types demonstrate that they are similarly impacted, so assuming that the sites surveyed here are representative of their respective types, this is less of a concern now.

L547-557. "Two further drawbacks..." I appreciate the addition of this topic to the Conclusion section. However, these sentences provide too much fine detail for the final paragraph, and introducing new drawbacks as the final big idea is not the ideal parting message for the reader. These would be better if woven into the Results/Discussion, section 3.5.

Author Response

Please see attached rebuttal letter (#2). Briefly, thank you so much for checking this over again, which has been especially important given that I have no co-authors. I believe I have addressed all comment except for one on moving methods from the results section to the methods section, not because I don't think it's a good idea but because those line numbers don't exist in my version of the last document submitted to MDPI, so I instead went through the Results and Discussion and moved any "methods" text to the methods. Of note, the meta-analysis description was previously mentioned in both the Methods and again in the Results and Discussion, so I have removed it from the latter. I also moved the "caveats" section to the section where I discuss sources of error and drawbacks, not at the very end (which would have been a terrible take-home message!). Thank you again for your efforts in improving this article. 

Author Response File: Author Response.docx