Coral Reef Benthos Classification Using Data from a Short-Range Multispectral Sensor

Round 1

Reviewer 1 Report

Review of “Coral Reef Benthos Classification Using Data From a Short-range Multispectral Sensor.”

This manuscript reports the results of using a small, 5-band, multispectral imaging sensor to acquire and classify images of corals and other reef-associated taxa. The topic is highly relevant, the writing is clear (pending a couple of clarifications, discussed below), the background and context are well researched and described. Overall, I think this is worth publishing, but I suggest three revisions.

The first revision should be an adjustment to the accuracy assessment (if I understand correctly the method used). To review, this is how the classifications were done. Three images were classified, which were two subsets of mosaic B (B1 and B2) as well as the entire mosaic B (BFull). The approach in each of these 3 cases was to hand-draw 40 polygons on the image, use 20 polygons as training data, run the supervised classification (actually several algorithms were tested but the same polygons were used for all of them), and evaluate the other 20 polygons for assessment.  The assessment was done with error matrices, overall accuracy, kappa and tau coefficients. (As an aside, putting the error matrices in the supplementary information is very appreciated! It would be even better to add one for BFull also.)  Sub-image B1 had 12 benthic classes, B2 had 8 classes, and BFull had 15 classes. Unfortunately, there is one major problem with the way these data were used for accuracy assessment. The problem is that the sampling unit used in the error matrices was pixels, but the sampling unit should have been polygons. The assumption used in the overall accuracy, kappa, tau type error analysis approach is that the samples used to construct the error matrix are independent. But the pixels all within one hand-drawn polygon are NOT independent samples because of spatial autocorrelation. Typically, for accuracy assessment, one would put random, or stratified random, points across the image and assess the classes at those independent points. If you randomly sample individual pixels like this, then, yes, the entries in the error matrix should be counts of pixels. But from what I can tell this sort of random sampling is not how it was done in this paper. It is OK to use polygons as sampling units and it is also OK to fractionally distribute them in an error matrix as a sort of fuzzy assessment but when doing so, the sum of the number of samples in the error matrix needs to be the number of polygons (20, in this case) not the number of pixels (6089 for B1, 2526 for B2, unknown number for BFull). Twenty independent points is too few to generate an error matrix for 8, 12 or 15 classes. Therefore my first recommendation is that the error matrices be recomputed with a sufficient number of independent samples.

The second revision has to do with the spectral bands. From what I can tell from the MicaSense web site, the spectral bands for their cameras are fixed, and in general these are tuned for agricultural remote sensing. It would be better if we could pick our own bands and use spectral regions identified years ago by Hochberg, Holden, and others to be particularly relevant for coral reef remote sensing. I understand this is not feasible, but at least the manuscript should discuss this issue. Moreover, the channels that were used were red (R) green (G) blue (B) plus red edge (RE), which is the very far red and very near infrared. This is basically a regular camera (RGB) plus one extra spectral channel. However, it is astonishing how much better the accuracy of classification is with this data than with other approaches that just use regular RGB cameras. The interesting question is, why? Is it because the spectral bands are narrow? Is it because of the extra RE band? Could it be an artifact of spatial autocorrelation in the error matrix? It would add a lot to this paper to sort these questions out and that could be easily done by repeating the classification experiment but dropping out the NE band. If the results are basically the same with RGB as with RGB+RE then that supports the idea that narrow bands are the key. If the results are much worse with RGB than with RGB+RE then that supports the idea that the RE channel contains a lot of discriminatory power. If the results change with better sampling to create the error matrix then that supports the idea that it is an artifact.

A third revision is slight adjustment of the way reflectance is discussed. For natural illumination, normalization to reflectance is susceptible to clouds, wave refraction as noted (line 420) but also self-shadowing by the instrument and diver. For artificial illumination, these factors will be reduced for deeper sites but never totally eliminated (except at night). Artificial illumination introduces another complication, however, which is non-uniform distribution of lighting across the field of view. It may look uniform but it’s not. Moreover, for both natural and artificial lighting, even tiny variations in distance between the camera and substrate (due to diver motion or 3-D structure of the seabed) are critically important, particularly in the longer wavelengths (> 500 and certainly > 600 nm).  Computing reflectance the way it was done in this project (by reference to the calibrated reflectance panel – lines 138-140) gives an approximate reflectance but it is important to note that this reflectance is not corrected for variable lighting and sensor-seabed geometry (i.e. distance variations between the light-seabed-sensor that are different from what it was when measured with the calibrated panel).  These caveats should be noted.

Finally, very minor: Figure 6 is a histogram not spectral curves. Also, make labels bigger.

In summary:

1.       The accuracy assessment has to be fixed to incorporate sufficient independent samples.

2.       It would be ideal to add a simple experiment to try to understand how much the RE channel is adding. In any case it would be good to try to explain why this system works so much better than a regular RGB camera.

3.       It should be noted that reflectance is not compensated for variable lighting nor light-target-sensor geometry.

Author Response

Reviewer 1

Authors' responses are underlined

We sincerely thank and appreciate the thorough revisions and constructive comments by the reviewers; which -we feel- have enhanced enormously the clarity and potential of this manuscript.

Review of “Coral Reef Benthos Classification Using Data From a Short-range Multispectral Sensor.”

This manuscript reports the results of using a small, 5-band, multispectral imaging sensor to acquire and classify images of corals and other reef-associated taxa. The topic is highly relevant, the writing is clear (pending a couple of clarifications, discussed below), the background and context are well researched and described. Overall, I think this is worth publishing, but I suggest three revisions.

We appreciate the reviewer’s kind comment, and we look forward to fully address the suggested revisions.

The first revision should be an adjustment to the accuracy assessment (if I understand correctly the method used). To review, this is how the classifications were done. Three images were classified, which were two subsets of mosaic B (B1 and B2) as well as the entire mosaic B (BFull).

Yes, that is correct.

The approach in each of these 3 cases was to hand-draw 40 polygons on the image, use 20 polygons as training data, run the supervised classification (actually several algorithms were tested but the same polygons were used for all of them), and evaluate the other 20 polygons for assessment.  

No, that is not correct. To clarify: Initially 40 polygons were hand-drawn for each class (benthos component) identified in each subset/image. 20 polygons per class were used as training fields, and the other 20 per class, were used for the accuracy assessment. This was edited in lines 216-221.

The assessment was done with error matrices, overall accuracy, kappa and tau coefficients. (As an aside, putting the error matrices in the supplementary information is very appreciated! It would be even better to add one for BFull also.)  

Thanks for your comment. The absence of B Full’s confusion matrix was an unfortunate oversight when we attached an erroneous version of the supplementary materials. In the original version submitted there were mistakes even in the names of the classes. We have uploaded de correct version of the Supplementary Materials with all proper and updated data.

Sub-image B1 had 12 benthic classes, B2 had 8 classes, and BFull had 15 classes. Unfortunately, there is one major problem with the way these data were used for accuracy assessment. The problem is that the sampling unit used in the error matrices was pixels, but the sampling unit should have been polygons.

Yes, we understand your point, we used the “Confusion Matrix Using Ground Thruth ROI” tool in ENVI 5.3, which can use points (single pixels) or polygons (clusters of pixels), but in the end it assesses each individual pixel from these ROIs vs. the corresponding pixels in the classified product. As a note, we have updated the classification scheme and number of classes in subplot B II and B Full (Figures 8 and 9, Tables 4-7).

The assumption used in the overall accuracy, kappa, tau type error analysis approach is that the samples used to construct the error matrix are independent. But the pixels all within one hand-drawn polygon are NOT independent samples because of spatial autocorrelation. Typically, for accuracy assessment, one would put random, or stratified random, points across the image and assess the classes at those independent points. If you randomly sample individual pixels like this, then, yes, the entries in the error matrix should be counts of pixels. But from what I can tell this sort of random sampling is not how it was done in this paper. It is OK to use polygons as sampling units and it is also OK to fractionally distribute them in an error matrix as a sort of fuzzy assessment but when doing so, the sum of the number of samples in the error matrix needs to be the number of polygons (20, in this case) not the number of pixels (6089 for B1, 2526 for B2, unknown number for BFull). Twenty independent points is too few to generate an error matrix for 8, 12 or 15 classes. Therefore my first recommendation is that the error matrices be recomputed with a sufficient number of independent samples.

We fully understand this concern. Originally, we used 40 hand-drawn polygons as training fields for each defined class, on each image/subset, i.e. Subplot B I has 12 classes, there were 480 polygons, 240 for classification and 240 for accuracy assessment. We assumed that the use of 20 polygons per class for accuracy assessment did have a statistical representativity, but we totally agree with the spatial autocorrelation of the samples, thank you for noting this. So, we kept the 20 polygons per class for the training and classification of suplots B I and B II, and set a stratified-random pixels procedure for recomputing the error matrices. A clarification was addressed improving the accuracy assessment description (lines 242-245, 374-381).

For the classification of plot B Full we changed the polygons and used points (7,993 individual pixels) for training the 14 classes, so the accuracy assessment was based on similar inputs: 9,375 individual pixels from a random stratified selection (lines 410-411).

The new accuracy values are reported in the updated Tables 4 and 6. Consequently, the Z-test was also updated (Table 7), and the updated confusion matrices are reported on the new version of the Supplementary Materials.

The second revision has to do with the spectral bands. From what I can tell from the MicaSense web site, the spectral bands for their cameras are fixed, and in general these are tuned for agricultural remote sensing. It would be better if we could pick our own bands and use spectral regions identified years ago by Hochberg, Holden, and others to be particularly relevant for coral reef remote sensing. I understand this is not feasible, but at least the manuscript should discuss this issue.

Yes, unfortunately the sensor has fixed spectral resolutions defined specifically for a very different application to the one we used it for (although in the end we are looking pretty much at photosynthetic pigments both in the zooxanthellae and in the benthic algae). We agree with the need for further discussion (added in lines 614-621).

Moreover, the channels that were used were red (R) green (G) blue (B) plus red edge (RE), which is the very far red and very near infrared. This is basically a regular camera (RGB) plus one extra spectral channel. However, it is astonishing how much better the accuracy of classification is with this data than with other approaches that just use regular RGB cameras. The interesting question is, why? Is it because the spectral bands are narrow? Is it because of the extra RE band? Could it be an artifact of spatial autocorrelation in the error matrix? It would add a lot to this paper to sort these questions out and that could be easily done by repeating the classification experiment but dropping out the NE band. If the results are basically the same with RGB as with RGB+RE then that supports the idea that narrow bands are the key. If the results are much worse with RGB than with RGB+RE then that supports the idea that the RE channel contains a lot of discriminatory power. If the results change with better sampling to create the error matrix then that supports the idea that it is an artifact.

Yes, thank you for this comment. Although we believe the spectral resolution and accuracy of the MicaSense data from its R, G and B sensors are a bit more advanced than those on normal digital cameras; that’s why the RedEdge-M is used in agricultural assessments instead of the very advanced UAV digital cameras used for photogrammetry. We also shot the A and B plots with a GoPro (Yes, I totally understand that a GoPro is not comparable with full-frame digital cameras like the ones used by Edwards et al. 2017) and while we were able to resolve adequately the orthomosaics they lacked detail and sharpness to even try to compare between products, hence we didn’t mention this in the manuscript.

We agree with the need of a comparison between RGB and RGB+RE products. It was included both in methods and in results (Lines 232-236, Figure 4, lines 394-399, Table 4). The results show a better discriminating power when using the RGBRE, over the RGB.  Also, as note, the Jeffries-Matusita separability values between class pairs are another worthy indicator of the discrimination power of the 4-band image vs. the 3-band one, which supports the accuracy of the classifications.

A third revision is slight adjustment of the way reflectance is discussed. For natural illumination, normalization to reflectance is susceptible to clouds, wave refraction as noted (line 420) but also self-shadowing by the instrument and diver. For artificial illumination, these factors will be reduced for deeper sites but never totally eliminated (except at night).

We concur with this assessment; while in the field, we did have the idea of conducting these surveys at night, but never get to do it because of the increase in scuba dive operation costs and logistics. I believe with proper planning it should be doable in the future. We added clarifications on lines 475-481, 493, 518.

Artificial illumination introduces another complication, however, which is non-uniform distribution of lighting across the field of view. It may look uniform but it’s not. Moreover, for both natural and artificial lighting, even tiny variations in distance between the camera and substrate (due to diver motion or 3-D structure of the seabed) are critically important, particularly in the longer wavelengths (> 500 and certainly > 600 nm).  Computing reflectance the way it was done in this project (by reference to the calibrated reflectance panel – lines 138-140) gives an approximate reflectance but it is important to note that this reflectance is not corrected for variable lighting and sensor-seabed geometry (i.e. distance variations between the light-seabed-sensor that are different from what it was when measured with the calibrated panel).  These caveats should be noted.

We fully agree with this comment, and caveats were added (Lines 534-543).

Finally, very minor: Figure 6 is a histogram not spectral curves. Also, make labels bigger.

Yes, you are right, thanks for the comment. Regarding comments by other two reviewers, this figure was eliminated, and a statistical analysis was included, as well as variability charts of O. franksi scene-relative reflectance values for the three distances (methods section, lines 192-199; results section, lines 290-304, Supplementary Data figures SD1-SD5).

In summary:

1. The accuracy assessment has to be fixed to incorporate sufficient independent samples.

It was implemented as suggested, the accuracy assessment values decreased 10% (0.1) in average, thus pointing to the existence of a spatial autocorrelation artifact in the original assessments, but accuracy values remained high nonetheless in the new assessments.

2. It would be ideal to add a simple experiment to try to understand how much the RE channel is adding. In any case it would be good to try to explain why this system works so much better than a regular RGB camera.

Yes! We did it, it does work better with the RE band than without! Thank you for the suggestion.

3. It should be noted that reflectance is not compensated for variable lighting nor light-target-sensor geometry.

Yes, we believe we addressed it appropriately in the discussion.

Reviewer 2 Report

The authors provide preliminary results of an instrumentation development project. Specifically, they integrate a multispectral imager designed for UAV-based agricultural mapping into an underwater housing and diver-operated instrument for high resolution mapping of coral reefs.

Overall this is an interesting project. The application of commercial off-the-shelf equipment into novel uses for remote sensing is a timely subject and of interest to the community. There are several edits and modifications to content and  structure that must be made before I can recommend this article for publication.

In general, there are multiple errors in text style that must be fixed throughout. For example, "meters squared" is rendered as m2 when it should be m².

There are many grammar and diction errors, though the english is quite good for a second-language paper. These do not affect my ability to understand the paper but this would improve the presentation. There are also many run-on sentences that should be broken into concise sentences.

Overall the citations are appropriate but there are some omissions. For example, the authors do a good job of comparing their work to RGB based mapping techniques, but they neglect to mention hyperspectral imaging used in benthic classification. The authors should note that their technique has advantages over the RGB techniques and is cheaper and less data intensive than hyperspectral. For example:

Chennu et al. 2017, A diver-operated hyperspectral imaging and topographic surveying system for automated mapping of benthic habitats.

Summers et al. 2022, Underwater Hyperspectral Imaging of Arctic Macroalgal Habitats during the Polar Night Using a Novel Mini-ROV-UHI Portable System

I am concerned that the authors mention a constant distance to the surface, but they measure this with a dive computer that will indicate depth from surface (and not always with a sufficient precision or accuracy). This should be addressed.

The authors mention the use of LED lights - I would suggest that these kind of white lights are not suitable for hyperspectral or spectrometer techniques but work well for multispectral. Another advantage of their technique.

The "reflectance" method that they present seems dubious to me, or at least the way it was described. The authors may have done this appropriately but I cannot tell from the description. Part of this is due to the instrument. The drone imager they are using is intended not for high quality, absolute reflectance measurements of the type used in rigorous remote sensing science, but rather a relative measure that is internally self consistent and suitable for agricultural mapping, classification, etc. In that regard the process is fine and appropriate. The authors should either discuss the limits of their technique and perhaps describe it as a "scene-relative" reflectance, or improve their description of the methods. They could compare a known coral reflectance spectrum and convolve it with the bands of the micasense to produce a reflectance value that can compare with their produced values. I understand that this may not be the purpose of this study but it would allow other authors to assess their data for comparison across sites and databases. Also, I would like more information on the reflectance standard used. What is the material? The reflectance of a greyscale standard under water is NOT the same as above water. See Russell et al. 2016, Spectral Reflectance of Palauan Reef-Building Coral with Different Symbionts in Response to Elevated Temperature.

Figure 6: This is not a spectral curve as coral reef optical scientists understand it. It is a distribution histogram of their data set and while valid I'm not sure it's useful. I would like to see actual multispectral reflectance curves generated for end member classes (wavelength on the X, reflectance on the Y) or perhaps reef mean  values with appropriate standard deviation bars. That would be useful to place this method in the context of traditional coral reflectance studies.

Table 1: Missing red edge band info

Line 331 through 338 - font size changes.

Line 440 and elsewhere: separation - units? metric?

Conclusions: Good section, overall well written. The authors should make the point that despite the lack of NIR data, this method has a red edge band not found in RGB photogrametry which can be important for assessing pigment content etc.

Author Response

Reviewer 2

We sincerely thank and appreciate the thorough revisions and constructive comments by the reviewers; which -we feel- have enhanced enormously the clarity and potential of this manuscript.

The authors provide preliminary results of an instrumentation development project. Specifically, they integrate a multispectral imager designed for UAV-based agricultural mapping into an underwater housing and diver-operated instrument for high resolution mapping of coral reefs. 

Yes, thank you for your correct appreciation

Overall this is an interesting project. The application of commercial off-the-shelf equipment into novel uses for remote sensing is a timely subject and of interest to the community. There are several edits and modifications to content and structure that must be made before I can recommend this article for publication.

Thank you very much for your kind comments, we hope we had addressed adequately all your suggestions.

In general, there are multiple errors in text style that must be fixed throughout. For example, "meters squared" is rendered as m2 when it should be m².

Text was thoroughly checked and edited accordingly.

There are many grammar and diction errors, though the english is quite good for a second-language paper. These do not affect my ability to understand the paper but this would improve the presentation. There are also many run-on sentences that should be broken into concise sentences.

The English language was revised.

Overall the citations are appropriate but there are some omissions. For example, the authors do a good job of comparing their work to RGB based mapping techniques, but they neglect to mention hyperspectral imaging used in benthic classification. The authors should note that their technique has advantages over the RGB techniques and is cheaper and less data intensive than hyperspectral. For example:

Chennu et al. 2017, A diver-operated hyperspectral imaging and topographic surveying system for automated mapping of benthic habitats.

Summers et al. 2022, Underwater Hyperspectral Imaging of Arctic Macroalgal Habitats during the Polar Night Using a Novel Mini-ROV-UHI Portable System 

We thank your comment, and agree, Chennu et al. 2017 was included in our introduction/background section, as well as Summers et al. 2022 (line 66). Now your valuable suggestions are included in the discussion (lines 587-607)

I am concerned that the authors mention a constant distance to the surface, but they measure this with a dive computer that will indicate depth from surface (and not always with a sufficient precision or accuracy). This should be addressed.

Reviewer 1 also raised this point. Yes, we agree that it is a relatively constant distance to the substrate, clarification on this was added in the methods and discussion sections. (Lines 102, 534-543)

The authors mention the use of LED lights - I would suggest that these kind of white lights are not suitable for hyperspectral or spectrometer techniques but work well for multispectral. Another advantage of their technique.

We concur with your comment, we did an extensive market research trying to find and acquire the best all-around lights in terms of consistent (and compatible) spectral output, and by the time we were writing this project, flood non-LED underwater lights (i.e. halogen) were pretty scarce, low powered, and extremely expensive. The reworked discussion addresses this on lines 608-613.

The "reflectance" method that they present seems dubious to me, or at least the way it was described. The authors may have done this appropriately but I cannot tell from the description. Part of this is due to the instrument. The drone imager they are using is intended not for high quality, absolute reflectance measurements of the type used in rigorous remote sensing science, but rather a relative measure that is internally self consistent and suitable for agricultural mapping, classification, etc. In that regard the process is fine and appropriate. The authors should either discuss the limits of their technique and perhaps describe it as a "scene-relative" reflectance, or improve their description of the methods.

We understand and agree with this concern, clarifications and improvements have been made in the methods (lines 145-150) and discussion sections (lines 475-481), and we have adopted your spot-on suggestion of using “scene-relative reflectance” (lines 480, 493, 518). Also, we have toned down the concept of spectral discrimination throughout the manuscript so it is not misleading.

They could compare a known coral reflectance spectrum and convolve it with the bands of the micasense to produce a reflectance value that can compare with their produced values. I understand that this may not be the purpose of this study but it would allow other authors to assess their data for comparison across sites and databases.

Yes, we concur with your comment and had thought about it before. This scenario was explored by one of my Ph.D. students in collaboration with Dr. James Goodman, in an attempt to automatize the classification of these orthomosaics. As you correctly point out, our reflectance data is scene-relative, which is not compatible at all with “real” and accurate reflectance values measured with spectrophotoradiometers measuring reflectance almost by contact with the objects (i.e. GER 1500).  We hope the clarifications in lines 475-481 of the discussion, arisen from this and the previous comment, has addressed this topic adequately.

Also, I would like more information on the reflectance standard used. What is the material? The reflectance of a greyscale standard under water is NOT the same as above water. See Russell et al. 2016, Spectral Reflectance of Palauan Reef-Building Coral with Different Symbionts in Response to Elevated Temperature. 

Yes, thank you for noting this, we agree, we tested this using first the calibrated reflectance panel (CRP) included with the camera, captured on the boat previous to the dive, and later using the same panel captured underwater at the measured distance of 50 cm and illuminated with both keldan lamps at its maximum output. This was added to the methods section (lines 145-150), and also addressed in the discussion (lines 475-481). We did not find any reference on the material composition of the panel. We even asked, to no avail, to the technical support at MicaSense. Available information on the MicaSense CRP can be found here:

https://support.micasense.com/hc/en-us/sections/4420300203287-Calibrated-Reflectance-Panel-CRP-

Figure 6: This is not a spectral curve as coral reef optical scientists understand it. It is a distribution histogram of their data set and while valid I'm not sure it's useful. I would like to see actual multispectral reflectance curves generated for end member classes (wavelength on the X, reflectance on the Y) or perhaps reef mean  values with appropriate standard deviation bars. That would be useful to place this method in the context of traditional coral reflectance studies.

Yes, thank you very much for your comment, Reviewer 1 also pointed out our erroneous use of the term in the figure 6 caption. This figure was eliminated regarding comments of all reviewers.

Unfortunately, these O. franksi datasets are the only data we have comparing the same object at different distances, captured within a few seconds when the environmental conditions were similar. As you point out, the reflectance values are scene-relative, and they cannot be compared with other datasets as the spectral libraries by Roelfsema & Phinn (2013) or Lucas & Goodman (2015).

Roelfsema CM & SR Phinn (2013): Spectral reflectance library of selected biotic and abiotic coral reef features in Glovers Reef, Belize. https://doi.org/10.1594/PANGAEA.824861

Lucas MQ & J Goodman (2015) Linking Coral Reef Remote Sensing and Field Ecology: It’s a Matter of Scale. J. Mar. Sci. Eng. 2015, 3, 1-20. https://doi.org/10.3390/jmse3010001

Nevertheless, we did multivariate analysis that pointed to the significant difference in these values for the 0.5m vs 0.75m and 1m distances (Methods Lines 192-199, Results Lines 290-305), and Supplementary Data figures SD1-SD5).

Table 1: Missing red edge band info

An oversight on our part when pasting the table. Table 1 is complete now.

Line 331 through 338 - font size changes.

Yes, thank you for noticing, also an oversight when pasting text into the journal’s .docx template. Corrected

Line 440 and elsewhere: separation - units? metric?

Indeed, an oversight too. Added the Jeffries-Matusita separation index information where needed. (lines 397-398, 553, 555).

Conclusions: Good section, overall well written. The authors should make the point that despite the lack of NIR data, this method has a red edge band not found in RGB photogrametry which can be important for assessing pigment content etc.

Yes, thank you for this suggestion, it was added accordingly in the conclusion section (lines 637-641).

Reviewer 3 Report

In this manuscript, the authors tried to classify coral reef benthos using multispectral under-water remote sensing images. The topic is useful but some major problems seriously influenced the quality of this research. The first problem is that the organization of this manuscript is more like a technique report rather than an article. It seems that the authors captured the images under water, mosaicked them and classified them. All these processing steps were carried out by mature commercial software. The scientific problem was not pointed out clearly. The artificial lighting test was interesting and I was hoping of some solid discussion on it when I firstly saw it in figure 1. But the contents of section 3.1.3 were so inadequate that the authors didn’t even describe their reason of setting the final height to 0.5 m clearly. The second major problem is the classification process. As the authors claimed in section 2.6, they selected 40 samples to train the classification model and validate the classification result. However, the SVM method in ENVI software is a pixel-based classification method. The result validation indices were also calculated based on all the pixels but not the digitized regions. A proper way to do the classification and validation here is to select a large quantity of sample points randomly in the images as training and validation samples. 40 samples are far from enough for this research because the class number is 14. Here are some minor comments.

Line 36. “AGRRA” needs full name at its first appear.

Line 191. “(subsets I and II)” I suggest to add a citation of figure 7 here.

Figure 1. This flow chart needs a remaking. Some important information such as preprocessing steps were not mentioned. The A and B routes after preprocessing seems to be two independent parts in this figure. But I believe A is to determine a proper classification method for B through the contexts. At last, this is a flow chart, what the readers really want to see is the main steps to reach the results of the research but not the exploring steps of the whole project.

Section 3.1.2. I suggest to reorganize this section. 1^st trial was not mentioned before. Why don’t the authors just use “Image of Plot A”?

Line 238. “25 m2” I believe it should be “25 m²”

Line 254. “The procedure revealed that the 0.5 m distance was the best option to capture useful pictures on all 5 bands”. You mentioned 5 bands here but you just plotted the histograms of 4 bands in figure 6. In addition, please be more specific of why 0.5 m is “the best option”.

Line 323. The classification result in figure 9 contain obvious salt-and-pepper errors. I suggest to add majority analysis as post classification process.

Line 340. Please add scale bar in this figure.

Line 363. The discussion needs to be reorganized. The current version is difficult to read.

Author Response

Reviewer 3

Authors' responses are underlined.

We sincerely thank and appreciate the thorough revisions and constructive comments by the reviewers; which -we feel- have enhanced enormously the clarity and potential of this manuscript.

In this manuscript, the authors tried to classify coral reef benthos using multispectral under-water remote sensing images. The topic is useful but some major problems seriously influenced the quality of this research. The first problem is that the organization of this manuscript is more like a technique report rather than an article.

Thanks for your assessment. Yes, we believe including all the steps is a necessary measure to ensure the replicability. And yes, we are aware that the first version lacked development of and discussion about important points. We hope this new revised version addresses all the concerns.

It seems that the authors captured the images under water, mosaicked them and classified them. All these processing steps were carried out by mature commercial software.

Yes, that is correct

The scientific problem was not pointed out clearly.

Yes, we understand this issue. The original paragraph was modified and now Lines 63-80 on the introduction have a better description of the scientific problem, which is the objective of this research.

The artificial lighting test was interesting and I was hoping of some solid discussion on it when I firstly saw it in figure 1. But the contents of section 3.1.3 were so inadequate that the authors didn’t even describe their reason of setting the final height to 0.5 m clearly.

Yes, thank you for the observation. We included additional multivariate analysis of the scene-relative reflectance (we adopted the term at the specific suggestion of one of the other reviewers) of the coral colony of Orbicella franksi, where values are clearly separated (Methods Lines 192-199, Results Lines 290-304), and Supplementary Data figures SD1-SD5)

The second major problem is the classification process. As the authors claimed in section 2.6, they selected 40 samples to train the classification model and validate the classification result. However, the SVM method in ENVI software is a pixel-based classification method. The result validation indices were also calculated based on all the pixels but not the digitized regions. A proper way to do the classification and validation here is to select a large quantity of sample points randomly in the images as training and validation samples. 40 samples are far from enough for this research because the class number is 14.

Yes, we agree. This point was raised also by another reviewer, and methods, results and discussion have been clarified and enhanced. We applied a stratified random point selection for the accuracy assessments, and for Plot B Full we used points (individual pixels) as training fields, and the same stratified random point selection for accuracy assessment. Thus, avoiding the spatial autocorrelation of the clusters of pixels within the original polygons for the accuracy assessment. The values decreased 10% in average for overall accuracy, and 0.1 for Kappa and Tau coefficients, but remained high.

Updated section number 2.7 in methods, lines 216-221, 242-245.

Results, lines 374-381, 410-418, Figures 7,8,9 and tables 3-6

Discussion, lines 547-559, 571-572, 585, 595-598.

Here are some minor comments.

Line 36. “AGRRA” needs full name at its first appear.

Yes!, thanks for the observation, it was added (line 37)

Line 191. “(subsets I and II)” I suggest to add a citation of figure 7 here.

We understand your suggestion, but as noted on the journal’s Instructions to Authors, all figures and tables must be presented shortly after mentioning them in the text. Figure 7 is part of the Results section, and line 191 (now line 210) is in the methods section, some 130 lines before figure 7.

Figure 1. This flow chart needs a remaking. Some important information such as preprocessing steps were not mentioned. The A and B routes after preprocessing seems to be two independent parts in this figure. But I believe A is to determine a proper classification method for B through the contexts. At last, this is a flow chart, what the readers really want to see is the main steps to reach the results of the research but not the exploring steps of the whole project.

Thanks for your observation, and apologies for the misnumbering, this should have been labeled as figure 4. And, yes, we agree that the original figure needed some clarification. After yours and the other’s reviewers comments the figure was reworked. Nevertheless, we chose to include all significant steps to help the reader understand the whole approach. The tests without adequate results are in grey. (New Figure 4)

Section 3.1.2. I suggest to reorganize this section. 1^st trial was not mentioned before. Why don’t the authors just use “Image of Plot A”?

Thank you for this observation. Yes, we changed the titles of subsections 3.1.2 and 3.1.4 accordingly.

Line 238. “25 m2” I believe it should be “25 m²”

Yes, thank you for your observation, several of these typos were corrected throughout the manuscript.

Line 254. “The procedure revealed that the 0.5 m distance was the best option to capture useful pictures on all 5 bands”. You mentioned 5 bands here but you just plotted the histograms of 4 bands in figure 6. In addition, please be more specific of why 0.5 m is “the best option”.

Yes, Figure 6 was eliminated, it had several shortcomings. As mentioned above in our response to your earlier comment about the artificial lighting tests, additional multivariate tests (ANOSIM and SIMPER) were applied to sets of 1150 pixel values, extracted from each band -R, G , B, RE, NIR- of each multispectral orthomosaic resolved at three different distances (0.5 m, 0.75 m and 1 m).

These tests defined the existence of significant differences between the dataset at 0.5 m distance to the datasets at 0.75 and 1 m distances. Pixel values were higher on the 0.5 m distance, and visually objects were better represented in comparison with the orthomosaics at other distances (Discussion section, lines 458-461).

Line 323. The classification result in figure 9 contain obvious salt-and-pepper errors. I suggest to add majority analysis as post classification process.

Yes! Thank you for your brilliant suggestion, majority analysis was applied systematically to all supervised classifications produced by the orthomosaics preprocessed with the local sigma filter. (Methods section lines 236-238, Results lines 375-382, 415-418, figures 7-9, Tables 4-7, Discussion 566-569).

Line 340. Please add scale bar in this figure.

Indeed, an oversight on our part. The new figure 9 has the scale bar.

Line 363. The discussion needs to be reorganized. The current version is difficult to read.

Thanks to the keen observations and suggestions from you and other two reviewers, the discussion section of the manuscript has been enhanced and hopefully is clearer and more useful now.

Round 2

Reviewer 2 Report

The authors have adequately addressed my concerns, as well as those of the other reviewers, and the paper is greatly improved.

There are still some minor spelling/grammar/formatting errors but they are not substantial.

I do agree with reviewer 3's assessment that it would be interesting to see spectral shapes of each class, and more information about the difference in shape with variable illumination distance.

Author Response

A: We appreciate the new comments and suggestions, we are sure they improved further the quality of this manuscript.

As both reviewers had similar concerns, we are addressing both reviewers in this response letter.

R2: The authors have adequately addressed my concerns, as well as those of the other reviewers, and the paper is greatly improved.

R3: The authors reviewed the manuscript well and answered most of my questions. The overall quality of the article is improved, the flow chart is clearer and readable, and the illustration of the results are better. But the current version still has some problems.

A: Thank you very much to you both for your kind comments!

R2: There are still some minor spelling/grammar/formatting errors but they are not substantial.

R3: I suggest the authors review the current version very carefully because there are still many typos and formatting issues.

A: Yes, English language has been thoroughly revised, and some lines have been re-phrased or expanded to improve readability.

R3: The readers will like to see some variables that could influence the method/technique. In this article, for example, I am curious about the typical spectral shapes of different classes in images that been captured from different heights (0.5 m, 0.75 m, and 1 m). With illustrating the spectral shapes, some statistical indices will also help to deepen the discussion, rather than the current simple description.

R2: I do agree with reviewer 3's assessment that it would be interesting to see spectral shapes of each class, and more information about the difference in shape with variable illumination distance.

A: Yes, this was an issue raised by the 2^nd reviewer during the 1^st round. Here is a copy of that answer and we expand the explanation below:

“…Unfortunately, these O(rbicella). franksi datasets are the only data we have comparing the same object at different distances, captured within a few seconds when the environmental conditions were similar.”

These O. franksi images were the image sets acquired specifically for the artificial lighting test at different distances. The hypothesis we were trying to test in that moment, which defined the sample we took was: There is a significant decrease in the (scene-relative) reflectance values of artificially illuminated substrate on all 5 bands, when distance increases from 0.5 m to 0.75 m and to 1 m.  Which we confirmed with the multivariate statistical analysis in the last version of the manuscript. This statistical test is robust in terms of sampling effort and illustrates perfectly what we were trying to test. So, we believe there is no need for additional statistical analysis.

We do not have full imagery sets of any individual plot (comparable to A or B plots of this study) surveyed at different distances.

As a compromise between the request by the reviewers and our data availability, we are including scene-relative Spectral Signatures of O. franksi at the three distances on the 5 bands (Supplementary Data figure 6 a,b,c) so the multivariate analysis is reinforced, and the effect of distance can be compared.

Also, scavenging through the field data folders, it turns out that we did have an additional image data set at different distances with artificial illumination; it was acquired a few seconds later than the O. franksi dataset. Previously this image set was disregarded because the coral colonies within the images were small, but we believe they do work for the spectral signatures illustrative purpose.  So, we also include scene-relative spectral signatures of five benthic components observable within this image set: corals Agaricia agaricites, Montastraea cavernosa and Stephanocoenia intersepta, along with two different brown algae genus Padina and Dyctiota (Supplementary Data figure 7).

Discussion was extended addressing these comparisons (lines 557-568).

R3: The discussion section needs a reorganization. If necessary, subsections will also improve the readability.

A: Discussion was reorganized  and subsections were added as suggested.

Reviewer 3 Report

The authors reviewed the manuscript well and answered most of my questions. The overall quality of the article is improved, the flow chart is clearer and readable, and the illustration of the results are better. But the current version still has some problems. I suggest the authors review the current version very carefully because there are still many typos and formatting issues. The discussion section needs a reorganization. The readers will like to see some variables that could influence the method/technique. In this article, for example, I am curious about the typical spectral shapes of different classes in images that been captured from different hights(0.5m, 0.75m, and 1m). With illustrating the spectral shapes, some statistical indices will also help to deepen the discussion, rather than the current simple description. If necessary, subsections will also improve the readability.

Author Response

A: We appreciate the new comments and suggestions, we are sure they improved further the quality of this manuscript.

As both reviewers had similar concerns, we are addressing both reviewers in this response letter.

R2: The authors have adequately addressed my concerns, as well as those of the other reviewers, and the paper is greatly improved.

R3: The authors reviewed the manuscript well and answered most of my questions. The overall quality of the article is improved, the flow chart is clearer and readable, and the illustration of the results are better. But the current version still has some problems.

A: Thank you very much to you both for your kind comments!

R2: There are still some minor spelling/grammar/formatting errors but they are not substantial.

R3: I suggest the authors review the current version very carefully because there are still many typos and formatting issues.

A: Yes, English language has been thoroughly revised, and some lines have been re-phrased or expanded to improve readability.

R3: The readers will like to see some variables that could influence the method/technique. In this article, for example, I am curious about the typical spectral shapes of different classes in images that been captured from different heights (0.5 m, 0.75 m, and 1 m). With illustrating the spectral shapes, some statistical indices will also help to deepen the discussion, rather than the current simple description.

R2: I do agree with reviewer 3's assessment that it would be interesting to see spectral shapes of each class, and more information about the difference in shape with variable illumination distance.

A: Yes, this was an issue raised by the 2^nd reviewer during the 1^st round. Here is a copy of that answer and we expand the explanation below:

“…Unfortunately, these O(rbicella). franksi datasets are the only data we have comparing the same object at different distances, captured within a few seconds when the environmental conditions were similar.”

These O. franksi images were the image sets acquired specifically for the artificial lighting test at different distances. The hypothesis we were trying to test in that moment, which defined the sample we took was: There is a significant decrease in the (scene-relative) reflectance values of artificially illuminated substrate on all 5 bands, when distance increases from 0.5 m to 0.75 m and to 1 m.  Which we confirmed with the multivariate statistical analysis in the last version of the manuscript. This statistical test is robust in terms of sampling effort and illustrates perfectly what we were trying to test. So, we believe there is no need for additional statistical analysis.

We do not have full imagery sets of any individual plot (comparable to A or B plots of this study) surveyed at different distances.

As a compromise between the request by the reviewers and our data availability, we are including scene-relative Spectral Signatures of O. franksi at the three distances on the 5 bands (Supplementary Data figure 6 a,b,c) so the multivariate analysis is reinforced, and the effect of distance can be compared.

Also, scavenging through the field data folders, it turns out that we did have an additional image data set at different distances with artificial illumination; it was acquired a few seconds later than the O. franksi dataset. Previously this image set was disregarded because the coral colonies within the images were small, but we believe they do work for the spectral signatures illustrative purpose.  So, we also include scene-relative spectral signatures of five benthic components observable within this image set: corals Agaricia agaricites, Montastraea cavernosa and Stephanocoenia intersepta, along with two different brown algae genus Padina and Dyctiota (Supplementary Data figure 7).

Discussion was extended addressing these comparisons (lines 557-568).

R3: The discussion section needs a reorganization. If necessary, subsections will also improve the readability.

A: Discussion was reorganized  and subsections were added as suggested.