A Gaussian Mixture Model to Separate Birds and Insects in Single-Polarization Weather Radar Data
Round 1
Reviewer 1 Report
Dear Authors
The citations and referring to Figs and Tables must be fixed BEFORE submission.
The discussion is very weak, and should be strengthened. Also, a conclusion section MUST be added.
Other comments and changes are denoted on the attached PDF.
Comments for author File: 
Comments.pdf
Author Response
Dear Authors,
The citations and referring to Figs and Tables must be fixed BEFORE submission.
We are sorry for these errors. We believe that they resulted from a problem in the online conversion to the pdf from the word document submitted. We sadly have no control over this (e.g., not a problem here: https://www.biorxiv.org/content/10.1101/2021.03.08.434434v1.full), but we will directly submit a pdf version in this submission.
The discussion is very weak, and should be strengthened. Also, a conclusion section MUST be added.
We have modified the discussion section to make it stronger and created a conclusion section. However, as technical notes underlie strong word limits, we had to remain concise in revising both sections.
Other comments and changes are denoted on the attached PDF.
We have changed the colours of the figures for better readability and accepted all the minor suggestions proposed, except L.17 (“nocturnal”) because we want to make sure that readers understand that this study only looks at nocturnal data.
Reviewer 2 Report
This manuscript has been submitted as a technical note, which is defined as short articles (less than 18 pages) on new developments, significant advances and novel aspects of experimental and theoretical methods and techniques relevant to the scope of Remote Sensing. While this manuscript does fit the scope for a technical note by proposing a novel method for determining the composition of birds vs insects within weather surveillance radar (WSR) data, it is extremely short (11 pages) and lacks some methodological details (e.g., no reporting of sample sizes, unclear description of what a “datapoint” is, no model fit statistics) and empirical validation of the proposed method. Thus, it is difficult to evaluate the rigor and accuracy of this method. Considering the methods on just theoretical terms, it has great appeal and intuitive results despite relying on many simplifying assumptions. There is still room to provide more details about the methods, which should help bolster adoption of this method by others and the impact of the manuscript. However, this method will remain empirically unvalidated without the addition of data simulations or field data to determine if the estimated proportions of insect vs bird composition are accurate and subsequent adjustments/estimates of speed of birds and insects are useful. I encourage the authors to consider using field data from one of their BirdScan radars to evaluate how well the mixing models estimate the proportion of insect vs birds. The BirdScan radar can identify birds from insects by wingbeat frequency, albeit at a much small extent of airspace than a WSR. I would think it could provide some useful field data for comparison to WSR data.
Section 2.2 – Can you please describe what exactly you were “cleaning” from the data? Even though you reference the appendix of a published paper, it would help to briefly summarize your cleaning procedure here. Are you eliminating precipitation and ground clutter (human infrastructure) echoes from the data? What about reflectivity returns in low tilt data caused by anomalous beam propagation (i.e. refraction of the beam towards the ground)? How do you identify AP? Are you eliminating entire scans from the dataset that are contaminated by non-biological data? Or are you filtering out individual sampling volumes within scans when deriving vertical profiles? More details are needed here.
Section 2.3 – More details are needed to help explain the wind data the you used and how you integrated it with radar-based ground speeds. What were the heights of the wind data that you used? Did you match the winds above the ground to the 200m height bins of the radar data in your vertical profiles? What is the native temporal and spatial resolution of the wind data? Did you extract wind data just at the radar location (treat radar as a point) or within the entire radar domain (treat the area around the radar as a polygon of a given radius)?
Section 3.1 – Model fitting – There are major gaps in explanation of methods here. You state the “we learn the distribution of airspeed and radial velocity standard deviation signature of birds and insects.” How is this done exactly? At what scale are you defining what is bird or insect? At the sampling volume scale or the scan level? In other words, what is your independent sampling unit that comprise your “datapoints”? Please report sample size of datapoints. There would be a lot of temporal autocorrelation in using successive radar scans (i.e., every 5 minutes). Is this a concern?
Section 3.2 – Please include citations that back up your statement that insects are less active during the winter and that mist tends to occur during winter (I will accept that snow is more common in the winter without a citation). While I expect that the frequency of snow varies with latitude, there is no explanation as to why in figure 5b that some radars attribute all “insect-like” reflectivity to insects in February while others not until May. There is no clear explanation of why these lines vary for different radars. Do you change what the peak timing of winter and summer are among radars?
Results – Please provide statistics of how well the mixture models fit the data.
Can you validate that what you identified as snow (slow-moving low-magnitude reflectivity in winter) was actually snowfall?
Discussion - There is no empirical validation of the identity of bioscatterers or the estimated proportion of mixture of insect and birds aloft. While this approach capitalizes on integrating air speed and variance in radial velocity, which have both been shown to be useful for bioscatter identity, it isn’t clear that your approach to estimate proportions of birds vs insects is accurate. As I already mentioned, adding data simulations of known mixtures of birds and insects or field validation data to gauge the accuracy and precision of the model for estimating bird/insect composition and impacts on adjusted speed calculations would be helpful. Without such validation, you should temper your claim that “the method can also correctly estimate the airspeed of birds by accounting for the contribution of insects to the average airspeed”, since your estimates are only theoretical.
Supplemental: The proportion values of the amplitude ratio seem inverted in the supplemental figures. I believe they should get larger with greater proportion of birds?
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
The manuscript provides new insight into discriminating between birds and insects using readily available weather radar observations. The manuscript is clearly and concisely written and except for minor suggestions given below, is deserving of publication. Minor Suggestions The authors have undoubtedly followed the journal’s suggested format. Omission and inclusion of their conclusions within Sections 4: Results and 5: Discussion, does not emphasize the nature and value of their findings. Multiple repetition of the absence of citations are presumably due to editorial review. This procedure is disruptive to say the least and is in many cases out of place since no published material may be available. This reviewer considers these inserts the responsibility of the authors and reviewers. In two locations, lines 71 and 90, the term “learn” is used. Perhaps “derive” or “determine” would be a better choice. Acronyms are used throughout the manuscript. Acronyms should not be used in an abstract even though they may well be known to scientists in the journal’s readership community. The first use of an acronym in the body of the manuscript should be defined. In the case of this manuscript where numerous acronyms are used throughout the paper, perhaps a tabular listing is desirable. Once defined, the definition should not be repeated (see RTR, lines 202 and 218). Discussion should be avoided in figure captions (see, in particular Fig. 1). Lines 83-86 should go into the text. Figure captions should normally be free of citations (Fig. 6, line 197). Figure 7 would benefit from using clearly contrasting (bright) colors. Figures 4 and 5 use “radar” redundantly in the captions. The second use of “radar” in parenthesis, line 123, Fig. 4, and line 140, Fig. 5, is unnecessary. The personal pronoun “We” is used some 21 times (and as many as 5 times on pg. 6). The use of an introductory phrase such as “We estimate”, line 66, or “We first learn”, line 71, can be avoided by simply starting the sentence with the action to be described, such as line 66, “The proportion”, line 71, “The typical signature”, etc.
Author Response
The manuscript provides new insight into discriminating between birds and insects using readily available weather radar observations. The manuscript is clearly and concisely written and except for minor suggestions given below, is deserving of publication.
Minor Suggestions
The authors have undoubtedly followed the journal’s suggested format. Omission and inclusion of their conclusions within Sections 4: Results and 5: Discussion, does not emphasize the nature and value of their findings.
We have modified the discussion section to make it stronger and created a conclusion section. However, as technical notes underlie strong word limits, we had to remain concise in revising both sections.
Multiple repetition of the absence of citations are presumably due to editorial review. This procedure is disruptive to say the least and is in many cases out of place since no published material may be available. This reviewer considers these inserts the responsibility of the authors and reviewers.
We are sorry for these errors. We believe that they resulted from a problem in the online conversion to the pdf from the word document submitted. We sadly have no control over this (e.g., not a problem here: https://www.biorxiv.org/content/10.1101/2021.03.08.434434v1.full), but we will directly submit a pdf version in this submission.
In two locations, lines 71 and 90, the term “learn” is used. Perhaps “derive” or “determine” would be a better choice.
We replaced the term “learn” with “derive”.
Acronyms are used throughout the manuscript. Acronyms should not be used in an abstract even though they may well be known to scientists in the journal’s readership community. The first use of an acronym in the body of the manuscript should be defined. In the case of this manuscript where numerous acronyms are used throughout the paper, perhaps a tabular listing is desirable. Once defined, the definition should not be repeated (see RTR, lines 202 and 218).
We have removed the use of the following acronyms GUI, PDF, single-pol and ENRAM and removed the duplicate RTR.
Discussion should be avoided in figure captions (see, in particular Fig. 1). Lines 83-86 should go into the text.
As suggested, we have removed lines 83-86 as the content is already presented in the main text (section 3 Methodology). We have also removed a sentence in Figure 2.
Figure captions should normally be free of citations (Fig. 6, line 197).
“Error! Reference source not found.“ is an internal reference to Figure 1 and not a citation to publish material. We corrected this.
Figure 7 would benefit from using clearly contrasting (bright) colors.
We have changed the colours of the figures for better readability.
Figures 4 and 5 use “radar” redundantly in the captions. The second use of “radar” in parenthesis, line 123, Fig. 4, and line 140, Fig. 5, is unnecessary.
We have removed the redundant words.
The personal pronoun “We” is used some 21 times (and as many as 5 times on pg. 6). The use of an introductory phrase such as “We estimate”, line 66, or “We first learn”, line 71, can be avoided by simply starting the sentence with the action to be described, such as line 66, “The proportion”, line 71, “The typical signature”, etc.
Most of the “we” were removed.
Round 2
Reviewer 1 Report
My comments were adequately addressed.
Author Response
We thank the reviewer for the help provided.
Reviewer 2 Report
I feel the authors have sufficiently addressed my comments in their revised draft. I would only request that the authors consider adding the analysis that they shared in their response to my comment, but didn't include in the revision. Namely, the analysis of observed tracks of animals made with their portable radar unit provides nice evidence to support their method for separating birds from insects. I understand there may be caveats to the representativeness of their portable radar observations, but they demonstrate that the mixing model is very similar to that of the weather radar observations. I recommend accepting the revised manuscript for publication with or without including the new analysis of portable radar data.
Author Response
"I think some sort of empirical validation for the method is needed here to back up the claims that estimates of the proportion of birds vs insects in radar data are accurate. Theoretically, the methods make sense, but are untested. Also, the lack of reporting of sample sizes, model fit statistics, and clear identification of what independent samples represent is troublesome. These are rather fundamental components to report. Poorly fit models may not be very useful or accurate - we just don't know how well the model they developed performs outside of theoretical considerations. This manuscript feels a bit premature and further validation of the methods is needed."
We understand that this comment was made prior to the reviewer seeing our response. For clarity purposes, we copy the relevant extract from our previous response for each of the topics mentioned above. Based on the second set of comments (see below), we believe that reviewer 2 is satisfied with these responses.
Sample size:
We have added the number of datapoints used in the study in line 56 “The resulting dataset used in this study consists of 6.8 million datapoints.”
Model fit statistics:
We have added the fitted parameter values of the two Gaussians. However, we are not aware of any statistic to properly assess the absolute fit a gaussian mixture. Indeed, we are fitting an empirical distribution (surface in coloured scale in Fig 1.) and not classify data of bird and non-bird. A visual assessment of the fit can be seen in the subfigure of Figure1.
Independent sample:
We used the vertical profile time series (vpts) as no scan data are available through ENRAM. A datapoint is therefore used for a single value in the vertical profile time series (vpts) “Each radar provides spatial and temporal datapoints, with a vertical resolution of 200m (0-5000m a.s.l.), a temporal resolution of 5 min.” (l. 33).
Validation:
We agree with the reviewer that validating the method with an external dataset (i.e., classified echoes as insect and bird without using speed) would have been ideal and bring a significant plus to the method. Unfortunately, such dataset is currently not readily available (See below the issue about Birdscan).
However, a solution for eliminating insect contamination from bird is badly needed (and used) in aeroecology at this stage. While previous methods (using threshold) have also never been validated, we believe that the method presented here is a significant improvement that will be extensively used and we are therefore convinced that a publication of this method will be appreciated by the aeroecological community (rather than full paper if validation was done).
The groundspeed accuracy of Birdscan data is currently not reliable enough for properly validating the method. It relies on the duration of the transit of birds/insects through the beam, and as such varies with altitude, target size and variation in track while in the beam. Nevertheless, we have performed a preliminary comparison with Birdscan-data that makes some basic assumptions. We accounted for the difference of sampling volume. The resulting joint pdf of airspeed and standard deviation (left subplot) is relatively similar to the fitted Gaussian of the method presented (right subplot). We have removed the word “correctly” from the sentence mentioned.
"I feel the authors have sufficiently addressed my comments in their revised draft. I would only request that the authors consider adding the analysis that they shared in their response to my comment, but didn't include in the revision. Namely, the analysis of observed tracks of animals made with their portable radar unit provides nice evidence to support their method for separating birds from insects. I understand there may be caveats to the representativeness of their portable radar observations, but they demonstrate that the mixing model is very similar to that of the weather radar observations. I recommend accepting the revised manuscript for publication with or without including the new analysis of portable radar data."
We fully understand and would also ideally include this analysis in the manuscript. Unfortunately, the accuracy of the estimation of speed from Birdscan is currently under discussion. In the analysis sent to the reviewer, the threshold used to select the tracks of insects and birds in the Birdscan dataset is very subjective and cannot be properly justified. We therefore believe that this current analysis is not up to publication standard. Nevertheless, as suggested by reviewer 2, we believe that this work is sufficiently coherent to be published in its current form. We therefore would like to opt for the second option left by reviewer 2, that is, to publish the manuscript without this analysis. We are hoping to publish a manuscript focusing on the estimation of speed with Birdscan at a later stage.
We have added the following sentence in the conclusion: “Future work could include validation of this approach using polarimetric data or tracking radar.”
