Advanced Leak Detection and Quantification of Methane Emissions Using sUAS

Round 1

Reviewer 1 Report

This paper is a good timely comprehensive review and summary of state of the art technologies for methane leak detection and quantification.  However the paper has little relevance to sAUS deployment of these technologies.  With some revision, the paper should be published in a journal other than Drones.

A major technical flaw making the paper unsuitable for publication in its current form is in Figure 11.  The inclusion of the Bridger data with the other nine data plots is scientifically inappropriate and suggests an author bias.  The Bridger data were not collected as part of the Stanford/EDF mobile monitoring challenge.  They were collected at METEC during the ARPAe MONITOR program.  Other MONITOR participants have published data comparable to Bridger.  This reviewer questions why these other MONITOR data and their underlying technologies are not also included here?

Author Response

Response to Reviewer 1:

Thank you for your careful review of our paper with constructive comments and technical advice.

1. The authors’ initial focus was more on the general application of quantification techniques for sUAS, including current state of the art and potential techniques for sUAS. Deployment of sUAS, however, requires specifics of the sUAS platform, such as the autopilots, control strategies, co-pilot systems (e.g. secondary computer to facilitate payload communication, read in peripheral sensors, etc.), payload integration, and so on. For these reasons we think this topic deserves its own paper. The authors have also added some details regarding payload integration and deployment as well as a figure showcasing some of the typical payload configurations. We also pointed to a paper on how to select the sUAS platforms.
2. The authors agree with the reviewer here and appreciate the attention to details. The Bridger Photonics data has been removed to another figure with the addition of other ‘available’ MONITOR participants data at METEC. After checking ARPA-E website, there are 11 participants in the MONITOR program. After searching for academic papers, the only reports the authors could find are from [1], where 6 of the MONITOR participants data was aggregated together with 6 other industry participants. Due to confidentiality agreements, they noted that the aggregated results do not illustrate the performance of any individual solution. Additionally, the MONTIOR METEC participants of (1) Bridger Photonics, (2) University of Colorado, Boulder, (3) IBM and Princeton, and (4) University of Houston, Physical Sciences, and Heath Consultants are shown as only publicly available results the authors could find.

[1] Bell et al. 2020. Evaluation of next generation emission measurement technologies under repeatable test protocols. Elem Sci Anth, 8: 32 DOI: https://doi.org/10.1525/elementa.426

Reviewer 2 Report

This paper provides a complex review of methane emission investigation based on UAV sampling systems. The detection and quantification methods are thoroughly discussed.  Whereas, there are some points need to be addressed.

1. The "SENSORS AND EQUIPMENT" are not presented explicitly in section 3. The structure/content needs to be modified to be more readable.
2.  "8. Conclusions" section should conclude the major findings of this paper, e.g., what the major results of the comparisons between UAVs and conventional platforms are, what insights, contribution or suggestions of this study mainly provides. Abstract and conclusion are two most important sections for reader to quickly approach to the "whole picture" of one study. 
3. Line numbers are messed up making the content is hard to be mentioned.
4. Line 162: correct the typo "...as well as a establish a quantitative comparison..."
5. Figure 11: Correct the reference number of the caption of Figure 11 "....and self reported results from BridgerPhotonics group white paper [? ]."
6. Between Line 399-400:  "...I historical developments..." typo?
7. Table 2: what are the units of Avg Precision and Avg Accuracy?

Author Response

Response to Reviewer 2:

1. Thank you for your careful review of our paper with constructive comments.The initial aim of the paper was to capture quantification methodologies. Although the sensors and equipment are vital to mission success, they have been mentioned in other literature works (e.g. in [1]). For that reason, we decided to keep the section brief. Additionally, the structure and content has been modified to be more readable. Two paragraphs have been added outlining sUAS integration and deployment as well as one figure showcasing sensor mounting.
2. A discussion has been added on the results from the comparisons between the UAVs and conventional platforms.
3. The authors have corrected this issue by wrapping the equations with the `linenomath*’ function.
4. This typo has been resolved. Thanks for spotting this.
5. This reference link issue has been resolved. Thanks for the catch.
6. This grammar issue has been corrected. Again thanks.
7. The values given in Table 2 for average precision and accuracy are unitless, such that they are normalized based on the true source rate (or the estimated source rate). An explanation of this was added to the Table 2 caption.

[1] Burgués, J.; Marco, S. Environmental chemical sensing using small drones: A review.Science of The Total Environment 2020, p. 141172.

Reviewer 3 Report

1. The relations (namely, the suitability) between the methods mentioned in Section 7 and the use of UAV should be better clarified.
2. Since quantitative figure of merits are defined for comparison purpose (Table 1), a final ranking of the discussed methods, in terms of the best suitability for UAV-based leak detection, should be also included and discussed.
3. Section 1. It may be advisable including city and country of the headquarters of the companies mentioned in the Introduction (lines 39-41)
4. Some reference numbers are missing, and there is a question mark instead [?]. Please, check.
5. Acronyms should be spelt out the first time they are used (e.g. EC Towers). Please, check.
6. The title of Section 4.3 may be missing, whereas the current Section 4.3 should be 4.3.1 (and, accordingly, 4.3.1 --> 3.2)

Author Response

Response to Reviewer 3:

Thank you for your careful review of our paper with constructive comments.

1. The authors have revised section 7 with relations to how sUAS can be used.
2. A final ranking based on the figures of merit has been included in a table and discussed.
3. The city and country of the companies have been added
4. The reference linking issues have been resolved.
5. All of the acronyms are now spelled out first before using the abbreviated term.
6. The section was labeled incorrectly in the latex code. It has been adjusted to subsubsection, such that it is a subsection of `4.2 OPTIMIZATION-BASED' methods.

Reviewer 4 Report

First, I'd like to commend the authors for their thorough and relevant paper. Some comments aimed at driving further improvement in the text:

1. The Introduction spans 150 lines and gets pretty far down in the weeds. While this information is very informative, consideration should be given to relocating some of the content to an appendix to make the reading of the body of the paper more straightforward.
2. I'm not sure that a compelling argument was made that the content is restricted to UASs or even more specifically, sUASs. I mean this as a compliment as the authors sort of inadvertently are narrowing the applicability of the content. Many of these techniques could also be leveraged by non-UAS platforms. Perhaps the title should be something like, "Advanced Leak Detection and Quantification of Methane Emissions using Various Techniques."
3. While the paper is very thorough, one of the sections that is the lightest on details is the "Sensors and Equipment" section. It is recommended that more material should be included in this section.
4. Lines ~269 through 400 are very interesting an important to the context of the paper. However, that material is (understandably) dense and hard to navigate. I would suggest moving most of the content to an appendix (which would serve as a handy reference for those interested) and replacing with a page-long summary in the body of the report.
5. There are several missing (noted as [?]) references in the paper.
6. In Table 3, consider adding the actual spelled-out method in addition to the abbreviation. I found myself having to scroll back and forth between the abbreviation section and this table, which resulted in making it harder to evaluate.

Author Response

Response to Reviewer 4:

Thank you for your careful review of our paper with constructive comments.

1. The introduction has been revised to improve readability and flow.
2. The authors appreciate the comment. The authors initial aim was to provide a broad comparison of methods for quantification between conventional approaches and sUAS. Several lines have been added to the introduction justifying the use of sUAS, such as, the advantages over ground-based mobile and static measurements. The authors also acknowledge that these methods can be applied to UAS but are intended for sUAS due to lower cost and easier deployment.
3. The initial aim of the paper was to focus on quantification methods of sUAS and although the sensors and equipment are vital to mission success, they have been mentioned in other literature works (e.g. in [1]). For that reason, we decided to keep the section brief. However, we did add some details on sUAS integration and deployment, as well as a figure showcasing payload configurations.
4. The authors appreciate the comment and have added a section overview (summary) at the beginning of the section `Advanced Leak Detection and Quantification Methods’ to improve understanding and readability.
5. The reference linking issues have been resolved.
6. In order to preserve the size of the table the authors added the method descriptions in the beginning of the section `Summary of Methods’ where Table 3 is referenced.

Round 2

Reviewer 1 Report

This revision is much improved compared to the original.  My previous objections to publication in Drones have mostly been addressed, particularly with the addition of Figure 13 and the related text.  The Bridger text of Fig.13(c) should be removed for consistency with the other plots and to eliminate the commercialism.

Author Response

Please see the attached.

Author Response File: Author Response.docx