Safety Occurrence Reporting amongst New Zealand Uncrewed Aircraft Users

Round 1

Reviewer 1 Report

This manuscript objective is to examine the prevalence of UA safety occurrences, how these safety occurrences are being reported, and why they are being reported using particular systems.  The objective, in fact three objectives, is only partly reached.  The methodology, based on an online survey, does not address the prevalence of UA safety occurrences, especially with the small sample.  Only one approach is used for the data collection and therefore no verification of the data is possible.  The literature review could be considered as an additional method, but the section 2 is too long and not organized to enable a comparison with the online survey.  Also, the authors does not mention the use of a mixed-method (litt review and survey) to reach the objective. The results are however interesting and could be worthy of publication if major modifications to the manuscript are done.  The manuscript length should be reduced to focus on the main finding and avoid repetitions.

Abstract: a sentence should be added at the start to explain the context.

Introduction: The methodology, presented in the abstract, should be repeated after the objective in the Introduction(line 52).  The last sentence, Line 58-60, does not belong to introduction but to the discussion or the conclusion.  After the methodology, the authors should explain the manuscript organization.

Literature Review: This section is too long, and I do not understand the utility of it for the following sections.  If the goal is to convince the reader that there is a need to improve the occurrence reports, or that not enough data are collected about UA occurrences, only two to three short paragraphs are needed.  Specifically, how section 2.4 is relevant for the present manuscript.  Why not only focus on New Zealand, as this is the survey sample? The information about other regulations is not used for this study. Then, section 2.5 brings some qualitative information about safety occurrence, but hardly related to the quantification data from the survey.  I do not see the relevance of section 2.6 (The importance of safety occurrence reporting and barriers to safety reporting) for the manuscript.  Maybe this section should belong to the discussion section.  Most of the topics discussed are repeated in section 5. Lines 340-342, lines 354-356, line 394-396, lines 404-406, lines 417-419 repeat information previously mentioned.  The information should be grouped by topic: actual legislation, need for reporting, obstacle for reporting, examples.

Finally, additional literature review is needed for the method justification. Is it the usual practice to collect information.

Methods: Why is this method the best way to reach the objective?  The participants recruitment introduces some bias, could you discuss it. Are the 110 responses obtained representative of the UA operators in NZ.

Discussion: Even the study indicates that a very large portion of reportable occurrences in NZ are going unreported, it does not have information about the prevalence of UA safety occurrence. Also, the results are valid only if the survey sample is representative of NZ.  You need to discuss this point. Some of the discussion results are already in the sections 1 and 2 of the manuscript. Also, since your main finding was expected, it gives the impression that you have a bias in your investigation.  Line 798-803 sentences are not related to the survey finding. 

Globally, the survey results are interesting, but mixed-methods should be used. For example, literature data could be used to verify the results. However, the manuscript should be organized consequently. The length should be shortened to focus on the main contribution.

Author Response

We thank the reviewer for their efforts in reviewing our manuscript. Please find our responses to each comment in the attached document.

Author Response File: Author Response.pdf

Reviewer 2 Report

General comments

The paper has two parts

•         A part relative to the importance of “reporting to learn” from incidents/accidents that have occurred

•         Another part related to demonstrating that there is no culture of providing this information in the drone world.

If those were the objectives, both have been achieved; however, when wanting to do a statistical analysis with few data, the results can be accepted or not. Obtaining such mean values ​​with large SD is due to the few data available and for this reason perhaps one cannot rely much on the quantitative value.

Results have been obtained through surveys that could be expected even before doing the surveys. It is true that they have been given a statistical result, but since the value of the sample is low, I do not know if these values ​​can be accepted.

On the other hand, a distinction must be made between recreational use and professional use. In recreational use, integration into controlled airspace is not foreseen, professional use is. I believe that data (recreational/professional) should be treated completely independently and should never be mixed. Certain requirements should not be asked for recreational use since in that case the requirements could prevent the development of the activity. Obliging to belong to certain groups can mean high costs and perhaps for this reason prevent the development of recreational use.

It is true that users must be informed/trained, particularly recreational ones, but it could be through official websites (CAA) in which things are explained, in which airspace you can fly and in which not. What is controlled airspace, and which is not, etc. It is not about requesting flight licences; it is about officially informing users so that they do not have to resort to private websites or internet forums without rigorous information.

A balance must be struck between the requirements and the development of the activity. Imposing excessive requirements certainly increases safety, but one could go so far as to prohibit the activity to achieve maximum safety. This situation is like what occurs with free flight (paragliders) which, due to their lack of training, sometimes enter controlled airspace, or climb more than they should, and many times it is due to lack of information in official websites, which makes them consult user forums on the internet with untrue information.

In my opinion, the requirement to report should be requested for those incidents that have occurred with drones with a certain certification which will be used, in general, for professional activities. The “small” RPAs for recreational use should be out of this requirement.

Some particular comments:

Section 2. Literature review

It is 7 pages length. Although its content is very interesting, perhaps too long.  

Section 4. Results

Just 92 valid responses may be not statistically representative. Then some subgroups are very small, so the quantitative results may be not acceptable. This idea is mentioned also by authors in section 7.

Section 5 Discussion

In general, I agree with the provided discussion. Some ideas have already been mentioned in section 2.

Section References: 79 references are very good

Comments for author File: Comments.pdf

Author Response

We thank the reviewer for their efforts reviewing our manuscript. Please find our responses to each comment in the attached document.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

All my requested corrections have been done, thank you.