Establishing Training and Certification Criteria for Visual Observers of Unmanned Aircraft Systems

Dolgov, Igor

doi:10.3390/safety4020015

Abstract

Safe integration of Unmanned Aircraft Systems (UAS) into airspace generally occupied by manned aircraft and other aviation stakeholders is a pressing global challenge. In the United States, efforts are being made to integrate small and large UAS into the National Airspace System (NAS). Whereas regulations for the civil operation of small UAS (25 kg and lighter) have already been adopted, those for larger unmanned systems are still being crafted. Thus, a two-part mixed methods study was conducted to examine three pivotal issues in the safe operation of large UAS: (1) What kind of visual observer skills are needed to execute safe UAS operations; (2) Should visual observers involved in UAS operations receive formal training; and (3) Should visual observers be required to pass a certification exam? In the first phase, subject matter experts identified various vigilance, trajectory estimation and communication skills that were vital to performing visual observer duties successfully and elaborated on their training regimens. In the second phase, survey participants were approximately evenly split on the need for formal classroom/online and hands-on visual observer training. Furthermore, participants generally favored visual observers having to pass a classroom/online certification exam, whereas they were against a practical (hands-on) exam.

Keywords:

aviation; safety; UAS; unmanned aircraft system; Drone; visual observer; training; certification; integration; National Airspace System

1. Introduction

The establishment of visual observer training and certification requirements for unmanned aircraft system (UAS) operations is a component in ensuring the safe and efficient integration of UAS into the U.S. National Airspace System (NAS). As stated in the Federal Aviation Administration’s (FAA) Unmanned Aircraft Systems (UAS) Operational Approval policy notice [1], visual observers are expected to be responsible for: (1) Helping UAS pilots keep the aircraft within visual line of sight and (2) Exercising see-and-avoid responsibilities by preventing the unmanned aircraft from creating a collision hazard and maintaining compliance with federal regulations, including Operating Near Other Aircraft [2], Right-of-Way Rules: Except Water Operations [3] and Right-of-Way Rules: Water Operations [4]. So, to ensure the safety of UAS operations, visual observers must be able to visually scan the airspace effectively, track aircraft and make accurate and reliable estimates of (relative) aircraft position, assess the need for a potential avoidance maneuver and communicate that need to the UAS pilot in a timely manner [5].

In 2015, the FAA released a Notice of Proposed Rulemaking for civil UAS operations [6]. The proposed language stated that flights were to be limited to small UAS (sUAS) operated within visual line of sight for the sUAS pilot during daylight hours in visual meteorological conditions. Some simple operational tasks, like flying a predefined route in dedicated airspace, allow sUAS pilots to maintain the aircraft in their line of sight throughout flight. However, more complex operational tasks, like flying an ad-hoc route in shared airspace, typically require the pilot to actively maneuver the aircraft. This can result in fewer opportunities to maintain visual contact with the sUAS and perform see-and-avoid tasks, and, consequently, reduced situation awareness and increased cognitive load. So, in the following scenarios the Notice of Proposed Rulemaking suggested that sUAS operations would necessitate a visual observer [6,7]:

Operations conducted above 500 feet above ground level or beyond 1500 feet laterally from the pilot-in-command must have at least one dedicated visual observer, even if visual line of sight can be maintained by the pilot-in-command.
When the pilot-in-command determines that one or more visual observers are necessary for the flight crewmember to maintain the safety of the operation.
When the pilot-in-command is (expected to be) in a “heads-down” or any situation that precludes the ability to perform see-and-avoid duties.
When the pilot-in-command is within an enclosure, at least two visual observers are required.

The sUAS Notice of Proposed Rulemaking also specified the following guidelines for visual observers [6,7]:

In order to be certified, applicants for a visual observer certificate would be required to pass a practical test with either a certified sUAS pilot or instructor.
Medical standards and operational limitations in this proposed rule ensure that the pilot and visual observer are capable of scanning the airspace of intended operations. Aids to vision, such as binoculars, must be used with care to ensure that the total overall viewing of the airspace is not inadvertently limited. An FAA second-class medical certificate is required for commercial operations.
The pilot-in-command must be able to see or ensure that a visual observer is able to see the aircraft throughout the entire flight well enough to: Know its location, determine its attitude and direction to exercise effective control, observe the airspace for other air traffic or hazards and determine its altitude.
The visual observer will be required to always know where the sUAS is and to discern the attitude and trajectory in relation to conflicting traffic, obstacles, or inclement weather. Because of the level of vigilance that would be required in scanning the surrounding airspace, a visual observer would be prohibited from supporting more than one aircraft operation at a time.
The visual observer will need to be in close proximity (within 10 feet) to the pilot-in-command and should be able to communicate directly, exchange non-verbal signals and share the same relative visual references. A backup communications system is required for operations where the pilot-in-command is in an enclosure and cannot directly see at least one visual observer.
Operations that involve the hand-off of UAS control and/or see-and-avoid duties among multiple pilots and/or visual observers, sometimes referred to as ‘daisy-chain,’ ‘relay,’ or ‘leap-frogging,’ would not be authorized under this proposed rule.

In 2016, many of these guidelines were reiterated in the FAA’s Small Unmanned Aircraft Regulations [8] for operations in the NAS. Similar to the Notice of Proposed Rulemaking, these regulations state that flights are limited to sUAS (25 kg or less) operated within visual line-of-sight in visual meteorological conditions. However, the proposed operational ceiling was lowered by 30 m and, notably, all certification requirements for visual observers were dropped. This has resulted in a potential regulatory gap.

Moreover, while systems larger that 25 kg are allowed to operate in the NAS with special certification from the FAA, general guidelines for such systems are still under development. Much care must be taken with this process because operating larger unmanned platforms inherently carries more risk due to the increased momentum of the aircraft and sturdier construction compared to their smaller counterparts. In addition, larger UAS platforms are flown with the pilot-in-command inside an enclosure much more frequently. Thus, it follows that flying such aircraft in the NAS will require visual observers to maintain visual line of sight in a variety of operational scenarios and settings. Furthermore, the added risk in such operations provides a clear impetus to standardize visual observer training and establish performance benchmarks such that visual observers can become certified to serve in the operation of large UAS (heavier than 25 kg). Although prior experimental work has examined visual observers’ performance in field settings (e.g., [5]) and unmanned crewmember aviation training studies exist (e.g., [9]), no prior research has investigated training and certification of visual observers.

Thus, a two-phase mixed methods study was conducted to address this timely and complex issue. Phase 1 was a phenomenological study that consisted of guided subject matter expert interviews. Phase 2 was a descriptive study which consisted of a broad survey of aviation stakeholders that focused on two pivotal issues: Should visual observers receive formal training and should visual observers be required to pass an exam?

2. Materials and Methods

The initial phase of this research (New Mexico State University Institutional Review Board #12314) was a phenomenological inquiry and consisted of interviewing three subject matter experts who were licensed manned aircraft pilots and also performed the roles of UAS pilot, visual observer and mission commander. These individuals were selected due to their deep understanding of all parts of manned and unmanned aircraft operations in the NAS. The interviews lasted approximately 60 min and were guided by the research questions noted in Appendix A. These interviews were transcribed and coded to examine subject matter experts’ background, training, assessment of vital skills and technologies needed to perform visual observer duties, assessment of UAS operation risks in various conditions and assessment of various current and potential UAS regulations (see Table A1, Table A2, Table A3, Table A4, Table A5 and Table A6).

The second phase (New Mexico State University Institutional Review Board #15418) of this research consisted of a brief descriptive survey of aviation stakeholders (see Appendix B) that was distributed in aviation forums, social networking site groups of NAS stakeholders and via email (snowball sampling).

3. Results

3.1. Phase 1

All subject matter experts reported that they underwent a combination of computer-based and practical hands-on training before engaging in visual observer duties. The lengths and particulars of the training programs varied but all were completed over a few days. All subject matter experts were licensed manned and unmanned aircraft pilots prior to visual observer training, thus the majority of the training was redundant with their prior knowledge.

Open-coding was performed on the interview transcripts (available in the Supplementary Materials); tables of the generated research questions and codes are noted in Appendix A. The subject matter experts unanimously stated that visual observers should be well-versed in FAA rules and regulations regarding manned and unmanned aircraft operations in the NAS. As noted in Table A5, subject matter experts stated that visual observers must establish proficiency in the following skills that are needed to successfully accomplish see-and-avoid duties:

Tracking unmanned and manned aircraft in various lighting and meteorological conditions:
1.
Must be able to maintain visual-line-of-sight to the aircraft
2.
Must be able to re-engage visual contact after loss and/or distraction
Scanning airspace for approaching air traffic:
3.
Must be able to shift visual depth of field
Informing pilot of impending near midair collision or some other danger with enough time for the pilot to take appropriate action:
4.
Must maintain cockpit discipline
5.
Must use appropriate verbiage when communicating with the pilot
6.
Must be able to use global bearings and local landmarks to identify positions of UAS and other air traffic
7.
Must be able to estimate aircraft flight paths, altitudes and closure rates in order to determine the likelihood of a near midair collision
8.
Must be able to determine and communicate correct course of action and a safe deviation from the flight path to avoid a potential near midair collision

3.2. Phase 2

Phase 2 consisted of a broad survey of aviation stakeholders (see Figure 1 for participants’ roles in aviation). As shown in Table 1, participants were approximately evenly split when answering the questions of whether visual observers involved in operations of UAS 25 kg and heavier should receive formal classroom/online and hands-on training (survey data available in the Supplementary Materials). While participants favored having to pass a formal classroom/online exam, the difference in Yes/No response counts was not significant. Furthermore, participants were generally against a formal practical exam; this was the only significant difference identified by χ² (chi-squared) tests.

Figure 1. Phase 2: Bar graph depicting participants’ reported roles in aviation; the total number of included participants is 112. The label ‘frequent flyer’ applies to pilots or passengers and the acronym ‘ATC’ stands for air traffic controller. Categories were not mutually exclusive.

Table 1. Yes/No counts for survey responses and results of statistical (χ²) tests in Phase 2. A 2 × 2 contingency table was constructed for each question, comparing the observed number of Yes and No responses to the expected values of 56 Yeses and 56 Nos.

Multinomial logistic regression and maximum likelihood ratio tests were then conducted to determine whether participants’ roles in aviation and age predicted their responses to the questions in Table 1. The models could not significantly predict Yes/No responses to the first three questions. However, participants’ roles did significantly predict responses to the final question of whether visual observers should take a formal practical exam (χ²(12) = 25.4, p = 0.013) and accounted for approximately 28% of the variance (Nagelkerke pseudo R² = 0.28). More specifically, commercial UAS pilots (χ²(1) = 3.6, p = 0.058) were marginally more likely to respond ‘No,’ whereas air traffic controllers (χ²(1) = 3.2, p = 0.073) and aircraft technicians (χ²(1) = 8.1, p = 0.004) were significantly more likely to say ‘Yes.’ Furthermore, older participants were marginally more likely to respond ‘Yes’ (χ²(1) = 3.55, p = 0.06).

Fewer than half of the participants provided reasons for their Yes/No answers. When examining the reasons for why participants felt that training and examination were necessary, participants noted the usefulness of classroom training in understanding NAS regulations and manned/unmanned aircraft operations, as well as increased risk due to platform size. For example, one participant stated, “Visual observers must be familiar with aeronautical information and performance standards in order to judge performance and assess hazards properly.” Another stated, “Knowledge of airspace, meteorology and human factors and crew resource management will help make for a more competent and safe crew.” Those participants which felt that practical training was needed stated that hands-on knowledge is irreplaceable. For example, one participant noted that, “(Visual observers need to) practice acquiring and re-acquiring UAS at different distances and altitude, (and to) understand how to determine the attitude and distance from the (ground control station) of the UAS,” and another said, “(Hands-on training is needed) because practical lessons provide a more realistic perspective to real world situations.”

When recommending training regimens, participants mainly suggested online/print materials, such as the sUAS airmen certification standards manual [10], the ALC-451: small Unmanned Aircraft Systems (sUAS) online course [11], the Aeronautical Information Manual [12] and commercially-available visual observer self-study guides. One participant that thought practical training was necessary mentioned simulator training as an option. Furthermore, another participant stated, “(Hand-on testing should be a) continuation of the formal practical training, another day would be the test day, where a spotter would be paired up with an operator who will then ensure the spotter goes through the list of operations correctly and may even test them by trying something not allowed.”

4. Discussion

To ensure safe UAS operations, visual observer duties primarily require them to assist UAS pilots in effective see-and-avoid. To date, visual observer training regimens have yet to be standardized and feature both practical and classroom/computer-based components. As Williams and Gildea [7] note, there are no adequate human perceptual models of aircraft detection, primarily due to the large number of variables involved. Thus, given some acceptable level of visual competency (such as medical clearance), it is likely that a visual observer training regimen should focus on non-perceptual aspects of operations, such as rules, regulations, procedures and related matters. Although 14 CFR § 107 [8], the Small UAS Rule, did not specify a requirement for visual observer certification, visual observer certification may be appropriate for personnel engaged in operations involving larger UAS, which carry added risk.

While aviation stakeholders acknowledge the additional risk of operating a UAS larger than 25 kg, there is currently a lack of consensus regarding whether training and/or certification needs to be mandated. Although participants in Phase 2 somewhat favored classroom/online certification, the difference in Yes/No counts was not statistically significant. However, this trend is indicative of numerous factors that make operations of such aircraft more dangerous than their lighter counterparts. Notably, significantly more participants thought that practical (hands-on) testing was not necessary than those that did. Moreover, while UAS and manned aircraft pilots were well represented in the sample, other NAS stakeholders were not. This limitation underlines the need for additional research in this domain that carefully considers the needs of all those that occupy the sky.

Furthermore, due to the human factors of UAS operations [5,7,13,14], one training and certification solution may not be optimal. Licensed pilots have already mastered many, if not all, of the essential skills needed to carry out visual observer duties. In addition, licensed pilots have already been trained and certified in their knowledge of rules and regulations pertaining to operations in the NAS. Likewise, manned aircraft pilots have a multitude of hours performing seek-and-avoid duties, are well-versed in aviation verbiage and are trained in maintaining proper cockpit discipline. Thus, it appears that pilots and non-pilots should require different training regimens and certification criteria, akin to the distinction between licensed pilots and non-licensed individuals made by the FAA when persons apply for sUAS certification.

Supplementary Materials

Interview transcripts and survey data are available online at http://www.mdpi.com/2313-576X/4/2/15/s1.

Acknowledgments

The Federal Aviation Administration has sponsored this project through the Center of Excellence for Unmanned Aircraft Systems. However, the agency neither endorses nor rejects the findings of this research. The presenting of this information is in the interest of invoking technical community comment on the results and conclusions of the research.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A

Table A1. Research Question: What roles/duties have participants carried out in the context of manned and unmanned aircraft operations?

Table A2. Research Question: What types of training and certification did participants receive prior to engaging in aviation duties?

Open Code	Properties	Example
Manned Pilot Training Program	Training program involves both classroom and field training	“I mean the pilot training was excellent and I had previously taken ground school before I did that. … So, the pilot training and just actually working with aircraft, the flight training, the hands-on stuff.”
Manned Certification	Licensed FAA pilot, certified military pilot	Experimenter: “And are you or were you a licensed pilot?” Participant: “I am.”
UAS Pilot Training	Training program involves both classroom and field training	“Of course with the Predator I did the pilot training some of that included mission training because to fly the airplane you have to be able to work with the payload operator, move the aircraft around so the payload guy can see what he’s looking at, the communications, airspace, all the things that go into operations in a war zone.”
UAS Pilot Certification	Military, FAA and manufacturer certification	“Airforce check ride. Ground check ride and then a flight check ride.”
UAS Mission Commander Training	Training program primarily involves classroom/manual-based training	“… with the Predator, I did the pilot training some of that included mission training …. So, that carried through my CFI training which is also safety based. Then to get to the university, since I was trained at least going through the courses for each aircraft I knew what the safety hazards were …. So, I just threw all those things together in order to be able to be mission commander.”
UAS Visual Observer Training	Training program involves both classroom and field training	“We have PowerPoints then we have actual on the job training …. So, I can’t really think of anything that’s rocket science and you have got to teach a visual observer. If they’ve got decent hearing and decent eyesight and they can stay vigilant and communicate clearly that’s all there is to it.”
UAS Visual Observer Certification	No certifications were noted	“Section 333, as of right now, there’s no requirements for the visual observer. It has no age requirement, has no training requirement, has no minimum …”

Table A3. Research Question: What aspects of training were most beneficial?

Table A4. Research Question: What aspects of training were lest beneficial?

Table A5. Research Question: What are essential skills/knowledge for visual observers?

Table A6. Research Question: What aspects of UAS operations impact risk?

Open Code	Properties	Example
Momentum	Weight; Speed	“I thought that the 55-pound weight was too big for the speed that they’re allowed. That’s a lot of mass and can cause a lot of damage really fast and danger…. There’s no doubt that the training required to fly the Predator is a lot more extensive than what is required to fly a multi-copter. It was, it is and has to be.”
Components/Configuration	Fixed wing aircraft vs multi-rotor aircraft; Rotor guards	“I have a multi-copter and I’ve crashed it twice and in both cases I threw a propeller blade. I didn’t do anything wrong; I didn’t hit anything…. you’re just flying along and one blade came off. It didn’t hit anybody it just turned upside down … I’ve never had a wing come off an airplane, if it did that’d be catastrophic so you got to weigh that.”
Distance/Line of Sight	Ability of UAS pilots and/or visual observers to see the UAS and scan surrounding airspace; Weather; Time of Day; Proximity to Hazards	“We’re flying at low altitude and for our flights we generally have three observers. We have one on each end of the flight area and one would be external pilot. So, we have people around that are an extra set of eyes. And the external pilots found that even with the larger aircrafts they didn’t want it as far away as they could possibly see it. And then the radio controls too, the larger one is probably large enough that it could get out of range of the radio controls.” “So the size [matters], an Aerostar or Tigershark [has] got all the light systems on it with a strobe then the risk I don’t really foresee that risk being any different than during the day. But if you’re looking at something like the Phantom, it’s got lights on it, you can see it from the ground but is a pilot in the air going to see it? Nope. So large UAS, sure I don’t see any reason why a large system can’t fly beyond line of sight with light systems on it at night, as long as they’re equipped with transponders, ASB, something of that sort and well-lit but small systems to be honest there is no reason to fly them at night.”

Appendix B

Survey of Aviation Stakeholders

Please try to answer all questions to the best of your ability. If you do not feel comfortable answering any of the questions (or their parts), please feel free to skip them.

(1)

Demographics (used U.S. government-provided definitions for items C and D).

a. Age b. Gender c. Race d. Ethnicity

(2)

What is your (current or past) role in the field of aviation? (Please check all that apply)

Private Manned Aircraft Pilot
Commercial Manned Aircraft Pilot
Private Unmanned Aircraft System Pilot
Commercial Unmanned Aircraft System Pilot
Unmanned Aircraft System Visual Observer
Other Unmanned Aircraft System Crewmember (mission commander, payload operator, etc.)
Air Traffic Controller at a Regional Airport
Air Traffic Controller at an International Airport
Manned/Unmanned Aircraft Technician
Frequent Flyer
Other

(3)

Currently, 14 Code of Federal Regulations Part 107 states that unmanned aircraft, 55lbs and under, may operate in Class G airspace (and classes B, C, D and E with ATC approval), provided that they do so during visual meteorological conditions and within visual line of sight. In certain scenarios (pilot in heads-down position or in an enclosure), a visual observer is required during operations. Currently, no certification or training is required for visual observers involved in such operations? Do you think that visual observers involved in unmanned aircraft system operations greater than 55lbs should be required to:

Receive formal classroom/online training? (Y/N)
Receive formal practical (hands-on) training? (Y/N)
Pass a formal classroom/online certification exam? (Y/N)
Pass a formal practical (hands-on) certification exam? (Y/N)

(4)

If you answered ‘Yes’ to any of the items in question #3, please explain the reason why. (Please write N/A for corresponding items to which you answered ‘No’ in Question #3.)

a. b. c. d.

(5)

If you answered ‘Yes’ to any of the items in question #3, please explain what kind of training/certification procedures would be ideal. (Please write N/A for corresponding items to which you answered ‘No’ in Question #3.)

a. b. c. d.

References

Unmanned Aircraft Systems (UAS) Operational Approval; National policy notice #8900.277; Federal Aviation Administration: Washington, DC, USA, 2013. Available online: http://fsims.faa.gov/wdocs/notices/n8900_227.htm (accessed on 13 November 2017).
Operating Near Other Aircraft, 14 CFR § 91.111. 2016. Available online: https://www.gpo.gov/fdsys/granule/CFR-2011-title14-vol2/CFR-2011-title14-vol2-sec91-111 (accessed on 3 April 2018).
Right-of-Way Rules: Except Water Operations, 14 CFR § 91.113. 2016. Available online: https://www.gpo.gov/fdsys/granule/CFR-2014-title14-vol2/CFR-2014-title14-vol2-sec91-113 (accessed on 3 April 2018).
Right-of-Way Rules: Water Operations, 14 CFR § 91.115. 2016. Available online: https://www.gpo.gov/fdsys/granule/CFR-2012-title14-vol2/CFR-2012-title14-vol2-sec91-115 (accessed on 3 April 2018).
Dolgov, I. Moving towards Unmanned Aircraft Systems integration into the National Airspace System: Evaluating visual observers’ imminent collision anticipation during day, dusk and night sUAS operations. Int. J. Aviat. Sci. 2016, 1, 41–56. [Google Scholar]
Notice of Proposed Rulemaking: Operation and Certification of Small Unmanned Aircraft Systems; Docket No.: FAA-2015-0150; Notice No. 15-01; Federal Aviation Administration: Washington, DC, USA, 2015. Available online: https://www.faa.gov/regulations_policies/rulemaking/recently_published/media/2120-AJ60_NPRM_2-15-2015_joint_signature.pdf (accessed on 13 November 2017).
Williams, K.; Gildea, K. A Review of Research Related to Unmanned Aircraft System Visual Observers; Technical report #DOT/FAA/AM-14/9; Federal Aviation Administration Office of Aerospace Medicine: Washington, DC, USA, 2014.
Small Unmanned Aircraft Regulations, 14 CFR § 107. 2016. Available online: https://www.gpo.gov/fdsys/search/pagedetails.action?collectionCode=CFR&browsePath=Title+14%2FChapter+I%2FSubchapter+F%2FPart+107&granuleId=CFR-2017-title14-vol2-part107&packageId=CFR-2017-title14-vol2&collapse=true&fromBrowse=true (accessed on 3 April 2018).
Craighead, J.; Murphy, R.; Burke, J.; Goldiez, B. A survey of commercial & open source unmanned vehicle simulators. In Proceedings of the IEEE International Conference Robotics and Automation, Roma, Italy, 10–14 April 2007; pp. 852–857. [Google Scholar]
Remote Pilot—Small Unmanned Aircraft Systems Airman Certification Standards; Document # FAA-S-ACS-10; Federal Aviation Administration: Washington, DC, USA, 2016. Available online: https://www.faa.gov/training_testing/testing/acs/media/uas_acs.pdf (accessed on 13 November 2017).
ACL-451: Part 107 Small Unmanned Aircraft Systems (sUAS) Online Course. Available online: https://www.faasafety.gov/gslac/ALC/course_content.aspx?cID=451 (accessed on 13 November 2017).
Federal Aviation Regulations. FAR AIM 2018 Aviation Regulation Manual; Federal Aviation Administration/ASA: Washington, DC, USA, 2018; ISBN 978-1619545366.
Dolgov, I.; Hottman, S. Human factors of UAS. In Introduction to Unmanned Aircraft Systems, 1st ed.; Barnhart, R., Hottman, S., Marshall, D., Shappee, E., Eds.; CRC Press: Boca Raton, FL, USA, 2011; pp. 165–181. ISBN 978-1439835203. [Google Scholar]
Dolgov, I.; Kaltenbach, E.; Khalaf, A.; Toups, Z. Measuring human performance in the field: Concepts and applications for Unmanned Aircraft Systems. In Human Factors in Practice: Concepts and Applications, 1st ed.; Cuevas, H., Velasquez, J., Dattel, A., Eds.; CRC Press: Boca Raton, FL, USA, 2017; pp. 37–54. ISBN 978-1472475152. [Google Scholar]

© 2018 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Article Metrics

Citations

Article Access Statistics

Journal Statistics

Multiple requests from the same IP address are counted as one view.

Open Code	Properties	Example
Manned Pilot	Certified military, commercial, or private manned aircraft pilot	“Yes, I am a licensed pilot.”
UAS Pilot	Certified military, commercial, private unmanned aircraft pilot	“I have acted as an internal pilot at the computer controls for flights and ... mission commander... and, also, I act as an observer.”
UAS Mission Commander	Mission commander for commercial (university) UAS operation	See previous.
UAS Visual Observer	Safety-related visual observer in UAS Operations	See previous.
UAS Flight Instructor	Flight instructor in UAS flight school	“I was hired by {company} as a flight instructor.”
Radio Controlled Aircraft Pilot	Recreational pilot of radio-controlled aircraft	“So, recreationally, I fly RCs. I’ve been flying since I was nine on that.”

Open Code	Properties	Example
Apprenticeship	Being trained by an experienced instructor / pilot	“… a new guy gets back from Iraq who’s been over there for 3 or 4 months and he’s kind of burned out and tired and they put him in with these students but they were the best people with all the information. Not just what the book said but what really happened what really worked.”
Hands-on Training	Practical training with real aircraft	“Hands on training for me I think is the most valuable being able to fly the aircraft safely and being able to decide whether it’s safe or not to operate an aircraft because sometimes proponents don’t know when to take a step back and think about safety.”

Open Code	Properties	Example
Effective See-and-Avoid	Visually tracking aircraft; Disengaging to scan for other air traffic and re-engaging; Identifying potential near midair collisions; Orienting using local landmarks and cardinal directions	“If you’re just doing regular commercial operations, then yeah I mean they can be within line of sight… You look around if you see an airplane, [then you estimate] distances and position. Whether you feel it’s going to be a factor or not to the flight of the aircraft do we need to divert or not. That’s really about it.”
Effective Communication	Communicating with pilot in person or via radio; Using global and local landmarks to relay aircraft position	“How to operate the radios, what you’re looking for…’Here’s south, figure out where it’s coming from. If you’re not sure give someone a call.’” “As long as you can get the message across to the pilot to maneuver the aircraft in time then yeah that should be fine.”
Aircraft Operations Rules	Knowledge of airspace and FAA regulations	“The other day I was at the RC field and a guy was flying a parasail. Don’t need a pilot license, don’t need to have any training on airspace and he’s circling our RC field … and he’s down at like 150 feet but that’s safe according to the FAA. But flying a smaller UAS below that altitude, we have to have a pilot’s license.”

Should Visual Observers:	Yes	No	χ² Value	p Value
(1) Receive formal classroom/online training?	58	54	0.07	0.71
(2) Receive formal practical (hands-on) training?	55	57	0.16	0.57
(3) Take a formal classroom/online exam?	61	51	0.64	0.26
(4) Take a formal practical (hands-on) exam?	43	69	5.79	0.00006