Next Article in Journal
A CFAR-Enhanced Spectral Whitening Method for Acoustic Sensing via UAVs
Previous Article in Journal
Post-Logging Estimation of Loblolly Pine (Pinus taeda) Stump Size, Area and Population Using Imagery from a Small Unmanned Aerial System
Open AccessArticle

Analysis of Autonomous Unmanned Aerial Systems Based on Operational Scenarios Using Value Modelling

InfraDrone LLC., 2411 Grand Ave, Des Moines, IA 50312, USA
Aerospace Engineering, Iowa State University, Ames, IA 50011, USA
Author to whom correspondence should be addressed.
Received: 1 November 2017 / Revised: 20 November 2017 / Accepted: 21 November 2017 / Published: 23 November 2017
(This article belongs to the Special Issue The Use of Drones at Field Stations and Research Reserves)


In recent years, the use of UAS (Unmanned Aerial Systems) has moved beyond the realm of military operations and has made its way into the hands of consumers and commercial industries. Although the applications of UAS in commercial industries are virtually endless, there are many issues regarding their operations that need to be considered before these valuable pieces of equipment are allowed for widespread civil use. Currently, UAS operations in the public domain are guided and controlled by the FAA Part 107 rules after overwhelming public pressure caused by the earlier 333 exemption. In order to approach such larger issues, this paper will exploit the use of value models, which will help to quantify how the different environmental and operational scenarios play a role in UAS operations based on the task being performed. The primary aim of this research is to use the attributes from key factors of the UAS such as the autonomy levels (AL) and technology readiness levels (TRL) along with their operating scenario factors, such as the environmental complexity and task complexity, based on the operating environment in which a UAS performs its task. To analyze the performance of autonomous UAS in different operational scenarios, the physical characteristics and class of a UAS may be linked to its AL and TRL. Using these parameters, the risks faced by the UAS in a particular mission are quantified and a value is assigned to the abstract entities involved. Although there are many critical questions with respect to good practices to be followed by UAS operators in order to obtain valuable data and information on the structures being scanned and monitored, there are many other challenges with regards to large scale operations of UAS such as the ethical, legal and societal implications that have to be addressed.
Keywords: UAS; structural health monitoring; civil UAS applications; value modelling; operational scenario evaluation; value-based systems engineering; value driven design UAS; structural health monitoring; civil UAS applications; value modelling; operational scenario evaluation; value-based systems engineering; value driven design

1. Introduction

The introduction of UAS into the National Airspace System (NAS) comes with a lot of advantages and some disadvantages which translate to gains and losses for different stakeholders involved. The challenges faced by the government and multiple stake holders who wish to use UAS technology on a large scale are issues tied to safety, policy, law, ethics and privacy. This research study involves understanding the different factors that affect the deployment of UAS on a large scale by taking into account operational constraints and issues associated with them. Understanding these factors from the viewpoints of the users and operators may be used to aid in amending and developing policies for UAS operations, which are directly related to the ongoing controversy regarding the regulation of UAS.
With the recent increase in the number of UAS in the skies, measures have to be taken to properly regulate and safely carry out the transition into this new realm of aerial vehicle operation. This process will require the coordination and involvement of academia, industry, government agencies like the Federal Aviation Administration (FAA) and Department of Defense, local authorities, and private sectors alike in order to successfully implement a safe, efficient and robust system for commercial UAS operations [1,2,3,4]. The ultimate goal of such a high level partnership between these agencies would be to successfully integrate small UAS (sUAS) operations into the NAS in order to enhance the use of UAS for commercial applications, while considering the ethical, legal, societal and environmental implications of these technologies. The current UAS boom for civil applications must be cautiously carried out without violating any rights or safety of individuals or the environment.
The primary goal of this research study is to create a value model for UAS by taking into account the technical, environmental and operational scenarios involved in order to possibly create a decision analysis simulation to analyze and rate different UAS based on their components, autonomy level and technology readiness level. This in turn may be used in the future to understand the misalignment between stakeholders’ preferences and the regulations due to policy requirements imposed on UAS operations. The research will involve using a combination of various tools and frameworks such as multidisciplinary design optimization (MDO), value-driven design (VDD), and decision analysis (DA) to analyze and understand the various factors involved while designing and operating UAS on a large scale for the use of structural health monitoring (SHM) and civil applications.
The methodology for this research will necessitate identifying different classes and types of UAS that can be used for commercial purposes and relating the physical characteristics of these UAS to virtual characteristics such as AL and TRL in order to evaluate these systems for the purpose of SHM, while staying compliant with the FAA Part 107 regulations. These evaluation methodologies of the software and hardware of UAS can in turn be used to rate their performance and guide operational activities based on the tasks being performed around the different operational scenarios. This will allow the value model to be used to correlate a UAS to its operational environment. These characteristics can be represented in the form of attributes and design variables to create a value model that can capture and incorporate the technical design aspects of the UAS to the legal aspects of deploying UAS. In order to rate the performance capability of the UAS in its operational scenario and to abide by the legal constraints, the autonomy level and technology readiness levels of the associated and relevant components play a key role in the value model.
The Federal Aviation Administration is the operational branch of the Department of Transportation (DOT) that is tasked with all aspects of aerospace governance and safety within the United States [5]. Part 107 is the regulation over the operation of small unmanned aircraft systems (sUAS) [6]. The new Part 107 rules put forth by the FAA regarding the operations of sUAS will be used as the primary platform to setup the different scenarios to use either a single UAV or a swarm of UAVs to monitor different types of structures autonomously in order to provide a holistic view of the condition of the structure being surveyed. In order to implement this system in an efficient manner, the authors apply VDD to systems engineering [7]. This methodology will provide an interdisciplinary approach to the implementation of the system while being able to communicate the design and operational requirements with stakeholders’ preferences. Examples of such stakeholders include the UAS pilots, operators and the FAA. This will help identify relationships between the subsystems of the overall system while including economic theories to better optimize the design and operation of UAS based on the mission scenario, while incorporating the preferences of the stakeholders [7].

2. Unmanned Aircraft System Model

The following UAS model will lay the groundwork for the formulation of a multi-objective optimization function. A two tiered breakdown of a multi-copter UAS model has been developed as a part of this research in order to serve as a test bed for the execution of different operational scenarios required to perform different tasks critical to SHM. The hierarchical model shown in Figure 1 describes a two-tier hierarchical decomposition for a multi-copter UAS. The simplified model of a multi-copter UAS has five primary subsystems in its first tier: propulsion, power, GNC (Ground Navigation and Control), structures and payload. The second tier of subsystems comprise of motors, battery, IMU, GPS (united states implementation of a GNSS), flight computer, 3D FPV camera, arms, body and landing gear [8]. Since this is a relatively large model, each of the primary disciplines and their subsystems namely, GNC, structures, power, propulsion and payload will interact with each other. In order to capture the couplings of these interactions, a DSM will be used as shown in Figure 2.
A Design Structure Matrix (DSM) is a representation of a system decomposition that illustrates the relationships of the identified entities (subsystems) that make up an entire system of interest based on the interface media of interest such as, material interfaces, energy interfaces, etc. These interfaces are commonly represented as feed-forwards and feed-backs between subsystems to represent connectivity [9,10,11,12]. The DSM shown has the capability to visualize the connections between the subsystems as couplings, feed forwards and feedbacks [13]. This makes it easier for the user to model and analyze the system, and shows the dependencies among the subsystems. Using this DSM and the hierarchical breakdown models, we can list out the performance attributes that are affected by the first tier subsystems as shown in Figure 3. As seen from the figure below, propulsion affects the altitude, endurance and payload mass that can be carried by the UAS; power from the battery affects endurance and range; GNC affects range, stability and autonomy level, while the structure affects payload volume, payload mass, proximity and maneuverability. The payload attached to the UAS can be removed based on its mission profile and task, thereby affecting the capability and tasks that can be performed by the UAS. The payload attributes can be changed based on the types of sensing equipment that will be attached to the UAS for SHM.
In order to set up the optimization process, we needed to develop an objective function that captures the goals of the stakeholders and abide by the legal constraints set by the FAA. Equation (1) is the formulation for the minimization of mass approach. The mass is assumed to be a summation of all of the subsystems that have a mass which could affect the system.
m i n   f ( x ) = M a s s t o t a l = i = 1 N M a s s A i
Side constraints are developed to fit within physical bounds, such as to keep the mass of the UAS from dropping below zero or making it too heavy for takeoff. Inequality constraints, on the other hand, are based directly on the FAA Part 107 rules for UAS operations, such as maintaining a total mass of the UAS to be less than 55 lbs., maximum speed less than 100 mph, maximum flight altitude of 400 ft. above ground level and flight missions must be carried out within visual line of sight [14]. These constraints used in the traditional MDO formulation represent the desires of the stakeholder, which in this case is the government, and highlights the regions in the design space that are deemed infeasible to use while designing the fixed wing UAS.
The stakeholders in this setup will represent the government (FAA), UAS designer and the UAS operator. Based on their preferences, each stakeholder will have different values, so the value given only makes sense to the stakeholder. The true preferences of the stakeholders interested in taking advantage of UAS can be obtained by analyzing the viewpoints of stakeholders obtained from multiple sources as demonstrated in Table 1. This table is a representation of the true preferences of UAS operators from different industries; the data presented in the table is extracted from the NTIA convened multi-stakeholder discussions due to a presidential memorandum issued by former President Obama for “Promoting Economic Competitiveness While Safeguarding Privacy, Civil Rights, and Civil Liberties in Domestic Use of Unmanned Aircraft Systems” [15]. This meeting comprised of members and entities from academia, industry, government sectors and civil society, thereby giving a holistic view of their combined opinions for the development of regulation with regards to UAS.
In Table 1, the examples of agricultural associations representing farmers, energy and construction companies, university researchers representing academia and media houses representing the journalist’s point of view are used. These potential UAS operators have different operations and tasks based on their mission scenario, whether it is scanning a field, spraying pesticides or fertilizers on farmland, inspecting tanks for gas leaks or for capturing footage of an event to show on television. Due to the nature of the different operations, the requirements, needs and issues faced by each of the operators also vary from case-to-case based on the operational environment and tasks to be performed. UAS designers play a critical role in the cycle of designing the UAS with sufficient AL and TRL to oversee and possibly restrict these operators from violating the rules and regulations put forth by the FAA. This relationship between the primary stakeholders in this case, namely, the government, operator and designer is shown in Figure 4. The model for understanding the required AL and TRL for a UAS based on tasks and operational environments will be discussed in the following chapter using the example of SHM.

3. Value Model Formulation

This section will provide a detailed overview of the relations that were drawn between different tasks that a UAS will be performing with respect to the environmental scenario of the operations as well as the physical characteristics of a UAS. This section will also explain the approaches to quantifying the relationships between the UAS characteristics and operational scenarios. This section serves as a launch pad for the creation of risk and bargain models.
The primary parameters involved in the operations involving a UAS herein are identified as: The mission scenario, mission objective and the UAS characteristics, as shown in Figure 5. The mission scenario is defined by the meteorological conditions and the structure type; the mission objective is defined by the data acquisition method used and mission tasks and the UAS characteristics are divided by their size, class, AL and TRL. The structure type and the data acquisition method will have a set of attributes and metrics that will help to determine complexity. This methodology can be used as a universal model for analyzing the operational scenarios for UAS in other applications such as precision agriculture and photojournalism, but in this study, we will use the example of SHM to understand the effects of the operational scenario on the UAS. This example will be used to demonstrate complexities involved in the process of UAS based SHM and how improved performance may be achieved.

3.1. Value Model for Operational Scenario

The two main parameters that are used to evaluate the operational scenario in our model are: environmental complexity and task complexity. The environment complexity is further subdivided based on the meteorological conditions and the infrastructure type around which the UAS will be operating. The task complexity is subdivided into data acquisition type and vehicle type. This model with the subdivisions of each of the categories of the operational scenario is illustrated in Figure 6 shown below. A combined analysis of each of these parameters will be used to understand the effects of the operational scenario on the UAS while performing its tasks. In order to quantify each of these parameters, common types of scenarios were studied based on their tasks and environment, then metrics were created for the two main subdivisions of these parameters that define the operational scenario: infrastructure type and data acquisition type. These metrics are supported by weighted attributes and these weights are assigned based on the value of its relevance to the attributes.
The infrastructure type is divided based on the structure that will be surveyed by the UAS while performing SHM. The meteorological conditions will be based on factors surrounding the infrastructure type where the UAS will be performing and will be evaluated based on temperature, precipitation, wind speeds and humidity. Given tasks in the proposed model, the weather conditions are incorporated as one of the attributes contributing to the infrastructure type’s complexity. As for task complexity, vehicle type describes the types of UAS that can be used to perform the operations. The data acquisition types are broadly divided into three different categories: Imagery (2D), 3D modelling (Photogrammetry) and non-destructive evaluation (NDE). In the context of this work, imagery refers to the raw data collected by use of cameras attached to UAS that do not require further processing and will therefore have a 2D image as the final deliverable. 3D modelling refers to the process of synthesizing information collected from multiple 2D images for the purpose of providing a three dimensional space as a final deliverable when using a UAS. These data acquisition types vary in levels of difficulty and are selected based on tasks that a UAS may be capable of performing in order to collect information regarding the health of a structure as well as whether or not data can be processed by the UAS as opposed to requiring secondary and tertiary processing.
Although there are four major subdivisions for the two parameters pertaining to the operational scenario, in this model, we will only be quantifying the subdivisions that directly affect the UAS operating environment and not the type of UAS itself. This analysis will focus on quantifying the infrastructure types and data acquisition methods to compare it to the AL and TRL of the UAS. The reason for using select subdivisions in our analysis is to directly co-relate the environment factors to the autonomy of the UAS.
The first parameter explored with regards to the operational scenario is the environmental complexity. Table 2 was put together with a list of different infrastructure types that can be scanned by the UAS. This table of infrastructure type has a set of metrics ranging from 1 to 10 for a broad range of infrastructures varying in complexities. The value of the metrics increases as the complexity of the infrastructure type increases from open unpaved rural roads to suspension bridge superstructures. These metrics are assigned to attributes that correlate to the complexity of the infrastructure and they include size of the structure, physical obstacles around the structure, weather when the UAS operations will be conducted, geography of the surrounding location and factors that may cause signal interference to the link between the UAS and the pilot or ground station. These attribute scores will further be rated from 1 to 10 based on the comparative impact it has on each of the attributes.
Having these metrics and attributes for these infrastructure types setup, a function is created that can provide a value by combining these metrics, attributes and weights. In this model, equal weights are assigned to all the five attributes for the types of infrastructure, although the weights assigned to each of the attributes can be changed based on its importance compared to other attributes. It must be noted that the total sum of the weights of all attributes must add up to be 1. Each of these attributes also has its own set metrics ranging from 1 to 10, and this will be based on the impact of that particular attribute on the infrastructure type. For example, the metric value for the ‘size of structure’ attribute for a wind turbine depends on the comparative size of the wind turbine being inspected with regard to other wind turbines.
E ( x ) = M E i = 0 n w i × A i
Once the infrastructure being scanned is identified and all the metrics and weights have been assigned to the attributes, a mathematical function is formulated as seen in Equation (2), which quantifies the environmental complexity. This was done by multiplying the infrastructure type metric (ME) with the summation of the product between the assigned attribute weights (wi) and the attribute metric (Ai). This formula quantifies the environmental complexity value from a minimum of 1 to a maximum value of 100 as shown in Equation (2).
The next parameter explored in detail with regards to the operational scenario is the data acquisition method. Table 3 was put together with a list of different types of data collection methods that may be performed by the UAS as a part of its SHM tasks. This table of data acquisition methods has a set of metrics ranging from 1 to 10 for SHM tasks that the UAS performs based on its complexity. The value of the metrics increases as the difficulty of the tasks increases from distant low resolution imagery to ultrasonic contact NDE. These tasks are subdivided based on the type of data it obtains such: Imagery, 3D modelling, non-contact NDE and contact NDE. Each of these divisions of data acquisition methods has its own set of attributes based on the way data is collected.
Similar to the environmental complexity, the data acquisition methods are correlated to attributes that contribute to the complexity of the data acquisition methods seen in Table 4. There are four different sets of attributes, one for each data acquisition class.
Once the metrics and attribute weights for the different data acquisition methods have been set up, a function is created that can provide us with a value by combining these metrics, attributes and weights. In this model, different weights are assigned to all the attributes for the data acquisition methods based on their relative importance compared to other attributes. The total sum of the weights of all the attributes of each data acquisition class must add up to 1. The attribute weights of all four classes are further rated by metrics ranging from 1 to 10 based on the comparative impact of that particular attribute on the data acquisition method. For example, the metric value of ‘equipment weight’ for collecting imagery data depends on the weight of the camera mounted on the UAS compared with the weight of other cameras.
T ( x ) = M T   i = 0 n w i × A i
Once the task to be performed on the infrastructure has been identified and all the metrics and weights assigned to the attributes are finalized, a mathematical function is formulated as seen in Equation (3), which quantifies the task complexity based on the data acquisition methods. The formula can be applied for the other tasks and the greatest task complexity value amongst the multiple tasks may be used as the final task complexity value to conduct any further analysis. In this way, the formula developed in Equation (3) quantifies the task complexity value from a minimum of 1 to a maximum value of 100, thereby allowing task complexity to be scaled to the environmental complexity.

3.2. Value Model for UAS Rating and Evaluation

Autonomous mobile robotic technology has rapidly evolved over the last decade and has begun to impact multiple sectors across various industries [16]. In order to characterize, compare and evaluate the advancements in autonomy of a vehicle, common taxonomies and terminologies will need to be put in place. Currently, multiple government agencies such as the Department of Defense Joint Program Office (JPO), the U.S. Army Maneuver Support Center, and the National Institute of Standards and Technology (NIST) have separate but related on-going efforts to describe levels of robotic behaviors [4]. Although most of these are military related projects for combat systems, similar autonomous behavioral models can be modified for commercial operations. Although there are multiple references and metrics that have been created for UAS autonomy evaluation, here, a version of the autonomy and technological readiness assessment (ATRA) is used and can be retrofitted into the larger model for decision analysis of UAS operations explained earlier.
The two parameters that are used to evaluate the autonomy and robustness of a system being implemented are, autonomy level (AL) and technology readiness level (TRL). These parameters are unified to form the ATRA, which can be used to provide a holistic view of the software component of UAS [4]. The formation of the network between the AL and TRL to form the ATRA is shown in Figure 7. The use of this framework to evaluate autonomous UAS can be used to make both qualitative and quantitative comparisons that may be accomplished with the help of a formulated value function. The creation of such a framework enables evaluation that can be systematically used to compare the autonomy of one UAV to another.
The ATRA framework can be used to measure the maturity and robustness of the UAS [17]. In this study, the framework is used to investigate how they can be used in SHM applications to determine the optimal level of autonomy for different tasks to be performed. Here, we will use the definitions of AL, TRL and the ATRA, defined by NiST as autonomy levels for unmanned systems (ALFUS) [17,18] with necessary modifications from other sources to fit our SHM applications agenda. The example of a UAV that was assembled with off-the-shelf components and control system is used and evaluated based on the ATRA framework developed to demonstrate its usage and effectiveness in the evaluation of autonomous UAS, while also keeping track of its operational environment.
The first parameter explored in detail is the autonomy level (AL). This is a set of metrics that is used to systematically evaluate and measure the autonomous capabilities and limitations of a UAS [18]. These metric ranges from 1 to 10, representing the accession of autonomous behavior from remote-controlled UAS with rudimentary first order flight control up to fully autonomous capabilities with no-human-pilot decision-making involved at all. The AL of a UAS is mainly based on it GNC functions, which are primary attribute to enabling autonomy in the hardware component of the UAS. The GNC capabilities of the UAS are directly proportional to the autonomous capabilities of the UAS. The list of the different levels of autonomy for our example is compiled in Table 5.
The next parameter we will explore is the technology readiness level (TRL), which is a set of metrics that is used by many companies and government agencies to systematically evaluate and assess the reliability and maturity of the technology being used, which in this case is the autonomous UAS [4]. This metric ranges from 1 to 10 and is related to the advancement of systems development, integration, validation and deployment, ranging from the formulation of basic principles of the system up to its deployment and full operation status. The TRL of a UAS is mainly based on its systems integration, test environment, mission scenario and performance, which goes hand in hand with the reliability of the combination of autonomy and hardware that can be implemented in the UAS [4]. The levels of TRLS that we will be using for our example here are shown in Table 6.

3.3. ATRA Formulation for a Hexacopter

A hexacopter UAS created by assembling off-the-shelf components to demonstrate the use of the AL and TRL evaluations in order to create an ATRA graph is used herein. In order to evaluate the AL of this UAS, there is a need to evaluate its GNC capabilities of its flight controller component. The flight controller used in this UAV is the 3DR Pixhawk mini [19], which is a commercially available flight controller for UAS designed and manufactured by Berkeley based drone company 3D Robotics [20]. This flight control system comes with an inbuilt auto flight stability system, which is assisted by a gyroscope and altimeter to adapt to dynamic changes in its environment such as varying wind speeds to keep it stable during flight.
The flight controller allows for the addition of a modular GPS and compass unit to the main flight controller CPU to allow it to provide accurate positioning via its ground station software—px4 autopilot pro [21]. The mission planner uses GPS-assisted waypoint navigation that allows a user to pre-program a path and allow the UAS to execute it autonomously. On referring to the AL table, and based on the GNC capabilities of the Pixhawk mini flight controller, it is observed that it is fully capable of performing tasks up to AL 5, which is ‘Path planning and execution’. As the AL increases to level 6 and beyond, capabilities such as real time path planning and dynamic mission planning can only be tested and experimented on by modifying the flight control system or ground station software. The addition of hardware, such as obstacle avoidance sensors, to the flight control system will make the system capable of enabling these autonomous capabilities into the UAS. Since these capabilities have not been implemented, the AL for ‘safe operations’ will be set at 5 as shown in Table 7.
Since TRL does not only depend on the critical autonomy enabling GNC components, but also on the systems integration and operational reliability of the whole UAS itself; it is necessary to consider the other hardware components used in the assembly of the UAV when evaluating the TRL. For AL 5, this UAS has been flown on a regular basis to capture images and videos using its integrated gimbal and camera system using a preplanned path. This puts its TRL at 8, which according to the metric is ‘Prototype operation in realistic mission scenario’, since path planning operations were used to execute its mission of capturing images and videos and has been carried out only using this one prototype. If a full UAS mission deployment must be considered, the UAV must be mass produced and its missions must be carried out on a regular basis with a track record exhibiting proof of operational success in performing its tasks. Based on this, the hexacopter used in this example receives a TRL of 8 and an AL of 5 as highlighted in Table 7.
From the results obtained using the AL and TRL metrics, the ATRA graph is created by plotting the AL in the X-axis and the TRL as the Y-axis. The Figure 8 above illustrates the ATRA graph for the hexacopter. The highlighted circle in the block of AL 5 and TRL 8 with a red square to signify the extent of safe operational autonomous capabilities of the UAV. The ‘+’ and ‘−’ subscripts in each of the blocks are used to denote whether that AL and TRL has been implemented on the system. In this ATRA graph, for all ALs above 6, the TRL remains below 6 due to the fact that it does not fulfill the requirements needed to be implemented on the UAS. For a system to be able to be implemented, the TRL of the UAS must be at least 7. Therefore, a UAS with TRL below 7 may only have a ‘−’ subscript; this threshold is denoted using a red line under the TRL 7 row.

4. Application of Developed Value Model

In the previous section, created metrics and functions helped to quantify abstract factors such as the environment, difficulty of tasks, autonomy and reliability of the UAS performing the mission. In this section, the use of functions and metrics created to evaluate risks and the value of a particular operation are utilized, which can then be used to guide the UAS operators before the flight missions. The risk analysis and operations discussed can be used for purposes ranging from authorizing UAS flights safely based on different operational scenarios to aiding in pricing and cost analysis of missions based on the environment and tasks. The application and usage of the value model developed earlier will be applied and demonstrated in the following operational scenarios.
The operational scenario explored in the following example will use a realistic mission scenario defined by the task performed, payload carried, the type of infrastructure being scanned and the flight conditions, which are abstract entities that will be quantified to provide a mission operation value and a risk value. The resulting operations value can be used as a basis for the creation of a dollar-based pricing model for commercial UAS operations, while the risk value can be used as a model used to aid in decision making for the UAS operator prior to the UAS operation by using the balance between the UAS characteristics and the operational scenario. In this study, the ‘risk analysis’ will merely be used as a pseudo measure of likelihood of operational mishap occurring. The results from these models can be used as a starting point to research and understand effects of other factors on UAS operations, such as the human factors, ergonomics, detailed UAS physique and failsafe mechanisms.

Operations Value and Risk Analysis

Every UAS operation is different in its own way, and in order to create a standardized method that can be used to quantify the value of these operations, the authors have created an operations value formulation. This formulation is made by combining the four major metrics developed in the previous chapter, which are, the environmental complexity—E(x), task complexity—T(x), autonomy level—AL and technology readiness level—TRL. In order to effectively incorporate these different metrics and provide us with the operations value, the environmental complexity value and task complexity value are added to each other and this then added to the sum of ten-times the AL and TRL in order to scale up the raw values obtained from the respective metrics.
V ( x ) = E ( x ) + T ( x ) + 10 × [ A L + T R L ]
The mathematical formulation of the operations value function is shown in Equation (4). This way, all the four primary parameters are equally represented in the operations value formulation and thereby play an equal role in determining the final operations value. Using this formulation, the total operations value can range from a minimum value of 22 to a maximum value of 400. The operations value V(x) in Equation (4) quantifies the overall difficulty and complexity of the UAS operations to be performed by taking into account both the physical and autonomous UAS attributes, along with the operational scenario. This aides the user in obtaining a holistic view in terms of operational decision making. The ‘risk analysis’ in this case is used as a decision-making mechanism and does not include probability of occurrences.
Similar to the development of metrics used to assign a single value that quantified the UAS operations, the same set of metrics can be used to perform risk analysis on the UAS operations. This will help the UAS operator in the decision making process before flight. The main purpose of this risk analysis will be to provide valuable information to UAS operators about whether the capabilities of the UAS they will be using to perform the respective mission are appropriate to complete the mission operations in the particular operating environment. Therefore, the risk analysis model used here will focus on simply balancing the operational scenario parameters with the UAS evaluation parameters.
0.1 × [ E ( x ) + T ( x ) ] A L + T R L
The exact risk analysis logic developed for this purpose is shown in Equation (5), where the 0.1 times the sum of the environmental complexity value E(x) and task complexity value T(x) on one side is balanced against the sum of AL and TRL on the other side. If the value obtained from the operational scenario parameters on the left side is less than or equal to the UAS evaluation parameters on the right-hand side of the inequality formula, then the UAS operation may be considered safe to execute, otherwise, the operational scenario is too difficult for the UAS to perform and hence is unsafe for operations unless a more advanced UAS can be used.

5. Evaluation of UAS Operational Scenario Using Value Function

In this section, the developed operations value and risk analysis formulations are used on a mission scenario to demonstrate the benefits of these value models for handling pricing and decision-making in the UAS operations for performing SHM applications.

Operational Scenario

A state department of transportation wants to perform an aerial high definition survey of a five mile stretch of an inner city road using a Canon EOS 70D DSLR camera mounted underneath a fixed wing UAV. The UAV is capable of executing a pre-planned path from the ground station and the operation is planned to take place on a high visibility and low wind speed day.
To conduct this type of analysis, one can use the operational scenario metrics of the infrastructure types and data acquisition methods to obtain the environmental complexity and task complexity values. The inner city roads to be scanned in this case comes under the ‘paved urban roads’ category that has a value of 2 in our infrastructure metrics. All the attributes in this analysis are equally weighted and the attribute values are determined by comparing the paved city road being scanned in this scenario with other paved city roads. The length of this inner-city road is assigned an attribute value of 5 based on the length of the road to be surveyed, since the UAV will be operating above the road in the city, there may be a few obstacles, and hence, the physical obstacles attributes value is 4. Since the survey is conducted on a relatively pleasant day above the city, the weather is assigned a value of 2 and the geography is given a value of 4. Due to the amount of metallic structures such as antennas and telecom towers present in the surroundings which may cause radio interferences, the signal interference may be moderate and is assigned a value of 5. Table 8 shown below contains the attribute weights and values for each of the five attributes that help us quantify the environmental complexity. Using these weights and values of the attributes obtained from Table 8, we can calculate the environmental complexity value as shown in Equation (6). The final environment complexity obtained using this equation is 8.
E ( x ) = M E i = 0 n w i × A i
Similarly, we can calculate the task complexity value using the data acquisition metrics and the set of attribute weights and values associated with it. On referring to the list of data acquisition metrics, the ‘distant HD aerial imagery’ task being performed here has a value of 3. For this task, the four different attributes along with their assigned weights and values are shown in Table 9. The equipment weight attribute is assigned a weight of 0.3 and given an attribute value of 7 due to the Canon EOS 70D DSLR being quite heavy as compared to other HD cameras. The image composition is assigned a weight of 0.2 and an attribute value of 5, since HD aerial imagery requires a medium level detail of composition to capture larger defects on the road. The proximity is weighted at 0.3 and assigned a value of 2, since the fixed wing will be flying at a relatively high altitude above the road while performing the aerial survey. Since the stability of the image from this mission needs not be highly stable and is supplemented by the relatively stable flight characteristics of the fixed wing aircraft, it is assigned a value of 4. Using these weights and values of the attributes shown in Table 9, we can calculate the task complexity value using Equation (7) below. The task complexity value obtained using this equation for distant HD aerial imagery is 13.5.
T ( x ) = M T i = 0 n w i × A i
Once the operational scenario has been quantified, the factors to evaluate the UAS characteristics must also be quantified. In order to do so, the two parameters we use are the autonomy level (AL) and technology readiness level (TRL). The flight controller of the UAS in this situation uses a pre-programmed waypoint navigation and has an AL value of 5. The UAS used here is being deployed on a mission of capturing HD images and is assigned a TRL value of 9. Using all the values obtained from the operational scenario and UAS evaluation, we can find out the final operations value for this mission. Using the formula shown in Equation (8), we get the operations value to be 35.5, which implies that this UAS operation is simple and the UAS available for fulfilling this task is not sophisticated. Based on this low operations value, it is safe to assume that this mission will be relatively cheap to perform and the UAS needed for this task need not be highly sophisticated.
V ( x ) = E ( X ) + T ( x ) + 10 × [ A L + T R L ]
Using the environmental complexity, task complexity, AL and TRL values, a simple risk analysis can also be performed that may be used to determine if the mission being performed is safe to execute purely based on the operational scenario and the UAS characteristics. The risk analysis for this operational scenario is performed by balancing the operational scenario and the UAS characteristics. In this case, the operation to be performed is safe with the available UAS configuration and the environment and task are not too complex to perform.

6. Conclusions and Future Work

This research study gives an overview of the different types of SHM applications using UAS and how the value and risk of these operations can be quantified using a value based analysis approach. The value model we constructed incorporates critical aspects of a UAS performing a SHM mission, such as the environment of the mission, the objectives and task of the mission, as well as the various physical, software and technical attributes of the UAS. The hardware and technical features of a fixed wing and multirotor UAS were broken down into its subsystems in a DSM to show the relation between all components. Similarly, in order to quantify the software and ‘intelligence’ attributes of the UAS, the table AL and TRL were used. The environment and task complexity were also represented as functions supported by attributes and metrics to incorporate into the final operations and risk values.
Using these four primary factors, the final value model that was created and laid the foundation for quantifying and analyzing the operations value and the risk value of the UAS operation. The risk model further evaluates whether or not the UAS operator should proceed to perform the flight mission while prioritizing safety. The operations value provides the UAS operator with a final numerical value that can be used to create pricing models for missions. Finally, the sensitivity analysis performed by changing the assigned weights of the attributes associated with the task and environmental complexities on value and risk models demonstrated the robustness and reliability of these models.
The value model created in this study to broadly quantify the value of operations and perform an elementary risk analysis may be further researched and expanded to include a dollar-based analysis for the different operational scenarios. This detailed value function can be used to determine insurance rates based on scenario, true cost of a UAS crash and the probability of mishaps that may lead to a catastrophic operational failure. An expected utility function may also be developed to perform risk analysis based on the probability and severity of possible occurrences, which may be supported by results from stochastic simulations that identify different failure modes during operations. This can be further iterated by creating a detailed value model comprising the UAS subsystem along with a comprehensive list of all its major parts and how their characteristics affect the final performance of the UAS in an operational scenario.
Another form of research that could stem out of this could be to perform qualitative and quantitative comparison of UAS-based SHM techniques to other traditional SHM methodologies, such as manual inspection, embedded sensors and aircraft or helicopter based surveying. The pros and cons of each of these methods can be compared by using operations cost, accessibility, quality of data, total time required, etc. as some of the benchmark attributes. This can be used to synthesize a quality/dollar value contrast of data collected through the different methodologies, and then, use a bargain model to explore an optimized combined approach for performing effective SHM over a long period of time and on a larger multi-tiered scale.
Lastly, there will be continued study into the ethical side of the value model. This ethical side will serve to be an ethical governance mechanism [22] for the mission that will help decide if a mission should be conducted based on ethical considerations, quantifying the ethics in a way similar to the quantification done in this study. These ethical considerations will range from concepts of privacy to social benefit in deciding if a mission should be executed or not. This will differ from the practice of evaluating a UAV during its use, but instead, evaluating the implications of creating it for a certain purpose during its design before it is put into practice.


The work was funded in part by Iowa State University through start-up funds.

Author Contributions

C.L.B., A.V. and R.P. worked on conceptualizing the value modelling; A.V., R.P. and B.J.K. worked on developing the value model, A.V. and C.L.B. worked on developing case scenarios to demonstrate the model. All authors contributed in writing of the paper.

Conflicts of Interest

The authors declare no conflicts of interest.


  1. Valavanis, K.; Oh, P.; Piegl, L.A. Unmanned Aircraft Systems: International Symposium on Unmanned Aerial Vehicles, UAV’08; Springer Science & Business Media: New York, NY, USA, 2008. [Google Scholar]
  2. Eaton, C.; Chong, E.; Maciejewski, A. Multiple-scenario unmanned aerial system control: A systems engineering approach and review of existing control methods. Aerospace 2016, 3, 1. [Google Scholar] [CrossRef]
  3. González-Jorge, H.; Martínez-Sánchez, J.; Bueno, M.; Arias, P. Unmanned aerial systems for civil applications: A review. Drones 2017, 1, 2. [Google Scholar] [CrossRef]
  4. Kendoul, F. Towards a unified framework for uas autonomy and technology readiness assessment (atra). In Autonomous Control Systems and Vehicles: Intelligent Unmanned Systems; Nonami, K., Kartidjo, M., Yoon, K.-J., Budiyono, A., Eds.; Springer: Tokyo, Japan, 2013; pp. 55–71. [Google Scholar]
  5. FAA. Federal Administration Aviation. Available online: (accessed on 1 November 2017).
  6. FAA. Part 107 Small Unmanned Aircraft Regulations. Available online: (accessed on 1 November 2017).
  7. Collopy, P.; Bloebaum, C.L.; Mesmer, B.L. The distinct and interrelated roles of value-driven design, multidisciplinary design optimization, and decision analysis. In Proceedings of the 14th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Indianapolis, IN, USA, 17–19 September 2012. [Google Scholar]
  8. Norris, D. Build Your Own Quadcopter: Power up Your Designs with the Parallax Elev-8; McGraw Hill Professional: New York, NY, USA, 2014. [Google Scholar]
  9. Browning, T.R. Applying the design structure matrix to system decomposition and integration problems: A review and new directions. IEEE Trans. Eng. Manag. 2001, 48, 292–306. [Google Scholar] [CrossRef]
  10. Carrascosa, M.; Eppinger, S.D.; Whitney, D.E. Using the design structure matrix to estimate product development time. In Proceedings of the ASME Design Engineering Technical Conferences (Design Automation Conference), Atlanta, GA, USA, 13–16 September 1998; pp. 1–10. [Google Scholar]
  11. Yassine, A. An introduction to modeling and analyzing complex product development processes using the design structure matrix (dsm) method. Urbana 2004, 51, 1–17. [Google Scholar]
  12. Web, D. DSM Web. Available online: (accessed on 1 November 2017).
  13. Bhatia, G.V.; Wenger, C.; Basha, N.S.; Subramanian, T.R.; Shihab, S. AERE 568X Final Project: A Comparison of the Traditional Systems Engineering Approach with Value-Driven Design and Incorporation of Uncertainties—An Aircraft System Case Study; MIT: Cambridge, MA, USA, 2015. [Google Scholar]
  14. FAA. Fact Sheet—Small Unmanned Aircraft Regulations (Part 107); FAA: Washington, DC, USA, 2014.
  15. National Telecommunications and Information Administration (NTIA). Multistakeholder Process: Unmanned Aircraft Systems; NTIA: Washington, DC, USA, 2016.
  16. Siegwart, R.; Nourbakhsh, I.R.; Scaramuzza, D. Introduction to Autonomous Mobile Robot; MIT: Cambridge, MA, USA, 2011. [Google Scholar]
  17. Huang, H.-M. Autonomy Levels for Unmanned Systems Framework; Volume I: Terminology, Version 1.1; NIST Special Publication: Gaithersburg, MD, USA, 2004; p. 1011. [Google Scholar]
  18. Huang, H.-M.; Pavek, K.; Ragon, M.; Jones, J.; Messina, E.; Albus, J. Characterizing Unmanned System Autonomy: Contextual Autonomous Capability and Level of Autonomy Analyses. In Unmanned Systems Technology IX, 2007; International Society for Optics and Photonics: San Diego, CA, USA, 2007; p. 65611N. [Google Scholar]
  19. 3DR. 3DR Pixhawk Mini. Available online: (accessed on 1 November 2017).
  20. Robotics, D. Drone. Data. Decisions. Available online: (accessed on 1 November 2017).
  21. Open Source for Drone. Available online: (accessed on 1 November 2017).
  22. Arkin, R.C.; Ulam, P.; Wagner, A.R. Moral decision making in autonomous systems: Enforcement, moral emotions, dignity, trust, and deception. Proc. IEEE 2012, 100, 571–589. [Google Scholar] [CrossRef]
Figure 1. Hierarchical decomposition of a multi-copter UAS (Unmanned Aerial System).
Figure 1. Hierarchical decomposition of a multi-copter UAS (Unmanned Aerial System).
Drones 01 00005 g001
Figure 2. Design structure matrix of a multirotor UAS.
Figure 2. Design structure matrix of a multirotor UAS.
Drones 01 00005 g002
Figure 3. Multi-copter UAS breakdown and connection to performance attributes.
Figure 3. Multi-copter UAS breakdown and connection to performance attributes.
Drones 01 00005 g003
Figure 4. Primary stakeholders in UAS operations.
Figure 4. Primary stakeholders in UAS operations.
Drones 01 00005 g004
Figure 5. Parameters for the operational scenario and UAS rating.
Figure 5. Parameters for the operational scenario and UAS rating.
Drones 01 00005 g005
Figure 6. Model representing composition of environment and task complexities.
Figure 6. Model representing composition of environment and task complexities.
Drones 01 00005 g006
Figure 7. Model relating autonomy level and technology readiness level.
Figure 7. Model relating autonomy level and technology readiness level.
Drones 01 00005 g007
Figure 8. ATRA graph for the hexacopter UAV.
Figure 8. ATRA graph for the hexacopter UAV.
Drones 01 00005 g008
Table 1. Preferences of UAS operators from various industries.
Table 1. Preferences of UAS operators from various industries.
Farmers & Ag AssociationsEnergy & ConstructionUniversity ResearchersMedia Houses & Journalists
OperationPrecision AgricultureInspection & MaintenanceResearch & EducationPhotojournalism
TaskPhotos—RGB, IR, NDVIPower line inspectTool for R&D in multiple fieldsPhoto and video footage for news
Gas leaks detection
EnvironmentRural FarmlandUrban & Rural areasConfined zonesBusy urban areas
IssuesOwnership of scan data.Current rules are too constrainingTreat institutes different from commercial entitiesNo reqs to disclose details.
Intrusion over private land.Privacy laws need to be loosenedTime for training & registration
RequirementsTraining programs neededSpecial operations exemptions for inspection & maintenanceUse IRB regulations for schools not FAANo additional privacy regulations for drone use
UAS must have ID for tracking.
Table 2. List of infrastructure types with attributes and metrics.
Table 2. List of infrastructure types with attributes and metrics.
Infrastructure TypeMetric
Open Unpaved Rural Roads1
Paved Urban Roads2
Houses/ Small Buildings4
Wind Turbines7
Regular Beam & Truss Bridges8
Off-Shore Platforms9
Suspension Bridge Superstructures10
Table 3. List of data acquisition methods with metrics.
Table 3. List of data acquisition methods with metrics.
Data Acquisition TypeData Acquisition MethodMetric
ImageryDistant Low Resolution Aerial Imagery1
ImageryClose-Up Aerial Imagery (<2m)2
ImageryDistant HD Aerial Imagery (18MP+)3
ImageryClose-Up HD Aerial Imagery4
3D Modelling3D Photogrammetry Rendering5
3D ModellingLight Detection & Ranging (LiDAR)6
Non-Contact NDENon-Contact NDE: Infrared Thermography7
Non-Contact NDENon-Contact NDE: GPR (Ground Penetrating Radar)8
Non-Contact NDENon-Contact NDE: Laser Shearography9
Contact NDEContact NDE: Ultrasound10
Table 4. List of attributes with weights for data acquisition methods.
Table 4. List of attributes with weights for data acquisition methods.
Imagery3D ModellingNon-Contact NDEContact NDE
Proximity0.3Point Density0.2Proximity0.2Penetration0.2
Stability0.2Acquisition Time0.2Acquisition Time0.2Scanned Area0.2
Table 5. List of autonomy levels (AL) with descriptions.
Table 5. List of autonomy levels (AL) with descriptions.
Autonomy Levels
1Remote Control
2Automatic Flight Control
3System Fault Adaptive
4GPS Assisted Navigation
5Path Planning & Execution
6Real Time Path Planning
7Dynamic Mission Planning
8Real Time Collaborative Missing Planning
9Swarm Group Decision Making
10Full Atonomous
Table 6. List of technology readiness levels (TRL) with description.
Table 6. List of technology readiness levels (TRL) with description.
Technology Readiness Levels
1Basic Principles
2Application Formation
3Technology Concepts & Research
4Tech Development & Proof of Concept
5Low Fidelity Component Testing (Labs)
6System Integration & Flight Tests
7Prototype Demonstration & Operation
8Prototype Operation in Realistic Mission Scenario
9UAS Mission Deployment
10Fully Operational Status
Table 7. Selection of AL and TRL values for ATRA.
Table 7. Selection of AL and TRL values for ATRA.
Technology Readiness Levels
1Basic Principles
2Application Formation
3Technology Concepts & Research
4Technology Development & Proof of Concept
5Low Fidelity Component Testing (Labs)
6System Integration & Flight Tests
7Prototype Demonstration & Operation
8Prototype Operation in Realistic Mission Scenario
9UAS Mission Deployment
10Fully Operational Status
Table 8. Attribute weights and values for paved urban road.
Table 8. Attribute weights and values for paved urban road.
Paved Urban Road
Size of Structure0.25
Physical Obstacles0.24
Signal Interference0.25
Table 9. Attribute weights and values for distant HD aerial imagery.
Table 9. Attribute weights and values for distant HD aerial imagery.
Distant HD Aerial Imagery
Equipment Weight0.37
Back to TopTop