On the Dominant Factors of Civilian-Use Drones: A Thorough Study and Analysis of Cross-Group Opinions Using a Triple Helix Model (THM) with the Analytic Hierarchy Process (AHP)

Abstract

This study explores the experts’ opinions during the consultation stage before law-making for civilian drones. A thorough literature study is first undertaken to have the set of influencing factors that should be suitable for the investigation from the perspective of designing and selecting civilian drones. Several rounds of surveys using the Delphi method, followed by an analytic hierarchy process (AHP), are performed to conform to the organized tree structure of constructs and factors and to obtain the knowledge about the opinions of the expert groups, with the expert sample being intentionally partitioned into three opinion groups at the beginning: academia (A), industry (I), and research institutes (R). Doing so facilitates a “mind-mining” process using the triple helix model (THM), while the opinions across the groups can also be visualized and compared. This exploits a new set of knowledge for the design and selection of civilian drones on a scientific yet empirical basis, and the observed differences and similarities among the groups may benefit their future negotiations to propose the drafts for regulating the design, manufacturing, and uses of civilian drones. As several significant implications and insights are also drawn and gained from the abovementioned results eventually, some possible research directions are worthwhile. The proposed hybrid methodological flow is another novelty.

1. Introduction

With the mature development of the Internet of Things (IoT) and mobile communication technology, especially 5G, civilian drones are widely used in many applications. The drone industry is rapidly developing, causing drones with higher technical capabilities at a lower cost to enter the market. Combined with their ease of use, drones could become the primary players in the field of surveying for commercial, government, and scientific entities [1].

A forecast of global commercial and private drone market sizes indicates that in 2020, the global drone market generated almost 22.5 billion USD in terms of global revenue. By 2025, the global drone market is expected to generate over 42.8 billion USD, with annual growth projected to beat 13.8% [2]. The market forecast also shows the market will keep growing, and the major key factor comes from civilian consumers. This means that many drone-based applications will be available for enterprises and other organizations in the future. Therefore, for the government, how to control the use of civilian drones becomes a critical issue. On the other hand, for the aeronautics industry, the design issues of advanced civilian-use drones have also become a hot topic valued by many researchers and manufacturers. Therefore, a nexus of these possible conflicts is law-making. For example, Sah et al. [3] have pointed out that regulations are the most critical barriers to implementing drones in the logistics sector, an important application domain of civilian drones. Therefore, the significance of drone regulations is highlighted.

In Taiwan, for example, as in many developed countries or regions, many drone applications have gradually emerged recently. Many enterprises and organizations could use drones to perform different kinds of jobs for their business. Given the gradual rise of demand for drone applications and regulations, the Civil Aeronautics Administration (CAA), Taiwan R.O.C., is planning relevant regulations and certification mechanisms to manage drone applications.

However, in the recent studies related to drones, most of those studies explored the issues of applications, technologies, markets, safety, privacy, and others. Some of these studies also explored the cooperative relationships among industry (I), government (G), and academia (A) regarding military and civil drones with the triple helix model (THM) [4].

The THM is the key theory for this study. A THM model usually refers to a triangle that involves the corporations of I, G, and A parties (i.e., the vertices) to achieve some certain objective (O), e.g., for the public interest to design a large (mega) construction project, as shown in the left model in Figure 1. It is a “conceptual model” because a party’s name remains non-uniformed, e.g., party I is sometimes called “Business” (B) and party A is sometimes called “University” (U), varying case by case. Despite the name varying, the interchangeable names usually connote the same party without any ambiguity. Given this model, relations and actions that happen between each pair of parties can be analyzed along with the edge in the model. Moreover, the intersections between the three parties may form yet another interesting matter to be observed. However, in this study, the THM model is slightly modified for the research purpose, and those relations and intersections (and gaps) between the group opinions that affect the law-making actions are analyzed.

Before drafting the relevant regulations for law-making, the CAA, the institution in charge (of the G party), consulted and collected the opinions from the other three parties, namely, academia (A) (e.g., the university-level academics), industry (I) (e.g., the largest aeronautic manufacturer), and research institutes (R) (e.g., the largest government-funded aeronautic R&D institution), in Taiwan, R.O.C. These three parties, namely, A, I, and R, may constitute another meaningful triangle in THM rather than industry, government, and academia (I, G, and A). Problems related to designing civilian drones and making reasonable legislation can be dissected and studied using the proposed and renewed THM model. In other words, among the players in the THM model, G is replaced by R, and such a replacement should be reasonable because of the following two main reasons.

First, the original THM was used to explore the military-, civil-, and dual-use drone development issues [4], so THM is based on the cooperative relationship between I, G, and A. However, for this study, it is the case that G usually would like to entrust A, I, and R to draft the relevant regulations, and G is then responsible for legislating and implementing the law later. This is because the R party always performs some cutting-edge research on drones, so it can also give G more suggestions about newly developing directions, limits, and applications of drones. Hence, there is a change in the players in THM, i.e., for “which are triple”, as required by the nature of the main aim.

Second, according to the surrogate model [5] in the engineering domain, replacing G with R to explore the design factors of civilian drones is also reasonable, because the R, if properly selected, is also G. Consider the largest domestic R&D institution chosen to be R for opinion collection in this study: could such an institution not represent G, as it is government funded? In other words, in the A-I-R THM model, R can be carefully chosen to concur with the original structure of the I-G-A THM model without breaking its rule for G. These are the reasons why we replace G with R in the “modified THM” in this study. For clarity, in this text, we refer to THM as this modified THM model in most of the cases.

Additionally, in this study, following the abovementioned, we argue that a systematic study of the design factors of civilian drones for the makers (or the selection factors for users, equivalently) may help to understand the opinions of these three parties in the THM and to clarify the differences in their opinions. Such knowledge may address the emphasis of the subsequent advice given to the government for law-making.

To begin with, a set of design factors are clearly established through a thorough literature study, and the Delphi method can be used to confirm whether this set of factors is effective or not. When we stand on the user’s side, the temporarily civilian drone selection is a typical multiple-criteria decision-making (MCDM) problem. The analytic hierarchy process (AHP) is a very suitable approach that can be applied to understand how each group of experts (i.e., the three parties) may perceive the importance of these factors (i.e., for mind-mining [6,7]). That is, in terms of MCDM, civilian drone selection can be treated as a multi-attribute decision-making (MADM) problem that usually involves many consideration factors and the priority among these consideration factors (i.e., the significances of them), and usually, these consideration factors are the “decision criteria” in terms of MCDM, so the information about the “criteria weight vector” (CWV) can be obtained using AHP.

Prior to this, we perform a comprehensive literature study to seek and summarize a set of these factors, mounting them under some meaningful “constructs” in a tree-formed hierarchy, and confirm whether this set of factors we have included is effective or not by reference to the opinions of the domain experts using the Delphi method. Note specifically that the use of the abovementioned Delphi–AHP approach in studying the design factors has been shown to be effective in practice for other aircraft types with military purposes, e.g., next-generation fighters and MALE UAS [8,9], and/or for other UAV subjects, e.g., in Ulloa et al.’s work [10] to design a lightweight, portable, and flexible air-based PV-T module for UAV shelter hangars and in Song et al.’s work [11] to evaluate the comprehensive performance of UAV-based granular fertilizer spreaders (GFSs).

In short, this study explores the main design factors for the civilian applications of drones. It adopts several research methods, such as a literature study to collect and establish the initial set of factors, the Delphi method to confirm and solidify the set of factors, and AHP to understand the priority over these factors and how the opinions may differ (or not) among the three interest groups (A, I, and R). Inasmuch as the results would be helpful to understand the thoughts in the minds of the different groups of experts pertaining to the set of civilian drone design factors and the priority over these factors before the three groups of experts give some advice regarding the relevant regulations in the law, this study may perfectly meet the requirement and the core spirit of data-driven decision-making (DDDM) [12].

Therefore, the research question of this study is worth exploring, and the possible contributions of this study should be two-fold. First, as there are relatively few studies on the market and applications of civilian drones that are driven by a law-making requirement, the results of this study may offer valuable knowledge in an empirical sense. Second, to the authors’ knowledge, there seems to be no research that applies the THMs with the decision analysis (MCDM) methods, such as AHP, to explore some know-how further for drones. This study fills the gap by offering a new hybrid way to conduct the research in the methodological sense.

This introductory section placed the study in a broad context and highlighted the reason why it was conducted. Section 2 reviews the relevant literature, including a thorough literature study for the influencing factors. Section 3 explains the processes performed by using the methods and the results obtained from using these methods. Section 4 provides the discussions and implications drawn. Section 5 gives the concluding remarks and the recommendations for future works.

2. Literature Study and Methods

This section starts with the history of drones and the related works about drones (Section 2.1). This is followed by an in-depth review of the possible consideration factors (Section 2.2). In addition, except for the THM theory, which should be clear so far, a review about Delphi and AHP is also given to link these used methods with the study (Section 2.3).

2.1. Drones and Their Civilian Uses

Drones originated in 1917 when the U.S. military began conducting research and tested them during World War I. Therefore, relevant developing drivers and phases are reviewed by using this country as the example here for space reasons. In addition, the comprehension of public datasets that are available is another reason (i.e., data availability) to adopt this country as an example, at least for the researchers of this study.

After the early stage of the drone’s invention, it was not taken into combat until after the Vietnam War. However, since then, advanced communication technology has increased the bandwidth of military communications satellites and the development of navigation technology, improved the remote-control capability of unmanned aerial vehicles (UAV), and made the remote operations of UAV more practical. Additionally, the geographical nature of the wars affecting Iraq and Afghanistan has increased the need for the Organization of American States to identify, locate, and attack hidden targets through continuous surveillance and rapid strikes, while minimizing collateral damage. In these applications, unmanned aircraft systems (UAS) provide asymmetrical technical advantages in these conflicts [13]. In general, over the past decades, UAS have played a critical role in non-military operations, such as supporting humanitarian relief operations in Haiti or for mine detection and chemical, biological, radiological, and nuclear reconnaissance.

The U.S. Federal Aviation Administration (FAA, USA) clearly defines the term UAV as “equipment used or intended for use in air, without an on-board pilot”. In other words, UAV include all categories of aircraft, helicopters, airships, and refractive lifts without on-board pilots, and this implies that a “drone” flies autonomously or remotely without a pilot operating the aircraft. In the development history of drones, they have different designs and functions for military missions and civilian and commercial applications [14]. Because many missions are boring, dirty, or dangerous (3D) for pilots, drones are better suited for some tasks. For example, many types of drones, either large or small, are widely used by government departments or research institutes to carry out different tasks and research work [15].

Additionally, UAV can integrate with ground control stations and data links to form UAV systems (UAS). Therefore, drones involve command, control, and communications (C3) systems and must support those who control them [16]. UAS can be considered a system that includes multiple subsystems, including aircraft (often referred to as drones), payloads, control stations (often with other remote stations), aircraft launch and recovery, support, communications, transmission, etc. With advanced navigation and communication technologies, UAS have become a “new capability” available to the government (public) and commercial (civil) aviation sectors [17]. As the developing technology of drones is becoming more and more mature, the civilian applications are growing day by day.

Table 1. The uses of drone technology in the civilian domain.

For Civilian Use
Aerial photography	Film, video, and stills
Agriculture crops	Monitoring and spraying, herd monitoring, and cattle driving
Coastguard	Search and rescue; coastline and sea-lane monitoring
Conservation	Pollution and land monitoring
Customs and excise	Surveillance for illegal imports
Electricity companies	Power line inspection
Fire services and forestry	Fire detection and incident control
Fisheries	Fisheries protection
Gas and oil supply companies	Land survey and pipeline security
Information services	News, pictures, and featured pictures, e.g., wildlife
Lifeboat institutions	Incident investigation, guidance, and control
Local authorities	Survey and disaster control
Meteorological services	Sampling and analysis of atmosphere for forecasting
Traffic agencies	Monitoring and control of road traffic
Oil companies	Pipeline security
Ordinance survey	Aerial photography for mapping
Police authorities	Search for missing persons, security, and incident surveillance
River authorities	Water course level monitoring and flood and pollution control
Survey organizations	Geographical, geological, and archaeological surveys
Water boards	Reservoir and pipeline monitoring

lists the related applications of drones in the civilian domain [18].

Drone manufacturers and UAS suppliers worldwide are developing industry-specific solutions to meet customers’ business needs effectively. Advancements in drone technology have enabled manufacturers to produce different models in sizes, weights, and shapes that can carry various devices and payloads to play roles in a wide range of applications. However, some safety issues and drone traffic management issues are the factors that could challenge the growth of the commercial drone market to some extent. The demand for drones in the commercial sector is increasing, because they increase productivity through improvements in graphical visualization and an overall reduction in project costs. Due to a significant improvement in drone control accuracy, many drone application requirements are emerging quickly in the commercial sector. Various applications related to cost reduction and time have led to the increasing use of drones. This trend could be expected to create the overall value of commercial drones. Figure 2 presents another forecast of the drone growth trend, in addition to the forecast made for 2020–2025 [2] (see Section 1), in the commercial sector in the U.S. from 2014 to 2025 [19].

Except for the U.S., many countries around the world also used drones in the civilian sector. Currently, at least 90 countries or non-state organizations are known to operate drones [20]. However, different types of drones have different usages. For example, micro and small drones are commonly used in low-altitude and unregulated airspace. Typically, light drones smaller than 150 kg are suitable for monitoring tasks in many real-world applications.

Regarding the classification of drones, Gupta et al. argued that there is no uniform classification due to the diversity of drone capabilities, size, and operational characteristics [16]. That means that we can have a categorical list of drones by the types of drones seen so far.

Table 2. The classification of drones by drone type.

Classification	Weight	Normal Operating Altitude	Radius of Mission	Endurance	Normal Employment	Typical Uses
Mirco	<2 KG	Up to 200 ft	5 KM	Few hours	Tactical platoon (single operator)	R, I, S
Mini	2~20 KG	Up to 3000 ft	25 KM	Up to 2 days	Tactical subunit (manual launch)	S, DG
Small	20~150 KG	Up to 5000 ft	50 KM	Up to 2 days	Tactical unit (employs launch system)	S, DG
Tactical	150~600 KG	Up to 10,000 ft	200 KM	Up to 2 days	Tactical formation	S, DG
MALE	>600 KG	Up to 45,000 ft	Unlimited	Days/weeks	Operational/Theatre	S, CT
HALE	>600 KG	Up to 65,000 ft	Unlimited	Days/weeks	Strategy/National	S, DG, SR
Strike/Combat	>600 KG	Up to 65,000 ft	Unlimited	Days/weeks	Strategy/National	S, DG, SR

Table note for legends: R: reconnaissance, I: inspection, S: surveillance, DG: data gathering, CT: cargo transportation, SR: signal relay.

lists the categories and their associated parameters. These include the maximum total take-off weight (UAV with payload), regular operating altitude, mission radius, endurance, general use, and purpose of use.

Grimaccia et al. proposed an effective method to classify drones according to their functionalities [21]. However, the U.S. Department of Defense provided other classifications, while NASA provided another “classification matrix” [15], as shown in

Table 3. U.S. DoD’s classification for drones.

Weight (kg)	Normal Operating Altitude (ft.)	Mission Radius (km)	Endurance (hrs)	Representative UAV
<2	<400	5	<1	Black Widow, Raven
2~25	<3000	25	2–8	Aerosonde, Scan Eagle, Puma
25~150	<5000	50	4–12	Manta B
150~600	<10,000	200–500	8–14	SIERRA, Viking 400, Tiger Shark
>600	<18,000	1000	>20	Ikhana (Predator B)
>600	>18,000	5000	>24	Global Hawk

and

Table 4. NASA’s drone classification matrix.

Category	I	II	III
Weight	<=55 lb (25 KG)	55–330 lb (25–150 KG)	>330 lb (150 KG)
Airspeed (kt)	<=70	<=200	>200
Type	Model or sUAS	sUAS	UAS

, respectively. In Table 3, these classifications are based on weight, altitude, mission radius, and duration [13,22].

In Table 4, drones are classified by weight, airspeed, and type. However, it can be seen that, although there are different terms, weight can be used as the only common classification criterion, as other characteristics of drones are usually weight-related. Additionally, some studies [9,16,18] provided another means of drone classification, which is descriptive, telling the “feature descriptions” rather than the numerical parameters above.

Table 5. The categorization of drones with the “feature description” method.

Category	Feature Description
NAV (Nano Air Vehicles)	NAV are recommended for radar obfuscation or ultra-short-range monitoring if the camera, propulsion, and control subsystem can be small enough.
MAV (Micro UAV)	MAV were originally defined as a drone with a wingspan of no more than 150 mm. MAV are mainly used for operations in urban environments, especially when flying slowly in buildings, preferably stopping to sit on walls or columns. Micro-aircraft usually need to be launched manually because they are very susceptible to atmospheric turbulence, so the payload of a fixed-wing aircraft is very low. Such problems may exist in all types of MAV.
MUAV (Mini UAV)	Most of them refer to UAV that are less than a certain mass. They may be less than 20 kg, but they are not as small as micro-UAV. They can be launched manually and fly about 30 km at most. They are used by mobile battle groups and also for various civilian purposes.
Close-Range UAV	Usually, mobile forces use these drones to perform relatively simple military operations and a variety of civilian purposes. They typically execute a range of approximately 100 km and perform tasks such as reconnaissance, targeting, surveillance, airport security, ship-to-shore surveillance, power line inspection, crop spraying, and traffic monitoring.
TUAV (Medium-range/Tactical UAV)	Its operation range is from 100 to 300 km; compared to HALE and MALE, these drones are smaller and have a simpler control; they could be operated primarily by the Army and Navy.
MALE UAV (Medium-altitude long-endurance UAVs)	Its flight altitude is between 5000 and 15,000 m and its endurance time is 24 h. It functions like a HALE system, but it usually runs in a shorter range that still exceeds 500 km. It needs to operate at a fixed base.
HALE UAV (High-altitude long-endurance UAV)	Its flight altitude is more than 15,000 m; its endurance time is over 24 h. It can perform extremely long-range reconnaissance and surveillance. It is a trend for HALEs to arm them with weapons. Usually, the Air Force operates them at fixed bases.

shows the categorization of drones with this method.

2.2. Review for the Consideration Factors

In the last subsection, the history and classifications of drones, as well as the civilian uses of drones, were reviewed in general. In this subsection, a deepened review for the design and selection factors for civilian-drone applications is provided, because subsequent studies (i.e., the confirmation of the set of factors, the solidified basis of the AHP hierarchy that is established, and the final results obtained using the AHP) may heavily rely upon a set of factors that is grounded. Additionally, after the review of works, we found that the surveyed literature involves several categories, e.g., cost, performance and applications, operation, and maintenance, and we present the review to the relevant works by reference to these categories in order. In fact, as will be shown later, these four factor categories form exactly the four constructs to establish the AHP hierarchy.

2.2.1. Cost

Cost is always a concern for designing and applying any equipment and facilities. This also holds for drones. Many studies have explored related costs of drones. Aragão et al. [23] proposed a UAV selection model, and initial investment and maintenance cost are two main factors in their proposed selection model. In an exploration of life cycle costing (LCC), Kianian et al. [24] argued that the LCC should contain the acquisition costs, operation costs, maintenance costs, and disposal costs. In the study related to a stochastic facility location model for drones, Kim et al. [25] argued that, besides the purchase cost, it is critical whether the operation cost and maintenance cost are reasonable or not when considering the costs of a drone. They also mentioned that the total relevant costs of a drone include the costs of opening drone facilities and the operation and maintenance of drones and drone facilities. Discussing the sustainability of small UAV, Figliozzi [26] highlighted some important trends, such as UAV tare, payloads, battery energy, purchase costs, and energy consumption per unit of time flown. He also argued that the cost of operating a drone should include UAV operation staff costs, maintenance costs, ground costs, energy costs, purchase costs, battery costs, software costs, and communications costs. In the study recommending the use of UAV platforms in precision agriculture in Brazil, Jorge et al. [27] mentioned that reduced operating costs is one of the consideration factors when purchasing a UAV. In another study about UAV’s technology-supporting maintenance operations, Miari [28] emphasized the importance of UAV operation and maintenance costs in UAV usage. As for a drone’s purchase cost and operation cost, Yu et al. [29] mentioned that these are very economical, with 1/5 of the purchase cost and 1/10 of the operation cost of unmanned helicopters.

Moreover, the study related to maintenance cost estimation of Royal Canadian Navy ships described that operating costs included three major categories: the cost of personnel operation, the cost of operation consumables, and the costs of all activities that support the system’s operation. In addition, the study also had a description of maintenance costs. It was stated that maintenance costs covered all planned and unplanned activities to keep or return the system to a given state or provide the additional operational capability. These maintenance activities include detection; inspection; troubleshooting; prevention; testing and calibration; overhaul; and replacement of parts, components, or assemblies performed by the crew, by specialist repair personnel, by a depot or agency, and by the industry [30]. Based on the airlines’ perspective, Dožić et al. [31] chose evaluation criteria to solve the aircraft type(s) selection problem. The chosen criteria include aircraft seat capacity, reflecting the measure of matching demand and capacity; aircraft price describing needed investment; total baggage related to the earning possibility from cargo transport; maximal take-off weight (MTOW), which is the main unit for calculation of airport and navigation fees; payment conditions describing payment advantages offered by different manufacturers or leasing companies; and total cost per available seat miles (CASM), indicating the operational costs and aircraft performances. Gomes et al. [32] proposed a novel approach to imprecise assessment and decision environments (NAIADE) method. This proposed method is based on three criteria, financial, logistics, and quality, to select an aircraft for regional chartering. In a study about cost–benefit assessment and implications for service pricing of electric taxies, Wang et al. [33] argued that the total life-cycle cost model for cost–benefit assessment should consider purchase cost, usage cost, and other operation costs. Additionally, Yeh and Chang [34] claimed that purchase cost and operating cost are the two criteria that can be used to evaluate an aircraft’s fuzzy rating performance.

In a study related to LCC, Woodward [35] mentioned that purchase costs are just one of the initial capital costs, and they should include land, buildings, fees, furniture, and equipment. It means that purchase costs should contain the costs of the purchase of equipment and facilities related to the equipment. Operation cost is usually to ascertain the costs of providing or operating a service. This method of costing is applied by those undertakings that provide services rather than the production of goods [36]. The role of operational cost is also evident in expected-value-approach (EVA) studies, while determining the best purchase portfolio, e.g., in the photovoltaic manufacturing industry [37]. Relatedly, in Bressani-Ribeiro et al. [38], it was shown that low operational cost is one of the primary reasons for users to adopt new technology. In their study, Nachimuthu et al. [39] mentioned that total maintenance costs should include maintenance personnel costs, maintenance ship costs, dedicated repair ship costs, spare parts costs, and production losses due to downtime.

Thus, it can be asserted that purchase cost, operation cost, and maintenance cost are the primary consideration factors for a civilian drone design and selection from the discussions above. For drones, the purchase costs would include flight vehicles, manipulation devices, and devices required to perform related applications. The operation costs would contain fuel/electricity, mission-related consumables, and the use of landing sites, and the maintenance would consist of flight vehicle and primary equipment maintenance and component replacement. In conclusion, this completes the review of the factors that should be included in the “cost” category.

2.2.2. Performance and Applications

A civilian drone’s performance and applications are perhaps the most complicated category. Although a construct can be established to recapitulate all relevant factors, a thorough study should be exerted to clearly specify the initial set of factors that are to be included under the construct and sent for experts’ approval during the Delphi process.

Performance and applications are usually the primary consideration factors when users try to adapt to any new equipment. For drone usage, many studies also discussed drones’ performance and applications. In discussing drone performance, Hwang et al. [40] concluded that the key performance of a drone should include speed, altitude, range, payload, and specific operational activity. In the study about selection of UAV for precision agriculture, Petkovics et al. [41] argued that flight duration time, flight speed, on-board computer, sensors, payload, coverage area, and operational time are the UAV selection consideration factors for precise agricultural usage. In the study about imagery collection to aid Aedes aegypti (mosquito) breeding site identification, Aragão et al. [23] proposed a UAV selection model. The proposed model adopted weight, dimension, technique, performance, speed, and investment as the main criteria. There are 12 performance-related sub-criteria, such as maximum take-off weight, maximum payload weight, wingspan, take-off, landing, maximum range, maximum mapped area, wind resistance, maximum altitude, cruise speed, stall speed, and maximum level speed. For last-mile delivery drone selection, Nur et al. [42] proposed an evaluation model. There are 19 sub-criteria related to the “performance” main criterion in the proposed model: drone’s overall size, weight, drone type, fuel type, internal computing components, location and proximity accuracy, communication and data quality, traceability, reliability, required delivery distance, maximum flight time, charge and fuel usage rate, maximum load, maximum carry dimensions, maximum reachable altitude, drone speed, adaptability to a dynamic assignment, package handling flexibility, and delivery flexibility.

For the selection of the most proper UAV for transportation in emergency operations, Ulukavak and Miman [43] used eight factors to obtain an evaluation process: payload, UAV weight, maximum altitude, maximum ground speed, approximate flight time, remote controller range, landing field, and ease of use. For visual inspection, monitoring, and analysis of infrastructure using UAV, Duque et al. [44] also select drones that can meet their requirements based on eight evaluation factors, which are flying time, an additional camera on top of a drone, camera resolution with low illumination, video resolution, payload capacity, drone lights, remote control range, and price. For inspecting bridges with a drone, Duque et al. selected a drone with various consideration specifications. These specifications include user-controls and interface, maneuverability, software capability, adaptability, size, and payload [45]. Hoyas Ester [46] selected a drone for his study with another eight selection criteria: price, body size, drone weight, flight time, radio frequency (RF) range, lens field of view (FOV), and lens aperture.

Cesnik et al. [47] mentioned that payload mass, flight speed, fuel mass, time, flight altitude, and landing and flying distance are the important indicators to evaluate the vehicle flight performance of a drone. Chen et al. [48] argued that designers of a drone must consider the high mobility and flight time while minding the limited battery life; the flight duration time of a drone is a key performance indicator. In the study related to drone flight capabilities, Ajanic et al. [49] discussed the importance of manipulative ability for a drone. Therefore, in drone selection, the manipulative ability of a drone is a critical evaluation factor. Yang et al. [50] mentioned the operation range of a drone, the control distance of a drone’s controller, the transmission bandwidth for a drone, and the signal interference of a drone are a drone’s critical performance issues. Shakeri et al. compared the advantages and disadvantages of multi-UAV and single-UAV systems with several features: targeted area coverage, cost, task time, radar cross-section, power, network topology, application, and security [51].

In discussing disaster management with UAV, Erdelj et al. [52] also argued that airborne operation duration of a drone is an important consideration factor in disaster process operation. In fact, the application of UAV now plays a critical role in not only disaster management but also law enforcement and first responders, and these can be mapped to “Fire Service and Forestry” and “Police Authorities” in Table 1. Whilst the essence of these applications is quickness (i.e., for launching actions just in time for law enforcement (police), effective interventions made by the first responders (e.g., police officers, firefighters, and disaster managers), etc.), the design of drones may also address such a noteworthy feature (i.e., “quick reconnaissance”). For this specific domain, we cite Laszlo et al.’s work in 2018 [53] and the work of Restas in 2015 [54], which have shown the importance of this feature in the aforementioned application domains.

Aljehani and Inoue [55] discussed the coverage problem of a UAV and mentioned that the communication system’s performance in a UAV is important for a UAV’s flight control and operation. In the study related to design and trajectory control of universal drone system, Yıldırım et al. [56] argued that controllers and sensors of a drone are the components that directly affect the vehicle’s flight performance, while other components directly affect the payload of a drone. Besides these basic components, other devices on a drone may be added to perform some specific requirements. Additionally, Amiri [57] argued that the manipulation’s agility and stability for the required maneuvers are the key performance indicator for a drone. Pai et al. [58] mentioned that the drone operator should be familiar with the operation interface of a drone, as it involves the manipulative ability of a drone. Therefore, the operation interface of a drone might affect its manipulative ability. Liu et al. [59] also discussed the manipulative ability of robots. Manipulative ability refers to the manipulative ability of robots to perform certain tasks. In terms of maneuverability, different types of rescue robots have different performances. Robots must be controllable and easy to control. The global manipulative ability embodies survivability, mobility, sensors, communication, and the human–machine interface.

Gomes et al. [32] used 12 sub-criteria to evaluate the selection of an aircraft. The performance-related sub-criteria include range, flexibility, cruising speed, landing and take-off distance, comfort, and avionics. Bruno et al. [60] proposed an aircraft evaluation model based on the airlines’ requirements. Their proposed evaluation model contains four main criteria and eight sub-criteria. The one main criterion, technical performance, and two sub-criteria, cruise speed and autonomy, are related to a performance consideration for an aircraft evaluation. Yeh and Chang [34] used three main criteria and 11 sub-criteria to evaluate each aircraft’s fuzzy rating performance. One of the three main criteria, technological advances, and two of the 11 sub-criteria, aircraft reliability and maximum range, are related to an aircraft’s performance. See et al. [61] used speed, range, and the number of passengers as criteria to select the best aircraft among a set of alternatives with a multi-attribute methodology.

In the study about the classifications, applications, and design challenges of drones, Hassanalian and Abdelkefi [62] mentioned that drones’ applications would cover a wide range of civilian and military fields. This is important. Drones can perform outdoor and indoor missions in very challenging environments. Drones can be equipped with a variety of sensors and monitors for intelligence, surveillance, and reconnaissance missions. Drone applications can be categorized into different channels. They can be based on mission type (military and civilian), flight area type (outdoor and indoor), and environmental type (underwater, water, ground, and air and space). Types of drone applications are shown in Figure 2. Depending on the type of drone, there are more than 200 drone applications in the future, including search and rescue missions, environmental protection, mailing and delivery, shooting and reconnaissance, performance, bird repellent, cleaning, agricultural spraying, missions at sea or on other planets, and other miscellaneous applications.

In discussing disaster management with UAV, Erdelj et al. [52] mentioned the applications of UAV, and they divided these applications into six categories: monitoring, forecasting, and early warnings; disaster information fusion and sharing; situational awareness, logistics, and evacuation support; support for a standalone communication system; support for search and rescue (SAR) missions; and damage assessment. Figure 3 shows a classification of drones’ applications [62].

In particular, Hassanalian et al. [62] mentioned that different types of drones use different types of power sources, such as fuel, battery, solar cells, and laser power beaming technology. They discussed the advantages and disadvantages of these power sources. Nur et al. [42] also mentioned that the types of power sources provided by UAV include batteries, solar energy, hydraulic fuel cells, internal combustion engines, and tethered and laser transmitters. In the invention related to the field of radiation detection and the “CdZnTe” aerial inspection system, Zhang et al. [63] emphasized the importance of a long operation duration. Pickett [64] also emphasized the importance of extended operation duration for applications of a drone in the patent about unmanned aerial vehicle boosters. Moreover, in rescue applications with an energy-constrained UAV, Liang et al. [65] argued that the longest operation duration of a UAV is a constraint factor in rescue applications. Erdelj et al. [52] also argued that airborne operation duration of a drone is an important consideration factor in disaster process operation in discussing disaster management with UAVs.

From the above long survey so far, we can find several critical factors that are significant to a drone’s performance and applications. These critical factors include vehicle flight performance, manipulative ability, flight power, main application, and operation duration.

2.2.3. Operation(s)

Operation is another important category of concern when users use drones. Many studies discussed issues of drones’ operation, and several patents are also related to this category.

There are several patents related to drone operation convenience. Zhang et al. [66] emphasized that operation convenience and operation security are critical for a drone; they had a rotary-wing invention to improve the operation convenience and the operational safety of the rotary-wing drones to a large extent. Ng [67] invented a mobile vehicle charging system that can improve the operation convenience of drones. Tang et al. [68] invented a remote controller of a UAV with a handle structure. This remote controller can simplify the control interface of a UAV to enhance the operation convenience of a UAV. From the above inventions related to UAVs, we can find that drone operational convenience is significant for users. Drone manufacturers can use those new inventions of UAV to improve the operation convenience of drones. Additionally, Zhou et al. [69] argued that the size of the landing site would affect the convenience of drone operations.

Xiao et al. [70] mentioned that UAV offer advantages, such as flexibility, ease of operation, convenience, safety, reliability, and low costs. Thus, ease of operation and safety are two of the reasons why users want to use UAV. In an exploration of the features of consumer UAV, Mao et al. [71] used the DJI UAV products to illustrate the characteristics of consumer UAV availability that include convenience, easy operation, safety, intelligence, and entertainment. That means safety is a critical consideration factor for a consumer UAV. In the study related to the risk, vulnerability, and safety improvement of the industrial UAV transportation system, Johnsen and Evjemo [72] explored the safety of UAS (unmanned aviation systems) based on drones and other UAVs. Safety is related to accidental harm, and security is related to intentional harm. They mentioned that drones have often been used in tasks that are dangerous, dirty, or dull. Thus, safety and security have often been the principal drivers when using UAS. Therefore, we must ensure high reliability, safety, and security when we use UAS in critical operations such as the transport of medical supplies. In the study related to operator engagement during UAV operation, Roy et al. [73] argued that bio-cybernetic systems could adapt the UAV operator’s mental status to optimize UAV performance and increase UAV operation safety. Therefore, how to improve the safety of UAV operation is a critical issue. Jorge et al. [27] also mentioned that ease of operation and safety are two of the consideration factors in purchasing a UAV. Moreover, they also explored the advantages and disadvantages of UAV with type, applications, and operations.

In the book about UAV design, Gundlach [74] described the supportability of unmanned aircraft. He argued that the supportability covers activities required to sustain operations. That includes support equipment, spares management, and other items. Both the customer and the contractor can perform support activities. Hwang et al. [40] also argued that supportability is a key performance indicator of a drone. In the study related to UAV design, Sadraey [75] listed the design-related requirements as follows: performance, stability, handling qualities, operational requirements, low weight, affordability, reliability, maintainability, producibility, evaluability, usability, safety (airworthiness for aircraft and operator), supportability and serviceability, sustainability, disposability, marketability, environmental requirements, detectability (i.e., stealth), standards requirements, and legal requirements. In the design consideration of a precision aerial delivery system, Hall [76] argued that the design team should consider the operational supportability of a precision aerial delivery system.

Since drone applications kept growing for both civilian and military purposes, Candeloro et al. [77] also mentioned that this situation exacerbated the noise pollution problem caused by drones. In the study about a drone’s noise scattering, Jiang et al. [78] described that both commercial and civil domains had many applications that caused growing environmental concerns on the noise emission. Thus, drone noise received much attention in recent years. In the study about an exploratory investigation of combustion and NVH emissions signature of a drone jet engine, Soloiu et al. [79] mentioned that the largest pollution source of greenhouse gas (GHG) emissions comes from aerospace transportation. The pollution caused by drones’ fuel exhaust is also getting worse. Gaynutdinov and Chermoshentsev [80] mentioned that the avionics of UAV uses analog, digital, and RF devices to manipulate a UAV simultaneously over a wide frequency range (up to several gigahertz) of voltages and currents that lead to expansion of the paths’ electromagnetic interactions. That obviously leads to deterioration in the electromagnetic environment. Petrov et al. [81] mentioned that mass electrification of UAV creates massive mobile sources of electromagnetic interference. This is worse in dense urban environments in particular.

From the abovementioned literature, we can find many factors that are related to drone operations. However, through this comprehensive literature survey, we can further locate several key consideration factors, such as convenience, safety and security, supportability, and environmental impact, for drone operations.

2.2.4. Maintenance

Users should consider the maintenance issue when they adopt new equipment or new facilities. Design factors under this category should also be critical for drones, so the literature is also abundant with articles about relevant issues.

In discussing the U.S. Army’s light tactical vehicle solution, Metzler argued that the design of the U.S. Army’s light tactical vehicle should keep maintenance requirements low and keep maintenance costs low as well. A possible design solution is to adopt commercial off-the-shelf parts found in civilian vehicles [82]. In a study related to the neural network control system of the UAV altitude dynamic, Muliadi et al. mentioned the reasons why a UAV becomes a popular type. The easiness of maintenance is one of the reasons [83]. Edgell et al. [84] argued that the development of modular aircraft results from highly reliable and easy-to-maintain systems. That will minimize logistical requirements and simplify maintenance requirements. They also mentioned that locally manufactured parts allow the maintenance process to reduce their dependence on the parts supply and described the importance of the acquisition processes for critical components.

In the study about trickling filters following anaerobic sewage treatment, Bressani-Ribeiro et al. [38] argued that maintenance simplicity is the key consideration factor for new technology adoption. In the study about the role of nanoparticles upon the productivity of solar desalination systems, Rashidi et al. [85] mentioned the advantages of solar distillation systems, including low costs of construction, repair, and maintenance, simplicity, portability, and use of solar energy resources. In the study related to a time service improvement scheme for a nuclear power plant, Zhai and Bai [86] proposed the improvement scheme of time service system to solve the problems of redundant backup, system networking, convenient maintenance, and power supply optimization. Therefore, maintenance convenience is a critical issue for the operations of a system. In the study about point cloud processing system development, based on the proposed, Dingning and Qiong [87] improved system security and maintenance convenience of the point cloud processing system. This study showed the significance of maintenance convenience for an information system. In the patent about container-type data centers, Zhao et al. [88] also emphasized the importance of maintenance convenience for a mini data center in their patent application.

Moreover, in a study to compare life-cycle costing and performance parts costing, Adebimpe et al. [89] mentioned that maintenance cost involves the usage of consumables and spare parts. In another patent about a spare parts and consumables management system, Dellar et al. [90] used the wafer manufacture as an example to illustrate the importance of spare parts and consumables. They mentioned that a lack of required spare parts or consumables at critical points could mean damage to expensive wafers in the process and other wafers in pipelines waiting to process with closed machines. Additionally, the time it takes for a machine to drop reduces the wafer output, which can be very expensive. They also emphasized that it is critical to ensure that needed spare parts are available when designing a system. In an illustration of the life cycle of durable manufactured products, Oliva and Kallenberg [91] also mentioned that the acquisition assurance of spare parts and consumables is a critical issue for users.

The above literature study has described many consideration factors systematically related to maintenance for drones. Further, in the review process, a set of common critical factors has been identified in the literature, including maintenance simplicity, maintenance convenience, and acquisition of parts and operation consumables. These should be the critical factors placed under the drone maintenance construct.

2.2.5. Short Summary

In this subsection, a suitable set of criteria (design factors) for the application and selection of civilian drones is filtered and obtained on a solid basis. The constructs, each of which cover a separate subset of these criteria, are also identified. Therefore, the hierarchical tree structure for AHP investigation (i.e., AHP hierarchy) is justified and developed concretely in Section 3, so all of the subsequent works can begin based on (i.e., the Delphi process to confirm the included criteria and the tree’s form) the main process of AHP to collect the opinions using the expert questionnaire designed to calculate the weights for the factors and prioritize them. The THM-integrated analysis is for the preferential orders toward the criteria and constructs in each opinion group, and the implicative discussions identify the emphases for law-making advice. In other words, the thoroughness of the reviewed literature that has been studied underlies all later results’ credibility; even the comprehension of the process is, perhaps, sufficient to form a review study.

2.3. Methods: Delphi and AHP

After the previous use of THM in this field (see Section 1), this subsection gives a review at a higher level of Delphi and AHP, as they are the other two main methodological elements.

2.3.1. Delphi Method

Delphi is a survey method of research that aims to structure group opinions and discussions. The RAND Corporation developed the Delphi method. The purpose is to discuss and enable judgments on a specific topic, and then some synchronized decisions related to the specific topic can be made to represent a given group’s opinions and views. The Delphi method can break the limitations when traditional methods were used to obtain a specific group’s opinion or judgments for policy making [92]. Woudenberg [93] evaluated the characteristics of Delphi, which are listed as follows:

• Anonymity: Participants, almost all being experts, are interviewed by mail or a computer network.
• Iteration: There are rounds for a Delphi process. The first round is inventory (probing). The number of rounds can be determined in advance or “on the fly” according to the stability of individual judgments.
• Feedback: The results of the first (inventory) round are usually sent back to all participants. In the second round, the Delphi asks participants to make arguments about their deviational estimations for the criteria set fixed in the first round. It will feedback these arguments and the statistical results to all participants before any third and subsequent rounds begin.

Delphi is also helpful when using other methods would be not adequate or inappropriate. The following application contexts wherein it is particularly useful are summarized [94]:

• The problem is not suitable for precise analytical techniques but can benefit from collective subjective judgments.
• Individuals who contribute to the review of broad or complex issues do not have a sufficient history of communication and may have different backgrounds in experience and expertise.
• In face-to-face communication, people need to communicate effectively.
• Time and cost make frequent group meetings not feasible.
• The supplementary group communication can improve a face-to-face meeting’s efficiency.
• Disagreements between individuals are serious or politically unpleasant, so the communication process must be reviewed and anonymity ensured.
• Dominance due to quantity or personality power must be avoided.

Turoff and Linstone [95] have also summarized several application areas of Delphi, which are as follows:

• To collect current and historical data, although the data cannot be accurately known.
• To evaluate possible budget allocations.
• To explore urban and regional planning options.
• To build an educational model.
• To describe the pros and cons associated with potential policy options.
• To distinguish and clarify real and perceived human motivations.
• To explore priorities of personal values, social goals, etc.

In this study, Delphi is used to confirm the criteria set, the proposed constructs, and the tree form of the AHP hierarchy, for the following studies and results to be made and justified on a solid basis (see Section 3.1).

2.3.2. Analytic Hierarchy Process (AHP)

AHP is a theoretical and methodological framework of humans’ mental measurements by using a “pairwise comparison” repeatedly. It depends on the judgment of experts to get the priority and measure the relative values of intangible things, e.g., assets, alternatives, criteria, etc. Comparisons are carried out with an absolute “judgment scale” that represents the extent to which one element dominates another with respect to a given concept or content [96].

Specifically, if the things being compared are criteria in the MADM context, those relative values would mean the relative importance among the criteria (or alternatives). The vector-based form consisting of these values is called a “criteria weight vector,” or CWV, which can be used to prioritize the criteria (or alternatives) (i.e., to order them or rank them) [7,97].

For AHP, the judgments may be inconsistent. How to measure the inconsistency is another focus of AHP. Saaty breaks down the decision-making process into several steps, including a step called “CR-validation” to verify whether one’s pairwise judgments made individually for each pair of things, when put together, are consistent or not [98].

Zhuang et al. [12] proposed another “two-phases” separation for the entire MADM process using AHP, namely, a first phase of “CWV-determination” and a second of “alternative ranking”. As the research question of this study only concerned the first phase, we also refer to [12] for a summary of the formulas for the calculation process of CWV-determination. The other summary for these calculations is also seen in another mathematical expression form [8]. We also refer to [6] for a demonstration of CR-validation using a numerical example. The mathematical details of these are omitted here for space reasons, but similar full applications of AHP for the CWV-determination purpose (with CR-validation) have been illustrated in [8] and [9] to study the design factors for fighters and MALE UAS.

3. Processes and Results

3.1. Confirming the Factors with Delphi Method

According to Section 2.3.1, in the first round of Delphi in our study, three “almost experts” who are not familiar with each other were intentionally chosen to meet the selection standard of the method, and these participants gave their qualitative estimates. Their communication used e-mails. This confirmed that the 15 factors that were distilled from the literature review work in Section 2.2 in the original factor set—purchase cost, operation cost, maintenance cost, vehicle flight performance, manipulative ability, flight power, primary application, operation duration, operation convenience, operation safety and security, operation supportability, and operation environmental impact—are all effective, and there was no comment given to augment this factor set by adding any other factor. In addition, none of them opposed the four constructs (categories) that were established by the authors after the literature study and agreed with how each individual factor (criterion) could be placed (mounted) under its upper-level construct. This means that the hierarchical structure of the tree has been confirmed at this stage, and the “decision hierarchy” can therefore be ascertained in Figure 4 for AHP.

However, as in the first round the operational definitions for each factor were sent along with the data for polling, and they were slightly modified by the participants after returning, we present the modified version in

Table 6. Operational definitions of the factors for civilian drone design/selection.

Criterion	Operational Definition
Purchase cost	The cost of purchase of drone flight vehicles, manipulation devices, and devices required to perform related applications.
Operation cost	The costs of using drones to perform related missions, such as the cost of flying power such as fuel and electricity, the cost of mission-related consumables, and the costs that may be involved in the use of landing sites.
Maintenance cost	The cost of maintenance and replacement of components for the flight vehicle and primary equipment that ensure the proper operation of a drone.
Vehicle flight performance	The critical flight performance considerations of a drone include flight speed, range, endurance, flight altitude, payload, and take-off and landing way.
Manipulative ability	The important considerations of a drone manipulation performance include operation range and distance, manipulation agility and stability, operation interface, and anti-interference of wireless control signal.
Flight power	The types of drone flight power include fuel, electricity, hybrid fuel and electricity, solar energy, and other possible power; different power systems have a relative impact on the flight performance of a drone.
Main application	The main applications of drones include logistics and transportation, agricultural plant protection, surveillance and security, geographic mapping, and aerial photography and performance.
Operation duration	The duration of time to perform the main application, such as spraying, monitoring, shooting, etc.
Operation convenience	The convenience of drone operation is restricted by the flight vehicle take-off and landing field and the open condition of the operating airspace also affects the convenience of drone operation.
Operation safety and security	Drone operations should consider the safety and security of flight and personnel.
Operation supportability	Related support required for drone operations, such as flight logistics support, hardware and software support for related equipment required to perform tasks, etc.
Operation environmental impact	The possible impact of drone operation on the surrounding environment, such as noise, pollution from fuel exhaust, electromagnetic wave emission, etc.
Maintenance simplicity	The maintenance simplicity of drone flight vehicles and related equipment means that maintenance personnel can easily repair drone flight vehicles and related equipment through modular components.
Maintenance convenience	The maintenance convenience of drone flight vehicles and related equipment means that the suppliers of drone flight vehicles and related equipment provide relevant maintenance and maintenance support in close and convenient.
Acquisition of parts and operation consumables	Drone flight vehicle and related equipment components and related consumables needed for operation are not in short supply.

.

Then, the second round of Delphi fed back the whole group’s results on the previous round to all participants for subsequent evaluation. These included the hierarchical structure of the tree weaving the constructs and the criteria (i.e., the decision hierarchy) as shown in Figure 4 and the set of factors (although no change was made) and the revised operational definitions for the factors in Table 6. Finally, all participants returned with no other comments, and this confirmed that all the delivered data (i.e., the set of factors, the constructs, the decision hierarchy that reveals their relationships, and the operational definitions) are effective. Therefore, fortunately, the Delphi process does not involve a third round.

3.2. Prioritizing the Factors with AHP Method

Based on the confirmed decision hierarchy, as shown in Figure 4, a set of expert questionnaires using the pairwise-comparison scales of 9:1, 7:1, 5:1, 3:1, 1:1, 1:3, 1:5, 1:7, and 1:9 is designed for investigating the individual opinions of the experts (or decision-makers (DMs5)).

According to the THM, we would like to understand the opinions of the domain experts in the industry, academia, and research institutes. For this sake, the entire sample is designed to have 27 experts, and it is intentionally partitioned into three groups, with each group containing 9 experts. The researchers visited these 27 experts and performed investigations in terms of AHP using the designed questionnaire set, during the period from 3 January to 28 February 2021.

Eventually, for each expert interviewee (respondent), we successfully received five pairwise comparison matrices of the five AHP expert questionnaires that are consistent in the results. Note that among the five matrices, one of them connotes the results of pairwise comparison of the constructs (i.e., with respect to CDSEM, or the civilian drone selection evaluation model as shown in Figure 4), and for the remaining four, each connotes the results of a pairwise comparison of the individual factors with respect to some construct.

Table 7. Background statistics of the interviewed experts.

	Type	#DMs	Percentage
Gender	Male	24	88.89%
	Female	3	11.11%
Degree	Master’s	14	51.85%
	Ph.D.	13	48.15%
Group of Expert	I: Industry	9	33.33%
	A: Academy	9	33.33%
	R: R&D Institute	9	33.33%
Age	31–40	7	25.93%
	41–50	11	40.74%
	51–65	9	33.33%
Years in Service	1–10	5	18.52%
	11–20	12	44.44%
	>21	10	37.04%

gives the statistics for the backgrounds of the 27 interviewed experts overall, as the “respondent profiles” are also polled along with the AHP-style survey using an initial but anonymous block of questions. As can be observed, 24 of these experts are male and three are female; 13 have a Ph.D. and 14 have a master’s degree. In terms of age, they are all 31–65 without exception. Finally, five of them have experienced the service for <10 years, 12 have experienced 10–20 years, while 10 of them have worked for over 21 years. The A, I, and R partitions of the sample of the experts are also clear in this table.

Surprisingly from these sample stratifications, no respondent in this survey possesses only a bachelor’s degree (or under), even for the respondents in group I (industry). After a post-survey consultation, the fact is interesting. For group I, we interviewed respondents from a maker that is partially government-funded, because it is not only the largest aeronautic manufacturer in the studied country (as mentioned in Section 1) but also where people who have the decisive/influencing power for drones’ law-making may come from. As such, the respondents whom we interviewed are all (very) high-level staff, but in such institutions, only people who possess a master’s or Ph.D. degree can be promoted to these higher positions.

As usual, the investigation process required one to three rounds of interviews for all pairwise comparison matrices filled in by an expert to pass the CR-validation process, with a CR (consistent ratio) threshold set at 0.1 (see Section 2.3.2). Additionally, we also found that most DMs could pass the CR-validation easily with one round of interviews when they were requested to answer the questionnaires containing three items to be compared pair-wisely (i.e., for the three factors under construct CA and for another three under construct CD). Otherwise, many of them passed the CR-validation with two to three rounds of interviews, especially when they were asked to compare the five factors under CB pair-wisely.

After the consistencies in the results were all guaranteed, calculations to obtain the CWVs based on the data in the five pairwise comparison matrices were performed for each expert using the Expert-Choice software (and it was also installed on a laptop as a mobile office during the investigation to record the experts’ answers in real time).

By reference to THM, we further combined the CWVs with respect to the same thing (i.e., CDSEM or one of the four constructs) for all experts in each expert group to obtain an “aggregated CWV.” For more details, please see Appendix A. This aggregated CWV connotes a “group opinion” with respect to “that thing”. As the different groups (i.e., A, I, and R) give various “group opinions” for “the same thing”, the variety and heterogeneity can be observed, analyzed, and compared. The following sub-subsections present these results in a visualized manner.

3.2.1. Academia (A) Group’s Opinions

The weights of the four main constructs are shown and ranked in Figure 5 based on the group opinion according to the “aggregated CWV” over the individual CWVs of the nine experts in the “A” group, with respect to CDSEM.

Reviewing Figure 5, we can find the significance rankings of those four constructs: CB: Performance and Application ≻ CC: Operation ≻ CD: Maintenance ≻ CA: Cost. The sum of the relative weights of the two decision constructs, CB and CC, exceeds 70%; especially, the performance and application construct received nearly half of the relative weight, which is 0.463.

Figure 6 shows the 15 consideration factors’ “absolute weights” when the aggregated CWV (for a subset of factors) with respect to each construct is multiplied with the associated element in the aggregated CWV with respect to CDSEM (i.e., a number in Figure 5, in its original non-ranked order), while the priority over these 15 factors is also visualized.

Examining Figure 6, we find that only the top five factors are “more important” and the 10 decision criteria remaining are less important, according to the academia (A) group’s overall opinion. The manipulative ability (cb2) and vehicle flight performance (cb1), both under construct CB, are the two most important factors. both absolute weights are over 0.1, and the sum of the two weights exceeds 30%. Relatively, operation convenience (cc1) is deemed the least important factor among the 15 by the experts in the A group; its absolute weight is less than 0.0335. Note that we have used “>0.7” as the rule to justify whether a factor is “more important” by seeing whether the absolute weight of the factor surpasses the threshold of 0.7 or not.

3.2.2. Industry (I) Group’s Opinions

The weights of the four main constructs are shown and ranked in Figure 7 based on the group opinion according to the “aggregated CWV” over the individual CWVs of the nine experts in the “I” group, with respect to CDSEM.

Reviewing Figure 7, we can find the significance ranking of those four constructs: CB: Performance and Application ≻ CA: Cost ≻ CD: Maintenance ≻ CC: Operation. Among them, the industry (I) group thinks that CB and CA are the two most significant constructs for designing and selecting civilian drones. Their relative weights, 0.288 and 0.287, are almost even. The operation construct (CC) is the least significant one, because its relative weight is only 0.180. However, CB and CA are just a bit more important than CD, and the weight of CC is just 0.07 away from a quarter (0.25).

Figure 8 shows the 15 consideration factors’ absolute weights, while the priority over these 15 factors is also visualized. The process to obtain this figure is analogous to what has been described in Section 3.2.1, but the studied sample group has been changed from A (academia) to I (industry).

Examining Figure 8, we can see that eight of the 15 factors are “more important” and the other seven factors are “less important”, according to the industry (I) group’s overall opinion, while the same threshold (0.7) is used to justify such a classification. Among the eight “more important” factors, we found that there are also five factors whose weights may dominate the other three to a certain extent, and their order is: cb5: Operation duration ≻ ca3: Maintenance cost ≻ cb1: Vehicle flight performance ≻ cb2: Manipulative ability ≻ cd3: Acquisition of parts and operation consumables. These five factors, in sum, may occupy 55% of the total importance. Relatively, among the seven less important factors, operation environmental impact (cc4) is the least important decision criterion. This factor receives a weight of only 0.014, which is obviously lower than the other six less important ones. This means that experts in the industry (from the I group) think the possible environmental impacts caused by operating the civilian-use drones are of little significance.

3.2.3. Research Institute (R) Group’s Opinions

Finally, this study explores the four main constructs’ weights as shown and ranked in Figure 9 based on the group opinion according to the “aggregated CWV” over the individual CWVs of the nine experts in the research institute (R) group, with respect to CDSEM.

Reviewing Figure 9, we can find that the significance rankings of those four constructs are CB: Performance and application ≻ CD: Maintenance ≻ CA: Cost ≻ CC: Operation. Among them, the research institute (R) group thinks that CB is the most significant factorial construct for designing and selecting civilian drones. Its relative weight, 0.315, may dominate the other three constructs, the importance of which is almost 1/3. In contrast, the cost construct (CA) and the operation construct (CC) are the less significant ones (see their weights, 0.221 and 0.211, in the figure). However, both of the weights are not far away from a quarter (0.25).

Figure 10 gives the absolute weights of the 15 consideration factors, while the priority over these 15 factors is also visualized further. The process to obtain this figure is analogous to what has been described in Section 3.2.1, too, but the studied sample group has been changed from A (academia) to R (research institute).

Given the results in Figure 10, seven of the 15 factors’ absolute weights have surpassed the used threshold, 0.7, so they can be classified as the “more important” factors. Besides these, the other eight factors should be less important for the design and selection of civilian drones. Further looking into these two groups, we find that the only three factors that have received a >0.10 or ~0.10 weight value, i.e., main application (cb4), vehicle flight performance (cb1) of civilian drones, and acquisition of parts and operation consumables (cd3), are the most important, as regarded by the experts who came from the R&D institutions. These factors should contribute a lot to CB and CD, as CB and CD have been shown to be the two more important constructs in Figure 9.

Relatively, the only two factors that received a weight value less than or equal to 0.03, i.e., operation convenience (cc1) and operation environmental impact (cc4), are deemed the two least important factors in minds of those experts who came from the research institute (R) group. The fact that the two least important factors are under the operation construct (CC) is reflective of the fact that CC is the least important construct in Figure 9.

3.3. A Short Summary

In this section, the process we have performed to conduct the analysis is demonstrated, and the results are shown and justified based on the empirical survey data, for retrieving a set knowledge about the design and selection factors of civilian drones, i.e., it is a mind-mining process for the experts’ key opinions that will be reflected in law-making. This process involves the use of multiple theories and methods, including THM, Delphi, and AHP.

In Section 3.1, the Delphi method is used to confirm that the established set of influencing factors is effective after the thorough literature study, that the operational definitions made for these factors are correct, that the postulated set of constructs used to cover these factors is reasonable, and that the defined tree structure (i.e., the AHP hierarchy) is plausible, with the help from the selected “almost expert” participants.

In Section 3.2, the AHP survey is designed in such a way that it is linked with the THM, i.e., three groups of experts, A, I, and R, are intentionally organized and interviewed to understand the “group opinions,” rather than the individual opinions. These “group opinions” are aggregated based on each individual expert’s CWVs related to the civilian drone selection and design constructs and the selection and design criteria under each construct.

However, after the studies in Section 3.2, we may eventually see that there are heterogeneities and homogeneities in the opinions of the three groups as stratified by using the THM; the similarities and differences when these groups’ opinions are compared subject to “the same thing” may require further exploration. This is the purpose of the next section.

4. Discussion and Implications

The priority and relative weights for the four constructs and for the 15 criteria were explored group by group in the previous section by aggregating the individual opinions (CWVs) in each group. Given the information, questions were raised as to the homogeneities and heterogeneities that exist across the different expert groups (i.e., A for academia, I for industry, and R for research institutes).

The first insight gained is that regardless of the subject group, the experts have a consensus that “CB: Performance and application” is the most significant construct to be considered while designing and selecting civilian drones. Such an outcome is persuasive, because no exception is found according to the group opinions.

Following the first insight, the second insight is that there are varieties when the groups of experts are addressing the importance of another three constructs while it is to design a civilian drone; these constructs are CA: Cost, CC: Operation, and CD: Maintenance. Among these, group A feels that operation is more important than cost and maintenance. Meanwhile, group I feels that cost is more important than maintenance and operation. However, group R feels that maintenance is more important than cost and operation. The differences in “which construct is more significant than the other two” in the three expert groups’ mind tells a truth that for these constructs (CA, CC, and CD), the group opinions are quite diversified (except for the commonly agreed critical importance for CB).

Following from the former two insights, the third insight relates to a claim that no construct is meaningless. The I group’s opinion for the constructs (with respect to CDSEM) (Figure 7) shows that the importance of the least important construct, operation, is 18.0%. The A group’s opinion for the constructs (Figure 5) shows that the importance of the least important construct, cost, is 12.1% (still >10%), let alone for the operation construct that is regarded as the least important by the R group (whose relative importance is 21.1%; see Figure 9). These may imply that the set of constructs proposed by this study is effective, since each construct, either being critical or not, possesses a weight to a certain extent.

Before going on to discuss other insights about the consideration factors in more detail (in addition to those about the constructs), in

Table 8. Rankings for individual criteria and the ranking differences among the expert groups.

Construct	Criterion	Industry	Academia	Research	SRD *	ARD *
Cost	Purchase cost	9	14	12	10
	Operation cost	10	12	10	4	10
	Maintenance cost	2	10	4	16
Performance and application	Vehicle flight performance	3	2	2	2
	Manipulative ability	4	1	6	10	8.4
	Flight power	6	6	8	4
	Main application	8	5	1	14
	Operation duration	1	4	7	12
Operation	Operation convenience	13	9	14	10
	Operation safety and security	7	3	5	8
	Operation supportability	14	10	11	8	7.5
	Operation environmental impact	15	13	15	4
Maintenance	Maintenance simplicity	11	8	9	6
	Maintenance convenience	12	15	13	6
	Acquisition of parts and operation consumables	5	6	3	6	6

* Table legend: SRD: sum of ranking difference, ARD: average ranking difference.

, let us summarize the final ordinal ranks over all criteria (row-wise) as assessed by the three expert groups (column-wise), in terms of “rank order vector” (or ROV; see [7,12,99]) and analyze the ranking differences (or differences in rank alternative).

Scrutiny into this table finds the fourth insight that several factors (decision criteria) receive similar overall rankings from the three expert groups. Because the SRD (sum of ranking difference) aggregates the “difference in rank between each pair of groups” for some specific criteria, and there are always three pairs of groups in each case, SRD/3 can be used to measure the average diversity of the group opinions when the focus is a factor. If we set up a threshold “SRD/3≦2”, which means that the three expert groups give similar rankings to the same factor (in that for this vector, the average diversities of the group opinions are quite converged), to filter the penultimate SRD column in Table 8, there are seven factors are under this threshold. Therefore, we can gain the knowledge that regardless of the expert group, experts in these three groups have reached a consensus to a certain extent on the priorities of “operation cost” (ca2), “vehicle flight performance” (cb1), “flight power” (cb3), “operation environmental impact” (cc4), “maintenance simplicity” (cd1), “maintenance convenience” (cd2), and “acquisition of parts and operation consumables” (cd3). Note that here the “priority” of any factor, while being represented using a rank order number as shown in the middle three columns in Table 8, does not necessarily mean its importance in terms of absolute weight.

The fifth insight. As can be seen in the above list of seven factors, one is under CA and one is under CC, two are under CB, but three are under CD (and all criteria under construct D, CD, are listed). This tells a truth that under every construct, there is at least one factor that has received (very) similar rankings from the three groups of experts, and the real count varies construct by construct. Moreover, compared to the factors under CA and CB regarding which the expert groups sometimes give very diversified opinions but sometimes do not (see the gap in SRD, 12, for the factors under CA and the same gap size, 12, for the factors under CB), their opinions (among the groups) are relatively stable (with smaller variance; the gap in SRD is six) for all factors under CC, and no variance is observed for the SRD values assessed for all factors under CD (with the smallest possible variance, 0).

The sixth insight. The last column in Table 8, “ARD” (average ranking difference), provides another measure. Theoretically speaking, the ARD measures the degree to which the opinions for the factors included by a construct are diversified on average, i.e., 10, 8.4, 7.5, and 6.0 for construct CA (cost), CB (performance and application), CC (operation), and CD (maintenance), respectively. It is, in fact, a measure other than the “gap” between the highest and lowest of the SRD values used in the previous point to connote the variance (degree) in those SRD values for each construct. However, despite this, it is surprising that the order of the constructs, while justified using the ARD measure, roughly concurs with the order while justified using the “gap” measure to see the variances in SRD under the constructs. In other words, these outcomes may cross-validate with each other, for how the group opinions of experts are diversified under each construct within which the individual factors are included.

Then, following the fourth insight, the seventh insight relates to the set of the remaining eight factors, which have more opinion differences among the three groups of experts in terms of SRD. These include: “purchase cost” (ca1), “maintenance cost” (ca3), “manipulative ability” (cb2), “main application” (cb4), “operation duration” (cb5), “operation convenience” (cc1), “operation safety and security” (cc2), and “operation supportability” (cc3). Additionally, according to their SRD values (i.e., the degree to which their opinions diversify across the groups for each factor), these eight factors are sorted in descending order as: ca3 (SRD = 16) > cb4 (14) > cb5 (12) > (ca1, cb2, cc1) (10) > (cc2, cc3) (8). Both the set of these factors and the order of them are worthwhile knowledge for the design and selection of civilian drones. These are also important for the different groups of experts to make the advice for law-making, because integrating their diversified opinions for these factors while defining the legislations is usually a tough task, but the results of this study may offer some supplemental information about the opinion gaps before they sit down together with real lawmakers.

The eighth insight pertains to “what factors are the most critical ones to be watched for?” Either the field of big data analytics or MADM usually gives more emphasis to the items (i.e., criteria or alternatives, and the factors as in the case here) that are extraordinary (i.e., the data points, the opinions from DMs, and the opinions for the factors here). For example, an MADM model may assign a heavier weight to a DM whose opinion is away from the average by using the opinion weight vector (OWV) concept [8]. For this sake, based on the information of the “diversified order” of those factors obtained in the previous point, we also find that the three groups of experts have the largest opinion differences for the maintenance cost (ca3), main application (cb4), and operation duration (cb5) factors for designing and selecting civilian drones. Following from the former insight, this implies that they (i.e., the three groups) may reach little consensus while the relevant drafts pertaining to these three factors are discussed among them for regulating the civilian use of drones (after they sit down together), if no extra (or at least sufficient) effort is made to coordinate their preferences and intentions toward civilian drones’ design and selection and law-making.

5. Conclusions and Future Recommendations

This study, at the outset, aims to understand the expert opinions during a consultation process before law-making. These opinions will shape the related laws made and enacted to regulate the relevant matters of civilian drones (i.e., to control the design, manufacturing, and uses of civilian drones in Taiwan). As the consultation process involves polling the opinions from three expert groups, academy (A), industry (I), and research institutes (R), the main research question is thus about understanding their opinions and noting the similarities and differences in their group opinions. However, how can the relevant set of knowledge be explored thoroughly and “neatly”?

In this study, first, the literature was studied comprehensively to have a set of influencing factors for designing and selecting the civilian drones. Eventually, a set of 15 consideration factors was established, with four possible constructs covering these factors being summarized, i.e., CA: cost, CB: performance and application, CC: operation, and CD: maintenance. Therefore, a hierarchical tree weaving the 14 factors under the four constructs was postulated, and this formed the initial “AHP hierarchy” that awaited further confirmation. This rigorous process is critical, because drone technologies have been combined with the non-aeronautic emerging technologies for various application purposes nowadays, e.g., 5G [100], data integration techniques [101], the machine learning models [102], optimization routing algorithms [103], and the humanoid robots [104]. Thus, a multiplex of these advanced capabilities (and technologies) into existing areas of UAV [105] may become the interferences to exploit the key influencing factors.

In the next, the established initial AHP hierarchy was sent to several “almost experts” for evaluation using the Delphi process for two rounds. At last, the AHP hierarchy is confirmed to be effective, with minor corrections made for the operational definitions. Following this, the questionnaires were designed in AHP style, and using these, experts in the three groups were interviewed. Stratifying the respondent sample into three groups intentionally followed the proposed “modified THM” model, in which the “triple player parties” were altered from I, G, and A to A, I, and R. See former sections for the reasoning process.

The entire survey process spanned over two months in early 2021. It required one to three rounds of interviews for the expert opinions to pass the CR-validation check (of AHP). Eventually, the pairwise matrices collected from all of the 27 experts were used to calculate the CWVs (i.e., opinions in each expert’s mind) for the constructs (with respect to the total goal) and for the criteria (factors) (with respect to some construct). Based on these individual CWVs, the “group opinions” are aggregated in terms of “group CWVs” for the relative importance (of the four constructs, and of the different factors under each construct).

For all the three groups, their group CWVs justified for the four constructs were dissected and analyzed. Furthermore, within each group, its group CWVs were “synthesized” to obtain the “absolute weights” for the 15 factors overall, and these weights were also ranked in each group. These overall ranks of the factors became another target to be compared across the THM groups. Through a process to understand the former results, several practical implications for drone design/selection and for the advice made by the different expert groups for law-making were therefore drawn. As a short summary, the insights gained are summarized as follows:

• Regardless of the subject group, the experts have a consensus that construct CB, performance and application, is the most significant construct to be considered without exception.
• For the importance of the three remaining constructs, the opinions are quite diversified, because, except for CB, each group feels a different construct (either CA, or CC, or CD) to be more important than another two.
• No construct proposed by this study is meaningless, since even the construct with the lowest importance among the three groups has a weight of 12.1% (i.e., CA by party A).
• The three groups have reached a consensus on the overall rank (i.e., connoting the priority for its importance level) of many factors to a certain extent, among all of the 15 factors considered, which are ca2: operation cost, cb1: vehicle flight performance, cb3: flight power, cc4: operation environmental impact, cd1: maintenance simplicity, cd2: maintenance convenience, and cd3: acquisition of parts and operation consumables.
• Compared to the situation in which the expert groups sometimes rank the factors under CA and CB very diversely but sometimes do not, their opinions are relatively stable for all factors under CC, and no variance in rank is observed for all factors under CD using the SRD measure.
• Using ARD as the measure, CA (cost) is shown to have a larger average ranking difference (over all factors) than CB (performance and application), CC (operation), and CD (maintenance); such an order concurs with the order that is justified using the previous variance measure (in terms of the farthest distance in SRD between any two factors under the constructs). In other words, these outcomes may cross-validate with each other.
• There is another set of eight factors that received more opinion differences from the three groups of experts. According to the SRD values of these factors (i.e., the degree to which the opinions of the groups are diversified), these factors can be sorted in descending order as: ca3 (16) > cb4 (14) > cb5 (12) > (ca1, cb2, cc1) (10) > (cc2, cc3) (8). Both the set of factors and the order of them are worthwhile knowledge to close the opinion gaps before the experts sit down together.
• By reference to the theory in the field of big data or MADM, we put an emphasis on the opinions of the factors that are extraordinary. Therefore, the last insight is that the three groups of experts have the largest opinion differences on factors ca3: maintenance cost, cb4: main application, and cb5: operation duration. This implies that they may reach little consensus while the relevant drafts to regulate the civilian use of drones are discussed after they sit down together if no extra coordination about their preferences and intentions toward the factors is made.

As can be seen in the list of valuable insights gained, all this is critical for the different groups of experts to give their advice for law-making because investigating and integrating their opinions, either diversified or not, for these factors is usually a difficult task while defining the legislations. Moreover, the results may provide the knowledge to facilitate the communication processes and close the opinion gaps before the experts sit down together or even before a final draft is formally delivered for law-making. As from the perspective of operational research (OR) some law-relevant issues have just been addressed [106], these insights also encompass the general aim of this research line.

The THM has been popular for years, but studies based on it explored many other issues, such as innovation (or national innovation system), governmental aspects (e.g., smart city), the industrial revolution and cooperation, local economic or regional development, and knowledge production, transfer, and economy matters. Most of the studies were based on the academia (university)–industry–government interactions (i.e., A–I–G) to explore the concerned topics where appropriate. Analogously, this study proposed a “modified THM” to explore the factors for civilian drone design/selection in terms of academia–industry–research (i.e., A–I–R) and identify the relationships between the group opinions (attitudes) toward those constructs and factors, so as to understand the required knowledge for law-making. In previous THM studies, e.g., in the field of smart city, only the sphere of drone applications has been touched. In this sense, this study not only fills the gap by using the modified THM to offer another set of in-depth knowledge about the design/selection of civilian drones with a systematic study flow, but also is helpful for lawmakers to develop regulations on drones during the formation process of law-making.

The future research directions may involve:

• More deepened analysis about the group opinions, or even the individual opinions, based on the (human) data set as collected by this study; for this sake, the advanced visualization techniques [8,9] are perhaps helpful.
• As the focus of this study is on the importance of the constructs and the factors (either relative or absolute), this study uses AHP for CR-validation and CWV-determination; the last phase of AHP or other MADM methodologies can be applied to the real selection problem of drones [6,9,97]. As current data profiles of different brands of drones can be aggregated as a “decision matrix” (which means a set of computer data), together with the human data polled in this study, integrating the heterogeneous data sets, solving the selection problem, and performing the decision analysis based thereon should be an interesting DDDM study [12].
• Some techniques in the (big) data analytics field can be applied to test whether each pair of the group opinions of experts is “really dissimilar” or not, and recently this concept has become popular. For this, we refer to the non-parametric tests used by the “similarity confirmation method” [12,99]. Besides this, using other non-parametric-based measures and comparing the results should also be interesting.