Safety Enhancement of UAVs from the Signal Processing’s Perspectives: A Bird’s Eye View

Abstract

Unmanned air vehicles (UAVs) or drones have gained popularity in recent years. However, the US Federal Aviation Administration (FAA) is still hesitant to open up the national air space (NAS) to UAVs due to safety concerns because UAVs have several orders of magnitude of more accidents than manned aircraft. To limit the scope in this paper, we focus on large, heavy, and expensive UAVs that can be used for cargo transfer and search and rescue operations, not small radio-controlled toy drones. We first present a general architecture for enhancing the safety of UAVs. We then illustrate how signal processing technologies can help enhance the safety of UAVs. In particular, we provide a bird’s eye view of the application of signal processing algorithms on condition-based maintenance, structural health monitoring, fault diagnostics, and fault mitigation, which all play critical roles in UAV safety. Some practical applications are used to illustrate the importance of the various algorithms.

1. Introduction

The use of unmanned aerial vehicles (UAV) in the military and industry today is becoming more and more widespread. However, perhaps due to lower manufacturing standards and budget limitations, the mishap rates in unmanned aerial vehicles (UAVs) are several orders of magnitude greater than manned aviation [1]. Considering these high mishap rates, the US Department of Transportation’s Federal Aviation Administration (FAA) has initiated several programs and partnerships to enhance the safety and reliable operation of UAVs [2].

In this paper, we focus on large, heavy, and expensive (millions of dollars) UAVs such as cargo transport and search and rescue aircraft, not radio-controlled amateur drones. In general, UAV safety can be improved from the following perspectives. First, the UAV manufacturers need to use durable engines and communication equipment and strong structural materials. Reliable communication equipment will ensure drone safety, as a lost link between UAV and ground station is very dangerous [3]. Durable engines and strong materials will ensure reliable flight in rough weather conditions. Second, advanced conditioned-based maintenance (CBM) practice should be deployed [4,5,6]. Compared with traditional periodic preventive maintenance, CBM can be cost-effective but may require additional sensors to monitor some critical components such as the engine. Third, structural health monitoring (SHM) [7,8], especially the non-destructive type of SHM, should be frequently used to monitor cracks, loosened fasteners, etc.

The aforementioned ideas mostly require both hardware and software, and have been routinely used in UAVs to maintain reliability and safety. In addition to the above safety enhancement practices, we would like to emphasize that signal processing algorithms also play important roles in other critical areas. Figure 1 shows the general architecture of an integrated system that can enhance the safety of UAVs. Some monitoring operations such as SHM, CBM, are done off-line on the ground. The off-line process involves data collection and processing. Others such as sensor and actuator fault diagnostic, fault-tolerant control are done online. The sensor’s measurements (angles, speed, etc.) are all collected in real-time and processed in real-time. Contingency planning is done off-line. Moreover, as shown in Figure 1, we can clearly see the use of signal processing algorithms in fault detection, fault isolation, fault magnitude reconstruction, and fault mitigation. First, robust fault diagnosis algorithms [9,10,11,12,13,14,15,16,17,18] perform accurate fault detection and isolation. Failures in sensors and actuators can cause system instability. Diagnosing those faults will improve the overall safety of UAVs. Second, in fault mitigations, robust [19,20,21,22,23] and fault-tolerant controllers [24,25,26,27,28,29,30,31] are also critical for UAV safety. Robust controllers can tolerate some small perturbations in the UAVs due to aging or external disturbances. Fault-tolerant controllers can perform control reconfiguration to directly address sensor and actuator malfunctions during flight. Third, one critical aspect of fault mitigation is contingency planning for engine failures, which is the last line of defense for UAV safety. That is, all the aforementioned practices have been tried, but nothing works and the UAV is eventually on its course to crash due to engine loss. Under such an emergency, the goal of contingency planning is either to help UAV operators to glide the UAV or will make the UAV autonomously land itself to a crashing/ditching site or local airport runway if there is no reliable communication link available. In the case of an emergency due to full loss of thrust, wind plays a critical role with respect to reachability to the emergency landing site [32]. Upon full loss of thrust due to engine failure, because of the wind impact, the UAV may not reach the designated landing site and crash into populated areas causing loss of lives. Thus, the wind impact on reachability needs to be addressed in path planning for engine loss contingencies. It is also important that in the event of an emergency that might happen at high altitudes, the UAV should choose a forced landing path that does not violate no-fly zones or stormy weather air zones not to further complicate the situation.

The contributions of our paper is as follows:

• We provide a bird’s eye view of the importance of signal processing algorithms in enhancing the safety of UAVs.
• In each area, we highlight some recent advances in the literature.

Our paper is organized as follows. In Section 2 and Section 3, we briefly mention CBM and SHM practices. In Section 4 and Section 5, we focus on fault diagnostics and fault magnitude estimation. In Section 6, we address fault mitigation using robust and fault-tolerant controllers. Section 7 discusses the last line of defense: contingency planning for an emergency landing. Finally, some concluding remarks, limitations, and future directions are given in Section 8.

2. Condition Based Maintenance (CBM)

Traditionally, mechanical components such as engines, bearings, and gearboxes are maintained by using preventive maintenance, which performs periodic checks on the components. For example, engine oil for cars is replaced with new oil every three thousand miles or every six months. Preventive maintenance is certainly effective and has been widely used. However, one drawback is that it may not be cost-effective. When a car is not being used frequently, periodic maintenance may be wasteful because the oil quality may still be good.

In recent years, people have been advocating condition-based maintenance [33,34], which means some maintenance decisions should be made based on the condition of the component rather than time. Two survey papers [35,36] provided a good review of the relevant works in this area. Some papers have proposed the use of wireless sensor networks (WSN) for quantifying system conditions [37,38]. Figure 2 illustrates how WSN can be used for induction motor monitoring.

It should be noted the Internet of Things (IOT) is also a hot area that can be beneficial to CBM. In [39], some discussions talked about the various applications of IOT in CBM. IOT can also help online fault diagnosis as well.

3. Structural Health Monitoring (SHM)

As shown in Figure 3, Structural health monitoring (SHM) [7,8] is one way of nondestructive inspection (NDI). It can be done passively or actively. The purpose is to detect structural defects such as cracks on the wing, loosened bolt, and fasteners, etc.

In our opinion, SHM is also one form of CBM. To illustrate the basic idea of SHM, one can refer to Figure 4, which shows an array of sensors/actuators to monitor an aircraft wing panel. Ultrasonic pulses are generated from each sensor, and the rest of the array is receiving the transmitted signals. Once all the sensors are scanned, the collected signals from all the elements will be fed into an algorithm for fault localization. In [8], a RAPID algorithm was developed, which has been widely used in many papers.

The interrogation methods in [7,8] are bulky. Recently, there are new and recent advances in wireless sensing for SHM. As shown in Figure 5, passive low-power wireless node can be installed onto structures. Details can be found in [40].

4. Sensor and Actuator Fault Diagnostic Algorithms

Sensors and actuators are critical components in complex systems. For instance, in an airplane, effective flight control is impossible if sensors and/or actuators are malfunctioning [41,42]. Sensors and actuators can fail, and their failures have a significant impact on the performance of a system. In the worst case, the failure even can affect the safe operation of the system, leading to a catastrophic event. Sensor failures may include precision degradation, drift, frozen reading, and complete failure [43]. Similarly, actuator failures may include limited range of motion, e.g., valve stiction and complete failure [44]. It is a challenging task to detect sensor and actuator failures [41,42] because sensor outputs contain information from a multitude of sources: normal system outputs, faulty sensor signals, and signals due to noise and external disturbances. Isolating different signals requires the utilization of the input-output relationship of the system. Conventional approaches to increasing the reliability of aircraft systems include installing redundant sensors, which will add more weights, costs, complexity, and most importantly, additional reliability problems.

Given a linear multi-input multi-output (MIMO) system, the inputs, internal states, and outputs can be described by

$$\begin{array}{l}{x}_{k+1}=A{x}_k+B{u}_k+B{M}_u\Delta u_k+{\mathsf{\omega }}_k\\ y_k=C{x}_k+D{u}_k+D{M}_u\Delta u_k+M_y\Delta y_k+{\mathsf{\epsilon }}_k\end{array}$$

(1)

where $u_k$, $y_k$, and $x_k$, are system inputs, outputs, and internal states, respectively.$\Delta u_k$, $\Delta y_k$ denote actuator and sensor faults. We would like to detect if the magnitude of $\Delta u_k \mathrm{or}$ $\Delta y_k$ is non-zero. If the answer is positive, we have to estimate the magnitude and identify the failure direction matrix $M_u \mathrm{or} M_y$. Finally, the magnitudes of failed sensors or actuators will be reconstructed.

Sensor fault detection and diagnosis have been a research topic for decades, and many articles have been published. Interested readers are referred to the survey paper by Frank [45]. While fault detection is relatively easy, isolation of multiple faults is still a challenge to many existing schemes. Recent advances in fault diagnosis use kernel partial least square [46] and nonlinear techniques [47].

5. Sensor Magnitude Reconstruction

After the identification of faulty sensors/actuators, one has to estimate the fault magnitudes. Detailed procedures can be found in [48]. With estimated fault magnitudes, one can correct measurements of the faulty sensors. The measurements in faulty sensors are affected both by the actuator fault $\Delta u_k$ and sensor faults $\Delta y_k.$ In order to get the corrected measurements in faulty sensors, one can design a Kalman filter,

$$\begin{array}{l}{\widehat{x}}_{k+1}=A{\widehat{x}}_k+B{u}_k+K(y_k-M_y\Delta {\widehat{y}}_k-D{M}_u\Delta {\widehat{u}}_k-C{\widehat{x}}_k-D{u}_k)\\ {\widehat{y}}_k=C{\widehat{x}}_k+D{u}_k\end{array}$$

(2)

where ${\widehat{y}}_k$ is the corrected measurement of faulty sensors. A, B, C, D are the system matrices in the state-space model. K is the Kalman gain, $M_u \mathrm{and} M_y$ are the fault direction matrices [48].

Nowadays, UAV formation (swarm) control practice uses the Real-time Kinematic Global Navigation Satellite System-Inertial Navigation System (RTK-GNSS-INS) [49] for routing and navigation. Moreover, inter-UAV sensors such as Automatic Dependent Surveillance-Broadcast (ADS-B) [50], which provides altitude, aircraft flight ID, and vertical airspeed, have been used in advanced systems. ADS-B reduces the risk of runway collisions, even at night or during heavy rainfall. ADS-B applications being developed now will give pilots indications or alerts of potential collisions. Such systems may be used to dissolve the problem related to faulty sensors.

Here we describe some results of detecting GNSS sensor failure and how to use reconstructed sensor information to control the formation of UAVs. We will illustrate the effectiveness of a fault-tolerant formation control algorithm [30] using the rotary UAV developed by UC Berkeley. Two types of formations are considered: mesh and triangle. For each formation, we performed two types of flight: straight flight and 90-degree turning flight. In each case, we assume a fault occurs to the fifth UAV at 10 s. A GNSS/INS (Inertial Navigation System) combination is usually used in UAV navigation systems. In the presence of GNSS failure, the position information provided by the INS will diverge due to error accumulation. Motivated by this observation, the fault model under consideration simulates slowly divergent position measurements. A fault isolation scheme was designed to capture this GNSS failure. Then an observer was designed and used to estimate the positions of the UAV. Figure 6 shows the results, which clearly demonstrated that the reconstruction can still maintain the UAV formation.

6. Robust and Fault-Tolerant Control

As shown in Figure 7, there can be quite a few actuators for controlling the aircraft. For example, the rudder controls the yaw channel, elevators control the pitch channel, and flaps control the roll channel. From

Table 1. Primary, secondary and tertiary actuators for the roll, pitch, and yaw channels.

	Primary Actuator	Secondary Actuator	Tertiary Actuator
Roll Channel	Aileron	Rudder & Asymmetric Engine Thrust	N/A
Pitch Channel	Tailplane	Symmetric Aileron	Symmetric Engine Thrust
Yaw Channel	Rudder	Asymmetric Engine

, we can see that some actuators can be used as secondary actuators for some channels. This shows that different combinations of actuators can serve as backup actuators for different roll, pitch, and yaw channels. In general, fault tolerant control refers to the selection of substitute actuators when primary actuators fail to function.

A robust controller [19,20,21,22,23] can deal with parametric uncertainties to some extent. However, when some faults such as sensor and actuator faults occur, robust controllers are not strong enough to handle such situations. A fault-tolerant controller [24,25,26,27,28,29,30,31] is extremely useful for guaranteeing closed-loop control performance.

Figure 8 shows a schematic diagram of how fault-tolerant control works. First, a fault diagnosis system that performs on-line fault detection and isolation. Second, a controller suite consists of a primary nominal control system used under normal operating conditions (without faults) and a secondary adaptive fault-tolerant control system engaged only after fault detection. Third, a reconfiguration supervisor makes decisions regarding control system reconfiguration and control reallocation using the fault information provided by the diagnostic module.

Here, we briefly summarize one application of our neural net (NN) controller. Our controller suite includes the nominal controller and the NN-based adaptive fault-tolerant controller. The nominal controller is used when the system is in a fault-free condition. The NN adaptive controller is activated after a fault is detected to compensate for the effect of the fault and to maintain acceptable control performance even in the presence of a fault. The fault-tolerant controller performance was demonstrated by using a well-known RCAM (Research Civil Aircraft Model) developed by the Group for Aeronautical Research and technology in Europe (GARTEUR). The design of the nominal controller is based on standard approximate dynamic inversion. Figure 9 illustrates the proposed nonlinear adaptive control architecture: the aircraft (f), the command filter to provide desired handling qualities, approximate dynamic inverse (${\widehat{f}}^{-1}$), a conventional linear tracking controller as described above, and an online learning neural network to correct for errors and uncertainty in association with the inversion model.

One of the key advantages of the proposed fault-tolerant control scheme is its capability to handle any occurrences of new or unanticipated faults. The neural network-based adaptive controller activated after fault detection is still capable of compensating for the effect of the fault on-line and to maintain acceptable control performance before further pilot intervention.

Figure 10 and Figure 11 give the control performances of the nominal controller and nonlinear adaptive controller engaged after a fault detection, respectively. We can clearly see the benefit of controller reconfiguration using on-line diagnostic information.

In recent years, there has been some new progress in fault-tolerant control. In [51], a hybrid fault-tolerant controller was proposed to handle control surface damages. Compared to the nonlinear control approach in [18], the controller in [51] was based on linearized models. In [52], a nonlinear sliding mode controller was proposed to deal with actuator failure in quadrotor UAVs. The inner loop was the attitude control and the outer loop was the position control. Sliding mode control was applied to both inner and outer loops. The control is challenging because if one or two control actuators fail, the system is under-actuated. A special form of sliding mode control was proposed based on back-stepping. In [53], an observed-based fault-tolerant controller was proposed for carrier-based UAVs. Some states in the UAVs are assumed to be unmeasurable and hence the control problem is challenging. The controller was nonlinear and closed-loop stability was given.

7. Contingency Planning

This is the last defense for fault mitigation in UAVs. The UAV has lost its engine and it is on its way to crash. Emergency landing via the parachute systems is part of the contingency planning for some small-to-medium sized UAVs [54]. However, for big and heavy drones like Global Hawk (12 tons) [55], parachute is not feasible. Can we still do something to minimize the damage? In other words, for fixed wing UAVs, the hanging time of some UAVs such as Global Hawk can still be 30 min or more due to its high flying altitude and large wing span. Moreover, if one plans ahead, the UAVs can still glide to some safe landing places such as airports, non-populated places such as beaches, waterways, grassy areas, etc. A well-known example is the US Airways Flight 1549, which avoided a crash landing by gliding onto the Hudson River.

In our recent papers [56,57,58,59,60], we have provided detailed procedures for contingency planning for engine failures.

Figure 12 shows the workflow of contingency plan generation. It is an off-line process. Given a UAV and its associated flying capabilities (wingspan, gliding speed, descending rate, etc.) and also the theater of operations, we need the following two major steps: preprocessing and contingency plan generation. We will summarize those two steps in the next two sub-sections.

7.1. Preprocessing

There are multiple modules in the preprocessing step. First, landing place selection is needed. Based on the UAV’s size and gliding speed, an appropriate landing site needs to have enough length. Potential landing sites include airport runways, beaches, waterways, etc. In [58], we developed a landing site selection algorithms based on Google maps. Figure 13 shows a landing site.

Second, for each landing site, we need to assign some waypoints such as Touchdown point (TDPT), Initial Approach Fix (IAF), and Final Approach Fix (FAF). Detailed procedures are provided in [56].

7.2. Contingency Plan Generation

Given a primary flight plan containing hundreds or even thousands of waypoints, we need to generate a contingency plan for each waypoint. An example of a contingency plan is shown in Figure 14.

Suppose an engine failure occurs near a waypoint in the primary flight path, we will apply A* path planning algorithm [56] to generate a contingency path between the contingency point (CP) and the IAF. Some no-fly zones need to be bypassed. From Figure 14, the red line section shows the A* generated plan. In the contingency plan, when the UAV reaches the IAF, there may still be excessive altitude to lose. We have developed time-constrained path generation algorithm to lose excessive altitude [57]. Wind speed needs to be taken into account. Some extremal paths can be seen in Figure 15. RSR means right-straight-right; RSL means right-straight-left.

There are also some additional processing steps to deal with different excessive altitudes. Details can be found in [56].

One limitation of our contingency planning approach is that if the wind conditions deviate too much from the forecast conditions, which can happen in practice, then the pre-planned contingency paths may need to change on the fly. More research is needed in this direction.

7.3. Emergency Landing in Hudson River

Here, we demonstrate how we generate a contingency plan for US Airways Flight 1549, which lost both engines due to a bird strike. The plane was on route from New York City’s LaGuardia Airport to Seattle, Washington and ditched into Hudson River on 15 January, 2009. In the climb phase right after its takeoff, it struck a flock of Canada geese and lost its engine power. The pilots Mr. Sullenberger and Mr. Skiles glided the plane to a ditching in the Hudson River. At the time of the bird strike, Flight 1549’s airspeed was about 200 knots. The highest altitude right before the plane started sinking was 3034 feet (925 m). At this altitude, the plane was located at coordinates: Latitude: 40.861666 degrees, Longitude: −73.879722 degrees. Time was 3:27:29 p.m. The wind amplitude was around 13.4 knots. The wind direction was 320 degrees. Assuming the coordinates at the highest altitude corresponds to the coordinates of CP, we found the heading angle at CP using the waypoint coordinates at the highest altitude, the waypoint right before that, and the wind information.

We applied our contingency plan generation tool to this incident. Figure 16 and Figure 17 show the top view and 3D view of the generated plan.

In the movie “Sully”, various flight simulator runs also showed that it was possible to return to LaGuardia and Teterboro airports. Why did the pilots choose to land on Hudson river? The reason is that the pilots need to follow some procedures to make sure both engines were lost. During this period of checking, the plane has lost quite some altitude. Consequently, there was not enough altitude to glide it back to the airports. Based on his experience, the pilot, Sullenberger, made some rough calculations and decided to land on Hudson Bay; the decision saved over 150 people onboard.

8. Conclusions

UAVs are gaining popularity. However, the safety of UAVs is not on par with manned aircraft. In this paper, we present safety enhancements of UAVs using signal processing algorithms, which can help condition-based maintenance, structural health monitoring, sensor and actuator fault diagnostics, fault magnitude reconstruction, fault-tolerant control, and contingency plan generation. Some recent advances in the aforementioned areas are also highlighted.

It is important to emphasize that UAV safety requires an integrated approach that contains all of the above. More research and development effort is needed to produce an integrated safety system for UAVs. For instance, the contingency plans are generated off-line, and if wind conditions deviate a lot from the forecast data during actual flights, our current system may not be able to handle that scenario. One potential future direction in contingency planning is to deal with highly dynamic windy conditions. Another direction is to devise some online contingency planning strategies to complement the off-line generated contingency plans.