Integrating Actuator Fault-Tolerant Control and Deep-Learning-Based NDVI Estimation for Precision Agriculture with a Hexacopter UAV

Abstract

This paper presents an actuator fault-tolerant control (FTC) strategy for a hexacopter unmanned aerial vehicle (UAV) designed specifically for precision agriculture applications. The proposed approach integrates advanced sensing techniques, including the estimation of Near-Infrared (NIR) reflectance from RGB imagery using the Pix2Pix deep learning network based on conditional Generative Adversarial Networks (cGANs), to enable the calculation of the Normalized Difference Vegetation Index (NDVI) for health assessment. Additionally, trajectory flight planning is developed to ensure the efficient coverage of the targeted agricultural area while considering the vehicle’s dynamics and fault-tolerant capabilities, even in the case of total actuator failures. The effectiveness of the proposed system is validated through simulations and real-world experiments, demonstrating its potential for reliable and accurate data collection in precision agriculture. An NDVI test was conducted on a sugarcane crop using the estimated NIR to assess the crop’s condition during its tillering stage. Therefore, the main contributions this paper include (i) the development of an actuator FTC strategy for a hexacopter UAV in precision agriculture applications, integrating advanced sensing techniques such as NIR reflectance estimation using deep learning network; (ii) the design of a flight trajectory planning method ensuring the efficient coverage of the targeted agricultural area, considering the vehicle’s dynamics and fault-tolerant capabilities; (iii) the validation of the proposed system through simulations and real-world experiments; and (iv) the successful integration of FTC scheme, advanced sensing, and flight trajectory planning for reliable and accurate data collection in precision agriculture.

1. Introduction

Unmanned aerial vehicles (UAVs) have received substantial attention lately because of their diverse capabilities and potential applications in various fields, including aerial photography, surveillance, oil spill detection, and precision agriculture, among others [1]. Hexacopter UAVs, in particular, have emerged as a popular choice for their stability, payload capacity, and maneuverability [2]. However, the reliability and safety of these vehicles are critical concerns, especially in precision agriculture applications, where accurate data collection and autonomous operation are essential [3]. With the growing applications and extensive integration of automation technology, hexacopter UAVs are becoming more vulnerable to faults. These faults can significantly affect vehicle dynamics, compromising stability, reliability, and safety throughout the flight envelope. Implementing fault-tolerant control strategies and advanced sensing techniques has become increasingly vital for overcoming these challenges, especially in managing actuator failures in hexacopter UAVs [4]. Fault-tolerant control (FTC) methods can be employed to identify malfunctions in real-time and enhance the reliability and safety of UAVs. These FTC techniques are classified into two types: passive and active [5]. Active techniques involve adapting or reconfiguring controller parameters based on Fault Diagnosis (FD) system data, ensuring the system remains stable and performs acceptably.

Fault-tolerant control aims to maintain the stability and performance of a system in the presence of faults. In the context of multicopter UAVs, several studies have investigated FTC approaches to ensure vehicle reliability. For instance, Reference [6] proposed two FTC designs for a hexacopter subject to actuator Loss of Effectiveness (LoE) based on gain-scheduling control. The results of the experiment performed on the UAV show the robustness of the methods. Similarly, Reference [7] developed an active FTC by redistributing the control signals among the healthy actuators with a reconfigurable multiplexing and pseudo-inverse control allocation applied experimentally on a coaxial octocopter UAV. Reference [8] presented a fault-tolerant control framework for a quadcopter in the case of a partial motor fault based on a three-loop nonlinear controller along with a Fault Detection and Isolation (FDI) algorithm. The simulation results demonstrate that the proposed controller has satisfactory performance with actuator damages up to 50%.

Recent advancements in FD techniques have further enhanced the capabilities of fault-tolerant control systems for multicopter UAVs. Reference [9] proposed a novel fault diagnosis method for multicopters based on a machine-learning-based probabilistic framework, utilizing in-flight, out-of-plane strain measurements at each vehicle boom to detect, identify, and quantify rotor faults. Reference [10] developed a robust actuator fault diagnosis algorithm for hexacopter UAVs based on an adaptive exogenous Kalman filter (AXKF) tested in simulation. The results show that the proposed scheme can accurately estimate the actuator fault’s magnitude. These advancements in FD techniques have paved the way for more effective and reliable fault-tolerant control strategies.

Total actuator failures in hexacopter UAVs pose a significant challenge to the vehicle’s stability and controllability. In such scenarios, the UAV loses one or more of its six rotors, leading to a severe degradation in performance and an increased risk of catastrophic failure [11]. To address this issue, researchers have developed specialized fault-tolerant control strategies that can accommodate total actuator failures. For example, Reference [12] proposed a novel FTC scheme that switches from a nominal controller to a fault-tolerant controller, based on a sliding mode controller with control allocation, for a hexacopter UAV. The effectiveness and robustness of the FTC controller are demonstrated in simulation considering an actuator failure. Similarly, Reference [13] developed a fault-tolerant strategy by converting the hexacopter structure into reconfigurable ones by adding a minimum number of servomotors to deal with failures and significantly improve maneuverability under these conditions. An experimental demonstration is shown for the proposed strategy, which will perform well in the case of complete rotor failure. Recently, Reference [14] presented a learning-based tracking controller based on the Gaussian processes for an FTC to perform a recovery maneuver in a hexacopter. The controller is designed to learn and compensate for the modeling uncertainties after a failure with a control allocation reconfiguration. Experimental tests were carried out to evaluate the proposed algorithm. It was observed that the algorithm allows an improvement in the performance of the UAV system.

Precision agriculture relies on accurate data collection and analysis to optimize crop management practices. UAVs equipped with specialized sensors, such as multispectral cameras, have proven to be valuable tools for gathering high-resolution data on crop health and vigor [15]. The Normalized Difference Vegetation Index (NDVI) is a widely used metric for assessing vegetation health, calculated from the reflectance values in the near-infrared (NIR) and red bands [16]. However, acquiring NIR data typically requires specialized sensors, which can be costly and complex to integrate. To overcome this limitation, recent studies have explored the estimation of NIR values from standard RGB imagery. Reference [17] proposed a method for estimating NIR reflectance from RGB images using a radiometric calibration approach, enabling the calculation of NDVI from consumer-grade cameras. Reference [18] developed a deep learning-based approach for estimating NIR reflectance from RGB imagery, demonstrating its effectiveness in generating accurate NDVI maps for precision agriculture applications.

Trajectory planning is another critical aspect of UAV-based precision agriculture, as it directly impacts the efficiency and coverage of data collection. Several studies have focused on developing optimal trajectory-planning algorithms for agricultural UAVs. Reference [19] proposed a genetic-algorithm-based approach for generating optimal trajectories for UAVs in precision agriculture, considering factors such as energy consumption, coverage, and obstacle avoidance. Similarly, Reference [20] proposed a single framework for efficient visual data acquisition using UAVs that combines perception, environment representation, and route planning. The experimental results show the effectiveness of the proposed route planning method. Recently, Reference [21] provides a literature review of numerous strategies and approaches to addressing path-planning problems for multirotor UAVs in the context of precision agriculture. Also, this paper suggested that a visibility-graph-based method using an A* algorithm to address the path length, time efficiency, and energy efficiency challenges has the best performance for agricultural environments. In this context, Reference [22] proposes a coverage path planning method for a spraying drone that efficiently restricts the flight region of interest, avoiding potential collisions. The algorithm has been tested in simulation, showing that it is effective and covers more areas than other approaches. In addition to trajectory planning, developing advanced control strategies has been a critical focus in precision agriculture applications using multicopter UAVs. Reference [23] proposed a path-tracking controller for a class of agricultural multirotors with variable payload under model uncertainties and exogenous disturbances. A neural-network-based adaptive control is used to offset model uncertainties, while a sliding-mode control approach is adopted to ensure the tracking errors are converged. An adaptive technique is employed to compensate for payload changes.

Despite the significant advancements in fault-tolerant control and sensing techniques for multicopter UAVs in precision agriculture, several challenges and limitations still need to be addressed. These include the limited flight endurance of UAVs, the need for robust and efficient data-processing algorithms, and the integration of heterogeneous data sources for comprehensive crop health assessment [24]. Future research should focus on developing advanced energy management strategies, intelligent data fusion techniques, and standardized protocols for UAV-based precision agriculture systems [25]. Additionally, creating more sophisticated fault-tolerant control strategies that can handle multiple simultaneous actuator failures and adapt to changing environmental conditions will ensure the reliable operation of hexacopter UAVs in precision agriculture applications [6].

The main contributions of this paper are as follows.

1.

The development of an actuator fault-tolerant control strategy specifically designed for a hexacopter UAV in precision agriculture applications. The proposed approach integrates advanced sensing techniques, such as the estimation of NIR reflectance from RGB imagery using the Pix2Pix deep learning network based on conditional Generative Adversarial Networks (cGANs) to enable the calculation of the NDVI for crop health assessment. In this work, the NDVI is presented for a sugarcane crop using the estimated NIR to assess the crop’s condition during its tillering stage.

2.

The design of a trajectory planning that ensures efficient coverage of the targeted agricultural area while considering the vehicle’s dynamics and fault-tolerant capabilities, even in the presence of total actuator failures.

3.

The validation of the proposed FTC system through extensive simulations using MATLAB. The effectiveness of the system is demonstrated in a simulation by considering agriculture flight planning.

4.

The successful integration of advanced sensing techniques, FTC strategies, and trajectory planning to create a comprehensive solution for reliable and accurate data collection in precision agriculture applications using hexacopter UAVs, even in the presence of actuator failures.

These contributions address the challenges of ensuring the reliability and safety of hexacopter UAVs in precision agriculture applications. Accurate data collection and autonomous operation are essential, particularly in the context of actuator failures that can significantly impact the vehicle’s stability and performance.

The rest of the paper is organized as follows: Section 2 details the hexacopter UAV dynamics model and the proposed fault-tolerant control system, which includes the Fault Reconfiguration (FR) scheme, the Available Control Authority Index (ACAI) method for controllability analysis, and the attitude, altitude, and translational controllers. A simulation of the FTC system with agriculture flight planning using MATLAB version 2023b is presented. Also, Section 2 describes the data collection and flight planning process, focusing on the camera model, projection area calculation, and the estimation of agricultural NIR images from RGB data using the Pix2Pix deep learning network. Section 3 presents the simulation and experimental results, demonstrating the effectiveness of the proposed system in two scenarios: experimental flight planning and data collection for the experimental estimation of agricultural NIR band using the digital camera. Finally, Section 4 provides some conclusions and final comments.

2. Materials and Methods

2.1. Aircraft Description

This section provides a detailed description of the UAV platform and sensors used for precision agriculture applications. The hexacopter UAV platform is selected as an aerial imaging system for precision agriculture because it has a low–medium price range. This UAV platform further presents some advantages, compared with other types of aerial platforms, like low altitude flight capability, better stability, stable hover flight capability, high payload capacity, and the ability for vertical landing and take-off. A swarm of UAVs can be used for better control, pre-programmed flight plans, and better accessibility, among others [26]. The main components of the hexacopter UAV platform used in this work are shown in Figure 1, whose specifications are given in

Table 1. Components of the hexacopter UAV platform.

Component	Specification	Value
Motor	Turnigy multistar	600 kv (×6)
Propeller	Carbon fiber	13 × 4 in (×6)
ESC controller	Afro ESC	30 A (×6)
Power bank	Portable charger	10 Ah
Battery	Li-Po	10 Ah 3 s 25 c
Radio control	Turnigy 9X	2.4 GHz
Telemetry	3DR	915 MHz
Frame	Tarot 680 pro	–
Flight controller	Pixhawk 2.4.8	–
GPS	Ublox Neo-M8N	–
RGB camera	SJ9000	–
Multispectral camera	Parrot Sequoia	–

.

The hexacopter is equipped with a flight controller, a Global Position System (GPS), six brushless motors, drivers and propellers, Litio (Li-Po) polymer batteries, a radio control, a telemetry system, and two cameras for image acquisition. The flight controller used is the autopilot Pixhawk, which features an STMicroelectronics STM32F427 Cortex-M4F 32-bit main controller with a 180 MHz operation frequency, 256 KB SRAM, 2 MB flash memory, and a 32-bit failsafe co-processor. The embedded sensors on the motherboard include an MPU-6000 as the main accelerometer and gyroscope sensor, a 3-axis STMicroelectronics L3GD20H 16-bit gyroscope sensor, an STMicroelectronics LSM303D 14-bit accelerometer and magnetometer sensor, and a MEAS MS561 barometer sensor. An external Ublox Neo-M8N GPS receiver sensor is connected to the autopilot Pixhawk controller. The telemetry system allows for communication between the UAV and the ground control system, facilitating the transmission of in-flight data and control updates.

Figure 2 shows the scheme of the hexacopter UAV platform, the in-flight image acquisition, and the post-flight image processing blocks. The in-flight image acquisition is carried out when the vehicle flies over the field to be analyzed. It comprises two cameras that take images when a signal is received from the flight controller. Post-flight image processing is achieved by processing flight data. It consists of applying image processing algorithms to obtain the vegetation index and the orthomosaic to analyze the field.

2.2. Hexacopter UAV Dynamics Model

This section introduces the nonlinear dynamics model of the hexacopter aerial vehicle to design an effective fault-tolerant control strategy. Figure 3 illustrates the hexacopter scheme, showing the main forces acting on the vehicle, with the corresponding parameters listed in

Table 2. Parameters of the the hexacopter UAV.

Parameter	Value	Unit
Mass of the vehicle, $m_s$	$2.95$	Kg
Acceleration due to gravity, g	$9.81$	m/s2
Drag force coefficient, $d_f$	$0.3$	Ns
Drag torque coefficient, $d_t$	$0.7$	Nms
Moment of inertia about x, $J_x$	$0.064$	Kgm2
Moment of inertia about y, $J_y$	$0.061$	Kgm2
Moment of inertia about z, $J_z$	$0.312$	Kgm2
Ratio between torque and lift, $k_u$	$0.2$	−
Motor’s maximum thrust force, $f_{max}$	$10.12$	N
Motor’s constant, $w_m$	39	–

. The following assumptions are made to derive the dynamic model: the quadcopter structure is rigid and symmetrical, the center of mass coincides with the origin of the body frame $O_B$, and the propellers are rigid.

The hexacopter consists of six independent motors, each with a propeller that generates torques and thrusts along the propellers’ axis of rotation. Propellers $1,3$ and 5 rotate clockwise, while propellers $2,4$ and 6 rotate counterclockwise, as shown in Figure 3. In this figure, $\mathcal{I}=\{O_I,p_{x_I},p_{y_I},p_{z_I}$} represents an inertial frame, and $\mathcal{B}=\{O_B,p_{x_B},p_{y_B},p_{z_B}$} represents a rigid frame attached to the vehicle’s center of mass.

The dynamics of the hexacopter can be described as follows:

$$\begin{array}{cc}\hfill \dot{\xi }\left(t\right)& =v\left(t\right),m_s\dot{v}\left(t\right)=R_s\left(t\right)F\left(t\right),\hfill \\ \hfill {\dot{R}}_s\left(t\right)& =R_s\left(t\right)\widehat{\Omega }\left(t\right),J\dot{\Omega }\left(t\right)=-\Omega \left(t\right)\times J\Omega \left(t\right)+\tau \left(t\right),\hfill \end{array}$$

(1)

where t denotes time, $\xi \left(t\right)={[p_x\left(t\right),p_y\left(t\right),p_z\left(t\right)]}^{\top }\in {\mathbb{R}}^3$ represents the position vector, and $v\left(t\right)={[v_x\left(t\right),v_y\left(t\right),v_z\left(t\right)]}^{\top }\in {\mathbb{R}}^3$ represents the velocity vector of the vehicle in the inertial frame $\mathcal{I}$. The angular velocity in the body-fixed frame $\mathcal{B}$ is given by $\Omega \left(t\right)={[p\left(t\right),q\left(t\right),r\left(t\right)]}^{\top }\in {\mathbb{R}}^3$, and $m_s$ denotes the vehicle’s total mass. The Euler angles are represented by roll $\varphi \left(t\right)$, pitch $\theta \left(t\right)$, and yaw $\psi \left(t\right)$. The moment of inertia matrix is $J=\mathsf{diag}(J_x,J_y,J_z)\in {\mathbb{R}}^{3\times 3}$ in frame $\mathcal{B}$, with $\tau \left(t\right)$ representing the moments in $\mathcal{B}$. The matrix $\widehat{\Omega }\left(t\right)$ is the skew-symmetric of $\Omega \left(t\right)$, $R_s\left(t\right)$ is the rotation matrix from $\mathcal{B}$ to $\mathcal{I}$, and $F\left(t\right)$ denotes the forces generated by the motors. Based on the given assumptions and using the Newton–Euler formalism, (1) can be formulated as follows:

$$\begin{array}{cc}\hfill {\dot{p}}_x\left(t\right)=& v_x\left(t\right),\hfill \end{array}$$

(2a)

$$\begin{array}{cc}\hfill {\dot{p}}_y\left(t\right)=& v_y\left(t\right),\hfill \end{array}$$

(2b)

$$\begin{array}{cc}\hfill {\dot{p}}_z\left(t\right)=& v_z\left(t\right),\hfill \end{array}$$

(2c)

$$\begin{array}{cc}\hfill {\dot{v}}_x\left(t\right)=& \left(\mathit{c}\psi \left(t\right)\mathit{s}\theta \left(t\right)\mathit{c}\varphi \left(t\right)+\mathit{s}\psi \left(t\right)\mathit{s}\varphi \left(t\right)\right){\displaystyle \frac{1}{m_s}}{u}_z\left(t\right)-{\displaystyle \frac{d_f}{m_s}}{v}_x\left(t\right),\hfill \end{array}$$

(2d)

$$\begin{array}{cc}\hfill {\dot{v}}_y\left(t\right)=& \left(\mathit{s}\psi \left(t\right)\mathit{s}\theta \left(t\right)\mathit{c}\varphi \left(t\right)-\mathit{c}\psi \left(t\right)\mathit{s}\varphi \left(t\right)\right){\displaystyle \frac{1}{m_s}}{u}_z\left(t\right)-{\displaystyle \frac{d_f}{m_s}}{v}_y\left(t\right),\hfill \end{array}$$

(2e)

$$\begin{array}{cc}\hfill {\dot{v}}_z\left(t\right)=& -g+\left(\mathit{c}\theta \left(t\right)\mathit{c}\varphi \left(t\right)\right){\displaystyle \frac{1}{m_s}}{u}_z\left(t\right)-{\displaystyle \frac{d_f}{m_s}}{v}_z\left(t\right),\hfill \end{array}$$

(2f)

$$\begin{array}{cc}\hfill \dot{\varphi }\left(t\right)=& p\left(t\right)+\left(\mathit{s}\varphi \left(t\right)\mathit{t}\theta \left(t\right)\right)q\left(t\right)+\left(\mathit{c}\varphi \left(t\right)\mathit{t}\theta \left(t\right)\right)r\left(t\right),\hfill \end{array}$$

(2g)

$$\begin{array}{cc}\hfill \dot{\theta }\left(t\right)=& \mathit{c}\varphi \left(t\right)q\left(t\right)-\mathit{s}\varphi \left(t\right)r\left(t\right),\hfill \end{array}$$

(2h)

$$\begin{array}{cc}\hfill \dot{\psi }\left(t\right)=& \left(\mathit{s}\varphi \left(t\right)q\left(t\right)+\mathit{c}\varphi \left(t\right)r\left(t\right)\right){\displaystyle \frac{1}{\mathit{c}\theta \left(t\right)}},\hfill \end{array}$$

(2i)

$$\begin{array}{cc}\hfill \dot{p}\left(t\right)=& \left(q\left(t\right)r\left(t\right)(J_y-J_z)+u_{\varphi }\left(t\right)\right){\displaystyle \frac{1}{J_x}}-{\displaystyle \frac{d_t}{J_x}}p\left(t\right),\hfill \end{array}$$

(2j)

$$\begin{array}{cc}\hfill \dot{q}\left(t\right)=& \left(p\left(t\right)r\left(t\right)(J_z-J_x)+u_{\theta }\left(t\right)\right){\displaystyle \frac{1}{J_y}}-{\displaystyle \frac{d_t}{J_y}}q\left(t\right),\hfill \end{array}$$

(2k)

$$\begin{array}{cc}\hfill \dot{r}\left(t\right)=& \left(p\left(t\right)q\left(t\right)(J_x-J_y)+u_{\psi }\left(t\right)\right){\displaystyle \frac{1}{J_z}}-{\displaystyle \frac{d_t}{J_z}}r\left(t\right),\hfill \end{array}$$

(2l)

where $\mathit{s},\mathit{c}$ and $\mathit{t}$ correspond to the trigonometric functions $\mathit{sine},\mathit{cosine}$ and $\mathit{tangent}$, respectively. The state vector is given by $x\left(t\right)=[p_x\left(t\right),p_y\left(t\right),p_z\left(t\right),v_x\left(t\right),v_y\left(t\right),v_z\left(t\right),\varphi \left(t\right),\theta \left(t\right),\psi \left(t\right),$${p\left(t\right),q\left(t\right),r\left(t\right)]}^{\top }$ while the system inputs for translational and attitude dynamics are denoted by $u\left(t\right)={[u_z\left(t\right),u_{\varphi }\left(t\right),u_{\theta }\left(t\right),u_{\psi }\left(t\right)]}^{\top }$. The actual inputs acting on the system are the upward-lifting forces generated by each propeller, represented as $\overline{f}\left(t\right)={[f_1\left(t\right),f_2\left(t\right),\dots ,f_6\left(t\right)]}^{\top }$, where each lifting force is saturated as $f_i\in [0,f_{max}],$ with $i=1,\dots ,6$ (see Figure 3). According to the geometry of the hexacopter presented in Figure 3, the mapping from the rotor lift $f_i$ to the system total thrust and torque $u\left(t\right)$, also called the actuator mixer, is defined by:

$$\begin{array}{c}\hfill \left[\begin{array}{c}{u}_z\left(t\right)\\ u_{\varphi }\left(t\right)\\ u_{\theta }\left(t\right)\\ u_{\psi }\left(t\right)\end{array}\right]=\left[\begin{array}{cccccc}{\eta }_1& {\eta }_2& {\eta }_3& {\eta }_4& {\eta }_5& {\eta }_6\\ 0& -{\displaystyle \frac{\sqrt{3}}{2}}{\eta }_{2}l& -{\displaystyle \frac{\sqrt{3}}{2}}{\eta }_{3}l& 0& {\displaystyle \frac{\sqrt{3}}{2}}{\eta }_{5}l& {\displaystyle \frac{\sqrt{3}}{2}}{\eta }_{6}l\\ {\eta }_{1}l& {\displaystyle \frac{1}{2}}{\eta }_{2}l& -{\displaystyle \frac{1}{2}}{\eta }_{3}l& -{\eta }_{4}l& -{\displaystyle \frac{1}{2}}{\eta }_{5}l& {\displaystyle \frac{1}{2}}{\eta }_{6}l\\ -{\eta }_1{k}_u& {\eta }_2{k}_u& -{\eta }_3{k}_u& {\eta }_4{k}_u& -{\eta }_5{k}_u& {\eta }_6{k}_u\end{array}\right]\left[\begin{array}{c}{f}_1\left(t\right)\\ f_2\left(t\right)\\ f_3\left(t\right)\\ f_4\left(t\right)\\ f_5\left(t\right)\\ f_6\left(t\right)\end{array}\right],\end{array}$$

(3)

where $\overline{\eta }=[{\eta }_1,{\eta }_2,\dots {\eta }_6]$, with ${\eta }_i\in [0,1]$ as the effectiveness of the i-th actuator. If ${\eta }_i=1$, then a total fault is presented in the i-th actuator, ${\eta }_i=0$ indicates that the i-th actuator is healthy, and ${\eta }_i=[0,1]$ indicates an LoE in the i-th actuator. The ratio between the reactive torque and the lift of the rotors is defined as $k_u$, and l is the distance from the center of the rotor to the center of mass of the vehicle. It is considered that the force generated by each motor has the following first-order transfer function relation:

$$\begin{array}{c}\hfill f_{m_i}\left(t\right)={\displaystyle \frac{w_m}{s+w_m}}{f}_i\left(t\right).\end{array}$$

(4)

where $f_{m_i}\left(t\right)$ are the true forces to be replaced in (3).

2.3. Fault-Tolerant Control System

The rapid dynamics of hexacopter UAV systems pose a challenge for promptly detecting faults and reconfiguring the system before a crash occurs. Our interest is in validating in a simulation the feasibility of the proposed FTC algorithm for hexacopter UAV systems fulfilling a trajectory for precision agriculture. In order to achieve this goal, a Fault Reconfiguration (FR) scheme is proposed. Figure 4 presents the overall fault-tolerant control architecture for the hexacopter UAV system. The Fault Reconfiguration is applied exclusively to the vehicle experiencing actuator failure. This scheme consists of three main components: (i) the hexacopter UAV system, (ii) the nominal controllers, and (iii) the actuator fault diagnosis, which includes fault detection, isolation, estimation, and the reconfiguration system. To simplify the validation procedure, we have chosen a pole placement feedback controller with an integrator comparator as the nominal position controller, altitude, and attitude nominal controllers. These nominal controllers are used to show the effectiveness of the proposed fault-tolerant control scheme when an actuator of the hexacopter fails. The analysis of nominal controllers and actuator fault diagnosis is not within the scope of this work; additional information can be found in [27,28].

Only the nominal controller is used when the vehicle is operating without faults. When an actuator fault occurs, the fault diagnosis system detects, isolates, and estimates the fault. Typically, a fault is detected when the fault estimation signal exceeds a predefined threshold value. The sign value of the fault estimation signal is utilized to isolate the actuator fault, as detailed in [27]. Finally, the proposed FTC law is computed with a control reconfiguration system that changes the nominal controller to a new controller. Actuator faults may result from damage to the propeller or the motor itself, both of which can reduce motor effectiveness and lead to a loss of controllability in the hexacopter. To assess the controllability of the vehicle under a fault condition, the Available Control Authority Index (ACAI) method, as proposed by [29], is utilized.

To describe the proposed fault-tolerant control system, let us suppose a fault in the actuator 2. The results of the ACAI controllability method by using the parameters shown Table 2 are presented in

Table 3. Hexacopter UAV controllability analysis.

	$\overline{\eta }=[1,{\eta }_2,1,1,1,1]$
Fault ${\eta }_2$	1	$0.9$	$0.8$	$0.7$	$0.6$	$0.5$	$0.4$	$0.3$	$0.2$	$0.1$	0
ACAI *	$2.26$	$2.03$	$1.81$	$1.58$	$1.35$	$1.13$	$0.90$	$0.67$	$0.42$	$0.17$	0
CY *	CN *	CN	CN	CN	CN	CN	CN	CN	CN	CN	UCN *
ACAI-WY *	–	–	–	–	–	–	–	–	–	–	$2.49$

* ACAI: Available Control Authority Index, CY: Controllability, ACAI-WY: ACAI without yaw, CN: Controllable, UCN: Uncontrollable.

. According to this table, if ann LoE is presented, i.e., ${\eta }_2\in [0.1,0.9]$, then the hexacopter UAV is controllable with ACAI > 0. In this case, a nominal controller is enough to stabilize the vehicle. Nevertheless, if a total fault occurs, ${\eta }_2$ = 0, the system is uncontrollable with ACAI ≤0. Table 3 also indicates that losing the controllability of yaw dynamics (ACAI without yaw) can be a viable solution to safely recover a hexacopter UAV experiencing an actuator failure, as ACAI-WY $>0$. This approach involves relinquishing control of the yaw angle and utilizing the remaining forces to maintain a constant horizontal spin.

By using $\overline{\eta }=[1,0,1,1,1,1]$ and $f_2\left(t\right)=0$ in (3), $u_{\varphi }\left(t\right)$ can be expressed considering the actuator fault as

$$u_{{\varphi }_f}\left(t\right)=-{\displaystyle \frac{\sqrt{3}}{2}}l{f}_3\left(t\right)+{\displaystyle \frac{\sqrt{3}}{2}}l{f}_5\left(t\right)+{\displaystyle \frac{\sqrt{3}}{2}}l{f}_6\left(t\right).$$

(5)

Now, a reduced control input vector is introduced as $\overline{u}\left(t\right)=[u_{z_f}\left(t\right),u_{{\theta }_f}\left(t\right)$, $u_{{\psi }_f}{\left(t\right)]}^{\top }$, where (3) is rewritten as

$$\begin{array}{c}\hfill \left[\begin{array}{c}{u}_{z_f}\left(t\right)\\ u_{{\theta }_f}\left(t\right)\\ u_{{\psi }_f}\left(t\right)\end{array}\right]=\left[\begin{array}{ccccc}{\eta }_1& {\eta }_3& {\eta }_4& {\eta }_5& {\eta }_6\\ {\eta }_{1}l& -{\displaystyle \frac{1}{2}}{\eta }_{3}l& -{\eta }_{4}l& -{\displaystyle \frac{1}{2}}{\eta }_{5}l& {\displaystyle \frac{1}{2}}{\eta }_{6}l\\ -{\eta }_1{k}_u& -{\eta }_3{k}_u& {\eta }_4{k}_u& -{\eta }_5{k}_u& {\eta }_6{k}_u\end{array}\right]\left[\begin{array}{c}{f}_1\left(t\right)\\ f_3\left(t\right)\\ f_4\left(t\right)\\ f_5\left(t\right)\\ f_6\left(t\right)\end{array}\right].\end{array}$$

(6)

This implies that

$$\begin{array}{cc}\hfill f_1\left(t\right)& ={\displaystyle \frac{3}{14}}{u}_{z_f}\left(t\right)+{\displaystyle \frac{8}{21l}}{u}_{{\theta }_f}\left(t\right)-{\displaystyle \frac{5}{42k_u}}{u}_{\psi }\left(t\right),\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill f_3\left(t\right)& ={\displaystyle \frac{1}{7}}{u}_{z_f}\left(t\right)-{\displaystyle \frac{4}{21l}}{u}_{{\theta }_f}\left(t\right)-{\displaystyle \frac{4}{21k_u}}{u}_{{\psi }_f}\left(t\right),\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill f_4\left(t\right)& ={\displaystyle \frac{3}{14}}{u}_{z_f}\left(t\right)-{\displaystyle \frac{2}{7l}}{u}_{{\theta }_f}\left(t\right)+{\displaystyle \frac{3}{14k_u}}{u}_{{\psi }_f}\left(t\right),\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill f_5\left(t\right)& ={\displaystyle \frac{1}{7}}{u}_{z_f}\left(t\right)-{\displaystyle \frac{4}{21l}}{u}_{{\theta }_f}\left(t\right)-{\displaystyle \frac{4}{21k_u}}{u}_{{\psi }_f}\left(t\right),\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill f_6\left(t\right)& ={\displaystyle \frac{2}{7}}{u}_{z_f}\left(t\right)+{\displaystyle \frac{2}{7l}}{u}_{{\theta }_f}\left(t\right)+{\displaystyle \frac{2}{7k_u}}{u}_{{\psi }_f}\left(t\right).\hfill \end{array}$$

(11)

From (8) and (10) $f_3\left(t\right)=f_5\left(t\right)$, then (5) can be rewritten as follows:

$$\begin{array}{cc}\hfill u_{{\varphi }_f}\left(t\right)& ={\displaystyle \frac{\sqrt{3}}{2}}l{f}_6\left(t\right)\hfill \\ \hfill & ={\displaystyle \frac{\sqrt{3}}{7}}l{u}_{z_f}\left(t\right)+{\displaystyle \frac{\sqrt{3}}{7}}{u}_{{\theta }_f}\left(t\right)+{\displaystyle \frac{\sqrt{3}}{7k_u}}l{u}_{{\psi }_f}\left(t\right),\hfill \end{array}$$

(12)

where (12) is the reduced input $u_{\varphi }\left(t\right)$ for the system (2) when a failure in the actuator 2 is presented. The same procedure could be carried out for any individual actuator failure. With this consideration, the yaw angle $\psi \left(t\right)$ dynamics is ignored following the work presented by [30].

From (2), a new system can be obtained to design the controller for attitude and altitude dynamics by considering the state vector $\overline{x}\left(t\right)=[\varphi \left(t\right),\theta \left(t\right),p\left(t\right),q\left(t\right),r\left(t\right),p_z\left(t\right),$$v_z{\left(t\right)]}^{\top }$ and the reduced control input vector $\overline{u}=[u_{z_f}\left(t\right),u_{{\theta }_f}\left(t\right)$, $u_{{\psi }_f}{\left(t\right)]}^{\top }$ as follows

$$\dot{\overline{x}}\left(t\right)=\mathbf{f}\left(\overline{x}\left(t\right)\right)+\mathbf{h}\left(\overline{x}\left(t\right)\right)\overline{u}\left(t\right),$$

(13)

where

$$\mathbf{f}\left(\overline{x}\left(t\right)\right)=\left[\begin{array}{c}p\left(t\right)+\left(\mathit{s}\varphi \left(t\right)\mathit{t}\theta \left(t\right)\right)q\left(t\right)+\left(\mathit{c}\varphi \left(t\right)\mathit{t}\theta \left(t\right)\right)r\left(t\right)\\ \mathit{c}\varphi \left(t\right)q\left(t\right)-\mathit{s}\varphi \left(t\right)r\left(t\right)\\ \left(q\left(t\right)r\left(t\right)(J_y-J_z)\right){\displaystyle \frac{1}{J_x}}-{\displaystyle \frac{d_t}{J_x}}p\left(t\right)\\ \left(p\left(t\right)r\left(t\right)(J_z-J_x)\right){\displaystyle \frac{1}{J_y}}-{\displaystyle \frac{d_t}{J_y}}q\left(t\right)\\ \left(p\left(t\right)q\left(t\right)(J_x-J_y)\right){\displaystyle \frac{1}{J_z}}-{\displaystyle \frac{d_t}{J_z}}r\left(t\right)\\ v_z\left(t\right)\\ -g-{\displaystyle \frac{d_f}{m_s}}{v}_z\left(t\right)\end{array}\right],\mathbf{h}\left(\overline{x}\left(t\right)\right)=\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ {\displaystyle \frac{\sqrt{3}}{7J_x}}l& {\displaystyle \frac{\sqrt{3}}{7J_x}}& {\displaystyle \frac{\sqrt{3}}{7k_u{J}_x}}l\\ 0& {\displaystyle \frac{1}{J_y}}& 0\\ 0& 0& {\displaystyle \frac{1}{J_z}}\\ 0& 0& 0\\ \left(\mathit{c}\theta \left(t\right)\mathit{c}\varphi \left(t\right)\right){\displaystyle \frac{1}{m_s}}& 0& 0\end{array}\right].$$

Note that in (13), the states $\varphi \left(t\right),\theta \left(t\right)$ and $p_z\left(t\right)$ are independent of $\overline{u}\left(t\right)$. These states can be expressed as

$$\overline{\mathbf{f}}\left(\overline{x}\left(t\right)\right)=\left[\begin{array}{c}\dot{\varphi }\left(t\right)\\ \dot{\theta }\left(t\right)\\ {\dot{p}}_z\left(t\right)\end{array}\right]=\left[\begin{array}{c}p\left(t\right)+\left(\mathit{s}\varphi \left(t\right)\mathit{t}\theta \left(t\right)\right)q\left(t\right)+\left(\mathit{c}\varphi \left(t\right)\mathit{t}\theta \left(t\right)\right)r\left(t\right)\\ \mathit{c}\varphi \left(t\right)q\left(t\right)-\mathit{s}\varphi \left(t\right)r\left(t\right)\\ v_z\left(t\right)\end{array}\right].$$

(14)

The derivative of $\overline{\mathbf{f}}\left(\overline{x}\left(t\right)\right)$ can be calculated as follows:

$$\begin{array}{c}\hfill {\displaystyle \frac{d\overline{\mathbf{f}}\left(\overline{x}\left(t\right)\right)}{dt}}=\left[\begin{array}{c}\ddot{\varphi }\left(t\right)\\ \ddot{\theta }\left(t\right)\\ {\ddot{p}}_z\left(t\right)\end{array}\right]={\displaystyle \frac{\partial \overline{\mathbf{f}}\left(\overline{x}\left(t\right)\right)}{\partial \overline{x}\left(t\right)}}\dot{\overline{x}}\left(t\right)=\overline{\mathbf{J}}\left(\overline{x}\left(t\right)\right)\dot{\overline{x}}\left(t\right),\end{array}$$

(15)

where $\overline{\mathbf{J}}\left(\overline{x}\left(t\right)\right)$ is the Jacobian matrix. Then, by substituting (13) into (15) for $\dot{\overline{x}}\left(t\right)$, the following equation can be obtained:

$$\begin{array}{c}\hfill \left[\begin{array}{c}\ddot{\varphi }\left(t\right)\\ \ddot{\theta }\left(t\right)\\ {\ddot{p}}_z\left(t\right)\end{array}\right]=\overline{\mathbf{J}}\left(\overline{x}\left(t\right)\right)\mathbf{f}\left(\overline{x}\left(t\right)\right)+\overline{\mathbf{J}}\left(\overline{x}\left(t\right)\right)\mathbf{h}\left(\overline{x}\left(t\right)\right)\overline{u}\left(t\right),\end{array}$$

(16)

where

$$\begin{array}{c}\hfill \overline{\mathbf{J}}\left(\overline{x}\left(t\right)\right)={\displaystyle \frac{\partial \overline{\mathbf{f}}\left(\overline{x}\left(t\right)\right)}{\partial \overline{x}\left(t\right)}}={\left[\begin{array}{ccc}q\left(t\right)\mathit{c}\varphi \left(t\right)\mathit{t}\theta \left(t\right)-r\left(t\right)\mathit{s}\varphi \left(t\right)\mathit{t}\theta \left(t\right)& -r\left(t\right)\mathit{c}\varphi \left(t\right)-q\left(t\right)\mathit{s}\varphi \left(t\right)& 0\\ r\left(t\right)\mathit{c}\varphi \left(t\right)\left(\mathit{t}{\theta }^2\left(t\right)+1\right)+q\left(t\right)\mathit{s}\varphi \left(t\right)\left(\mathit{t}{\theta }^2\left(t\right)+1\right)& 0& 0\\ 1& 0& 0\\ \mathit{s}\varphi \left(t\right)\mathit{t}\theta \left(t\right)& \mathit{c}\varphi \left(t\right)& 0\\ \mathit{c}\varphi \left(t\right)\mathit{t}\theta \left(t\right)& -\mathit{s}\varphi \left(t\right)& 0\\ 0& 0& 0\\ 0& 0& 1\end{array}\right]}^{\top }.\end{array}$$

2.3.1. Attitude and Altitude Controller

In order to design a tracking trajectory controller for attitude and altitude dynamics, the following errors can be defined:

$$\begin{array}{c}\hfill \left[\begin{array}{c}{e}_1\left(t\right)\\ e_2\left(t\right)\\ e_3\left(t\right)\\ e_4\left(t\right)\\ e_5\left(t\right)\\ e_6\left(t\right)\end{array}\right]=\left[\begin{array}{c}\varphi \left(t\right)-{\varphi }_d\left(t\right)\\ \dot{\varphi }\left(t\right)-{\dot{\varphi }}_d\left(t\right)+k_1{e}_1\left(t\right)\\ \theta \left(t\right)-{\theta }_d\left(t\right)\\ \dot{\theta }\left(t\right)-{\dot{\theta }}_d\left(t\right)+k_3{e}_3\left(t\right)\\ p_z\left(t\right)-p_{z_d}\left(t\right)\\ {\dot{p}}_z\left(t\right)-{\dot{p}}_{z_d}\left(t\right)+k_5{e}_5\left(t\right),\end{array}\right],\end{array}$$

(17)

where $k_1,k_3$, and $k_5$ are positive constant control gains and the subscript d indicates the desired reference. Therefore, the error dynamics of (17) is as follows:

$$\begin{array}{c}\hfill {\dot{e}}_1\left(t\right)=e_2\left(t\right)-k_1{e}_1\left(t\right),{\dot{e}}_3\left(t\right)=e_4\left(t\right)-k_3{e}_3\left(t\right),{\dot{e}}_5\left(t\right)=e_6\left(t\right)-k_5{e}_5\left(t\right).\end{array}$$

(18)

The tracking trajectory’s objective is to design a controller for the tracking errors (17) that force the altitude and attitude subsystems to the desired reference. Based on (19) and the above control objective, the following control inputs are established:

$$\begin{array}{cc}\hfill \overline{u}\left(t\right)=& -\left(\overline{\mathbf{J}}(\overline{x}\left(t\right)\right)\mathbf{h}\left(\overline{x}\left(t\right)\right){)}^{-1}\overline{\mathbf{J}}\left(\overline{x}\left(t\right)\right)\mathbf{f}\left(\overline{x}\left(t\right)\right)\hfill \\ \hfill & -{\left(\overline{\mathbf{J}}\left(\overline{x}\left(t\right)\right)\mathbf{h}\left(\overline{x}\left(t\right)\right)\right)}^{-1}\left[\begin{array}{c}(k_1^2-1)e_1\left(t\right)-(k_1+k_2)e_2\left(t\right)+{\ddot{\varphi }}_d\left(t\right)\\ (k_3^2-1)e_3\left(t\right)-(k_3+k_4)e_4\left(t\right)+{\ddot{\theta }}_d\left(t\right)\\ (k_5^2-1)e_5\left(t\right)-(k_5+k_6)e_6\left(t\right)+{\ddot{p}}_{z_d}\left(t\right)\end{array}\right],\hfill \end{array}$$

(19)

where $k_2,k_4$ and $k_6$ are positive constants gains. Thus, the following theorem can be formulated to guarantee the stability of the tracking errors (17):

Theorem 1.

The attitude dynamics (2c) and (2f), and the attitude dynamics (2g), (2h), and (2j)–(2l), applying the control input vector (19), converge to the desired reference in the sense that the tracking errors (17) are exponentially stable if the control gains $k_j$, with $j=1,\dots ,6$, are positive.

Proof. 

Let us choose the following candidate quadratic Lyapunov function as

$$\begin{array}{c}\hfill V_1={\displaystyle \frac{1}{2}}\sum _{j=1}^6{e}_j^2\left(t\right).\end{array}$$

(20)

Then, the time derivative of the function (20) is given by

$$\begin{array}{c}\hfill {\dot{V}}_1=\sum _{j=1}^6{e}_j\left(t\right){\dot{e}}_j\left(t\right).\end{array}$$

(21)

By substituting (18) into (21), the following derivative of $V_1$ is deduced:

$$\begin{array}{cc}\hfill {\dot{V}}_1=& e_1\left(t\right)\left(e_2\left(t\right)-k_1{e}_1\left(t\right)\right)+e_2\left(t\right)\left(\ddot{\varphi }\left(t\right)-{\ddot{\varphi }}_d\left(t\right)+k_1(e_2\left(t\right)-k_1{e}_1\left(t\right))\right)\hfill \\ \hfill & +e_3\left(t\right)\left(e_4\left(t\right)-k_3{e}_3\left(t\right)\right)+e_4\left(\ddot{\theta }\left(t\right)-{\ddot{\theta }}_d\left(t\right)+k_3(e_4\left(t\right)-k_3{e}_3\left(t\right))\right)\hfill \\ \hfill & +e_5\left(t\right)\left(e_6\left(t\right)-k_5{e}_5\left(t\right)\right)+e_6\left({\ddot{p}}_z\left(t\right)-{\ddot{p}}_{z_d}\left(t\right)+k_5(e_6\left(t\right)-k_5{e}_5\left(t\right))\right).\hfill \end{array}$$

(22)

Substituting the time derivatives of $\dot{\varphi },\dot{\theta }$ and ${\dot{p}}_z$ from (2f) and (2j)–(2l) into (22) with the control inputs (19), the derivative of the Lyapunov function (22) is rewritten as follows:

$$\begin{array}{c}\hfill {\dot{V}}_1=-\sum _{j=1}^6{k}_j{e}_j^2\left(t\right).\end{array}$$

(23)

Now, by choosing the control gains $k_j>0$, with $j=1,\dots ,6$ in (23), then ${\dot{V}}_1\le 0$, and the altitude and attitude tracking errors (17) are exponentially stable. □

2.3.2. Translational Controller

The translational dynamics (2d) and (2e) can be expressed as

$$\left[\begin{array}{c}{\ddot{p}}_x\left(t\right)\\ {\ddot{p}}_y\left(t\right)\end{array}\right]=R_{\psi }\left[\begin{array}{c}\mathit{s}\varphi \left(t\right)\\ \mathit{s}\theta \left(t\right)\mathit{c}\varphi \left(t\right)\end{array}\right]{\displaystyle \frac{1}{m_s}}{u}_{z_f}\left(t\right)-\left[\begin{array}{c}{\displaystyle \frac{d_f}{m_s}}{v}_x\left(t\right)\\ {\displaystyle \frac{d_f}{m_s}}{v}_y\left(t\right)\end{array}\right],$$

(24)

with

$$R_{\psi }=\left[\begin{array}{cc}\mathit{s}\psi \left(t\right)& \mathit{c}\psi \left(t\right)\\ -\mathit{c}\psi \left(t\right)& \mathit{s}\psi \left(t\right)\end{array}\right],$$

where ${[{\ddot{p}}_x\left(t\right),{\ddot{p}}_y\left(t\right)]}^{\top }={[{\dot{v}}_x\left(t\right),{\dot{v}}_y\left(t\right)]}^{\top }$.

The errors for the translational subsystem (2a), (2b), (2d) and (2e) are defined as

$$\begin{array}{c}\hfill \left[\begin{array}{c}{e}_7\left(t\right)\\ e_8\left(t\right)\\ e_9\left(t\right)\\ e_{10}\left(t\right)\end{array}\right]=\left[\begin{array}{c}{p}_x\left(t\right)-p_{x_d}\left(t\right)\\ {\dot{p}}_x\left(t\right)-{\dot{p}}_{x_d}\left(t\right)+k_7{e}_7\left(t\right)\\ p_y\left(t\right)-p_{y_d}\left(t\right)\\ {\dot{p}}_y\left(t\right)-{\dot{p}}_{y_d}\left(t\right)+k_9{e}_9\left(t\right)\end{array}\right],\end{array}$$

(25)

where $k_7$ and $k_9$ are positive constant gains. The error dynamics of (25) is as follows:

$$\begin{array}{c}\hfill {\dot{e}}_7\left(t\right)=e_8\left(t\right)-k_7{e}_7\left(t\right),{\dot{e}}_9\left(t\right)=e_{10}\left(t\right)-k_9{e}_9\left(t\right),\end{array}$$

(26)

Now, to design the translational controller ensuring that the tracking errors (25) converge to zero, the following Equation (27) is stated by solving for ${[\mathit{s}\varphi \left(t\right),\mathit{s}\theta \left(t\right)\mathit{c}\varphi \left(t\right)]}^{\top }$ in (24) as a desired roll and pitch dynamics and by considering small angles, i.e., $\mathit{s}{\varphi }_d\left(t\right)\approx {\varphi }_d\left(t\right)$ and $\mathit{c}{\varphi }_d\left(t\right)\approx 1$. Then ${\varphi }_d\left(t\right)$ and ${\theta }_d\left(t\right)$ are chosen as

$$\left[\begin{array}{c}{\varphi }_d\left(t\right)\\ {\theta }_d\left(t\right)\end{array}\right]=R_{\psi }^{\top }{\displaystyle \frac{m_s}{u_{z_f}\left(t\right)}}\left(\left[\begin{array}{c}(k_7-1)e_7\left(t\right)-(k_7+k_8)e_8\left(t\right)+{\ddot{p}}_{x_d}\left(t\right)\\ (k_9-1)e_9\left(t\right)-(k_9+k_{10})e_{10}\left(t\right)+{\ddot{p}}_{y_d}\left(t\right)\end{array}\right]+\left[\begin{array}{c}{\displaystyle \frac{d_f}{m_s}}{v}_x\left(t\right)\\ {\displaystyle \frac{d_f}{m_s}}{v}_y\left(t\right)\end{array}\right]\right),$$

(27)

where $k_8$ and $k_{10}$ are positive constant gains. The following theorem is stated to ensure the stability of the translational tracking errors (25).

Theorem 2.

The translational dynamics (2a), (2b), and (24) converge to the desired reference in the sense that the tracking errors (25) are exponentially stable if the control gains $k_j$, with $j=7,\dots ,9$, are positive.

Proof. 

The proof follows the same steps as in the proof of Theorem 1 and is not given here for compactness of the presentation. □

2.4. Validation of the Fault-Tolerant Control Scheme

In order to show the feasibility of the proposed fault-tolerant control algorithm in the hexacopter UAV suffering an actuator failure, an agriculture flight planning is developed. In this validation, an actuator failure in motor 2 is introduced at time 1035 s while the hexacopter is flying along a predefined path at a non-hover position. Before the failure, the vehicle operates with the nominal controllers. Upon detecting and isolating the fault, the nominal controllers are replaced with the proposed position, altitude, and attitude controllers, utilizing information from the fault diagnosis system, as illustrated in Figure 5, Figure 6 and Figure 7.

The gains used in the attitude and altitude controllers are $k_1=k_3=1,k_5=1.5,k_2=k_4=0.85$, and $k_6=0.3$. The gains for the translational controller are $k_7=k_9=2$ and $k_8=k_{10}=0.3$. We impose that the fault is detected and isolated after 1 s of occurrence (see Figure 7). Sensor noise was introduced using a random function with a normal distribution, with a standard deviation of $0.005$ and a mean of zero.

Figure 5 depicts a 3-D position comparison of the hexacopter vehicle. This figure illustrates that without an FTC strategy, the hexacopter crashes to the ground. In contrast, the FTC system enables the vehicle to complete its path successfully. Furthermore, Figure 6 shows the position and attitude dynamics. Figure 6b presents the roll $\varphi \left(t\right)$ and pitch $\theta \left(t\right)$ angles, which remain around zero, indicating a stable flight even in the presence of an actuator failure. When the FTC strategy is applied, the roll and pitch angles are around 15 degrees with stable dynamics, indicating that the failure is reconfigured. Additionally, Figure 6b shows that the yaw angle varies due to the uncontrolled state. Figure 6c displays the vehicle’s yaw angular velocity $r\left(t\right)$ during the flight test. Once fault $f_2$ is detected and isolated, the attitude controller induces a constant yaw rotation at a velocity of $r\left(t\right)=-200$ deg/s, as shown in Figure 6c.

Figure 7 illustrates the forces produced by the motors when the FTC algorithm is applied. It is evident that when the failure is introduced in motor 2, $f_2$, the motor opposite $f_5$ reduces its force to compensate. This adjustment continues with the nominal controller until the failure is detected and isolated. When the Fault Reconfiguration is applied, the opposite motor $f_5$ increases its magnitude to compensate for the failure. The remaining motors $f_3$ and $f_1$ also increase their forces while $f_6$ and $f_4$ reduce their forces quickly to achieve the required yaw velocity dynamics. After reconfiguring for the failure, the hexacopter UAV successfully follows the desired path without incurring any damage.

The following subsections will discuss the application of various advanced technologies in precision agriculture. This will include in-flight image acquisition and flight path planning for efficient data collection and field coverage. Additionally, the estimation of near-infrared reflectance to generate the normalized difference vegetation index and assess crop health will be explored. Finally, the application will be carried out on sugarcane crops during their tillering stage.

2.5. In-Flight Image Acquisition

2.5.1. Digital Camera

The SJ9000 camera used in this study was sourced from SJCAM, Shenzhen, China. The SJ9000 camera is chosen to collect the red–green–blue (RGB) imagining data because of the light weight and low cost of this type of camera (see Figure 8a). Low-cost action cameras, like a standard digital camera, have a Bayer color filter that allows capturing an RGB color image. The SJ9000 camera is equipped with an Aptina AR0330 CMOS sensor, the optical format is $1/3$-inch, and the sensor size is 4.51 mm × 3.6 mm, defined as $I_x$ and $I_y$, respectively. The image size of this camera is 4032 × 460 pixels, defined as $L_x$ and $L_y$. The focal length $f_c$ of the digital camera is 2.55 mm.

The digital camera is limited to visual bands, that is, visible red, green, and blue information. These cameras are used to detect plants’ outer defects, greenness, and growth monitoring. In addition, they are applied to calculate a range of vegetation indices to create high-resolution digital elevation models and vegetation height maps [31]. Furthermore, in this work, the proposed photographs for mapping are achieved by using the SJ9000 single-lens frame camera because it provides high geometric picture quality. The autopilot Pixhawk was used to control the camera trigger. The autopilot detects a trigger event and proceeds to capture the images with a predefined time $T_s$ according to the camera characteristics and flight velocity $v_{max}$. The SJ9000 camera was modified by adding an electronic switch circuit to digitally press the image capture button when the trigger event is detected from the autopilot, as shown in Figure 8a.

2.5.2. Multispectral Camera

A Parrot Sequoia camera mounted on the hexacopter UAV was used to acquire multispectral data (see Figure 8b). This camera has five separate imaging sensors with a global shutter, including four multispectral sensors and one RGB sensor. A sunshine sensor is included that is positioned on top of the UAV, so the Parrot Sequoia camera can be used in clear and overcast conditions. The multispectral sensors capture images in the green (G, 530–570 nm), red (R, 640–680 nm), red-edge (RE, 730–740 nm), and near-infrared (NIR, 770–810 nm) wavelength bands. The focal length of the Sequoia camera is 3.98 mm with an image size of 1280 × 960 pixels and a sensor size of 4.8 mm × 3.6 mm.

2.6. Data Collection and Flight Planning

2.6.1. Camera Model

The goal of our mission is to generate the photogrammetry image by scanning a given area with the digital camera and to extract multispectral data in order to evaluate different vegetation indexes with the Sequoia camera. Camera parameters can be used to calculate the projection area and the image dimensions. The angular field of view is the angle $\alpha$ subtended at the rear nodal point of the camera lens by the diagonal d of the picture format, as illustrated in Figure 9a.

The focal length of a camera measures the distance between the nodal point of the lens and the camera sensor, and it is expressed by $f_c$ in millimeters. Image resolution $L_x$ and $L_y$, are expressed in pixels for both sides of the image. Then, the angular field of view $\alpha$ can be calculated as follows [32]:

$$\begin{array}{c}\hfill \alpha =2{\tan }^{-1}\left({\displaystyle \frac{d}{2f_c}}\right).\end{array}$$

(28)

The distance is defined as $d=\sqrt{I_x^2+I_y^2}$, where $I_x$ and $I_y$ are the image format size, expressed in millimeters. From Figure 9b and [33], the projection area can be calculated by assuming that the camera is parallel to the ground at a defined height h, as follows:

$$\begin{array}{cc}\hfill x_m& =2h\tan \left({\displaystyle \frac{\alpha }{2}}\right),y_m={\displaystyle \frac{x_m{L}_y}{L_x}}.\hfill \end{array}$$

(29)

Spatial resolution R is used in order to calculate the maximum height $h_{max}$ that the hexacopter UAV can fly, applying the following equations:

$$\begin{array}{c}\hfill R={\displaystyle \frac{L_x}{x_m}}={\displaystyle \frac{L_x}{2h\tan \left(\frac{\alpha }{2}\right)}}.\end{array}$$

(30)

Then, by using (30), $h_{max}$ is calculated by

$$\begin{array}{c}\hfill h_{max}={\displaystyle \frac{L_x}{2R\tan \left(\frac{\alpha }{2}\right)}}.\end{array}$$

(31)

The reconstruction of the area of interest is achieved by using the parameters presented in Figure 10. This figure shows the waypoints and the flight trajectory that the hexacopter UAV needs to follow. The area of interest is decomposed into rectangles of size $x_m$ and $y_m$. The vehicle path must be programmed to follow the center of those rectangles. To generate a correct photogrammetric image, the acquired images must be overlapping, as illustrated in Figure 10, with $o_x$ and $o_y$ as horizontal and vertical overlaps, respectively. The horizontal distance between two horizontal points is defined by $d_s=x_m-o_x$, and the vertical distances between two vertical points are represented as $d_w=y_m-o_y$. By considering the percentage of overlap ($o_{x_p},o_{y_p}$), $d_s$ and $d_w$ can be computed as follows [32]:

$$\begin{array}{cc}\hfill d_s& =x_m\left(1-{\displaystyle \frac{o_{x_p}}{100}}\right),d_w=y_m\left(1-{\displaystyle \frac{o_{y_p}}{100}}\right).\hfill \end{array}$$

(32)

The maximum speed $v_{max}$ of the hexacopter UAV can be calculated by using the interval between two consecutive shots of the camera $T_s$ and the minimum specified overlap defined by the distance between two points of the path planning $(d_s$ or $d_w)$: $v_{max}{T}_s\le d_w$. Therefore the maximum speed of the vehicle can be defined as:

$$\begin{array}{c}\hfill v_{max}={\displaystyle \frac{y_m}{T_s}}\left(1-{\displaystyle \frac{o_{y_p}}{100}}\right).\end{array}$$

(33)

The limitation of the hexacopter UAV flight altitude depends on the camera’s characteristics used for data acquisition. This is an inherent limitation when performing data collection tasks with digital and multispectral cameras using UAVs. The maximum altitude and speed are the recommended values.

Note that weather conditions, such as wind, are not explicitly considered in the flight planning. However, the designed controllers are sufficiently robust due to the integrators included to handle such disturbances.

2.6.2. Estimation of Agricultural Near-Infrared Images from RGB Data

Usually, the near-infrared (NIR) band is used in precision agriculture to create the Normalized Difference Vegetation Index (NDVI) map that can provide the identification of damaged vegetation areas, monitoring plant growth and chlorophyll levels [34]. The NDVI is calculated using NIR and red band (R), as follows [35]:

$$\begin{array}{c}\hfill NDVI={\displaystyle \frac{NIR-R}{NIR+R}},\end{array}$$

(34)

where $NIR$ and R are spectral radiance measurements recorded with sensors in red (visible) and near-infrared regions, respectively. NDVI produces values ranging from −1 to 1. Negative values typically represent water bodies, clouds, and areas without vegetation. Values close to 0 indicate bare soil or areas with little vegetation. Higher values (close to 1) indicate densely vegetated areas. Usually, the sensor used is a multispectral or hyperspectral camera, both of which are expensive. Based on [36], an approach method for NIR data estimation with only information from RGB images using deep learning Pix2Pix network [37] is proposed in this work.

Pix2Pix uses Generative Adversarial Networks (GANs) in the conditional setting, also called cGANs. This algorithm applies a condition on an input image and generates a corresponding output image for image-to-image translation tasks. The image-creating network is called the Generator (G), and the fake image identifier is called the Discriminator (D). The architecture of Pix2Pix is based on the U-Net convolutional network [38] as a generator model and the discriminator is a convolutional PatchGAN [39].

The objective of the cGAN is to learn a mapping from observed image $x_p$ and random noise vector $z_p$, to the output image $y_p$, and it is defined as [37]

$$\begin{array}{c}\hfill {\mathcal{L}}_{cGAN}(G,D)={\mathbb{E}}_{x_p,y_p}\left[\log D(x_p,y_p)\right]+{\mathbb{E}}_{x_p,z_p}[\log (1-D\left(x_p,G(x_p,z_p)\right)],\end{array}$$

(35)

where ${\mathbb{E}}_{x_p,y_p}$ is the error between the input image and the ground truth image, and ${\mathbb{E}}_{x_p,z_p}$ is the error between the input image and the generated image calculated by the discriminator D. The generator G tries to minimize the objective (35) against an adversarial D that tries to minimize the following objective:

$$\begin{array}{c}\hfill G^{*}=\arg \underset{G}{min}\underset{D}{max}{\mathcal{L}}_{cGAN}(G,D)+\lambda {\mathcal{L}}_{L1}\left(G\right),\end{array}$$

(36)

with

$$\begin{array}{c}\hfill {\mathcal{L}}_{L1}\left(G\right)={\mathbb{E}}_{x_p,y_p,z_p}\left[\right||y-G(x_p,z_p)\left|\right|],\end{array}$$

(37)

and $\lambda$ is defined as the weight parameter to ${\mathcal{L}}_{L1}\left(G\right)$. The discriminator learns to classify fake and real images. Pix2Pix uses a real image to perform image-to-image translation training. Both the generator and discriminator use modules of the form convolution-BatchNorm-ReLu.

In this application, the real images are RGB-NIR samples. Initially, for training purposes, the RGB and NIR data were captured by the Sequoia multispectral camera. Once the training is complete, the input image is obtained from a low-cost RGB camera in order to validate the method for generating the NDVI map, as shown in Figure 11. This figure presents two main blocks: model training and NIR estimation. First, the multispectral images. Subsequently, only the real RGB and NIR images are extracted. These two images are then used to train the Pix2Pix neural networks. Once the system is trained, an input RGB image is applied to obtain the estimated output NIR image.

In the context of the Pix2Pix method, the RGB and NIR real images are initially misaligned due to the lens’s position in the multispectral camera. To address this, an alignment correction is implemented using rigid registration. This technique finds the optimal transformation (rotation and translation) that aligns one image to another, ensuring that the input and output images are correctly aligned for the Pix2Pix process. It also uses normalized cross-correlation to find the optimal alignment.

To evaluate the similarity between the real NIR and the estimated ones, the Structural Similarity Index (SSIM) [40] and the Peak Signal-to-Noise Ratio (PSNR) [41] are used. The SSIM is a metric used to assess the similarity between two images. It evaluates the pixel-wise differences and considers the structural information and luminance variations present in the images. The SSIM consists of three components: luminance $L(a,b)$, contrast $C(a,b)$, and structural $S(a,b)$ comparison. The SSIM equation is represented as follows:

$$\begin{array}{cc}\hfill SSIM(a,b)& =L{(a,b)}^{\beta }C{(a,b)}^{\gamma }S{(a,b)}^{\delta }\hfill \\ \hfill & ={\left({\displaystyle \frac{2{\mu }_a{\mu }_b+C_1}{{\mu }_a^2+{\mu }_b^2+C_1}}\right)}^{\beta }{\left({\displaystyle \frac{2{\sigma }_a{\sigma }_b+C_2}{{\sigma }_a^2+{\sigma }_b^2+C_2}}\right)}^{\gamma }{\left({\displaystyle \frac{{\sigma }_{ab}+C_3}{{\sigma }_a{\sigma }_b+C_3}}\right)}^{\delta },\hfill \end{array}$$

(38)

where a and b represent the local windows in the images; ${\mu }_a$ and ${\mu }_b$ are the means of a and b, respectively; ${\sigma }_a$ and ${\sigma }_b$ are the standard deviations of a and b, ${\sigma }_{ab}$ is the covariance of a and b, $C_1,C_2$ and $C_3$ are small constants to stabilize the division with weak denominator, and $\beta ,\gamma$ and $\delta$ are constants to weight the relative importance of the components. The SSIM index ranges between $-1$ and 1, where 1 indicates perfect similarity. Higher values imply greater similarity between the images.

The PSNR is a commonly used metric for evaluating the quality of a compressed or restored image in relation to the original image. It is expressed in decibels and quantitatively measures how similar a compressed or restored image is to the original image. The equation for calculating PSNR is as follows:

$$\begin{array}{c}\hfill PSNR=10{log}_{10}\left({\displaystyle \frac{MA{X}^2}{MSE}}\right),\end{array}$$

(39)

where $MAX$ is the maximum possible value in the original image, $MSE$ is the Mean Squared Error, which is calculated as follows:

$$\begin{array}{c}\hfill MSE={\displaystyle \frac{1}{mn}}\sum _{i=1}^m\sum _{j=1}^n\left|\right|f(i,j)-{g(i,j)\left|\right|}^2,\end{array}$$

(40)

where f is the original image, g is the compared image, and m and n and the number of rows and columns in pixels, respectively. A higher PSNR indicates less discrepancy between the images and, therefore, better quality of the compressed or restored image.

3. Results

For testing the effectiveness of the proposed strategy, two scenarios were considered. In the first scenario, the experimental flight planning, the data collection, and the orthomosaic generation are presented. In the second scenario, the estimation of the agricultural NIR band is established by using the digital camera. To clarify the research plan,

Table 4. Study parameters.

Type	Parameter
Independent	UAV flight altitude
Independent	Sensor type
Independent	Actuator fault scenarios
Dependent	NDVI values
Dependent	UAV stability
Dependent	Data accuracy
Crop	Sugarcane
Other	Tillering stage

defines the parameters, including independent and dependent variables, crop type, and other relevant parameters for greater clarity in the study.

3.1. Scenario 1—Experimental Flight Planning and Data Collection

The experiments were conducted in Ameca Jalisco, Mexico, in August 2023. The flight planning and data collection were carried out experimentally by using the hexacopter UAV as shown in Figure 12. In this scenario, the vehicle performs a flight over the ethnobiological garden of Jalisco, located at the University of Guadalajara, Los Valles University Center. The area of interest is a rectangle with an area of 101 m × 104 m, as shown in Figure 13. The digital camera has an angular field of view $\alpha =92.75$ deg. The maximum height the hexacopter can fly is $h_{max}=35$ m, with the horizontal distance between two horizontal points $d_s=14.68$ m and the vertical distances $d_w=11.01$ m. The percentage of overlap is specified to $o_{x_p}=o_{y_p}=80\%$ (see Figure 13). According to these values, the maximum speed that the vehicle can fly is $v_{max}=11.017$ m/s. The images are captured from the autopilot with a predefined time of $T_s=1$ s. Also, the images are georeferenced with the onboard GPS data. The flight planning is designed with the above parameters and programmed into the hexacopter UAV using the Mission Planner ground station.

The trajectory and parameters allow the collection of 63 images to create the orthomosaic using the software PIX4D mapper version 4.5, as shown in Figure 14. From this figure, it is clear that successful reconstructions with an effective overlapping and correct attitude flight reference according to the computed flight planning. The proposed grid trajectory is accurate enough to generate a good-quality orthomosaic to create an effective NDVI map.

3.2. Scenario 2—Experimental Estimation of Agricultural NIR Band

To perform the image-to-image translation training, Pix2Pix uses a pair of real images of RGB-NIR samples. The Sequoia multispectral camera generates five-band images captured experimentally on a flight by the hexacopter UAV. The flight was performed at the same altitude and velocity as the first scenario. The software MATLAB version 2023b is used to extract $256\times 256$ pixels of the RGB-NIR samples. A total of 1080 RGB-NIR pairs were extracted for training the Pix2Pix network, and the validation was carried out with 400 pairs, as shown in Figure 15. Also, from Figure 15, it can be noted that both images are misaligned with each other due to the lens’s position in the multispectral camera. To correct this, the alignment correction is programmed in MATLAB by using rigid registration. The pix2Pix method was implemented in Python using Keras, OpenCV, and Tensorflow libraries. The Pix2Pix network was trained in 400 epochs with $\lambda =100$ from (36). It used the Adam optimizing parameters.

Figure 16 presents the estimated NIR images using the Pix2Pix method. For the estimation of the agricultural NIR band, Pix2Pix uses RGB images and NIR images as inputs from the multispectral camera. When the network is trained, a test is performed with an RGB image to obtain the estimated NIR data in order to validate the proposed method. Figure 16a and Figure 16b represent the real RGB and NIR input images, respectively. Figure 16c presents the estimated NIR image. The reader can observe in Figure 16 that the real NIR image is similar to the NIR estimated image. The network training can be observed from the following link: https://www.youtube.com/shorts/dHMd0389B64 (accessed on 25 July 2024).

Table 5. Similarity results of the images from Figure 16.

Data	SSIM	PSNR
Image 1	$0.65$	$-12.4$ dB
Image 2	$0.53$	$-10.8$ dB

shows an SSIM value of 0.65 for image 1 and 0.53 for image 2. This indicates a moderate structural similarity between the images, meaning that while the images are not identical, they share significant common features. Additionally, if one of the images is a degraded or compressed version of the other, these SSIM values suggest that the degradation or compression has noticeably affected the image quality. Despite these differences, many important structures and visual details are still recognizable and similar between the images. From the same table, Table 5, negative PSNR values, such as $-10.8$ dB and $-12.4$ dB, indicate poor image quality. These values suggest that the reconstructed image is different from the original, with noise and distortion. However, these values are sufficient to construct an adequate normalized difference vegetation index.

In order to show the effectiveness of the estimated NIR in constructing the NDVI, a comparison between the normalized index is generated with real NIR and with the estimated NIR image. Figure 17 displays two side-by-side maps of the NDVI, likely representing the same geographical area. Figure 17a is the NDVI constructed using the NIR image from the multispectral camera, while Figure 17b is the vegetation index created with the estimated NIR image. From this figure, negative values correspond to water, snow, clouds, or non-vegetated surfaces. Values close to zero generally represent rock, sand, or barren areas with little to no vegetation. Low positive values indicate shrubs and grasslands, and high values indicate temperate and tropical rainforests.

Also, Figure 17 exhibits a high degree of similarity. The consistent patterns and distribution of NDVI values across both maps indicate that the vegetation health and density characteristics are nearly identical between the two images. Given the strong similarity between the two NDVI maps, it is possible to derive the same set of information and conclusions about the vegetation health, density, and distribution from either image.

Furthermore, in Figure 18, the orthomosaic and the NDVI created from the estimated NIR applied to a sugarcane crop are presented. These experiments were conducted in Ameca, Jalisco, Mexico, during May 2023. In this experiment, a flight was performed over a sugarcane crop at the tillering stage. The area of interest is 1916 m².

Table 6. Meaning of NDVI values [42].

Type of Objects	NDVI Value
High, dense vegetation	0.1–0.7
Sparse vegetation	0.5–0.2
Open soil	0.2–0.025
Clouds	0
Snow, ice, dust, rocks	0.1–−0.1
Water	−0.33–−0.42
Artificial materials	−0.5

presents the NDVI values depending on the object type. It can be observed that areas close to 1 indicate dense vegetation, while areas close to 0 indicate bare soil or areas with little vegetation. The average NDVI value in the sugarcane planting area is $0.47$, indicating the tillering stage. The growing stages of the 12-month cycle of sugarcane are as follows: germination with a mean NDVI value of 0.35–0.4; tillering: 0.4–0.65; grand growth: 0.65–0.7; and ripening: 0.65–0.3 [42]. As a result, in Figure 18, we can identify the areas that require additional attention or intervention. These interventions may include targeted fertilization, irrigation adjustments, or pest and disease management to improve the overall health and yield of the sugarcane crop.

4. Conclusions

This paper proposed and validated an actuator fault-tolerant control strategy for a hexacopter UAV using advanced sensing techniques and flight trajectory planning for precision agriculture applications. The findings demonstrate that the proposed approach, which integrates NIR reflectance estimation from RGB imagery using the Pix2Pix deep learning network based on cGANs and NDVI calculation, enables effective crop health assessment. The NDVI was presented for a sugarcane crop using the estimated NIR to assess the crop’s condition during its tillering stage. The flight trajectory planning ensures the efficient coverage of the targeted agricultural area while considering the vehicle’s dynamics and fault-tolerant capabilities. Despite the limitations of UAV flight endurance and the need for robust data-processing algorithms, the study contributes to understanding FTC systems in precision agriculture. It highlights the importance of developing sophisticated strategies for adapting to changing environmental conditions. The results of this study can be applied to improve the reliability and efficiency of UAV-based data collection in agriculture.

The presented FTC scheme introduces several novel contributions. (i) Our FTC strategy not only can ensure fail-safe landing but also maintains operational efficiency and data collection accuracy even in the event of total actuator failures. This is particularly critical for precision agriculture applications where continuous data acquisition is essential. (ii) We have developed flight trajectory planning that ensures the efficient coverage of agricultural areas. This system takes into account the hexacopter dynamics and fault-tolerant capabilities, thus maximizing the effectiveness of the UAV’s operations. (iii) Our study specifically addresses fault-tolerant control during precision agriculture trajectories and in non-hover positions, a scenario that existing references do not fully cover. This ensures that the hexacopter can handle faults during actual mission operations, not just in stationary or hover modes.

Further research should focus on advancing energy management strategies, intelligent data fusion techniques, and standardized protocols to address the challenges posed by total actuator failures in hexacopter UAVs for precision agriculture applications.