Real-Time Interval Type-2 Fuzzy Control of an Unmanned Aerial Vehicle with Flexible Cable-Connected Payload

Abstract

This study presents the design and real-time applications of an Interval Type-2 Fuzzy PID (IT2-FPID) control system on an unmanned aerial vehicle (UAV) with a flexible cable-connected payload in comparison to the PID and Type-1 Fuzzy PID (T1-FPID) counterparts. The IT2-FPID control has significant stability, disturbance rejection, and response time advantages. To prove and show these advantages, the DJI Tello, a commercial UAV, is used with a flexible cable-connected payload to test the robustness of PID, T1-FPID, and IT2-FPID controllers. First, the optimal coefficients of the compared controllers are found using the Big Bang–Big Crunch algorithm via the nonlinear UAV model without the payload. Second, once optimised, the controllers are tested using several scenarios, including disturbing the payload and the coverage path planning area to examine their robustness. Third, the controller performance results are evaluated according to reference achievement and point-based tracking under disturbances. Finally, the superiority of the IT2-FPID controller is shown via simulations and real-time experiments with a better overshoot, a faster settling time, and good properties of disturbance rejection compared with the PID and the T1-FPID controllers.

1. Introduction

Significant and rapid advances in unmanned aerial vehicle (UAV) technology have made them economically viable and easily maintainable. They have been used in research and in civil, military, and commercial applications, such as military UAVs, search and rescue, and agriculture [1,2,3,4,5]. Even though they can complete tasks, they still face many challenging problems because of the sensor accuracy and the coupled highly nonlinear system dynamics. Considering the controller of the UAV, which is one of the challenging problems, encouraged many researchers to take on such challenges and propose several control designs, such as proportional–integral–derivative (PID) [6,7], fuzzy PID control (FPID or T1-FPID) [8,9,10], linear quadratic regulator (LQR) [11,12], linear quadratic Gaussian and loop transfer recovery (LQG/LTR) [13], model predictive control (MPC) [12,14,15] and H∞ [16,17] control algorithms.

The controller algorithms and the calculation of their parameters are directly related to the physical dynamics of the system. However, the physical dynamics of the UAV represent a challenge for many of the control algorithms implemented on these UAVs, mainly when the control parameters are determined by using limited known dynamic model parameters. Moreover, knowing all parameters of the physical dynamics is almost impossible due to uncertainties or inaccessible parameters. Therefore, the stable controller parameter might cross to the unstable region in the presence of uncertainty or inaccessible parameters. To explain and illustrate the problem, it is scaled and tested using a mini UAV. For indoor experimental research, nano or mini UAVs have proven their efficiency concerning easy access, small sizes and weights [18]. As a result, researchers have used several commercial UAVs, such as Bitcraze Crazyflie 2.0 [8,19,20,21,22], Parrot Mambo Minidrone [9,10,12] and DJI Tello [23,24,25].

In many studies, the connection with the payload represents a further research challenge [23,26,27,28]; the control problem under uncertain payloads is solved using reinforcement learning. The UAV has a flexible cable, payloads, and magnets. When using the magnets, the payloads can connect to UAV. Hence, the weights cannot be known, adding additional uncertain parameters. Thanks to model-based meta-reinforcement learning, the system’s inputs and outputs are trained and created model-based data using a human-based remote controller. After designing a model-based meta-reinforcement learning dynamic model, a model predictive control algorithm is implemented to cope with such effects [23].

UAVs face other challenges not only due to the payload but also due to wind disturbances. In [27], a path following a control algorithm for a UAV with cable-connected payload is designed under wind disturbances. In addition, near hovering and cruising states are included in the control, as well as an employed position-based tracking control algorithm for the outer structure. A whole system model is developed with payload and disturbance effects, meaning that they are not unknown parameters in [27]. Considering unknown parameters, such as the payload mass, an excellent example of associated research work can be attributed to [26] on adaptive UAV control for a cable-suspended unknown payload. The system is tested with a nonlinear and adaptive PD controller. The coefficients/parameters of the UAV controllers are tuned via many different optimisation methods [29], and each of them has its advantages and disadvantages. In [30], a comparative study of different controllers for UAVs is investigated in terms of limitations and strength parts. In this study, we focus on fuzzy-based PID controllers since they have better accuracy than PID controllers and are computationally efficient, which makes them applicable to UAVs in real time [7].

Contributions

This study shows the successful implementation of the Interval Type-2 Fuzzy PID (IT2-FPID) controller under a nonlinear dynamic model and unknown/uncertain parameters of the UAV with a payload in simulation and real-time systems. The implementation comes from model-based tuned parameters to overcome an unknown weighted flexible cable-connected payload. It is aimed to eliminate the oscillating payload effect on DJI Tello UAV and to follow the given references by using these controllers. PID, T1-FPID, and IT2-FPID are tuned using the Big Bang–Big Crunch (BB-BC) optimisation method [31] on the dynamical model of the UAV without a payload. Due to the unmodelled dynamics, such as inaccessible parameters or uncertainties, the controller coefficients are fine-tuned on a real-time UAV without the payload before being tested with the flexible cable-connected payload. After that, it is expected that the UAV will be able to eliminate disturbing payload effects and demonstrate efficient tracking of the designed trajectory. Moreover, the performance of the IT2-FPID is compared with T1-FPID and PID under the same scenarios. The presented results of the study clearly show that the designed IT2-FPID control system is superior in comparison to its T1 and conventional counterparts. The video file provided in the Supplementary Material shows the real-time control performance of the deployed IT2-FPID.

The rest of the study is organised as follows: Section 2 provides the UAV model, including the DJI Tello model and the problem statement and challenges. Section 3 presents implemented controller methods, which are PID, T1-FPID and IT2-FPID, and the simulation results. Then, Section 4 shows the indoor testing area and the designed offline path. Moreover, the process of position localisation of the UAV is explained in this section. The real-time results of the PID, T1-FPID and IT2-FPID controllers are given in Section 5. Finally; Section 6 presents the conclusions and a perspective on future work.

2. UAV Model and the Problem Definition

This section presents the mathematical model for the dynamics of DJI Tello UAV, and the problem statement and challenges. The first part of this section describes the dynamic model of the UAV. This dynamic model is shown without any payload. After that, the problem of the UAV with a flexible connected payload is explained.

2.1. Model Dynamics of DJI Tello UAV

Before outlining the UAV model, we show the main coordinate frames used in this study: ${(.)}^I$ represents the global Earth-fixed Earth-centred inertial frame, and ${(.)}^B$ is the body-fixed frame, fixed at the UAV centre of mass (CoM) as given on Figure 1. There are two types of configurations for the UAV, which are × and + configurations [8,20,32]. In this study, the ×-type UAV is modelled and used.

The position ($p^I$) and attitude (${\eta }^I$) of the UAV with respect to the frame I are defined as follows:

$$\begin{array}{c}{p}^I={\left[xyz\right]}^T,{\eta }^I={\left[\varphi \theta \psi \right]}^T,\\ {\upsilon }^I={\dot{p}}^I=R{\upsilon }^B,{\omega }^I={\dot{\eta }}^I=T{\omega }^B,\end{array}$$

(1)

where the vector $p^I$ contains the x, y, and z Cartesian position coordinates of the UAV inertial frame, and ${\upsilon }^I$ and ${\omega }^I$ are the derivative of position and attitude values, respectively. The superscripts ${(.)}^I$ and ${(.)}^B$ denote, respectively, the inertial and body frames. The ${\eta }^I$ vector contains the angles of roll $\varphi$, pitch $\theta$, yaw $\psi$ of the UAV body frame, R is a rotational matrix, and T is a translation matrix used in frame conversions. These rotation and transformation matrices are used to calculate the transformation of the inertial frame to the body frame or vice versa. The rotation and transformation matrices are written in Equation (A1) of Appendix A.

The $\upsilon$ variable is used to represent the linear velocity, and $\omega$ is used to represent the angular velocity. In this study, the heading angle of the UAV, yaw reference (${\psi }_r$) is equal to ${\psi }_r=0^{\circ }$ in order to overlap the inertial and the body coordinate frames of the DJI Tello.

After the defining position and orientation, a nonlinear dynamic equation for the DJI Tello UAV can be written with the Newton–Euler formulation as given below [8,20]:

$$\begin{array}{c}{m}_{\mathrm{uav}}{\dot{\upsilon }}^I=F^I=F_{gravity}^I+F_{thrust}^I,\\ I^B{\dot{w}}^B=-w^B\times \left(I^B{w}^B\right)+{\tau }_{rotor}^B.\end{array}$$

(2)

In Equation (2), $m_{\mathrm{uav}}$ is the mass of the UAV to the CoM of the UAV, $F_{gravity}^I$ is the gravity force, $F_{thrust}^I$ is the total thrust force which comes from the propellers, $I^B$ is a moment of inertia, and $I^B$ is referred to as the inertia matrix. The cross product at the right-hand side of the angular velocity dynamics equation comes from the Coriolis force effect. Considering the DJI Tello UAV, this effect can be ignored because of the size of the UAV. Ref. [8] tested a fixed heading angle and ignored the Coriolis effect on the DJI Tello UAV, which has an almost similar size to the DJI Tello UAV. Therefore, the formula can be written as $I^B{\dot{w}}^B={\tau }_{rotor}^B$, and the UAV torques (${\tau }_{rotor}^B$) are expressed by ${\tau }_{\varphi }$,${\tau }_{\theta }$, and ${\tau }_{\psi }$, respectively:

$$\begin{array}{c}{F}_{gravity}^I={\left[00-m_{\mathrm{uav}}g\right]}^T,\\ F_{thrust}^I=R{F}_{thrust}^B.\end{array}$$

(3)

To overcome the gravity effect, the force is applied to the CoM of the UAV as $F_{gravity}^I$. The acceleration due to Earth’s gravity is denoted as g. The $F_{thrust}^I$ forces are generated by propellers, and the formula is written as Equation (4). The angular velocity of each rotor’s propeller is denoted as ${\Omega }_i$, and the thrust coefficient is referred to as b

$$\begin{array}{c}{F}_{thrust}^B={\left[00b\sum _{i=1}^4{\Omega }_i^2\right]}^T,\end{array}$$

(4)

$$\begin{array}{c}{\dot{\upsilon }}^I=\left[\begin{array}{c}\ddot{x}\\ \ddot{y}\\ \ddot{z}\end{array}\right]=\frac{b}{m_{\mathrm{uav}}}\sum _{i=1}^4{\Omega }_i^2=\left[\begin{array}{c}s\left(\varphi \right)s\left(\psi \right)+c\left(\varphi \right)s\left(\theta \right)c\left(\psi \right)\\ -s\left(\varphi \right)c\left(\theta \right)+c\left(\varphi \right)s\left(\theta \right)s\left(\psi \right)\\ c\left(\varphi \right)c\left(\theta \right)\end{array}\right]-\left[\begin{array}{c}0\\ 0\\ g\end{array}\right].\end{array}$$

(5)

In Equation (5), $s(.)$ and $c(.)$ denote the sine and cosine, respectively. In the equations above, the nonlinear dynamical model for the translational motion of the UAV is represented, and a detailed linear dynamical model of the UAV is written in Equations (A5)–(A7) of Appendix A. The angular velocity dynamics of the UAV also can be found in Equations (A3) and (A4). As aforementioned about the mass of UAV ($m_{\mathrm{uav}}$), it only gives the UAV weight.

In the above equations, mass equals $m_{\mathrm{uav}}=0.08$ kg, and the length between the CoM and propeller is $0.6$ m. The detailed parameters of the DJI Tello UAV are explained in Refs. [24,34].

2.2. Problem Statement and Challenges

In Figure 2, the UAV with the flexible cable-connected payload and its pendulum effects are shown. The figure shows the different perspectives of the UAV with cable-connected payload and the variable payload in detail. A 3D-printed part is created and mounted to provide a UAV connection with a payload cable. The cable length is 35 cm and is chosen as flexible. Taking into account the pendulum effects, forces $F_P$, $F_R$ and the angles $\alpha$, $\epsilon$, $\beta$, and $\lambda$ vary with the acceleration of the UAV due to Newton’s second law. The maximum carrying payload is measured with empirical methods, and its maximum value is determined as 75 g for the experiments. This weight ($m_{\mathrm{p}}$) is a significant value when considering the mini UAVs. Then, it is able to rewrite the equations as $m_{\mathrm{total}}=m_{\mathrm{uav}}+m_{\mathrm{p}}$. Instead of rewriting the dynamical equations and calculating the controller parameters using swinging $m_{\mathrm{total}}$, we propose that the compared controllers can eliminate the unknown payload effects and the variable CoM problems without redesigning or the calculation.

Understandably, unknown payloads and disturbances directly relate to the system dynamics. Therefore, the main objective of this work is to overcome these adverse effects. At this point, instead of re-writing the equations of the UAV dynamics and redesigning the controllers by considering the whole mathematical model with the additional dynamics of the swinging payload, the controllers are designed by considering the known dynamics of the UAV itself without taking the payload and disturbance effects into account so that the robustness of the three controllers is compared with each other, then the IT2-FPID controller gives the results in terms of system stability and robustness.

The most important question is as follows: why do we focus on uncertain parameters, such as unknown payload and flexible cable effects? This system is a simulation of a commercial agricultural UAV problem. Commercial agricultural UAVs may contain liquid or other payloads. However, the problem is that these payloads, such as seeds or pesticides, are unknown loads because of their variable weights and variable centre of masses during the flight.

3. Implemented Controllers

In this study, we design and deploy PID, T1-FPID and IT2-FPID controllers as x-y position controllers that generate the x-y velocity reference signals ${\upsilon }_x$, ${\upsilon }_y$ (m/s) to be tracked by the on-board velocity controller by DJI Tello UAV. In this context, let us first define the position errors to be minimised by the controllers on the x and y axes $e_x,e_y$ as follows:

$$\begin{array}{c}{e}_x=x_{ref}-x_{obs}\\ e_y=y_{ref}-y_{obs}\end{array}$$

(6)

where ($x_{ref},y_{ref}$) and ($x_{obs},y_{obs}$) are the references and the feedbacks (actual position of the UAV), respectively. In this study, the reference and feedback units are in meters (m) in the position-based controller. To minimise $e_x,e_y$, we designed two controllers that generate the velocity signals $u_x={\upsilon }_x$, $u_y={\upsilon }_y$. Within the scope of this study, we do not handle the altitude z and yaw $\psi$ control problems of the UAV and use the onboard controller directly. Throughout the study, we set altitude reference $z_{ref}=90$ cm and heading angle reference of the UAV ${\psi }_{ref}=0^{\circ }$.

In this section, as the x-y position controllers have the same structure, the subscripts $(x,y)$ in $e_x$, $e_y$, $u_x$, $u_y$ are dropped in the rest of the section for a better understanding.

3.1. PID Controller

The PID control law for the UAV is represented as follows:

$$\begin{array}{c}u=K_c\left(e+\frac{1}{T_i}\int edt+T_d\frac{de}{dt}\right),\end{array}$$

(7)

where u is a reference signal for the inner controller, e is represented as the error, and $T_i$, $T_d$, and $K_c$ are the integrator, derivative, and proportional gain, respectively.

3.2. Type-1 Fuzzy PID Controller

In this section, the general structure of the T1-FPID is presented. Before starting T1-FPID, the fuzzy PID control block for T1 and IT2 is shown in Figure 3 [35]. The position error (e), and its rate of change ($\Delta e$) are shown as controller inputs [8,9].

As shown in Figure 3, the input scaling factors (SFs) are denoted as $K_e$ and $K_d$, while the output scaling factors (SFs) are denoted as $K_0$ and $K_1$. The equation is in the form

$$\begin{array}{cc}E=K_{e}e,& \Delta E=K_d\Delta e,\end{array}$$

(8)

where E and $\Delta E$ denote the FLC inputs. The output of the FLC (U) is transformed to the actual control signal of the T1 or IT2 fuzzy PID controllers (u) as follows:

$$\begin{array}{c}u=K_{1}U+K_0\int Udt.\end{array}$$

(9)

The employed fuzzy logic controller (FLC) is designed with 3 × 3 rules as given in

Table 1. The rule table of T1- and IT2-FPID controller.

$\mathit{E}/\Delta \mathit{E}$	P	Z	N
P	PB:1	PM:0.65	Z:0
Z	PM:0.65	Z:0	NM:−0.65
N	Z:0	NM:−0.65	NB:−1

. The consequent part of the rules is described with crisp singletons. Moreover, they are represented with five linguistic fuzzy terms, which are defined as follows: PB—positive big; PM—positive medium; Z—zero; NM—negative medium; and NB—negative big.

In Figure 4, the membership functions (MFs) of the T1-FPID are presented for the UAV [8,9]. There are three MFs in T1-FPID, N, Z and P abbreviated from “negative”, “zero”, and “positive”, which are the linguistic fuzzy variables. The antecedents are defined with uniformly distributed symmetrical triangular MFs. In this study, triangular MFs are selected instead of Gaussian or other types of MFs. Triangular MFs provide a fast computational time and real-time implementation [7]. Moreover, the MFs values are determined based on empirical evaluations. The resulting control surface is shown in Figure 4. The implemented FLC uses the product implication and operates the centre of sets defuzzification method [36,37,38]. In this study, the fuzzy weighted average defuzzification method is used, and the equation is written as Equation (10):

$$\begin{array}{c}U=\frac{{\sum }_{i=1}^M{y}^i{f}^i}{{\sum }_{i=1}^M{f}^i}\end{array}$$

(10)

where U is the final output of the Takagi–Sugeno–Kang fuzzy inference system [39], f is the firing function of T1-FPID, y is denoted as consequent MFs, and M is the total number of rules. Table 1 is used for T1-FPID.

3.3. Interval Type-2 Fuzzy PID Controller

Just like the T1-FPID controller, the IT2-FPID controller is composed of choosing E and $\Delta E$ as inputs and the control action of the IT2-FLC U as output as displayed in Figure 3. The IT2-FLC has the same N = 9 rules and is presented in Table 1. The MFs of IT2-FPID are characterised by triangular MFs, and the control surface of the IT2-FPID is given in Figure 5. The linguistic fuzzy variables are N, Z, and P, respectively. Here, $m_n$ is the only parameter that creates the footprint of the uncertainty of the IT2-FPID [40]. We employ the following parameters: $m_2=\alpha$ and $m_1=m_3=1-\alpha$ as suggested in [36,40,41,42]. Therefore, the selection of the parameter ($\alpha$) is vital because it directly affects U.

Several methods have been used for the calculation of the controller output (U) [40,43]. In this study, the Biglarbegian–Melek–Mendel method is used [40]. The equation relating to this method is given as follows:

$$\begin{array}{c}\overline{y}=\frac{{\sum }_{i=1}^M{f}_h^i{y}^i}{{\sum }_{i=1}^M{f}_h^i}\\ \underline{y}=\frac{{\sum }_{i=1}^M{f}_l^i{y}^i}{{\sum }_{i=1}^M{f}_l^i}\\ \mathrm{U}=\zeta \underline{y}+(1-\zeta )\overline{y}.\end{array}$$

(11)

where $y^i$ shows consequent MFs, M is the total number of rules (Table 1 is used for IT2), $\zeta$ is a weighting parameter for the type reduced set ($[\underline{y},\overline{y}]$), and U is the output of FLC. Lastly, $f_l$ and $f_h$ are defined as the lower and upper firing functions of IT2-FPID, respectively. In Figure 5, the control surface, which selected $\zeta =0.25$ and $\alpha =0.25$ for IT2-FPID, is illustrated.

Parameters of T1-FPID and IT2-FPID are selected aggressively by using rule-based FLCs and MFs. The $\zeta$ and $\alpha$ parameters are selected by trial and error so that an aggressive control surface is created for IT2-FPID, then it is defined as $\zeta =0.5$ and $\alpha =0.25$ in the study. After that, a comparison of the control surfaces ($\Delta U$) of the T1-FPID ($U_{T1}$) and IT2-FPID ($U_{IT2}$) is presented in Figure 6, taking into account their aggressiveness and smoothness. The differences between them are formulated as $\Delta U=U_{T1}-U_{IT2}$. An aggressive control action is characterised by a relatively higher control action output compared to the others. Specifically, if a line lies above another line in the positive region where $E,\Delta E\ge 0$ (or below in the negative region where $E,\Delta E<0$), the resulting control action is deemed aggressive. Conversely, if this condition is not met, it is deemed a smooth control action.

3.4. Optimisation Results and Simulation

All implemented controllers are optimised via the BB-BC algorithm, described in Appendix B. The cost function of BB-BC is defined as

$$\begin{array}{c}\mathrm{ISE}=\int (e_x^2+e_y^2)dt.\end{array}$$

(12)

In the equation above, errors are defined for two axes as $e_x,e_y$ of the UAV. The error values are explained in Equation (6), and integral square error (ISE) is selected for the performance criteria of the optimisation algorithm. In the BB-BC algorithm, the population size is chosen as 50, and the maximum iteration number is selected as 200 for the optimisation process of each controller. The BB-BC optimisation algorithm is used to tune $K_c$, $T_i$, and $T_d$ for the PID controller, and the $K_e$, $K_d$, $K_0$, and $K_1$ parameters for fuzzy-based controllers are tuned using BB-BC, separately. It is noticed that $\alpha$ and $\zeta$ are defined empirically. The BB-BC approach evaluation over simulated data is performed within the Matlab environment [44]. The computation times for the controllers are measured between 1 and 4 ms; these values are negligible considering the sampling time in real-time and simulation experiments. The selected controllers and their parameters are in the Python environment [45]. Control signals ${\upsilon }_x$ and ${\upsilon }_y$ are constrained between $-12$ m/s and 12 m/s during the BB-BC search.

In

Table 2. Tuned parameters and their performance criteria results.

Controller	Parameters	ISE
PID	$K_c=0.0472$, $T_i=0.0052$, $T_d=0.0445$	82,133.4
T1-FPID	$K_e=0.019$, $K_d=0.0019$, $K_0=0.0001$, $K_1=30$	55,128.2
IT2-FPID	$K_e=0.019$, $K_d=0.0019$, $K_0=0.0001$, $K_1=30$, $(\alpha =0.25,\zeta =0.5){}^{\star }$	36,371.9

^⋆: These parameters have been selected before BB-BC.

, parameters of the controllers and the best performance criteria (ISE) obtained by BB-BC are listed. For tuning to the parameters of each controller, the position reference points are $x_{ref}=1.1$ m and $y_{ref}=0.52$ m for BB-BC optimisation.

The optimised controllers are tested on the simulation model, and the results can be seen in Figure 7. In the simulation, the reference signals are defined as $1.1$ m for the x-axis and $0.52$ m for the y-axis. They are the same as the references given in the controller parameter optimisation process using BB-BC.

These simulation results show that the IT2-FPID controller leads to the best results, with no overshoot, the slightest steady-state error, and the rise time. In addition, T1-FPID gives a smoother response than the PID controller. Considering the compared controllers, they do not have the same system responses in both axes due to the fact that the feedbacks of the UAV in both axes have different levels of noise. The aim of choosing different levels of noise is that the payload is not connected to the CoM of the UAV, and it will have a directly different effect on the axes. Oscillation and steady-state errors seen in Figure 7 are caused by the Gaussian disturbances added to the simulation model in the testing stage to show the robustness of the compared controllers against external disturbances. Gaussian distribution parameters for the x and y axis are defined as $N(0,5)$ and $N(0,15)$, respectively. After testing the controllers in the simulation, the controllers are implemented in the UAV.

4. Indoor Testing Area

In Figure 8, an overview of the system, including the proposed IT2-FPID-based position controller, is shown. Figure 8 presents the overall structure of the system, including the proposed IT2-FPID-based position controller. Reference signals and controls and position estimates are calculated on the computer controlling the UAV remotely. The computer used in the real-time experiments has Intel i7 CPU, 32 GB RAM and NVidia GTX 1650Ti GPU. The generated outputs of the computer are the reference signals of the UAV. Moreover, the Python 3.10 environment is used in the system [45]. The computer and the UAV communicate via a Wi-Fi communication protocol. The design process of the main system starts with the generation of the reference signals for the desired UAV trajectory. Considering the main system, the process starts with the reference path generation. There are two different reference points in the proposed approach. The first one is about disturbing the payload. It means that when the UAV tracks the reference points using a stick, it is disturbed, and the flex cable-connected payload is a disturbance. Then, the second test is initiated as coverage path planning (CPP). Next, the offline CPP is explained, and the resulting reference points are given [46,47,48]. The last part is about how to detect the UAV position with a camera and vision-based localisation algorithm. In the following two sections, these are explained intensively.

4.1. Intel RealSense Depth Camera and Vision-Based Localisation Method

In this study, an Intel RealSense depth camera is used for the experiments [49], as it is a low-cost localisation system [8,50]. There are several advantages to using this particular depth camera in terms of its size, camera specifications and 3D Cartesian coordinate detection. Moreover, it has a ${69}^{\circ }\times {42}^{\circ }(\alpha \times \beta )$ field of view (FoV) for the RGB sensor, $1280\times 720{\mathrm{px}}^2$ resolution, and a 30 fps RGB frame rate.

To find the localisation of the UAV, RGB colour detection is used to track it. A red-coloured circle piece is put on the region of interest of the area, and the colour parameters are measured as shown in

Table 3. Camera HSV parameters.

HSV	Hue	Saturation	Value
Maximum Values	160	60	0
Minimum Values	200	255	255

. These colour parameters are in the hue, saturation, and value colour space (HSV). Moreover, the HSV values have maximum and minimum limits, which are shown in Table 3.

After detecting the HSV parameters, some trigonometric formulas are used to find the transformed centre points of the UAV. Figure 9 shows the three-axis sizes of the testing area ($x,y,z:200,150,250$, respectively), camera position, UAV altitude level ($z_a:90$ cm), and detected surface on the UAV.

In Equation (13), the raw centre points of the UAV (${\widehat{x}}_{px}$ and ${\widehat{y}}_{px}$) are defined, which are converged to the transformed centre points. In the matrix, $s_x,s_y$, $s{h}_x,s{h}_y$ and $t_x,t_y$ stand for scale factors, shear factors, and translation factors, respectively. Moreover, these parameters are determined by using “Computer Vision Toolbox for Matlab” [44] and using the following model:

$$\begin{array}{c}\left[\begin{array}{c}{\widehat{x}}_{px}\\ {\widehat{y}}_{px}\\ 1\end{array}\right]=\left[\begin{array}{ccc}{s}_x& s{h}_y& 0\\ s{h}_x& s_y& 0\\ t_x& t_y& 1\end{array}\right]\left[\begin{array}{c}{x}_{px}\\ y_{px}\\ 1\end{array}\right]\end{array}.$$

(13)

Normally, this calculation is set for the two axes because the z-axis is fixed. For the calculation for the three axes, the trigonometric formula is used via Equation (14). In this equation, $M_x$ and $M_y$ show the total number of pixels for the x-y axes, and also ${\sigma }_x$ and ${\sigma }_y$ are defined as the relative mounting angles of FoV. Lastly, h (or z-axis) is used for the distance between the camera and the ground or floor. In Figure 9, the x-y-z-axes dimensions are given as 200 cm, 150 cm, and 250 cm, respectively, and by accounting for the following relations:

$$\begin{array}{c}{x}_{obs}=-z\tan \left({\sigma }_x+\beta \left({\widehat{y}}_{px}/M_x-1\right)\right),\\ y_{obs}=-z\tan \left({\sigma }_y+\alpha \left({\widehat{x}}_{px}/M_y-1\right)\right).\end{array}$$

(14)

4.2. Coverage Path Planning

This section presents the implemented CPP algorithm. CPP is another challenging problem for UAVs  [46,47,48]. The UAV aims to track the specific points defined by using the CPP method.

In Figure 10, the boundaries, starting point, and the specific reference of the CPP lines are shown and coloured yellow, black, and turquoise, respectively. This is because the UAV has a task: reaching the specific points, which are the endpoints of the path segments. The following relation is used:

$$\begin{array}{c}d=w.(1-\upsilon ).\end{array}$$

(15)

The shape, length, and direction of the path constructed by the CPP algorithm depend on d, w, and $l_s$ values, and these correspond to the distance among rows, the half of the camera footprint, and the width of the original polygon, which is swept, respectively, and the percentage of vertical overlap between the width of the camera footprints on consecutive rows are denoted by $\upsilon$ and with a total number n of turns equal to

$$\begin{array}{c}n=\left\{\begin{array}{cc}2.\lceil z/d\rceil \hfill & \mathrm{if}\left(z\mathrm{mod}d\right)\le w/2,\hfill \\ 2.(\lceil z/d\rceil +1)\hfill & \mathrm{if}\left(z\mathrm{mod}d\right)>w/2.\hfill \end{array}\right.\end{array}$$

(16)

The total number of turns n resulting from the CPP algorithm is given in Equation (16), and this algorithm aims to minimise n to some degree, as it can be seen that this number depends on the z value, which is a function of the polygon width that is $z=l_s-w/2$. Therefore, the direction in which the polygon width is defined determines the value of n.

If the gap between the last row of the created path and the vertex or the edge of the original polygon is greater than half of the camera footprint w, then an extra row is added to the end of the path to reduce the gap as mentioned earlier; therefore, the distance between the last row and the original polygon might be less than $w/2$. In Figure 11, the difference between the distances from the border of the polygon to the initial row and the last row of the CPP path is an example of this case. The optimal line sweep direction and the path are determined by the following algorithm given in Algorithm 1 [46,47].

Algorithm 1 Basic CPP algorithm.

Step 1: (Farthest vertex v from the edge e) For each edge, find the farthest vertex with respect to Euclidean distance and keep this (e, v) pair and the calculated distance value. Step 2: (Minimum distance pair (e,v)) Compare the distance values and find the (e, v) pair which gives the minimum distance value. Step 3: (Orthogonal direction vector) The optimal line-swept direction is the vector that is orthogonal to the edge from the (e, v) pair found in Step 2, and the rows constructed by the CPP algorithm will be parallel to this edge e. Step 4: (Path construction) The path is defined by the way points that are connected by line segments. These way points are placed in such a way that the line segments connecting them are parallel to the edge on which the polygon width is defined and the distance between a way point and the border of the polygon satisfies the camera footprint requirements.

The CPP algorithm is employed in the testing area offline as shown in Figure 11. After reference-point generation by the CPP algorithm, all the controllers implemented in this study are tested over the area.

In the following section, the training and the testing results of the compared controllers are given, and it is shown that the best performance results are obtained by the IT2-FPID controller.

5. Real-Time Experimental Results

In this section, the performance of the IT2-FPID controller is compared with the PID and the T1-FPID controllers by the data gathered from real-time experiments. All experiments are in two parts: without/with disturbance and payload on the set-point tracking. The tuned parameters without payload using the BB-BC algorithm are employed in the UAV. After that, the set-point tracking shows all the results of the compared controllers under the flexible cable-connected payload and disturbance effects. Comparison results for these controllers are shown as root mean square error (RMSE) values. The sampling time ($T_s$) is chosen as 055 ms. The selected optimal value of $T_s$ and the UAV IP address are communicated via Wi-Fi from the computer to the UAV. The latencies are taken into consideration, and these have a minimal impact on the real-time algorithm performance. The performance of the proposed approach can be observed via the video file provided as Supplementary Material.

The implemented controllers are tested for three cases. First, the UAV tracks the set point without any disturbance and the payload. Secondly, the controllers try to stabilise the system under disturbance and the payload. Additionally, the square reference points are given to the UAV. The last case is giving the reference path for the UAV by the CPP algorithm. In the CPP scenario, any external disturbance is not applied because when UAV follows the reference trajectory, the flexible cable-connected payload may generate disturbances on both axes due to the fact that the payload is not connected to the CoM of the UAV.

Disturbing the payload aims to investigate the controller responses under disturbance. In the last test, CPP aims to compare the controller performances for minimum time and maximum accuracy. Disturbing the payload and the CPP results show the robustness of the controller in real-time.

5.1. Set-Point Tracking without Disturbance and the Payload

Before testing with the unknown payload, all control methods and their control parameters are tested without a connected payload. The reason for this is that the defined controller parameters may need further fine-tuning.

In Figure 12 and Figure 13, the PID, T1-FPID and IT2-FPID position results can be seen. The reference, PID, T1-FPID, and IT2-FPID are coloured black, green, blue, and red, respectively. The starting point of the UAV is selected as $x_0=0.2\mathrm{m},y_0=0.2$ m, and the reference points are chosen as $x_{ref}=1.1\mathrm{m},y_{ref}=0.52$ m.

In Figure 12, the IT2-FPID results show the best performance compared to other controllers regarding settling time, oscillation, steady-state error, and overshoot. PID gives more overshoot and then arrives at coverage of the reference area after 31 s. T1-FPID has a slower response compared with IT2-FPID, but it also reaches the reference faster than PID.

In Figure 13, unlike the results on the x-axis, all the controllers do not achieve good results on the y-axis. Significantly, the PID controller has an overshoot problem and does not reach the reference or converge to the circle. Moreover, IT2-FPID looks faster than T1-FPID, and the system response is the best.

5.2. Set-Point Tracking with Disturbance and the Payload

The results of set-point tracking with disturbance and the payload consist of two different parts. The first relates to payload disturbing, and the second applies to CPP. In this part, all controller structures are tested under disturbance. The payload is disturbed using a stick to implement the disturbances as external disturbances, directly impacting the system stability. After that, in the CPP scenario, the disturbances are generated by themselves as explained in the aforementioned above section.

In Figure 14, the disturbance effects are seen on the x-axis. PID and T1-FPID give slow responses, and generally, they do not reach the reference points. IT2-FPID is not as affected as the others.

In Figure 15, the biggest problem for all controller types is on the y-axis. The PID cannot guarantee stability under disturbance, and T1-FPID and IT2-FPID show a slight overshoot.

After the disturbance-based results, some specific points are given for each controller. The second experiment of the IT2-FPID controller is coverage path planning. Each vertex of the coverage path planning is fixed and determined before the test, and these are given as reference points. The reference point changes to the next vertex when the UAV converges or reaches the current vertex. Considering the task compilation time, IT2-FPID, T1-FPID, and PID complete the task in $74.55$ s, $112.35$ s, and $120.15$ s, respectively. In addition to a faster completion time, IT2-FPID also shows better results in other ways.

The first experiment of the CPP is the PID controller structure for the real-time UAV with a flexible cable-connected payload as shown in Figure 16. PID shows a small steady-state error and slower system response on the x-axis. However, PID has a problem on the y-axis. It does not reach the reference points properly.

In Figure 17, T1-FPID does not show promising results compared with the PID results on the x-axis, but it completes the task time faster than the PID controller; these results are plausible. However, T1-FPID oscillates on the y-axis. This issue can be explained by the fact that the payload is not at the CoM of the UAV but nearer the front side. Therefore, the problem on the y-axis is expected for all of the controller’s structures.

The last and the best results achieved with the IT2-FPID are shown in Figure 18. It completes the CPP task in $74.55$ s. In addition, it has a better settling time, oscillation, and steady-state error in terms of minor errors and short time. Although the payload position problem on CoM exists for the other controllers, IT2-FPID overcomes the problem steadfastly.

The experimental results on each axis are analysed separately. RMSE is chosen as a performance index for the comparison of the controllers. In

Table 4. RMSE values on x-axis.

RMSE Values (m)	PID	T1-FPID	IT2-FPID
Set-Point Tracking	0.2275	0.2255	0.2218
Under Disturbance	0.5574	0.5490	0.4971
CPP Results	0.5719	0.5597	0.4540

, the results of RMSE on the x-axis are shown. The best results of each experiment are IT2-FPID on the x-axis.

As far as the RMSE values obtained in the set-point tracking scenario and in the testing against the disturbance, that is, disturbing the payload with a stick, are considered, IT2-FPID shows almost two times better results when compared to PID and T1-FPID as given in

Table 5. RMSE values on y-axis.

RMSE Values (m)	PID	T1-FPID	IT2-FPID
Set-Point Tracking	0.137	0.0964	0.0805
Under Disturbance	0.3103	0.1877	0.1810
CPP Results	0.1338	0.1298	0.1068

.

In summary, IT2-FPID has the best performance results in all two robustness tests. Considering the aim of the testing for IT2-FPID, the minimum task time, and RMSE values, maximum accuracy is obtained. However, although T1-FPID is not as good as IT2-FPID, it has better results than the compared PID controller. In the simulation, 20 independent tests are conducted, and then 5 independent tests are run in the real-time system.

6. Conclusions and Future Works

In this study, the flexible cable-connected unknown payload, which is a challenging engineering problem, is described. The challenging problems are related to the swinging unknown payloads and point-based reference trajectories. For this challenging engineering problem, a proportional–integral–derivative (PID) controller, type-1 Fuzzy PID controller (T1-FPID), and interval type-2 Fuzzy PID controller (IT2-FPID) are tuned and implemented on a real UAV. The control algorithms are generic and can be applied to different types and sizes of UAVs, although the experiments are performed using a DJI Tello UAV. The performance of the IT2-FPID is also compared against a PID and a T1-FPID controller. After set-point tracking and robustness testing via disturbing the payload with a stick and changing the reference points in coverage path planning (CPP), causing the payload to swing, the experimental results show that the IT2-FPID is better than the controllers. There are different aims for each test, such as minimum task time and RMSE. Disturbing effects of the flexible cable-connected payload on the UAV are shown throughout the experiments. These adverse effects can be eliminated by the three compared controllers to some point, but the results show the superiority of the IT2-FPID. As an example from the experiments, although the PID controller may solve the trajectory-tracking problem, the uncertainties, such as the unknown payload connection via flexible cable and disturbing the payload, lead to highly detrimental results for the PID controller. Thanks to implementing the IT2-FPID controller, these problems are eliminated considerably. Consequently, the system stability is guaranteed without considering the changes in the system dynamics equations and redesigning the controller.

After the controller design, the sensor part of the UAV will be improved to achieve better accuracy. In order to improve the sensor accuracy, the noise distribution will also be investigated, and then the system will be tested both in the simulation environment and in real time. According to the type of distribution, a filter design will be included to eliminate such noise and general disturbances. Lastly, a theoretical analysis of the performance of the proposed approach under internal and external disturbances is a topic of future work.