Iterative Learning Sliding Mode Control for UAV Trajectory Tracking

This paper presents a novel iterative learning sliding mode controller (ILSMC) with 1 application to trajectory tracking of quadrotor unmanned aerial vehicles (UAVs) subject to model 2 uncertainties and external disturbances. Here, the proposed ILSMC is integrated in the outer loop 3 of a controlled system. The control development, conducted in the discrete-time domain, does 4 not require a priori information of the disturbance bound as with conventional SMC techniques. 5 It involves only an equivalent control term for the desired dynamics in the closed-loop and an 6 iterative learning term to drive the system state toward the sliding surface to maintain robust 7 performance. By learning from previous iterations, the ILSMC can yield very good tracking 8 performance when a sliding mode is induced without control chattering. The design is then 9 applied to the attitude control of a 3DR Solo UAV with a built-in PID controller. Simulation results 10 and experimental validation with real-time data demonstrate the advantages of the proposed 11 control scheme over existing techniques. 12


Introduction
In recent years, the interest in developing and utilizing unmanned aerial vehicles (UAVs) has been growing, with numerous applications in practice, such as mapping [1,2], inspection, search and rescue [3,4], construction automation [5], and agricultural surveillance [6]. When a quadrotor drone performs a desired trajectory, accurate tracking is highly necessary. In trajectory tracking control, feedback linearization (FL) has been widely used [7,8]. This control method works well under the assumption of known system dynamics. When faced with large uncertainties and disturbances, FL-based approaches may lead to poor tracking performance, and other advanced control laws are required. An adaptive feedback linearization controller was applied in [9], allowing for adjustments of the control parameters to enhance control performance. Robust control methods were also developed to improve control performance [10]. A backstepping controller was introduced in [11] to improve the tracking accuracy and robustness of UAVs' attitude control, wherein the external disturbances are estimated using a nonlinear disturbance observer.
Sliding mode control (SMC), a well-known control method for improving system robustness, has been successfully applied to various control systems [12,13] in general and particularly to UAVs [14,15]. However, information on disturbance bounds is required in these techniques. Adaptive SMC was developed to overcome this requirement [16]. This approach still reveals the main disadvantage of SMC, i.e., control signals usually present a chattering behavior, especially when dealing with large uncertainties and disturbances, which often require excessively high control gains. Various techniques have been suggested as a remedy, mostly using an approximation of the sign function to avoid or reduce chattering with some trade-offs with system robustness for control signal smoothing. Deep-learning-based techniques with convolutional neural networks have recently shown promise in learning systems that are subject to highly complex and time-varying uncertainties, for example, in vision-based applications such as in robust face tracking [17]. For control in robotic manipulators, a deep convolutional neural network was developed to integrate learning schemes into a fractional-order terminal sliding-mode controller for enhancing tracking accuracy [18]. Although the deep learning methods can achieve highquality performance with continuous control signals, a major disadvantage is the high computational cost incurred for implementing a deep neural network.
Iterative learning control (ILC) is an effective technique for dealing with repetitive tasks. It allows for learning from system data to update the control input repeatedly to improve system performance [19]. Through trial-based learning, ILC is able to achieve highly accurate tracking performance even with large model uncertainties and disturbances. Unlike nonlearning control techniques, the system in ILC is reset to zero after the system reaches the final time, and then repeatedly follows the same reference again. Thereby, the control input can be adjusted through the repetitions to result in perfect tracking. Since ILC can learn from the system response to provide feedforward control in the iteration domain, it is more robust and can effectively compensate for model uncertainties and unknown disturbances, particularly iteration-invariant disturbances [20].
The application of iterative cybernetics, first proposed in [21], has emerged for iterative learning control in robotic systems [22], later developed for industrial control [23]. In the past decades, ILC has become an effective tool in various control systems, such as robot arm manipulators [24], chemical batch processes [25], wafer scanner systems [26], and video-rate atomic force microscopy [27]. Unlike other learning techniques such as artificial neural networks, which obtain the inverse dynamics from a training set [28] or adaptive controller, which tunes the control parameters [29,30], requiring a time-consuming process, ILC updates the control input from the information from previous executions and can hence converge quickly after a limited number of repetitions [19]. Moreover, as ILC does not require a system model, it is quite beneficial in practical applications that deal with unknown characteristics.
In this paper, integrating the learning capacity of ILC with the strong robustness of SMC, we propose a novel iterative learning sliding mode controller (ILSMC) to achieve high-accuracy trajectory tracking for UAVs while retaining strong robustness as well as alleviating control chattering. In terms of iterative control, several existing techniques have been introduced for UAVs. In [31], a plug-in controller was designed and implemented in aerial robots. Although the average tracking error is reduced for periodic reference trajectories, the technique does not concern the effect of disturbances. In [32], fuzzy PIDtyped ILC was introduced; where fuzzy logic is used to tune the control parameters, high-accuracy tracking performance is hard to attain in comparison to other rigorous control strategies. In [33], optimization-based ILC was developed to improve the UAV trajectory tracking performance. In this approach, learning and filtering schemes are formulated into convex optimal problems. Although the two-step convex optimization problem can be solved using software, it involves high computational complexity.
The proposed ILSMC offers a simpler design, and thus is more robust and effective. The main contributions of this work include (i) the comprehensive development of an iterative learning term with fast convergence after several iterations to compensate for system uncertainties and unknown disturbances, and (ii) the integration of ILC and SMC schemes to a built-in PID controller in cascade to yield high performance for quadcopter trajectory tracking. This paper is structured as follows: The control development of ILSMC is presented in Section 2. The convergence and stability analysis of the proposed learning algorithm is also provided in this section. Next, Section 3 presents system modeling, including kinematic and quadcopter dynamics. Then, the integration of ILSMC with PID control for the UAV is described in Section 4. Section 5 provides numerical simulation results, and Section 6 presents experimental validation with real-time data. Finally, a conclusion is drawn in Section 7.

Iterative Learning Sliding Mode Control
Iterative learning control (ILC) is a tracking control strategy for systems performing repetitive tasks, which are commonly required in industry. This technique aims to generate a feedforward control signal so that the system can learn from the previous responses to improve tracking performance and repeatedly eliminate disturbance after each iteration. The basic structure of an ILC is depicted in Figure 1 for an iteration number j. At this iteration, the input u (j) (k) and the deviation e (j) (k) between the reference y d (k) and the output y (j) (k) are stored to compute the control signal for the next iteration, with k starting from an initial time instant (k = 0). In this section, an iterative learning sliding mode control scheme is designed to deal with large uncertainties and disturbances.

ILSMC Design
Consider the following general discrete-time control system: where k is the time instant and ∆ t is the sampling period. The subscript j denotes the iteration index, also called trial, run, cycle, or repetition in the ILC literature. The system ∈ R m is the control signal, B ∈ R m×m is a positive definite matrix, and f (.) is a vector function. The influence of parameter variations and loading conditions, model uncertainties and external disturbances can be lumped into a vector d (j) (k). In each iteration, the input and state variables comprise an N-sample sequence each, where N is a finite number of samples.

Definition 1.
At iteration j, an exogenous input δ (j) (k) is called iteration-invariant if it occurs repeatedly over iterations, or persistent within the iteration domain, that is, δ (1) To proceed with the ILC methodology, the following assumption [20,27,34] is made: In this paper, the lumped disturbance d (j) (k) is assumed to be iteration-invariant.
In this paper, an ILSMC law is developed aiming to drive the tracking error asymptotically to zero from any initial condition and under external disturbances d (j) (k). The control algorithm consists of two steps. The first step is to induce a desired sliding surface to drive a learning sliding function to zero after some iterations regardless of external disturbance and system uncertainties. In the second step, the tracking error of the system is driven to zero in the sliding mode associated with the control sliding function. Instead of using a discontinuous gain as in the conventional SMC methodology, here, an iteration learning process is involved; hence, a priori information of the disturbance bound is not required, and chattering is fully alleviated.
The control design is initiated by considering the tracking error in an iteration as where T is the desired trajectory vector, which is also iterationinvariant during the execution of repetitive tasks. Let us define the control sliding function as: we have from (3): Substituting (1), (3), and (5) into (4) yields: By using the forward Euler method for discretization, we also have Applying (7) into (6) produces:

Equivalent Control
Now let us consider the following dynamics to be induced by the learning process: where µ > 0 is a control parameter. Substituting (8) into (9) yields: In nominal conditions of the system under no model error or disturbance, the equivalent control of the system is described by:

Learning Control
In the learning step, to drive the system trajectories toward the sliding surface (9) regardless of disturbances, an iterative learning scheme is introduced using the stored data from previous iterations as: where the initial iterations u ilc (0) (k) = 0 and λ > 0 are a design parameter for the learning rate.
From (11) and (12), the ILSMC law is described by: We summarize the ILSMC design in the following theorem: , under the iterative learning sliding mode control (13) comprising the equivalent control (11) and learning control (12), if the control parameter µ and learning rate λ are respectively chosen such that 0 < µ < 2/∆ t , 0 < λ < 2 and λ = 1, then the tracking error (2) is driven to zero at a sufficiently large number of iterations and the control system is asymptotically stable.
Proof. By substituting (8) and (11)-(13) into (9), we obtain: Similarly, According to the Assumption, as d (j) (k) is iteration-invariant, from (14) and (15), we have: From (16), the iterative learning algorithm converges to 0 at large values of the iteration number under the condition |1 − λ| < 1. Therefore, if the learning rate λ is selected to satisfy 0 < λ < 2 and λ = 1, we have By substituting ∆ σ (j) (k) into S (j) (k), Equation (9) can be rewritten as whereby S (j) (k) → 0, with a proper selection of the learning rate λ and at an adequate number of iterations j, the sliding function (3) becomes: where σ (j) (0) is the initial value of σ (j) (k) at the jth iteration. Therefore, given a positive constant µ with 0 < µ < 2/∆ t , the sliding function σ (j) (k) in (19) approaches zero at a sufficiently large value of k. Thus, since the sliding function σ (j) (k) as defined in (3) is driven to zero after some iterations j, a sliding mode is induced from the selection of parameter c > 0. It follows that lim Notably, the asymptotic convergence of the tracking error e (j) (k) here does not come from the switching of the control signal with a high discontinuous gain as in conventional SMC, but is a result of the proposed learning process (12). Hence, the sliding mode (3) induced for the tracking error can retain system robustness in the face of uncertainties and disturbances while avoiding the high-frequency switching of the control signal. This is the reason why the proposed ILSMC can achieve highly accurate tracking without control chattering. The tracking performance then depends on the convergence of the learning process, governed by the learning rate λ.
To verify the system stability, let us consider the control sliding function σ (j) (k) at iteration j. According to [35], the discrete-time control system will be asymptotically stable if, for all its entries [σ (j) (k)]: where Sign(·) is the signum function. To verify the above conditions, from (18), we have, We obtain, accordingly Therefore, the first condition of (21) is satisfactory as From (22), we also have and with the choice 0 < µ < 2/∆ t , the second condition of (21) is also satisfactory since Therefore, the control system is asymptotically stable. The proof is completed.

Remark 1.
From (16), to quickly induce a sliding surface, a high rate of convergence is required, subject to the condition |1 − λ| < 1. This condition is similar to the ILC convergence condition presented in the frequency domain [36]. On one hand, the closer λ is to 1, the faster the convergence in the learning step. On the other hand, under the effect of noise and nonrepeating disturbances, a rapid learning rate could affect robustness. In practice, one can choose λ as close to 1 for fast convergence and gradually lower this value if required to reduce the system sensitivity.

Remark 2.
The learning process can be terminated upon satisfaction of a required tracking performance index (TPI), e.g., when the integral time absolute error (ITAE) of the control error satisfies the requirement for tracking performance for a specific task of the system.

System Description and Modeling
The UAV employed in this study was a quadcopter with a symmetric rigid structure and driven by four motors, as shown in Figure 2. For the quadcopter, the pitch angle, varied in accordance with the quadcopter's longitudinal motion, is controlled by adjusting the front and rear propellers' velocities, which generate the force F 1 and F 3 . Meanwhile, its lateral displacement is governed by the roll angle, which is controlled through the right and left rotors' speeds, resulting in the forces F 2 and F 4 . Finally, the yaw angle, associated with the UAV yaw motion, is regulated by the difference between torques generated by these pairs of rotors. In this work, we focused on the attitude tracking control, and thus, only the quadcopter orientation is considered here. The torques acting on the quadcopter include the thrust forces τ, the gyroscopic torques caused by the rotation of the quadcopter's rigid body τ b and of four propellers τ p , as well as the torque due to aerodynamic friction τ a . Here, the propellers' gyroscopic effects and the drag from air resistance are considered external disturbances.

Kinematics
As shown in the configuration in Figure 2, an earth frame, {x e , y e , z e }, is fixed at the ground and a body frame, {x b , y b , z b }, is attached to the CoG of the quadcopter, both with the z axis pointing downward. The position of the UAV's mass center in the earth frame is defined by a vector P = (x, y, z) T . The UAV orientation is represented by angles (φ, θ, ψ) T , corresponding to roll, pitch, and yaw motion, respectively. For attitude control, these angles are limited as φ ∈ [−π/2, π/2], θ ∈ [−π/2, π/2] and ψ ∈ [−π, π]. With respect to the earth frame, the orientation of the quadcopter is obtained a rotation transformation resulting from successively rotating around the x b , y b and z b axes, and characterized by an orthonormal rotation matrix R [37]: where s x and c x denote sin x and cos x, respectively. Denoting the angular velocity vector of the quadcopter in the body frame as (ω φ ω θ ω ψ ) T , the rotational kinematics can be obtained as follows [38]: where

Quadcopter Dynamics
From the quadcopter description, the components of torque vector τ = [τ φ τ θ τ ψ ] T , corresponding to rotation in the roll, pitch, and yaw directions, respectively, are calculated as where l is the distance from each rotor to the CoG, and β is the apparent radius for converting the force into the yaw torque. From (29)-(31), the control inputs are described as: where u φ , u θ , and u ψ are the roll, pitch, and yaw torques, respectively; F = ∑ 4 n=1 F n is the lift force, representing the total thrust utilizing from the four motors. As the only attitude of the quadcopter is controlled, u z is assumed to balance with gravity.
The gyroscopic torque due to the rotation of the symmetric body of the quadcopter is described by [29]: where is a skew-symmetric matrix. As shown in the configuration in Figure 2 with the body frame assigned to the quadcopter, given a mass point m i with its coordinates (x i , y i , z i ) in the body, the quadcopter's inertia can be obtained as a diagonal matrix: Accordingly, Equation (33) can be rewritten as The gyroscopic torque due to the rotation of four propellers is determined as [29]: where I r is the moment of inertia of the rotor of each motor, and ω r = −ω r1 + ω r2 − ω r3 + ω r4 is the residual angular velocity, in which ω r1 , . . . , ω r4 are the corresponding angular velocities of the propellers. The air drag torque is calculated as [29]: where k ax , k ay , and k az are aerodynamic friction factors. The dynamics of the quadcopter in attitude control can then be represented as: Now, if the propeller gyroscopic and aerodynamic torques are considered as external disturbances, i.e., where d φ , d θ , and d ψ are disturbance components, then substituting (32), (35), and (39) into (38) yields:ω To express the quadcopter dynamics via the orientation angles, the model can be simplified by considering [ω φ , ω θ , ω ψ ] ≈ [φ,θ,ψ]. This approximation is acceptable since a minor model error can be adequately addressed by a good controller. Accordingly, the quadcopter model is obtained as:

Discrete-Time Model
In the discrete-time domain, by considering the difference approximation for first and second derivatives using the forward Euler method, the transformed discrete-time model can be obtained as below: Now, consider the system state vector x(k) = [x 1 (k) x 2 (k)] T defined by: From (46)-(50), we obtain the UAV state equation in discrete time of the form (1) as: where f (x(k)) = x 2 (k) + ∆ t I −1 τ b (k) and B = I −1 .

Integrated ILSMC for UAV Attitude Control
The proposed ILSMC was then applied to the outer loop of a quadcopter with a built-in PID controller in the inner loop for flight control. Here, the ILSMC is integrated in cascade control to improve the performance of the UAV trajectory tracking in dealing with noise, nonrepeating uncertainties, and disturbances. Figure 3 shows the block diagram of the proposed controller wherein the reference signal of a feedback controller is generated by the ILSMC signalû (j) (k) at a time instant k.

Inner-Loop PID Controller
As shown in Figure 3, the output of the quadcopter PID controller is computed as where • denotes the elementwise Hadamard product; are PID control parameters. The error of the PID feedback loop whereû (j) (k) is the ILSMC control signal. Substituting (54) into (53) yields: where

Outer-Loop ILSMC
As mentioned above, the proposed ILSMC was then added to the predesigned PID controller for improving the UAV tracking performance. Substituting (55) into (10) yields: The equivalent control of the outer loop is then given by: In the learning step, the iterative learning term is computed aŝ where S (j−1) (k) is obtained from the learning process at a previous iteration (j − 1) as per (15). This finally leads to the integrated iterative learning sliding mode control law (13) for the quadcopter:û (j) (k) =û eq (j) (k) +û ilc (j) (k). (61)

Implementation Procedure
In summary, a step-by-step procedure to implement the proposed control scheme is summarized as:

Simulation Results
This section provides the simulation results of the proposed ILSMC design. The parameters used for simulation were obtained from the 3DR Solo drone [29], as listed in Table 1. The selected control parameters are described in Table 2. Here, in the learning process, a suitable value for λ was chosen to obtain a fast convergence rate, so S (j) (k) was driven to zero quickly. In the control phase, coefficients c φ , c θ , c ψ ,and µ were chosen by the desired error dynamics described in (3). Initially, the iterative learning control signal was set to zero, u ilc (0) (k) = 0, and then updated after each iteration. To evaluate performance of the proposed controller, we compared it with other available techniques including the PD feedback controller, PD-type ILC [19], adaptive twisting sliding mode controller (ATSMC) [29], and adaptive finite-time control scheme (AFTC) [30].  c ψ 20 --

Step Response in Nominal Conditions
In this section, the performance of the proposed controller is evaluated via the step responses in nominal conditions where disturbances and uncertainties were set to zero. The desired reference attitude angles were set to φ d = −20 • , θ d = 20 • , and ψ d = 60 • at 1 s. The simulation results of the step responses and control signals are shown in Figure 4, in which the black step signal is the desired angle, and responses of ATSMC, AFTC, PD, PD-ILC, and the proposed ILSMC controllers are depicted in cyan, green, magenta, blue, and red, respectively. It can be seen from Figure 4 for all three orientation angles that while ATSMC and AFTC provide some oscillations in the control, and PD presents a slow response, both iterative-learning-based techniques, ILSMC and PD-ILC, exhibit fast responses with zero steady-state error. PDILC, however, incurs a large overshoot, whereas ILSMC is able to maintain the desired dynamics without overshoot owing to the merits of sliding mode control. Notably, the fast response of ILSMC in comparison to ATSMC, AFTC, PD, ILC, and PD is attributed to the choice of c = diag(c φ , c θ , c ψ ) and µ. A faster transient response, however, requires more control efforts that may exceed the physical limits imposed by the motors and power supply for the drone. Moreover, the proposed controller is chattering-free in the steady state. (a)

Trajectory Tracking Performance under Disturbances and Uncertainties
To evaluate the tracking performance of ILSMC with the presence of uncertainties and disturbances caused by load variations, the reference attitude angles in this simulation were set, in degrees, as below: For performance evaluation, the system was injected with a disturbance at t = 10 s whose components were: Considering 20% loading conditions, the model uncertainties were introduced by setting: whereÎ xx ,Î yy , andÎ zz are the estimation of I xx , I yy , and I zz , respectively. Figure 5 shows the tracking performance of the attitude angles, while Table 3 presents the TPI, for which the integral time absolute error (ITAE) is adopted here for all angles. It can be seen that the PD controller cannot cope with disturbances with large tracking errors at t = 10 s and high values of ITAE. As with PDILC, it can suppress disturbances but suffers from control overshoot and a noticeable error. Both the ATSMC and AFTC techniques present relatively accurate tracking performance with a small ITAE, between 1.40 and 5.03. The proposed ILSMC presents a relatively large tracking error at the first iteration (since u ilc (0) = 0), but owing to the learning ability, the tracking error decreases with increasing iterations by updating the iterative learning control term after each iteration. As the tracking performance is improved significantly, at the last iteration, ILSMC has the smallest ITAE among the considered techniques. Additionally, the absolute error is also the smallest, almost zero in steady state, as shown in the magnified figure, demonstrating the advantage of the proposed ILSMC.   Figure 6 shows the control efforts in the presence of disturbances and uncertainties. It can be seen that its magnitude increases after 10 s, which implies that more energy is required to compensate for the external disturbances. More importantly, the control efforts of ILSMC display oscillation only in the transient state, but no chattering in the steady state, which is beneficial for practical implementation. To evaluate the effect of the proposed learning mechanism, the ITAE values were computed for ILSMC after each iteration with different values of λ. The results up to 15 iterations are presented in Figure 7. They indicate that the ITAE of the all three attitude angle errors quickly decreases and converges to zero after several iterations. To induce a fast system sliding mode, a higher rate of convergence must be selected. It can be seen in Figure 7 that this can be obtained when λ is close to one. In this work, λ = 0.9 was chosen to achieve the desired control performance.

Experimental Validation
In this section, we evaluate the performance of the combined ILSMC and PID control algorithm in the trajectory tracking problem for our UAV testbed, in which a built-in PID was already employed.

Experimental Setup
The setup for the experiments is shown in Figure 8, using a 3DR Solo drone with its parameters described in Table 1 [39]. It consists of two Cortex M4 168 MHz processors used for low-level control and one ARM Cortex A9 processor used for running the Arducopter flight operating system. The drone is equipped with a camera, a laser scanner, and environmental sensors for data acquisition. During the experiments, communication data, including control reference signals and drone sensor outputs, were transmitted to the ground control station via the local network established by the drone system. Mission Planner software was connected to the network to upload the flight plan to the drone and log flight data for analysis. In the experiments, the PID gains were set to their default values implemented in the 3DR Solo. From the desired and actual roll, pitch, and yaw angles, the tracking error was computed.

Real-Time Data Validation Results
The steps for conducting the experiments to validate the trajectory tracking performance of the proposed ILSMC were as follows: First, a trajectory was predefined with a starting point being set at the home position of the drone in an absolute frame of reference, as depicted in Figure 9. After that, the longitude, latitude, and altitude of the waypoints forming the trajectory were imported into Mission Planner, as depicted in Figure 10. Next, those waypoints were uploaded to the 3DR Solo to fly automatically, as shown in Figure 11. Then, the reference and actual attitude angles were logged by Mission Planner, as shown in Figure 12, for comparison. The errors between those angles were used to update the iterative learning term. Finally, the trajectories of the 3DR Solo drone obtained by using the built-in PID controller were compared with the results obtained using ATSMC, AFTC, PD-ILC, and the proposed ILSMC.  The comparison was performed by setting the references obtained from the 3DR Solo drone under similar control settings as in the simulation. Figure 13 shows typical comparison results for the UAV roll, pitch, and yaw responses. It can be seen that the deviation between the reference and the actual roll angle controlled by the built-in PID is relatively high due to disturbances. Advanced techniques can improve the tracking performance in which ILSMC remains the best, as indicated by its smallest tracking error, as can be seen clearly in the insets in the figure. The results obtained confirm the validity and efficiency of the proposed approach.    From the real-time experiments, the recorded control efforts are shown in Figure 14, where the steady-state yaw torque is a constant as the quadcopter was controlled to lift up with a linearly increasing height while performing a circular trajectory during the test.

Conclusions
We proposed an effective control technique called ILSMC to address the tracking control problem experienced by quadcopters when subject to disturbances and uncertainties. The control signal consists of an equivalent term to control the system states within the desired sliding surface, and an iterative learning term to drive the system states toward the sliding surface and then remain in the sliding surface despite the presence of uncertainties and disturbances. The iterative learning signal is updated following some iterations to improve the tracking performance by using the data acquired from previous iterations. The simulation results showed, in the case of disturbances and uncertainties, that the iterative learning sliding mode controller presented the smallest tracking errors compared to some other existing control techniques used for quadcopter control. For UAVs with built-in PID controllers, the proposed control scheme can be integrated in a cascade structure to improve the trajectory tracking accuracy and robustness. Field tests were performed, and validation with real-time experimental data was conducted to confirm the advantages of the proposed approach. Our future work will focus on extending the learning mechanism to enable the control of multiple UAVs for real-time formation.