Quadcopter Trajectory Tracking Based on Model Predictive Path Integral Control and Neural Network

Abstract

This paper aims to address the trajectory tracking problem of quadrotors under complex dynamic environments and significant fluctuations in system states. An adaptive trajectory tracking control method is proposed based on an improved Model Predictive Path Integral (MPPI) controller and a Multilayer Perceptron (MLP) neural network. The technique enhances control accuracy and robustness by adjusting control inputs in real time. The Multilayer Perceptron neural network can learn the dynamics of a quadrotor by its state parameter and then the Multilayer Perceptron sends the model to the Model Predictive Path Integral controller. The Model Predictive Path Integral controller uses the model to control the quadcopter following the desired trajectory. Experimental data show that the improved Model Predictive Path Integral–Multilayer Perceptron method reduces the trajectory tracking error by 23.7%, 34.7%, and 10.3% compared to the traditional Model Predictive Path Integral, MPC with MLP, and a two-layer network, respectively. These results demonstrate the potential application of the method in complex environments.

1. Introduction

Recent research on quadcopters has established them as a significant platform for the development of unmanned aerial vehicles (UAVs) owing to their simple construction and maintenance. These quadcopters demonstrate exceptional maneuverability and are capable of hovering, taking off, flying, and landing in confined spaces, owing to their vertical takeoff and landing (VTOL) capability [1]. A quadcopter comprises four rotors positioned at the corners of a cross-shaped frame, and it is controlled by adjusting the rotational speeds of these rotors [2,3]. Quadcopters are employed in various applications such as surveillance, search-and-rescue missions, and hazardous operations in congested and Global Positioning System (GPS)-denied settings, such as dense forests, crowded offices, corridors, and warehouses. To perform these tasks effectively, quadcopters must have robust navigation abilities and a thorough understanding of their operational environment. Substantial research work examined various aspects of quadcopter dynamics and control, motion planning, and trajectory generation in unstructured environments [4,5,6,7,8].

Sampling-based optimal control methods have attracted considerable interest in path planning and trajectory tracking applications. These control techniques function by generating multiple trajectories at each control step through input sampling (referred to as rollouts) and subsequently determining a sub-optimal sequence of control inputs based on the sampled trajectories. This process involves executing a finite time-horizon optimization, utilizing the optimal input for the initial time step and then iteratively repeating the optimization for subsequent time steps. The path integral optimal control framework [9,10,11] provides a mathematical foundation for developing optimal control algorithms through stochastic trajectory sampling. The main principle is based on the transformation of the value function for the optimal control problem using the Feynman–Kac lemma [12,13] into an expectation over all possible trajectories, referred to as a path integral. This transformation enables the resolution of stochastic optimal control problems by a Monte Carlo approximation employing a forward sampling of stochastic diffusion processes. Furthermore, the utilization of path integral control theory has gained prominence in recent research.

Model Predictive Control (MPC) has been used on various robotic systems, such as self-driving cars [14] and quadcopters [15], and it is widely employed in industrial applications [16]. The path integral control framework can be integrated with the Model Predictive Control paradigm. In this scenario, an open-loop control sequence is continuously optimized in the background, whereas the system finds the current optimal estimate of the controller in parallel. One big problem with this approach is that it needs to sample a lot of different trajectories simultaneously, which is challenging when the systems being considered have complex dynamics. A possible solution to this issue involves simplifying the system model through a hierarchical scheme [17]. This employs path integral control to create trajectories for a simplified point mass model that a low-level controller can then follow. Model Predictive Path Integral (MPPI) control was introduced in [18], displaying its ability in aggressive autonomous driving. It explored information-theoretic dualities between free energy and relative entropy, which led to the formation of the Model Predictive Path Integral control logic. Williams developed the iterative path integral method, known as the Model Predictive Path Integral control framework [16]. That method iteratively modified the control sequence to find optimal solutions based on the importance of sampling trajectories. In [19], an alternative iterative method was proposed that removed the control and noise affine constraints inherent in the original Model Predictive Path Integral framework, which could accommodate non-affine dynamics. The central idea behind that extension was to use the information-theoretic interpretation of optimal control through the Kullback–Leibler (KL) divergence and free energy. It was different from the original method, which used a linearization of the Hamilton–Jacobi–Bellman (HJB) equation and applied the Feynman–Kac lemma. Despite these methods coming from different derivations, they were practically equivalent and theoretically linked. A learning model to generate informed sampling distributions was presented in [20], which was an extension of Williams’s information-theoretic-based method. In [15], the generation of time-optimal trajectories through multiple waypoints for quadrotors was investigated.

The MPPI controller has been mainly utilized in aggressive driving and UAV navigation in congested conditions. However, UAV navigation in natural conditions often involves incomplete state information. In environments such as forests and urban canyons, the navigation system cannot directly measure all state variables due to sensor limitations or occlusion problems [1,6]. Although most control schemes depend on the accuracy of the underlying models, specific adaptive control, sliding mode control, and robust control methods can remain effective despite model inaccuracies [21,22,23]. A comprehensive analysis of advanced aerodynamic effects controlling quadrotor flight, such as blade flapping and airflow dynamics, is conducted in [24]. These effects are fundamentally complicated to model, posing considerable challenges in controller design. The standard Model Predictive Path Integral approach may necessitate substantial computational resources or manual parameter adjustment to maintain control performance during dynamic environmental variations or severe system state fluctuations. Therefore, relying exclusively on the Model Predictive Path Integral control mechanism is inadequate for responding to modern complicated missions. Incorporating an adaptive mechanism into the Model Predictive Path Integral framework allows for the dynamic modification of essential parameters according to system states, and control performance enhance the robustness and efficiency of the control strategy. Moreover, to overcome modeling problems, data-driven and learning-based control strategies have been proposed [25]. A promising approach employs neural networks (NNs) to model system dynamics. Neural networks are acknowledged as universal function approximators, capable of modeling highly nonlinear functions and latent states directly from observed data, which may be challenging to model otherwise explicitly [26]. The study in [27] demonstrates that simple feedforward networks can proficiently learn and generalize system dynamics with high accuracy.

This paper proposes an adaptive control method that integrates the enhanced Model Predictive Path Integral (MPPI) method and a Multilayer Perceptron (MLP) neural network, which can dynamically modify essential parameters according to system states or control performance. In experiments, the method achieves a root-mean-square error (RMSE) in trajectory tracking that is more than 10% lower than alternative methods, surpassing the MPPI control without learning dynamics and the Model Predictive Path Integral method employed with a two-layer neural network. The paper is organized as follows: Section 2 explains the dynamic model of quadcopter; Section 3 presents the training and validation of the MLP; and Section 4 proposes the adaptive Model Predictive Path Integral approach, which can adjust its sampling based on trajectory error. Section 5 presents a comparison of Model Predictive Path Integral control without learning, utilizing two hidden layers of neural network (NN), to adaptive Model Predictive Path Integral control employing a Multilayer Perceptron. Experimental results prove that the adaptive Model Predictive Path Integral method utilizing a Multilayer Perceptron outperforms alternative methodologies.

2. Material

In this section, we first present the nonlinear dynamic model of the quadcopter to be used and the associated model uncertainties and external disturbances.

A general discrete-time stochastic nonlinear system is of the form [1,9]:

$$x_{t+1}=f\left(x_t,u_t+{\epsilon }_t\right)+{\omega }_t$$

(1)

where $x\in {ℝ}^N$ is the state, and $u\in {ℝ}^M$ is the control input. The term ${\epsilon }_t\sim N\left(0,{\sigma }_{\epsilon }^2\right)$ represents a noise disturbance with a normal distribution in the control signals generated in the lower-level controllers for real-world robot systems. The term ${\omega }_t$ represents the combined errors due to inaccurate model representations and purely external disturbances, such as wind in the case of quadcopters.

Quadcopter Model

Figure 1 shows the quadcopter model, with an inertial reference frame defined as $W$ and a body-fixed frame defined as $\mathrm{B}$, with the origin defined at the center of the quadcopter. The Euler angles to describe the quadrotor rotations in $W$ about the body-fixed axes $x_{\mathrm{B}}$, $y_{\mathrm{B}}$, and $z_{\mathrm{B}}$, are roll $\varphi$, pitch $\theta$, and yaw $\psi$, respectively.

The rotation from $W$ to $\mathrm{B}$ with a ZYX transformation is expressed as:

$${}^{W}R_{\mathrm{B}}=\left[\begin{array}{ccc}{c}_{\psi }{c}_{\theta }& c_{\psi }{s}_{\varphi }{s}_{\theta }-c_{\varphi }{s}_{\psi }& s_{\psi }{s}_{\varphi }+c_{\varphi }{c}_{\psi }{s}_{\theta }\\ c_{\theta }{s}_{\psi }& c_{\varphi }{c}_{\psi }+s_{\psi }{s}_{\varphi }{s}_{\theta }& c_{\varphi }{s}_{\psi }{s}_{\theta }-c_{\psi }{s}_{\varphi }\\ -s_{\theta }& c_{\theta }{s}_{\varphi }& c_{\varphi }{c}_{\theta }\end{array}\right]$$

(2)

where $s_x=\sin \left(x\right),c_x=\cos \left(x\right)$ for $\forall x\in \left\{\varphi ,\theta ,\psi \right\}$.

The body angular velocity $\Omega$ represents the angular velocity vector relative to the body frame and is defined as:

$$\Omega =\left[\begin{array}{c}p\\ q\\ r\end{array}\right]$$

(3)

where $p$ is the rotation rate around the $x_B$ axis; $q$ is the rotation rate around the $y_B$ axis; $r$ is the rotation rate around the $z_B$ axis.

The Euler angles $\left(\varphi ,\theta ,\psi \right)$ define the orientation of the quadrotor by rotating from the inertial frame to the body frame. The relationship between the body angular velocity $\Omega$ and the Euler angle derivatives $\left(\dot{\varphi },\dot{\theta },\dot{\psi }\right)$ is given by:

$$\Omega =K\dot{\Phi }$$

(4)

where $\dot{\Phi }=\left[\begin{array}{c}\dot{\varphi }\\ \dot{\theta }\\ \dot{\psi }\end{array}\right]$.

To derive the transformation matrix $K$, we decompose the rotation in the inertial frame. After applying the sequence of rotations, we can express $\Omega$ in terms of $\dot{\varphi },\dot{\theta },\dot{\psi }$ as:

$$\Omega =\left[\begin{array}{ccc}1& 0& -\sin \left(\theta \right)\\ 0& \cos \left(\varphi \right)& -\sin \left(\varphi \right)\cos \left(\theta \right)\\ 0& -\sin \left(\varphi \right)& \cos \left(\varphi \right)\cos \left(\theta \right)\end{array}\right]\left[\begin{array}{c}\dot{\varphi }\\ \dot{\theta }\\ \dot{\psi }\end{array}\right]$$

(5)

Taking the inverse, we can express the Euler angle rates as:

$$\dot{\Phi }=K^{-1}\Omega$$

(6)

where $K^{-1}=T$ is the required transformation matrix.

$$T=\left[\begin{array}{ccc}1& s_{\varphi }{t}_{\theta }& c_{\varphi }{t}_{\theta }\\ 0& c_{\varphi }& -s_{\varphi }\\ 0& {\displaystyle \frac{s_{\varphi }}{c_{\theta }}}& {\displaystyle \frac{c_{\varphi }}{c_{\theta }}}\end{array}\right]$$

(7)

The linear motion equations describe the translational motion in space. Using Newton’s second law, we have:

$$m\ddot{r}=F_{total}$$

(8)

where m is the mass of the quadrotor; $r$ is the position vector of the quadrotor; $F_{total}$ is the total force acting on the quadrotor.

The total force consists of two components: the gravitational force $-mg{e}_3$, where $e_3={\left[0,0,1\right]}^T$ is the unit vector pointing upwards, and the thrust force $F={}^{W}R_{\mathrm{B}}F{e}_3$, where the thrust is always directed along the $z_B$-axis of the body frame. The rotation matrix ${}^{W}R_{\mathrm{B}}$ converts that thrust into the inertial frame. The dynamics of the quadcopter are given by:

$$\dot{x}=v$$

(9)

$$m\dot{v}=-mg{e}_3+{}^{W}R_{B}F{e}_3+d_F$$

(10)

$$\dot{\Phi }=T^{-1}\Omega$$

(11)

$$J\dot{\Omega }=\Gamma +d_{\Gamma }-\Omega \times J\Omega$$

(12)

where m is the mass of the quadcopter, $J$ is the inertia matrix expressed in $\mathrm{B}$, $x={\left[x,y,z\right]}^T$ is the position vector, $v={\left[v_x,v_y,v_z\right]}^T$ is the velocity vector, $\Phi ={\left[\varphi ,\theta ,\psi \right]}^T$ is the Euler angle vector, $\Omega ={\left[p,q,r\right]}^T$ is the body angular rate vector, $e_3={\left[0,0,1\right]}^T\in {ℝ}^3,F\in {ℝ}^{+}$ is the body thrust input generated by the motors, $\Gamma ={\left({\tau }_x,{\tau }_y,{\tau }_z\right)}^T\in {ℝ}^3$ is the torque input to the vehicle in $\mathrm{B}$, and $L\in {ℝ}^{+}$ is the arm length of the quadrotor. The disturbances $\epsilon$ and $\omega$ are explicitly included in the quadrotor dynamics equations, affecting both thrust $F$ and torque $\Gamma$. $d_F$ represents the external force disturbance vector affecting the thrust $F$. $d_{\Gamma }$ represents the external torque disturbance vector affecting the torque $\Gamma$. The state of the system is defined as $x={\left[x,y,z,\varphi ,\theta ,\psi ,v_x,v_y,v_z,p,q,r\right]}^T\in {ℝ}^{12}$. Each rotor produces a vertical force $F_i=k_F{\omega }_i^2$ and moment $M_i=k_M{\omega }_i^2$, where ${\omega }_i$ is the angular velocity of $i^{th}$ rotor. We map the control inputs $F$ and $\Gamma$ to the system inputs $u=\left[{\omega }_1,{\omega }_2,{\omega }_3,{\omega }_4\right]$ as

$$\left[\begin{array}{c}F\\ {\tau }_x\\ {\tau }_y\\ {\tau }_z\end{array}\right]=\left[\begin{array}{cccc}{k}_F& k_F& k_F& k_F\\ 0& k_{F}L& 0& -k_{F}L\\ -k_{F}L& 0& k_{F}L& 0\\ k_M& -k_M& k_M& -k_M\end{array}\right]\left[\begin{array}{c}{\omega }_1^2\\ {\omega }_2^2\\ {\omega }_3^2\\ {\omega }_4^2\end{array}\right]$$

(13)

where $k_F$ is the thrust coefficient, $k_M$ is the moment (torque) coefficient, $L$ is the distance from the rotor to the center of mass.

We deal with the actuator dynamics by introducing a first-order lag in the control inputs. This accounts for the physical limitations and response times of the quadrotor’s actuators, ensuring that rapid changes in control commands do not lead to unrealistic actuator behaviors. Specifically, the actuator dynamics are represented as follows.

The actuator is modeled as a first-order linear system characterized by its time constant $\tau$:

$$\tau \dot{u}+u=u_{cmd}$$

(14)

where $\tau$ is the actuator time constant, $u$ is the actual control input, and $u_{cmd}$ is the commanded control input. Rearranging the equation, we obtain:

$$\dot{u}=-{\displaystyle \frac{1}{\tau }}u+{\displaystyle \frac{1}{\tau }}{u}_{cmd}$$

(15)

This first-order model captures the delayed and smoothed response of the actuators to control commands, reflecting real-world actuator behavior.

By simulating the actuator dynamics, the dataset more accurately reflects real-world quadcopter behavior, including the effects of rotor inertia and drag. In subsequent sections, we simplify the dynamics during control implementation because the neural network has already learned the influence of actuator dynamics. The inclusion of Equation (14) in the data generation process ensures that the learned model accurately captures the effects of actuators on the quadcopter’s overall dynamics.

3. Methods

3.1. Neural Network

This section describes the neural network architecture employed to mimic the quadrotor dynamics. The improvement in computational capacity enables the execution of large-scale batch predictions within minimal timeframes. The adequately trained neural networks can effectively generalize the system’s dynamic equations using supervised learning techniques. Subsequently, employing dynamic equations directly in iterative control schemes has disadvantages, as these methods frequently neglect model uncertainties and external disturbances. These constraints can be mitigated by replacing traditional dynamic models with neural networks, which can be continuously updated online as new data are acquired during quadrotor flights. This adaptability enables the neural networks to adjust changes in the model, hence improving control performance. This section provides a detailed explanation of the data collection process, neural network model design, training procedures, and the resulting training outcomes.

3.1.1. Data Generation

First, we created a dataset of input–output pairs, where the inputs were the current states and control inputs, and the outputs were the corresponding state derivatives. The input $x_{input}=\left[x,y,z,v_x,v_y,v_z,\varphi ,\theta ,\psi ,p,q,r,{\sigma }_1,{\sigma }_2,{\sigma }_3,{\sigma }_4,\gamma \right]$ to the neural network was 17-dimensional data, and the output $y_{output}=\left[\dot{x},\dot{y},\dot{z},{\dot{v}}_x,{\dot{v}}_y,{\dot{v}}_z,\dot{\varphi },\dot{\theta },\dot{\psi },\dot{p},\dot{q},\dot{r}\right]$ was the 12-state derivatives. Multiple input combinations were sampled from uniform distributions within the operational range defined for the quadrotor. The state and their ranges are specified in

Table 1. Inputs’ sampling and ranges.

Input	Range	Description
$x,y$	[−10, 10]	X-axis position, Y-axis position, simulates variations in horizontal position
$z$	[−10, 10]	Z-axis position, considers flight requirements below the reference plane
$\left(v_x,v_y,v_z\right)$	[0, 10]	Velocity
$\left(\varphi ,\theta \right)$	[−$\mathsf{\pi }/4,\mathsf{\pi }/4$]	Roll and pitch, prevent drone instability
$\left(\psi \right)$	[−pi, pi]	Yaw, covers all possible directions
$\left(r,p,q\right)$	[−2, 2]	Angular velocity
$\left({\sigma }_1,{\sigma }_2,{\sigma }_3,{\sigma }_4\right)$	[0, 900]	Rotor speed

.

The sampled variables $\left({\sigma }_1,{\sigma }_2,{\sigma }_3,{\sigma }_4\right)$ were not used as inputs to the NN directly for training. For a quadcopter, the thrust $T_i$ produced by each rotor can be represented as:

$$T_i=k_F{\omega }_i^2$$

(16)

where $k_F$ is the thrust coefficient, which represents the efficiency of thrust generation by the rotor, and ${\omega }_i$ is the angular velocity of the ith rotor.

The total thrust $F$ is the sum of the thrusts produced by all four rotors:

$$F={\displaystyle \sum T_i}=k_F\left({\omega }_1^2+{\omega }_2^2+{\omega }_3^2+{\omega }_4^2\right)$$

(17)

In addition to thrust, each rotor generates roll, pitch, and yaw moments. The moments consist of two parts: (1) roll and pitch moments and (2) yaw moments. These are moments caused by the eccentric position of the rotors. For roll and pitch, the moments are:

$${\tau }_x=L{k}_F\left(-{\sigma }_2^2+{\sigma }_4^2\right),{\tau }_y=L{k}_F\left(-{\sigma }_1^2+{\sigma }_3^2\right)$$

(18)

where $L$ is the distance from the rotor to the center of mass of the quadcopter.

The sign of the moment depends on the arrangement and rotation direction of the rotors.

The yaw moment: this moment is generated by the reaction torques of the rotors and can be expressed as:

$${\tau }_z=k_M\left(-{\sigma }_1^2+{\sigma }_2^2-{\sigma }_3^2+{\sigma }_4^2\right)$$

(19)

where $k_M$ is the coefficient representing the efficiency of moment generation.

Gyroscopic precession effect: differences in the speed of the rotors also lead to a net gyroscopic effect, which affects the system’s dynamic response.

$$\gamma ={\sigma }_1-{\sigma }_2+{\sigma }_3-{\sigma }_4$$

(20)

where $\gamma$ is the net effect due to differences in motor speeds, contributing to gyroscopic precession.

We used these sampled variables to calculate the following additional inputs to the NN as

$$\begin{array}{l}F=k_F\left({\sigma }_1^2+{\sigma }_2^2+{\sigma }_3^2+{\sigma }_4^2\right)\\ {\tau }_x=L{k}_F\left(-{\sigma }_2^2+{\sigma }_4^2\right)\\ {\tau }_y=L{k}_F\left(-{\sigma }_1^2+{\sigma }_3^2\right)\\ {\tau }_z=k_M\left(-{\sigma }_1^2+{\sigma }_2^2-{\sigma }_3^2+{\sigma }_4^2\right)\\ \gamma ={\sigma }_1-{\sigma }_2+{\sigma }_3-{\sigma }_4\end{array}$$

(21)

where $F$ is the sum of thrusts produced by all four rotors, ${\tau }_x$ is the roll torque causing a rotation around the X-axis, ${\tau }_y$ is the pitch torque causing a rotation around the Y-axis, ${\tau }_z$ is the yaw torque causing rotation around the Z-axis, $\gamma$ is the net effect due to differences in motor speeds, contributing to gyroscopic precession.

These inputs were passed through the quadrotor dynamics defined in Section 2. The training output for the NN was the state derivatives obtained for each input.

The use of neural networks allowed for the implicit approximation of disturbances in the dynamic model. During training, the neural network employed supervised learning, using input data, which included the effects of disturbances, and output data to learn the dynamic relationships of the entire system and establish an approximate dynamic model:

$$\widehat{f}\left(x_t,u_t\right)\approx f\left(x_t,u_t+{\epsilon }_t\right)+{\omega }_t$$

where $\widehat{f}$ is the dynamic model learned by the neural network.

By approximating ${\epsilon }_t$ and ${\omega }_t$ through the neural network, the design phase of the controller did not need to explicitly introduce stochastic disturbances. Instead, these disturbances were considered as part of the learned deterministic model.

3.1.2. NN Model and Training

This section describes the selection of a Multilayer Perceptron (MLP) neural network architecture to imitate the quadrotor dynamics [27,28]. The network consisted of six hidden layers containing 256, 256, 128, 128, 64, and 64 neurons, respectively, as shown in Figure 2. Every hidden layer employed the Leaky Rectified Linear Unit (LeakyReLU) activation function to incorporate nonlinearity and mitigate the vanishing gradient problem. The training process utilized a dataset consisting of 15,000 samples, processed over 2000 epochs with a batch size of 64. A Graphics Processing Unit (GPU) was employed to accelerate training, facilitating fast computation and supporting a gradual decrease in the loss function through multiple iterations, thus improving the model’s predictive performance. The optimization utilized the Adam optimizer in conjunction with a custom learning rate scheduler to enhance accuracy. The learning rate followed an exponential decay strategy, starting with an initial rate of 0.001 and reducing to 0.0001 over 1000 epochs. That method ensured stable convergence and prevented overshooting during training. The neural network model was developed with TensorFlow 2 and trained on a system comprising an Intel i9-13900 HX processor, 16 GB of RAM, and an NVIDIA GeForce RTX 3060 Ti GPU with 8 GB of memory. The model achieved a Mean Squared Error (MSE) of 0.013 and a Mean Absolute Error (MAE) of 0.047 on the test dataset on completion of training, which indicated high predictive accuracy and robustness.

The LeakyReLU activation function is as follows:

$$f\left(x\right)=\left\{\begin{array}{c}x\\ \alpha x\end{array}\right.\begin{array}{c}if\\ if\end{array}\begin{array}{c}x>0\\ x\le 0\end{array}$$

(22)

where $\alpha$ is a small positive constant, typically set to $\alpha =0.01$ or another small value. Specifically, when the input $x$ is positive, Leaky ReLU outputs $x$; when the output is negative, it outputs $\alpha x$, retaining a slight negative gradient.

The Mean Squared Error was used for the loss function; it can measure the average squared difference between the predicted and actual state derivatives:

$$L\left(\theta \right)={\displaystyle \frac{1}{N}}{{\displaystyle \sum _{\mathrm{i}=1}^N\left({\widehat{y}}_i-y_i\right)}}^2$$

(23)

where ${\widehat{y}}_i$ is the predicted state derivative from the MLP, $y_i$ is the ground-truth state derivative.

On completion of training, the network approximated the function $f:x_{input}\to y_{output}$, essentially functioning as a surrogate model for the quadrotor dynamics.

3.1.3. Validation

A distinct validation set was used to assess the model’s performance on unseen data after completing training, thereby preventing the model from overfitting and from memorizing the training data. The MSE and MAE outputs on the validation set indicated the model generalization capability. Figure 3 shows the desired value and actual value on validation set. Points situated along the diagonal line y = x in Figure 3 indicate flawless predictions. Figure 4 displays the actual position and predicted position of $\left(x,y,z\right)$ over time on the validation set. Figure 5 shows the actual angular and predicted angular velocity of $\left(r,p,q\right)$ over time on the validation set.

3.2. Improved Model Predictive Path Integral

For a discrete-time dynamical system with states $x_t\in \aleph$, control inputs $u_t\in U$, $\epsilon$ error in control input, $\omega$ as external disturbances, and mapping $f:\aleph \times U\to \aleph$ from the current state and input to the next state given by

$$x_{k+1}=f\left(x_k,u_k\right)$$

(24)

a general optimal control problem consists of finding a control policy $\pi \left(x\right)$, a map from the current state to the optimal input, $\pi :\aleph \to U$, such that a cost $J:\aleph \times U\to {ℝ}^{+}$ is minimized and is given by

$$\pi \left(x\right)=\underset{u}{\arg \min }J\left(x,u\right)$$

(25)

$$J\left(x,u\right)={\displaystyle \sum _{t=0}^T\left({‖x_t-x_{ref}‖}_Q^2+{‖u_t‖}_R^2\right)}$$

(26)

where $x_t$ is the current state, $x_{ref}$ is the reference state, $u_t$ is the control input, and $Q$ and $R$ are weighting matrices, subject to $x_0=x$

$$x_{k+1}=f\left(x_k,u_k\right)$$

(27)

$$x_k\in \aleph ,u\in U$$

(28)

The path integral control method for developing optimal control algorithms is based on stochastic trajectory samplings [29,30]. This method’s benefit is in its avoidance of using derivatives for the system dynamics or the cost function, thus providing robustness in estimating the system dynamics and formulating the cost function. Given a transition model F, a sample size N, number of time steps (horizon) T, and an initial control sequence $u_t^i=\left\{u_{t|t}^i,u_{t+1|t}^i,\dots u_{t+T-1|t}^i\right\}$, the algorithm calculates K rollouts for T time steps for the inputs and evaluates different possible trajectories by running them through the dynamic model F. The cost and weight for each of these rollouts are calculated using the provided cost function. The K weights are used to evaluate the contribution of each rollout to the final optimal input sequence, with only the first input being applied. The remainder of the sequence is used as the initial input for the subsequent iteration, and the process continues. The traditional MPPI method may require considerable computational resources or manual parameter adjustments to maintain control performance in response to dynamic environmental changes or significant swings in system states. Consequently, incorporating an adaptive mechanism into the MPPI control method enables the dynamic adjustment of its essential parameters according to system states or control performance, thereby enhancing the control’s robustness and efficiency.

At each time step t, multiple control sequences were generated by randomly sampling the control inputs. The sampled control input sequences were denoted as follows:

$${\left\{u_t^i\right\}}_{t=1}^N$$

(29)

Each control sequence $u_t^i=\left\{u_{t|t}^i,u_{t+1|t}^i,\dots u_{t+T-1|t}^i\right\}$ had the length of the prediction horizon T.

The real-time system error $e\left(t\right)$ as the distance between the current system state $x$ and the target state $x_d$ was defined as:

$$e\left(t\right)=‖x_d-x‖$$

(30)

The Euclidean distance was used to calculate the system error as follows:

$$e\left(t\right)=\sqrt{{\left(x_d-x\right)}^T\left(x_d-x\right)}$$

(31)

Based on the real-time error $e\left(t\right)$, we dynamically adjusted the sampling number N and the prediction horizon T:

$$N=N_o+k_{N}e\left(t\right)$$

(32)

$$T=T_o+k_{T}e\left(t\right)$$

(33)

where $N_o$ and $T_o$ are the baseline sampling number and baseline prediction horizon used when the error is small. $k_N$ and $k_T$ are gain coefficients that control the responsiveness of the sampling number and prediction horizon to the error.

When the error $e\left(t\right)$ increased, the sampling number N was increased to enhance sampling density and control precision. Conversely, when the error decreased, the sampling number was reduced to save computational resources. Similarly, the prediction horizon T was dynamically adjusted based on the error magnitude. An increased error led to a longer prediction horizon, allowing the controller to consider state changes over a more extended period. A decreased error shortened the prediction horizon, improving computational efficiency.

The cumulative cost for each trajectory was calculated as follows:

$$J^i={\displaystyle \sum _{k=0}^{T-1}\mathcal{l}}\left(x_{t+k|t}^i,u_{t+k|t}^i\right)$$

(34)

The weight for each trajectory was calculated as follows:

$${\omega }^i={\displaystyle \frac{\exp \left(-J^i/\lambda \right)}{{\displaystyle {\sum }_{j=1}^N\exp \left(-J^j/\lambda \right)}}}$$

(35)

Subsequently, the current control input using a weighted average of all sampled control inputs was updated as follows:

$$u_t={\displaystyle \sum _{i=1}^N{\omega }^i{u}_t^i}$$

(36)

After applying the updated control input $u_t$ to the system, we moved to the next time step t + 1 and repeated the above steps.

The MLP and improved MPPI control block diagram is shown in Figure 6:

Flight data and dynamic inputs: Provide a realistic dynamic description of the system, serving as the true data source in the simulation. These dynamics describe the motion behavior of the quadrotor UAV under external influences, including position, velocity, angular velocity, noise, and disturbance.

Multilayer Perceptron: The MLP is used to learn the system dynamics, effectively replacing traditional dynamic models to some extent. This approach is particularly effective in complex dynamic environments or under external disturbances. The neural network’s inputs are the current system state and control inputs, and the output is the derivative of the system state.

Improved MPPI controller: Based on the path integral method, the controller generates multiple possible trajectories through stochastic sampling and selects the optimal control input from them. By adaptively adjusting the sampling number and prediction horizon, the improved MPPI controller can better handle dynamic changes in the environment. When the error increases, the sampling number is increased to improve control accuracy; when the error decreases, the sampling number is reduced to save computational resources.

Trajectory output and control adjustment: The MPPI controller dynamically adjusts the control input based on real-time state errors, generating the trajectory output for the UAV. When there is an error between the actual trajectory and the target trajectory, the control adjustment module provides real-time feedback on this error and optimizes the process by dynamically adjusting the sampling number and prediction horizon, thereby enhancing control performance.

3.3. Stability Analysis

3.3.1. Neural Network Convergence Analysis

According to the universal approximation theorem, neural networks with appropriate structures and sufficient training data can approximate any continuous function with arbitrary precision. Therefore, we assumed that the MLP neural network satisfied the following after training:

$$‖\widehat{f}\left(x,u\right)-f\left(x,u\right)‖\le \epsilon ,\forall x,u\in D$$

(37)

where $\widehat{f}\left(x,u\right)$ is the neural network-approximated dynamics model, and $f\left(x,u\right)$ is the true dynamics model; $\epsilon$ is a small positive constant, and D is the system’s operating space.

During the training process, the neural network reduced the approximation error $\epsilon$ iteratively through learning, ultimately keeping it within a small bounded range. Through learning mechanisms and Figure 3, Figure 4 and Figure 5, it ensured that the approximation error always remained within a controllable range. The above analysis demonstrates that the neural network reliably converged to an accurate approximation of the quadcopter dynamics.

3.3.2. Stability Analysis of the Closed-Loop System

To prove the stability of the closed-loop system, we adopted a control framework based on passivity, following the method proposed by Ruggiero et al. [31]. The passivity-based control method is suitable for nonlinear systems and effectively handles system uncertainties and external disturbances.

Choose the control law as:

$$u=u_{MPPI}-Ke$$

(38)

where $u_{MPPI}$ is the control input obtained from the Model Predictive Path Integral (MPPI) controller, and $K$ is the gain matrix designed to stabilize the tracking error, $e$ is the error.

Define a Lyapunov function V to evaluate the stability of the system. For the quadrotor system, we selected the following candidate Lyapunov function:

$$V\left(e\right)=e^{T}Pe$$

(39)

where $e=x-x_{ref}$ is the tracking error of the system, and P is a symmetric positive definite matrix.

Compute the time derivative of the Lyapunov function:

$$\dot{V}=2e^{T}P\dot{e}$$

(40)

Based on the system dynamics:

$$\dot{x}=f\left(x,u\right)+d$$

(41)

where $d$ represents all external disturbances and model uncertainties.

Based on $e=x-x_{ref}$ and Equation (40), we obtained:

$$\dot{e}=f\left(x,u\right)+d-{\dot{x}}_{ref}$$

(42)

The neural network approximation becomes:

$$f\left(x,u\right)\approx \widehat{f}\left(x,u\right)-\epsilon \left(x,u\right)$$

(43)

Thus,

$$\dot{e}=\widehat{f}\left(x,u\right)-\epsilon \left(x,u\right)+d-{\dot{x}}_{ref}$$

(44)

Assuming that the MPPI controller ensures that in the absence of errors and disturbances:

$$\widehat{f}\left(x_{ref},u_{MPPI}\right)={\dot{x}}_{ref}$$

(45)

we can linearize the tracking error dynamics around $x_{ref}$:

$$\dot{e}\approx \left(A-BK\right)e-\epsilon \left(x,u\right)+d$$

(46)

Substituting $\dot{e}$ into $\dot{V}$:

$$\dot{V}=2e^{T}P\left(\left(A-BK\right)e-\epsilon \left(x,u\right)+d\right)$$

(47)

where $A$ and $B$ are the Jacobian matrices of the system dynamics with respect to $x$ and $u$, respectively.

Under the passivity-based control framework, the control law and Lyapunov function parameters are designed such that the Lyapunov derivative satisfies:

$$\dot{V}\le -\lambda {‖e‖}^2+\gamma {‖d‖}^2+\beta {‖\epsilon ‖}^2$$

(48)

where $\lambda >0,\gamma >0,and\beta >0$ are design parameters.

To ensure stability, we require the following:

$$\dot{V}\le 0$$

(49)

To achieve this, the matrix $P$ and the control gain matrix $K$ are selected by solving the Lyapunov equation.

Choose $P$ and $K$ so that they satisfy the following equation:

$${\left(A-BK\right)}^{T}P+P\left(A-BK\right)=-Q$$

(50)

where $Q>0$ is a positive definite matrix. This condition ensures that the linear part of $\dot{V}$ is negative definite:

$$2e^{T}P\left(A-BK\right)e\le -\lambda {‖e‖}^2$$

(51)

where $\lambda >0$ is the smallest eigenvalue of $Q$.

Apply the Cauchy–Schwarz inequality to the remaining terms:

$$-2e^{T}P\epsilon +2e^{T}Pd\le 2‖P‖‖\epsilon ‖‖e‖+2‖P‖‖d‖‖e‖$$

(52)

To maintain $\dot{V}\le 0$, we require the following:

$$-\lambda {‖e‖}^2+2‖P‖‖\epsilon ‖‖e‖+2‖P‖‖d‖‖e‖\le 0$$

(53)

To satisfy the above inequality, we can bound the terms involving $‖e‖$ by ensuring that

$$\lambda {‖e‖}^2\ge 2‖P‖‖\epsilon ‖‖e‖+2‖P‖‖d‖‖e‖$$

(54)

Dividing both sides by $‖e‖$ (assuming $‖e‖>0$):

$$\lambda ‖e‖\ge 2‖P‖\left(‖\epsilon ‖+‖d‖\right)$$

(55)

This condition holds if:

$$‖e‖\ge {\displaystyle \frac{2‖P‖\left(‖\epsilon ‖+‖d‖\right)}{\lambda }}$$

(56)

When the tracking error $‖e‖$ is sufficiently large, $\dot{V}\le 0$. For smaller errors, the bounded approximation error and disturbances are dominated by the negative definite term, ensuring that $\dot{V}\le 0$ overall.

From the above analysis, we conclude that the tracking error $e$ converges to a bounded region around zero, with the bound determined by the approximation error $\epsilon$ and disturbances $‖d‖$. The tracking error remains bounded all the time, ensuring robust performance despite disturbances and model uncertainties. The control law satisfies the passivity condition, preventing energy injection that could destabilize the system.

Therefore, the closed-loop quadrotor system under the proposed MPPI controller and MLP neural network approximation achieves global asymptotic stability, ensuring reliable trajectory tracking in the presence of disturbances and model uncertainties.

4. Results

The neural network was trained using TensorFlow 2 on Google Colab, utilizing local system resources. Drone simulations were conducted in MATLAB, leveraging the MATLAB Deep Learning Toolbox, which facilitates the import and utilization of pre-trained TensorFlow models. We implemented three control methods: (1) Model Predictive Path Integral (MPPI) control without learning dynamics, (2) MPPI control with dynamics learned by a simple fully connected neural network, (3) MPC with dynamics learned by a Multilayer Perceptron (MLP) and (4) MPPI control with dynamics learned by a Multilayer Perceptron (MLP). Simulations were performed across two distinct trajectories: (1) waypoint navigation and (2) tracking a figure-eight trajectory. In all simulations, the plant was integrated at a frequency of 100 Hz (every 0.01 s), and the controller operated at 20 Hz (every 0.05 s). The initial position of the quadcopter was set to [0,10] with an initial control input of [620.6108, 620.6108, 620.6108, 620.6108]. For the waypoint navigation task, the desired position was [3,10]. The figure-eight trajectory was defined with a scale factor of 2, and an additional spiral path was utilized to evaluate the algorithm’s performance further. All algorithms were evaluated using a comprehensive cost function that penalized deviations in position tracking, velocity magnitudes, angular rotations, deviations of control signals from the nominal hovering control (indirectly reflecting control effort), and changes in control signals. A critical penalty was imposed if the sampled trajectory descended below z = 0 (below the reference ground level). The Root-Mean-Square Error (RMSE) metric was employed to assess the experimental results across the three trajectory tasks:

$$RMSE=\sqrt{{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{e}_i^2}}$$

4.1. MPPI Control Without Learning Dynamics

(1) Waypoint navigation: The number of rollouts per step was 800, with a prediction horizon of 25. At each step, the controller took an average of 0.12 s to execute. The program execution time was 618.76 s. The final tracking error was 0.1426 m. The results are shown in Figure 7a–f. The RMSE was 0.7124 m.

(2) Figure-eight trajectory tracking: The number of rollouts per step was 800, with a prediction horizon of 25. At each step, the controller took an average of 0.21 s to execute. The program execution time was 938.49 s. The results are shown in Figure 8a–f. The RMSE was 1.9756 m.

4.2. MPPI Control with Learning Dynamics with Two Hidden Layers Connected to Neural Network

(1) Waypoint navigation: The number of rollouts per step was 800, with a prediction horizon of 25. At each step, the controller took an average of 0.48 s to execute. The program execution time was 723.15 s. The results are shown in Figure 9a–f. The RMSE was 0.7053 m.

(2) Figure-eight trajectory tracking: The number of rollouts per step was 800, with a prediction horizon of 25. At each step, the controller took an average of 0.5 s to execute. The program execution time was 986.37 s. The results are shown in Figure 10a–f. The RMSE was 1.3883 m.

4.3. MPC Control with Learning Dynamics by Multilayer Perceptron

(1) Waypoint navigation: The number of rollouts per step was 800, with a prediction horizon of 25. At each step, the controller took an average of 1.6 s to execute. The program execution time was 650.25 s. The results are shown in Figure 11a–f. The RMSE was 0.9747 m.

(2) Figure-eight trajectory tracking: The number of rollouts per step was 800, with a prediction horizon of 25. At each step, the controller took an average of 0.53 s to execute. The simulation time was 30 s, and the actual program execution time was 1253.27 s. The results are shown in Figure 12a–f. The RMSE was 1.8973 m.

4.4. MPPI Control with Learning Dynamics by Multilayer Perceptron

(1) Waypoint navigation: The number of rollouts per step was 800, with a prediction horizon of 25. At each step, the controller took an average of 1.8 s to execute. The program execution time was 819.68 s. The results are shown in Figure 13a–f. The RMSE was 0.6515 m.

(2) Figure-eight trajectory tracking: The number of rollouts per step was 800, with a prediction horizon of 25. At each step, the controller took an average of 0.45 s to execute. The simulation time was 30 s, and the actual program execution time was 1142.27 s. The results for the same are shown in Figure 14a–f. The RMSE was 1.1093 m.

5. Discussion

Although existing MPPI methods perform well in path planning and trajectory tracking, they typically require high computational resources and manual parameter adjustments to maintain control performance. In this study, we proposed an improved MPPI controller that incorporates an adaptive mechanism based on system errors, allowing a dynamic adjustment of the sampling number and prediction horizon, thereby improving the controller’s robustness and efficiency in complex dynamic environments.

This paper used a multilayer perceptron (MLP) neural network to learn the dynamics of the quadcopter and combined it with the MPPI controller. Compared to other control methods based on traditional dynamic models, our approach can handle modeling inaccuracies and external disturbances by learning the system’s nonlinear dynamic behavior. Compared to classical model-based approaches, this data-driven control method better adapts to real-world uncertainties.

Unlike other studies that use traditional Model Predictive Control (MPC) and robust control methods, our research continuously learns the system dynamics using a neural network and introduces an adaptive adjustment mechanism, enabling the system to maintain good performance even in the face of dynamic environmental changes and partial observability. This characteristic is particularly suitable for UAVs operating in unknown or dynamic environments, such as forests and urban canyons.

6. Conclusions

This paper presented an approach that integrated MPPI control and an MLP. The improved MPPI method could adjust the sampling quantity according to trajectory error to achieve a better trajectory. The MLP neural network could learn the dynamics of a quadrotor by its state parameter and model the quadcopter. The improved MPPI employed the model to guide and control the quadcopter along the desired trajectory. Three trajectories were employed to test the effectiveness of the proposed method. The results were compared with MPPI control without learning, MPPI control with two hidden layers, MPC with MLP, and improved MPPI control utilizing MLP. The experimental results showed that improved MPPI control with MLP had better performance than MPPI control without learning and MPPI control with two hidden layers. In the future, reinforcement learning methods could be integrated, allowing the neural network to automatically optimize control parameters based on flight variations, thereby improving adaptability and resilience to external disturbances. We could extend the improved MPPI method to not only perform single-objective trajectory tracking but also to execute multi-objective path planning and collision avoidance tasks in complex environments. Future work could incorporate graph search algorithms and global path optimization techniques to improve navigation efficiency and safety.