A Sensor Fusion Approach to Observe Quadrotor Velocity

Abstract

The growing use of Unmanned Aerial Vehicles (UAVs) raises the need to improve their autonomous navigation capabilities. Visual odometry allows for dispensing positioning systems, such as GPS, especially on indoor flights. This paper reports an effort toward UAV autonomous navigation by proposing a translational velocity observer based on inertial and visual measurements for a quadrotor. The proposed observer complementarily fuses available measurements from different domains and is synthesized following the Immersion and Invariance observer design technique. A formal Lyapunov-based observer error convergence to zero is provided. The proposed observer algorithm is evaluated using numerical simulations in the Parrot Mambo Minidrone App from Simulink-Matlab.

1. Introduction

Thanks to the advancement of unmanned aerial vehicles, these have gained ground in various applications, including law enforcement, precision agriculture, and architectural/industrial inspection. Additionally, researchers have chosen to use this aircraft type, which offers excellent academic advantages, to test new nonlinear control theory methods. Crewless aircraft have made a quantum leap, and the nonlinear control theory has evolved. The basis of the classical nonlinear control theory, built by, for instance, [1,2], has been adapted rapidly into this century’s new technologies, as reported in [3,4]. Hence, quadrotor aerial vehicles have become a worldwide standard platform for robotics research. In particular, the Parrot Mambo Minidrone App in Simulink-Matlab offers excellent functionality for testing new control and state observation algorithms. It provides semi-realistic inertial and visual sensors, making real-time implementation straightforward. In reference to the first developments of sensor fusion for unmanned aerial vehicles, we can find that in [5], the measurements from an Inertial Navigation System (INS) and GPS sensors are fused by using a Kalman filter. The information for the simulation experiment was used to examine the data of both sensors; tool implementations of the filter were considered. The research article [6] establishes a basic requirement for an autonomous robot merging the data of different sensors; odometric and sonar sensors are fused together by means of an Extended Kalman Filter (EKF). The adaptive algorithm performs a precise localization of the vehicle and can be established in a wide range of experimental situations. The researchers in [7] describe a simple yet powerful statistical technique for fusing information from different sensors that aims to obtain the spectral error densities of the navigation gravity model, completing a robust aerial model that absorbs vertical disturbances. The information is computed in the time-dependent frequency.

In [8], the authors present a real-time implementation of position controllers based on the Adaptive–Proportional–Integral Derivative (APID) method for the Parrot Mambo Minidrone. An adaptive mechanism based on a second-order sliding mode control is also included to modify the conventional parameters of the altitude controller. Simulations and experimental flights validate the success of this approach. The research reported in [9] synthesizes Proportional–Integral–Derivative (PID), Linear Quadratic Regulator (LQR), and Model Predictive Control (MPC) algorithms for a Parrot Minidrone, and the control algorithms’ strengths and weaknesses are analyzed. These control techniques were chosen considering that the MPC design for Parrot Minidrones is unavailable in the existing literature. The automatic code generation capabilities offered by the Simulink coder facilitated the experimental implementation of the proposed control algorithms. The use of a quadrotor linear model evidently causes a relatively large mismatch between simulations and experimental results. The work in [10] presented the real-time implementation of a novel Cartesian translational robust control strategy for the Parrot Mambo Minidrone. Multiple data were collected from actual flight experiments involving a set of four Parrot Mambo Minidrones. A first-order dynamic with time delay was identified as the mathematical model for the Cartesian translational dynamic. The control strategy was designed based on a parameter variant discrete-time linear system. Reference [11] reports the design and implementation of a cascade attitude controller for the Parrot Mambo Minidrone. The controller was designed using the triple-step and nonlinear integral sliding mode control NISMC method considering the three-DOF nonlinear model. In [12], a hand gesture-based drone controller for the Parrot Mambo Minidrone is developed; the controller can perform take-off, hovering, and landing operations. The delay between executing each command is only three seconds, and the system accuracy obtained for hand gesture detection is remarkably good. The system can be further extended by modifying the controller so that the UAV can move and perform multiple commands using hand gestures.

Current directions aim to develop nonlinear observer strategies for autonomous navigation using only onboard sensors. The work in [13] proposed an embedded fast Nonlinear Predictive Model (NMPC) algorithm. This controller ensures a stable and safe flight of micro aerial robots relying solely on onboard sensors to localize themselves. The implemented controller can drive the micro aerial vehicle to track a path and outstand external disturbances. The design of a computationally efficient optical flow algorithm for tiny multirotor aerial vehicles, also known as pocket drones, is the focus of the work reported [14]. The Edge-Flow algorithm uses a compressed representation of an image frame to match it with the one in the previous time step; the adaptive time horizon also enables it to detect sub-pixel flow, from which velocities can be observed. Reference [15] presents a fusion filter design using the Kalman Filter (KF) complemented with the No Motion No Integration Filter (NMNI) to reduce noise influence to observe the tilt angle using only one accelerometer and one gyroscope. This approach relieves the burden of multiple sensors for attitude tracking, reducing battery energy consumption and helping the drone obtain the precise angle under rotor vibrations.

Translational velocity observation for quadrotors is an appealing problem from practical and theoretical perspectives. In [16], a globally exponential convergent observer based on nonlinear adaptive techniques is proposed. The observer uses measurements from an Attitude and Heading Reference System (AHRS). Numerical simulations evaluate the observer algorithm that shows good performance in the presence of noisy acceleration measurements. Reference [17] reports the design of a deterministic quadrotor translational velocity observer employed to reconstruct the scale factor of the position determined by a Simultaneous Localization And Mapping (SLAM) algorithm. The translational velocity observer uses the translational acceleration and the non-scaled translational position; its performance is evaluated through numerical simulations. Translational velocity observation is also an essential issue for fixed-wing aircrafts. Reference [18] presents an exponentially stable nonlinear wind velocity observer for fixed-wing unmanned aerial vehicles. The proposed observer employs a GNSS-aided Inertial Navigation System (INS), an attitude observer, and a pitot-static probe measuring dynamic pressure and airspeed in the longitudinal direction. The proposed observer estimates wind velocity and, as by-products, the angles of attack, sideslip, and scaling factor of the pitot-static probe measurement without requiring UAV maneuvers with the Persistence of Excitation (PE). The algorithm is well-suited for embedded systems and was tested through simulations. Finally, in [19], a uniform, semi-global, exponentially stable nonlinear observer for attitude, gyro bias, position, velocity, and specific force estimation for a fixed-wing UAV is presented. The nonlinear observer uses inertial and visual measurements without any assumptions about the flight altitude or the structure of the terrain being recorded. Experimental data from a UAV test flight and simulated data show that the nonlinear observer performs robustly.

Examining the existing literature reveals a significant gap in quadrotor translational velocity observations. No deterministic observer complements inertial and visual measurements with a formal proof of convergence of the observation error and a semi-realistic numerical simulation. By incorporating novel methodologies such as the Immersion and Invariance method, this research fills these gaps and presents a technique to complementarily fuse both measurements, accompanied by a formal proof of the convergence to zero of the observation error. This research was driven by an aim to enhance quadrotor capabilities to fly autonomously using only onboard sensors. In relation to the autonomous performance of aerial vehicles, we can find in [20], mounted-based stations, which are expected to become an integral component of future intelligent transportation systems. The aerial vehicle, integrated with the ground vehicular network, is used to bridge coverage gaps, offer broader communication services, and improve the network connection stability. Its self understanding is the main task, which is performed by the autonomous unmanned vehicle. As mentioned, a fundamental goal of the autonomous UAV is the development of navigation systems that help to realize any kind of aerial operations; inside [21], a nano-UAV learns to detect and fly through programmed trajectories with an autonomous navigation system based on neural networks. This research article is extremely pretentious because, besides the autonomous requirements, it is needed to establish appropriate sensor fusion, as well as the overall perspective of the whole aerial environment. Key insights from this research include the complementary nature of the inertial and visual sensors. Moreover, the findings reported in this article complement the work presented in [17]. The nonlinear translational velocity observer of [17] degrades when the quadrotor remains in hover; this problem is overcome by adding the optical flow measurement. Adding the optical measurement also gives the proposed observer an advantage over the observer reported in [16]. Finally, it is important to observe that the nonlinear observers reported in [18,19] use different sensors.

This paper introduces a novel sensor fusion strategy for observing the quadrotor translational velocity. The proposed observer algorithm uniquely fuses inertial and visual measurements. This fusion strategy is synthesized following the Immersion and Invariance approach method introduced in [3]. The observer error convergence to zero is guaranteed using Lyapunov theory arguments, and its performance is evaluated through numerical simulations in the semi-realistic Parrot Minidrone app from Matlab-Simulink. It is demonstrated that the proposed observer operates effectively under noisy measurements and considering different quadrotor motions.

This work has the following structure: Section 2 presents the quadrotor model and describes the available measurements. Section 3 reports the sensor fusion algorithm and formally states the nonlinear observer design. Section 4 is devoted to the numerical simulation study, the observation error, and the stability properties of the designed observer; finally, Section 5 presents the conclusions of this work.

2. Quadrotor Mathematical Model and Available Measurements

2.1. Quadrotor Dynamics

The basic structure of the dynamic quadrotor model has been reported in various research papers and books. This work considers the dynamic quadrotor model reported in [17,22]. The model contains states expressed in two frames of reference, the inertial and the fixed body, and is described by the following set of differential equations. See Figure 1.

$$\begin{array}{cc}\hfill \dot{X}& =R{V}^b\hfill \\ \hfill m{\dot{V}}^b& =mg{R}^{\top }{e}_3-T_T{e}_3-\mu H{V}^b-mS(\Omega )V^b\hfill \\ \hfill \dot{R}& =RS(\Omega )\hfill \\ \hfill J\dot{\Omega }& =-S(\Omega )J\Omega +M^b\hfill \end{array}$$

(1)

where $X={\left[\begin{array}{ccc}x& y& z\end{array}\right]}^{\top }$ is the quarotor position in the inertial reference frame. $R\in SO\left(3\right)$ is the rotation matrix from the fixed body to inertial coordinates with

$$SO\left(3\right)=\left\{R\in {\mathbb{R}}^{3\times 3}|R^{\top }R=I,\det \left(R\right)=1\right\}$$

where $I\in {\mathbb{R}}^{3\times 3}$ is the identity matrix, and $V^b={\left[\begin{array}{ccc}u& v& w\end{array}\right]}^{\top }$ denotes the translational velocity expressed in the fixed body frame. Moreover, m represents the vehicle’s mass, g is the gravitational force constant, $e_3={\left[001\right]}^{\top }$, $T_T$ is the total thrust force produced by the four rotors,

$$H=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right],$$

and $S(·)$ is a skew symmetric matrix such that

$$a\times b=S\left(a\right)b,\forall a,b\in {\mathbb{R}}^3,$$

$J=\mathrm{diag}\{J_{xx},J_{yy},J_{zz}\}$ is the inertia matrix, $\Omega ={\left[\begin{array}{ccc}p& q& r\end{array}\right]}^{\top }$ is the quadrotor rotational velocity expressed in the fixed-body frame coordinates, and $M^b$ is the vector of moments generated by the differential rotor’s thrust and rotor’s reaction moment.

Figure 1. Mambo Parrot’s coordinate frames.

2.2. Available Measurements

This work considers that the quadrotor has an Inertial Measurement Unit (IMU), a monocular camera pointing downwards, and a height sensor. Thus, the following signals are available.

2.2.1. Quadrotor’s Specific Translational Acceleration

As the leading electronic sensor, multi-rotors carry an IMU, which provides information on the fixed body coordinates. An IMU delivers the specific translational acceleration $a^b$, the angular velocity $\Omega$, and the intensity of the Earth’s magnetic field. According to the work reported in [23], the specific translational acceleration measured by the IMU’s accelerometer mounted on board an aerial vehicle is given by

$$a^b=\frac{1}{m}{F}_T^b-g{R}^{\top }{e}_3$$

(2)

where $F_T^b$ models the total external forces acting on the vehicle where the IMU is mounted. From the second equation in (1), it follows that

$$F_T^b=mg{R}^T{e}_3-T_T{e}_3-\mu H{V}^b$$

(3)

Notice that the above equation does not include the term $-mS(\Omega )V^b$ because the Coriolis acceleration is an inertial force. Substituting (3) into (2), the specific acceleration force measured by an accelerometer onboard a quadrotor is

$$a^b=-\frac{T_T}{m}{e}_3-\frac{\mu }{m}H{V}^b$$

(4)

The first available measurement is the specific translational acceleration, which element-to-element reads in the following way,

$$y_1=a^b=\left[\begin{array}{c}{a}_x^b\\ a_y^b\\ a_z^b\end{array}\right]=-\frac{1}{m}\left[\begin{array}{c}\mu u\\ \mu v\\ T_T\end{array}\right]$$

(5)

2.2.2. Quadrotor’s Attitude and Angular Velocity

The IMU data processed through an estimation algorithm constitutes an Attitude and Heading Reference System (AHRS) that provides the quadrotor attitude R and angular velocity $\Omega$. The AHRS can deliver the attitude using a parameterization such as Euler angles or quaternions or directly delivering the rotation matrix R. Thus, one has

$$\begin{array}{ccc}\hfill y_2& =& R\hfill \\ \hfill y_3& =& \Omega \hfill \end{array}$$

(6)

2.2.3. Optical Flow

Nowadays, the second crucial sensor for autonomous navigation is a monocular camera. The pixel velocity can be measured and related to the quadrotor velocity by processing the camera image.

The cornerstone to interpreting an image inside a computer is brightness. The camera sensors integrate the irradiance from the scene so that $I(x_p,y_p)$ defines the brightness of the pixel located at $(x_p,y_p)$; thus, the k image captured at time t can be characterized as [24],

$$I_k(x_p,y_p,t):\mathcal{R}\subset {\mathbb{R}}^2\times \mathbb{R}\to {\mathbb{R}}_{+}$$

Computer vision deals with extracting meaningful information from images; this is extracting meaningful information from brightness. A much more straightforward problem is computing image motion information from brightness. Consider two images of the same pixel location, $I_1(x_p,y_p,t_1)$ and $I_2(x_p,y_p,t_2)$, taken from infinitesimally close vantage points and at infinitesimally consecutive time instants $t_2\approx t_1$; thus,

$$\begin{array}{ccc}\hfill I_1(x_p\left(t_1\right),y_p\left(t_1\right),t_1)& =& I_1(x_p\left(t_1\right),y_p\left(t_1\right),t_1)\hfill \\ \hfill I_2(x_p\left(t_2\right),y_p\left(t_2\right),t_2)& =& I_2(x_p(t_1+\Delta t),y_p(t_1+\Delta t),t_1+\Delta t)\hfill \end{array}$$

where $t_2=t_1+\Delta t$ with $\Delta t$ an infinitesimal time increment. Assume that only pixels belonging to flat and parallel image plane portions and moving parallel to the image plane are considered. Then,

$$\begin{array}{ccc}\hfill x_p(t_1+\Delta t)& =& x_p\left(t_1\right)+u_p\Delta t\hfill \\ \hfill y_p(t_1+\Delta t)& =& y_p\left(t_1\right)+v_p\Delta t\hfill \end{array}$$

with $u_p$ and $v_p$ the pixel velocity. As a result,

$$\begin{array}{ccc}\hfill I_1(x_p\left(t_1\right),y_p\left(t_1\right),t_1)& =& I_1(x_p\left(t_1\right)+u_p\Delta t,y_p\left(t_1\right)+v_p\Delta t,t_1+\Delta t)\hfill \end{array}$$

(7)

Expanding the right-hand-side of Equation (7) by the Taylor series, one has

$$\begin{array}{ccc}\hfill I_1(x_p\left(t_1\right),y_p\left(t_1\right),t_1)& =& {\displaystyle I_1(x_p\left(t_1\right),y_p\left(t_1\right),t_1)+\frac{\partial I_1}{\partial x_p}{u}_p\Delta t+\frac{\partial I_1}{\partial y_p}{v}_p\Delta t+\frac{\partial I_1}{\partial t}\Delta t+O(\Delta t^2)}\hfill \end{array}$$

(8)

Neglecting the high-order terms, it follows that

$$\begin{array}{ccc}\hfill {\displaystyle \frac{\partial I_1}{\partial x_p}{u}_p+\frac{\partial I_1}{\partial y_p}{v}_p+\frac{\partial I_1}{\partial t}}& =& 0\hfill \end{array}$$

(9)

this equation is known as the brightness constancy constraint [24] or the optical flow constraint equation [25,26]. Note that it is impossible to compute the speeds $u_p$ and $v_p$ perpendicular to the image gradient using the brightness constancy constraint; this drawback is known as the aperture problem [27]. It is necessary to evaluate the brightness constancy constraint in each pixel location belonging to a region where $u_p$ and $v_p$ can be assumed constant, for example, the window

$$\begin{array}{ccc}\hfill W_i(x_p,y_p)& =& \left\{(x_p,y_p)\in {\mathcal{R}}_W\subset \mathcal{R},\delta \in {\mathbb{R}}_{+}\left|\right|x_p-x_p^i|+|y_p-y_p^i|\le \delta ,\right.\hfill \\ & & \left.\mathrm{for} \mathrm{some}(x_p^i,y_p^i)\in {\mathcal{R}}_W\right\}\hfill \end{array}$$

with $\mathcal{R}$ the image plane. If the window $W_i(x_p,y_p)$ is fixed inside the image plane, the computed speeds $u_p$ and $v_p$ are known as the optical flow. Using the differential method proposed by Lucas–Kanade [28], the computation of constant values of $u_p$ and $v_p$ in each small neighborhood $W_i(x_p,y_p)$ can be implemented as the minimization of

$$\sum _{(x_p,y_p)\in W}Q{(x_p,y_p)}^2{\left[\frac{\partial I_1}{\partial x_p}{u}_p+\frac{\partial I_1}{\partial y_p}{v}_p+\frac{\partial I_1}{\partial t}\right]}^2$$

with $Q\left(x\right)$, a function giving more influence to constraints at the center of $W_i(x_p,y_p)$ than those on the boundary. As reported in [29], this method is one of the most reliable.

Consider a camera onboard the aerial vehicle monitoring several characteristic points located at $P_i={\left[\begin{array}{ccc}{x}_c^i& y_c^i& z_c^i\end{array}\right]}^{\top }$, as illustrated in Figure 2. It is worth mentioning that this Figure comes from [30] with slight additions. The velocity of each point relative to the camera is given by:

$${\dot{P}}_i=-y_3\times P_i-V^b$$

(10)

From the perspective projection condition, it follows that the location of each characteristic point projects on the image plane as follows

$$\begin{array}{ccc}\hfill x_p^i& =& {\displaystyle f\frac{x_c^i}{z_c^i}}\hfill \\ \hfill y_p^i& =& {\displaystyle f\frac{y_c^i}{z_c^i}}\hfill \end{array}$$

(11)

with f the focal length of the camera lens. Combining (10) and (11), one obtains

$$\left[\begin{array}{c}{u}_p^i\\ v_p^i\end{array}\right]=\left[\begin{array}{c}{\displaystyle -\frac{f}{z}u+\frac{x_p^i}{z}w+\frac{x_p^i{y}_p^i}{f}p-\left(f+\frac{{x_p^i}^2}{f}\right)q+y_p^{i}r}\\ {\displaystyle -\frac{f}{z}v+\frac{y_p^i}{z}w+\left(f+\frac{{y_p^i}^2}{f}\right)p-\frac{x_p^i{y}_p^i}{f}q-x_p^{i}r}\end{array}\right]$$

(12)

the pixel velocity $(u_p^i,v_p^i)$ registered by the camera due to the quadrotor motion.

Figure 2. Pinhole camera principle [30].

Figure 2 shows that $z_c=z$. Using the differential method proposed by Lukas-Kanade to determine $u_p$ and $v_p$ in a region $W_i(x_p,y_p)$, it is possible to obtain the quadrotor translational velocity as follows

$$\left[\begin{array}{c}u\\ v\end{array}\right]=\left[\begin{array}{c}{\displaystyle -\frac{z}{f}{u}_p+\frac{x_p}{f}w+z\frac{x_p{y}_p}{f^2}p-\frac{z}{f}\left(f+\frac{x_p^2}{f}\right)q+z\frac{y_p}{f}r}\\ {\displaystyle -\frac{z}{f}v+\frac{y_p}{f}w+\frac{z}{f}\left(f+\frac{y_p^2}{f}\right)p-z\frac{x_p{y}_p}{f^2}q-z\frac{x_p}{f}r}\end{array}\right]$$

(13)

in an image region $W_i(x_p,y_p)$ for some i.

Consider the following assumption,

Assumption 1.

The optical flow is computed in a region $W_i$ that contains the pixel image origin; this is, $x_p=y_p=0$. Moreover, the quadrotor has a height controller that ensures $z\approx \overline{z}$ for some constant $\overline{z}$ and $w\approx 0$.

Under Assumption 1, the Equation (13) becomes

$$y_4=\left[\begin{array}{c}{u}_p\\ v_p\end{array}\right]=-H_1{V}^b+H_2{y}_3$$

(14)

with

$$H_1=\left[\begin{array}{ccc}\frac{f}{\overline{z}}& 0& 0\\ 0& \frac{f}{\overline{z}}& 0\end{array}\right],H_2=\left[\begin{array}{ccc}0& -f& 0\\ f& 0& 0\end{array}\right]$$

a measurable signal.

2.2.4. Height Sensor

Ultrasonic or laser devices to measure distance can be mounted on quadrotors. Hence, it is assumed that the following signals are also measured

$$\begin{array}{ccc}\hfill y_5& =& z\hfill \\ \hfill y_6& =& w\hfill \end{array}$$

(15)

thus, the matrix $H_1$ is also measurable since it can be expressed as

$$H_1=\left[\begin{array}{ccc}\frac{f}{{\overline{y}}_5}& 0& 0\\ 0& \frac{f}{{\overline{y}}_5}& 0\end{array}\right]$$

Finally, note that in terms of the measurable signals, the second equation in (7) can be expressed as

$${\dot{V}}^b=g{y}_2^{\top }{e}_3+y_1-s\left(y_3\right)V^b$$

(16)

3. Nonlinear Observer Design

Sensor fusion has become essential for solving mobile robotics state observation problems [31]. Better spatial and temporal coverage, robustness to sensor failures, and increased state observation accuracy are the main desirable properties of sensor fusion algorithms. This research article proposes a sensor fusion algorithm to estimate the quadrotor Cartesian velocity based on the Immersion and Invariance observer design technique proposed in [3]. The proposed fusion algorithm can be classified as a complementary fusion across domains [32,33]. Both considered sensors measure the same quantity in different domains and in acceleration and pixel velocity, and they work in a complementary configuration. Figure 3 illustrates schematically the proposed sensor fusion. The sensor fusion algorithm is designed in the deterministic nonlinear time-invariant framework.

Figure 3. Sensor fusion.

The observer design method can be explained as follows. Consider the following non-linear, deterministic, time-invariant system [3].

$$\begin{array}{cc}\hfill \dot{\eta }& =f_1(\eta ,\mathbf{y})\hfill \\ \hfill \dot{\mathbf{y}}& =f_2(\eta ,\mathbf{y})\hfill \end{array}$$

(17)

where $\eta \in \mathcal{R}\subset {\mathbb{R}}^n$ and $\mathbf{y}\in \mathcal{Y}\subset {\mathbb{R}}^m$ are the unmeasured and measured states, correspondingly.

Definition 1.

The dynamic system

$$\dot{\widehat{\eta }}=\varphi (\widehat{\eta },\mathbf{y})$$

(18)

with $\widehat{\eta }\in {\mathbb{R}}^n$, is a sensor fusion observer for the unmeasured state η if there exists a mapping $\beta :{\mathbb{R}}^n\times {\mathbb{R}}^m⟶{\mathbb{R}}^n$ such that the manifold,

$$\mathcal{M}=\left\{(\eta ,\widehat{\eta },\mathbf{y})\subset {\mathbb{R}}^n\times {\mathbb{R}}^n\times {\mathbb{R}}^m|\beta (\widehat{\eta },\mathbf{y})=\eta \right\}$$

(19)

has the following properties,

• $\mathcal{M}$ is positively invariant;
• All trajectories of (17) and (18) that start in a neighborhood of $\mathcal{M}$ asymptotically converge to $\mathcal{M}$.

The design of the observer of the form given in Definition 1 requires additional properties on the mapping $\beta (\widehat{\eta },\mathbf{y})$, as stated in the following result.

Theorem 1.

Consider the system (17). Assume that the vector fields $f_1(\eta ,\mathbf{y})$ and $f_2(\eta ,\mathbf{y})$ are forward complete and that there exist differentiable maps $\beta :{\mathbb{R}}^n\times {\mathbb{R}}^m⟶{\mathbb{R}}^n$ such that

• A1 For all $\widehat{\eta }$ and $\mathbf{y}$ the map $\beta (\widehat{\eta },\mathbf{y})$ satisfies,

  $$det\left(\frac{\partial \beta }{\partial \widehat{\eta }}\right)\ne 0$$
• A2 The dynamic system

  $$\tilde{\dot{\eta }}=f_1(\tilde{\eta }+\beta (\widehat{\eta },\mathbf{y}),\mathbf{y})-f_1(\beta (\widehat{\eta },\mathbf{y}),\mathbf{y})-\frac{\partial \beta }{\partial \mathbf{y}}(f_2(\tilde{\eta }+\beta (\widehat{\eta },\mathbf{y}),\mathbf{y})-f_2(\beta (\widehat{\eta },\mathbf{y}),\mathbf{y}))$$

  (20)
  has a (globally) asymptotically stable equilibrium at $\tilde{\eta }=0$ uniformly in $\widehat{\eta }$ and $\mathbf{y}$.

Then, the system (18) with,

$$\varphi (\widehat{\eta },\mathbf{y})={\left(\frac{\partial \beta }{\partial \widehat{\eta }}\right)}^{-1}\left(f_1(\beta (\widehat{\eta },\mathbf{y}),\mathbf{y})-\frac{\partial \beta }{\partial \mathbf{y}}{f}_2(\beta (\widehat{\eta },\mathbf{y}),\mathbf{y})\right)$$

(21)

is a (global) observer for (17).

Remark 1.

The result in Theorem 1 is a simplified version of the general observer design theory reported in [3]. The proof of Theorem 1 was reported in [17]. The sensor fusion characteristics of the observer in Definition 1 are as follows. Assume that two measurements ${\mathbf{y}}_1$ and ${\mathbf{y}}_2$ contain information on the nonmeasurable state η. Then, it is possible to define a function $\gamma ({\mathbf{y}}_1,{\mathbf{y}}_2)$ that fuses both measurements, and then the function $\beta (\widehat{\eta },\gamma ({\mathbf{y}}_1,{\mathbf{y}}_2))$ integrates the fusion to the observer design procedure.

Cartesian Velocity Observer

Here, the quadrotor Cartesian velocity observer is designed. The observer fuses all measurements described in Section 2.2. First, using all available measurements, the following measurements are tailored

$$\begin{array}{ccc}\hfill {\overline{y}}_1& =& \left[\begin{array}{c}{a}_x^b\\ a_y^b\\ y_6\end{array}\right]={\overline{H}}_1{V}^b\hfill \\ \\ \hfill {\overline{y}}_4& =& \left[\begin{array}{c}{u}_p\\ v_p\\ y_6\end{array}\right]={\overline{H}}_2{V}^b+{\overline{H}}_3{y}_3\hfill \end{array}$$

(22)

with

$${\overline{H}}_1=\left[\begin{array}{ccc}-\frac{\mu }{m}& 0& 0\\ 0& -\frac{\mu }{m}& 0\\ 0& 0& 1\end{array}\right],{\overline{H}}_2=\left[\begin{array}{ccc}-\frac{f}{{\overline{y}}_5}& 0& 0\\ 0& -\frac{f}{{\overline{y}}_5}& 0\\ 0& 0& 1\end{array}\right],{\overline{H}}_3=\left[\begin{array}{ccc}-f& 0& 0\\ 0& -f& 0\\ 0& 0& 0\end{array}\right]$$

From the definition of the manifold (19), the observer error is defined as

$${\tilde{V}}^b=V^b-{\beta }_1({\widehat{V}}^b,\sigma )$$

(23)

with

$$\dot{\sigma }=\gamma ({\overline{y}}_1,{\overline{y}}_4)=\frac{1}{{\alpha }_1+{\alpha }_2}\left({\alpha }_1{\overline{y}}_1+{\alpha }_2{\overline{y}}_4\right)$$

(24)

with ${\alpha }_1$ and ${\alpha }_2$ scalar constants. Note that the function $\gamma ({\overline{y}}_1,{\overline{y}}_4)$ fuses the information of the same quantity, translational velocity, expressed in different domains, body axes acceleration ${\overline{y}}_1$, and optical flow ${\overline{y}}_4$. The scalars ${\alpha }_1$ and ${\alpha }_2$ modulate the fusion. The time derivative of the observer error ${\tilde{V}}^b$ reads as

$${\dot{\tilde{V}}}^b={\dot{V}}^b-\frac{\partial {\beta }_1}{\partial {\widehat{V}}^b}{\dot{\widehat{V}}}^b-\frac{\partial {\beta }_1}{\partial \sigma }\gamma ({\overline{y}}_1,{\overline{y}}_4)$$

(25)

Using (16) and (22), one obtains

$${\dot{\tilde{V}}}^b=g{y}_2^T{e}_3+y_1-S\left(y_3\right)V^b-\frac{\partial {\beta }_1}{\partial {\widehat{V}}^b}{\dot{\widehat{V}}}^b+\frac{1}{{\alpha }_1+{\alpha }_2}\frac{\partial {\beta }_1}{\partial \sigma }\left({\alpha }_1{\overline{H}}_1{V}^b+{\alpha }_2({\overline{H}}_2{V}^b+{\overline{H}}_3{y}_3)\right)$$

(26)

Now, to express (26) in terms of the observer error, the Equation (23) is solved for $V^b$ and substituted in (26). Thus,

$$\begin{array}{ccc}\hfill {\dot{\tilde{V}}}^b& =& g{y}_2^T{e}_3+y_1-S\left(y_3\right)\left({\tilde{V}}^b+{\beta }_1({\widehat{V}}^b,\sigma )\right)-\frac{\partial {\beta }_1}{\partial {\widehat{V}}^b}{\dot{\widehat{V}}}^b\hfill \\ \hfill & & {\displaystyle +\frac{1}{{\alpha }_1+{\alpha }_2}\frac{\partial {\beta }_1}{\partial \sigma }\left({\alpha }_1{\overline{H}}_1\left({\tilde{V}}^b+{\beta }_1({\widehat{V}}^b,\sigma )\right)+{\alpha }_2({\overline{H}}_2\left({\tilde{V}}^b+{\beta }_1({\widehat{V}}^b,\sigma )\right)+{\overline{H}}_3{y}_3)\right)}\hfill \end{array}$$

(27)

Defining

$${\beta }_1({\widehat{V}}^b,\sigma )={\widehat{V}}^b+\Gamma \sigma$$

(28)

with $\Gamma$ a matrix gain, the state observer dynamic can be defined as follows

$${\displaystyle {\dot{\widehat{V}}}^b=g{y}_3^T{e}_3+y_1-S\left(y_3\right){\beta }_1+\frac{1}{{\alpha }_1+{\alpha }_2}\Gamma \left({\alpha }_1{\overline{H}}_1{\beta }_1({\widehat{V}}^b,\sigma )+{\alpha }_2({\overline{H}}_2{\beta }_1({\widehat{V}}^b,\sigma )+{\overline{H}}_3{y}_3)\right)}$$

(29)

It is important to verify that the state observer dynamic depends only on available measurements and known parameters. Then, the following vector differential equation described the observer error dynamic

$$\begin{array}{ccc}\hfill {\dot{\tilde{V}}}^b& =& {\displaystyle -S\left(y_3\right){\tilde{V}}^b+\frac{1}{{\alpha }_1+{\alpha }_2}\Gamma \left({\alpha }_1{\overline{H}}_1+{\alpha }_2{\overline{H}}_2\right){\tilde{V}}^b}\hfill \end{array}$$

(30)

Hence, one has the following.

Proposition 1.

Consider that Assumption 1 holds. Assume that the quadrotor is equipped with a set of sensors to measure $y_i$, $i=1,\cdots ,6$. Assume that the quadrotor flies over a surface with enough visual characteristics and there is a region $W_i$ containing the pixel location $x_p=y_p=0$, where the optical flow is constant. Then, there exist constants ${\alpha }_1$, ${\alpha }_2$ and a matrix Γ such that the observer dynamic (29) complimentarily fuses the available measurements and the observer error ${\tilde{V}}^b$ exponentially converges to zero.

Proof. 

The function $\gamma ({\overline{y}}_1,{\overline{y}}_4)$ in (24) performs the complementary fusion of the available measurements directly related to the quadrotor translational velocity. From this point, the observer design follows the lines of Theorem 1.

To analyze the observation error’s stability properties, consider the following Lyapunov function [2]

$$\mathcal{V}=\frac{1}{2}{\left({\tilde{V}}^b\right)}^{\top }{\Gamma }_1{\tilde{V}}^b$$

(31)

with ${\Gamma }_1\in {\mathbb{R}}^{3\times 3}$ a diagonal positive definite matrix. Thus, one has

$${\lambda }_m\left({\Gamma }_1\right)\parallel {\tilde{V}}^b{\parallel }^2\le \mathcal{V}\le {\lambda }_M\left({\Gamma }_1\right){\parallel {\tilde{V}}^b\parallel }^2$$

(32)

with ${\lambda }_m\left(A\right)$ and ${\lambda }_M\left(A\right)$ the smallest and greatest eigenvalues of any matrix A.

The time derivative of (31) along the trajectories of the observer error dynamic (30) is given by

$$\dot{\mathcal{V}}={\left({\tilde{V}}^b\right)}^{\top }{\Gamma }_1\left[-S\left(y_3\right){\tilde{V}}^b+\frac{1}{{\alpha }_1+{\alpha }_2}\Gamma \left({\alpha }_1{\overline{H}}_1+{\alpha }_2{\overline{H}}_2\right){\tilde{V}}^b\right]$$

(33)

Since $S\left(y_3\right)$ is a skew symmetric matrix, it follows that

$$\dot{\mathcal{V}}={\left({\tilde{V}}^b\right)}^{\top }\left[{\Gamma }_1\frac{1}{{\alpha }_1+{\alpha }_2}\Gamma \left({\alpha }_1{\overline{H}}_1+{\alpha }_2{\overline{H}}_2\right)\right]{\tilde{V}}^b$$

(34)

It is straightforward to verify that there exist ${\alpha }_1$, ${\alpha }_2$, $\Gamma$, and ${\Gamma }_1$ such that the matrix

$$P=\left[{\Gamma }_1\frac{1}{{\alpha }_1+{\alpha }_2}\Gamma \left({\alpha }_1{\overline{H}}_1+{\alpha }_2{\overline{H}}_2\right)\right]$$

(35)

is negative definite. Thus,

$$\dot{\mathcal{V}}\le -{\lambda }_m\left(P\right){\parallel {\tilde{V}}^b\parallel }^2\le -\frac{{\lambda }_m\left(P\right)}{{\lambda }_M\left({\Gamma }_1\right)}\mathcal{V}$$

(36)

and the proof is concluded. □

4. Results

This section presents a semi-realistic numerical simulation study to validate the theoretical developments of the previous section. First, the available measurements—the body axes acceleration and the optical flow—must be computed. Then, the proposed observer is implemented. It is essential to underscore that the Parrot Minidrone simulator provided by MATLAB-Simulink incorporates realistic quadrotor dynamics and sensor models. The works in [8,9,10] illustrate that the experimental implementation is straightforward after performing numerical simulations using this simulator; see also the work in [34], where semi-realistic simulations are performed.

The Parrot Minidrone’s physical characteristics and the camera specifications are summarized in Table 1.

Table 1. Drone’s physical characteristics and camera specifications.

Dimensions	(13.2 × 13.2 × 4.1 ) cm.
Weight	(63) g.
Motor type	Brushless (4).
Flight time	(8) min.
Camera	(Monocular / Vertical).
Shutter speed	(60) fps.
Resolution	(120 × 160) px.
Focal length	(1) mm.

4.1. Determination of the Parameter $\mu$

Note that in Equation (24), two essential measurements to reconstruct the quadrotor velocity are ${\overline{y}}_1$ and ${\overline{y}}_4$. Table 1 shows that the quadrotor’s mass is available, but the parameter $\mu$ is not. Hence, the quadrotor follows a circular trajectory, recording, along the $0X^b$ axis, its acceleration $a_x^b$ and velocity u to determine $\mu$ from the following relationship

$$\mu =-\frac{a_x^b}{u}m$$

(37)

It is important to highlight that to compute $\mu$, the measured acceleration $a_x^b$ was filtered using a low-pass first-order filter, as recommended in [23]. Then, the value of $\mu$ was computed as the average value of the $\mu$ values obtained during the flight. Figure 4 shows (white line) the recorded acceleration $a^b$ and (blue line) the reconstructed acceleration; this is,

$${\overline{a}}_x^b=-\frac{\mu }{m}u$$

(38)

Hence, for this quadrotor, one has $\mu =0.0035$. The parameter $\mu$ is related to the blade’s aerodynamic profile and induced drag forces. This positive constant is known in helicopter literature as blade drag [23,35].

Figure 4. Measured acceleration $a_x^b$ and reconstructed acceleration ${\overline{a}}_x^b$.

Note that this is an open-loop reconstruction so it is not expected that $a_x^b$ and ${\overline{a}}_x^b$ are exactly coincident. However, as reported in [17,23], this procedure gives an adequate approximation of $\mu$.

4.2. Optical Flow Algorithm Design

The optical flow estimation algorithm is implemented in the Parrot Minidrone Competition simulation environment, specifically in the Image Processing subsystem in the Flight Control System block. The optical flow estimation algorithm is performed as follows: (a) The monocular camera information that arrives in the Y1UY2V format is transformed into the RGB format. (b) The image in RGB format is transformed to Grayscale and filtered using an FIR (Finite Response Impulse) filter represented as a 2D coefficient matrix or a pair of separable filter coefficient vectors. (c) The filtered image is used to estimate the optical flow, employing the Lucas–Kanade method. The block implementing the Lucas–Kanade method delivers the pixel displacement per frame as a complex number for each image’s pixel.

These calculated displacements are multiplied by the corresponding number of frames per second (intrinsic parameter of the camera) to obtain the pixel velocity. Finally, the pixel velocities are subjected to statistical processing consisting of selecting a region of interest on the image plane, which, considering Assumption 1, corresponds to the image origin. Figure 5 shows a block diagram of the optical flow algorithm.

Figure 5. Optical flow block design.

4.3. Quadrotor Trajectories

To test the proposed observer, two trajectories are considered. In the first one, the quadrotor tracks a circle while in the second one, the quadrotor visits two waypoints. Figure 6 shows the quadrotor’s path during the circle-tracking flight at a constant altitude. Figure 7 depicts the quadrotor’s path during the two-way point flight, which is at a constant altitude once again.

Figure 6. Aircraft tracking: circular-type trajectory.

Figure 7. Aircraft tracking: square-type trajectory.

4.4. Measurements

Note that the parameter $\mu$ is required to implement the proposed observer to reconstruct the quadrotor-specific force. The quadrotor acceleration is obtained directly from the IMU implemented in the simulator. Figure 8 and Figure 9 show the specific force along the $0X^b$ axis delivered by the IMU when the quadrotor tracks the circular and the square-type (the quadorotor moves first along the $0X^b$ axis and, 10 s after, moves along the $0Y^b$ axis) trajectories, respectively. It is essential to state that the IMU from Simulink is implemented considering that the accelerometers also measure the gravity force, contradicting Equation (4). Hence, the gravitational force is subtracted from the IMU’s accelerometer measurement to match (4).

Figure 8. Specific force measured along the $0X^b$ axis while the quadrotor follows a circular trajectory.

Figure 9. Specific force measured along the $0X^b$ axis while the quadrotor follows a square-type trajectory.

Figure 10 and Figure 11 depict the quadrotor optical flow along the $0X^b$ axis computed as described in Section 4.2 when the quadrotor follows the circular and square-type trajectories, respectively.

Figure 10. Computed optical flow along the $0X^b$ axis while the quadrotor tracks a circular trajectory.

Figure 11. Computed optical flow along the $0X^b$ axis while the quadrotor tracks a square-type trajectory.

From Equation (22), it is clear that the signals plotted in Figure 8, Figure 9, Figure 10 and Figure 11 contain information about the translational velocity ${\tilde{V}}^b$. It is not evident to identify a relationship between the specific force and optical flow measurements. However, the proposed observer fuses the information and extracts the translational velocity ${\tilde{V}}^b$.

4.5. Observer Evaluation

The observer state dynamic described by Equation (29) is implemented inside the Parrot Mambo Minidrone simulator. The observed velocity

$${\widehat{V}}^b+\Gamma \sigma$$

(39)

is compared with the velocity computed by the internal algorithm of the Parrot Mambo Minidrone simulator, which employs a Kalman filter.

Figure 12 and Figure 13 show the time history of the observed velocity along the $0X^b$ and $0Y^b$ axes, respectively, together with the speeds computed by the internal algorithm of the Parrot Mambo Minidrone simulator when the quadrotor follows the circular trajectory. The observer gain matrices were fixed as follows

$$\begin{array}{c}\hfill \Gamma =\left[\begin{array}{ccc}6& 0& 0\\ 0& 6& 0\\ 0& 0& -1\end{array}\right];{\Gamma }_1=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]\end{array}$$

The scalar constants ${\alpha }_1$ and ${\alpha }_2$ were fixed at the following values

$${\alpha }_1=-13;{\alpha }_2=0.4$$

(40)

Figure 12. Observed speed $\widehat{u}+{\Gamma }_{11}{\sigma }_1$ (blue line) and speed computed by the Parrot Mambo simulator algorithm u (yellow line).

Figure 13. Observed speed $\widehat{v}+{\Gamma }_{22}{\sigma }_2$ (blue line) and speed computed by the Parrot Mambo simulator algorithm v (yellow line).

Now, Figure 14 and Figure 15 present the observer error behavior when the quadrotor follows the square-type trajectory. The speeds computed by the Parrot Mambo Minidrone algorithm are also shown.

Figure 14. Observed speed $\widehat{u}+{\Gamma }_{11}{\sigma }_1$ (blue line) and speed computed by the Parrot Mambo simulator algorithm u (yellow line).

Figure 15. Observed speed $\widehat{v}+{\Gamma }_{22}{\sigma }_2$ (blue line) and speed computed by the Parrot Mambo simulator algorithm v (yellow line).

Figure 12, Figure 13, Figure 14 and Figure 15 illustrate that the selected observer gains performed adequately to identify both translational speeds.

Figure 16 and Figure 17 present the observation errors to examine the proposed algorithm’s performance more closely. Note that the observation errors only converge to a neighborhood of zero. This behavior results from the semi-realistic simulation that includes noise in the measurements. However, the strong result in Proposition 1 hints at this observer behavior.

Figure 16. Observed speeds errors for the circular trajectory. $\tilde{u}$ (blue line), $\tilde{v}$ (red line).

Figure 17. Observed speeds errors for the square-type trajectory. $\tilde{u}$ (blue line), $\tilde{v}$ (red line).

Remark 2.

Under mild assumptions, consider that measurement noise enters into the translational velocity dynamics (16), as follows

$${\dot{V}}^b=g{y}_2^{\top }{e}_3+y_1-s\left(y_3\right)V^b+\delta \left(t\right)$$

(41)

with $\delta \left(t\right)\le \overline{\delta }$ a bounded time variant disturbance modeling the noise in all measurements. Then, the observer error dynamic becomes

$$\begin{array}{ccc}\hfill {\dot{\tilde{V}}}^b& =& {\displaystyle -S\left(y_3\right){\tilde{V}}^b+\frac{1}{{\alpha }_1+{\alpha }_2}\Gamma \left({\alpha }_1{\overline{H}}_1+{\alpha }_2{\overline{H}}_2\right){\tilde{V}}^b-\delta \left(t\right)}\hfill \end{array}$$

(42)

Now, from the analysis in Proposition 1, one obtains

$$\begin{array}{ccc}\hfill \dot{\mathcal{V}}& \le & -{\lambda }_m\left(P\right)\parallel {\tilde{V}}^b{\parallel }^2+{\lambda }_M\left({\Gamma }_1\right)\overline{\delta }\parallel {\tilde{V}}^b\parallel \hfill \\ \hfill & =& -(1-\sigma ){\lambda }_m\left(P\right)\parallel {\tilde{V}}^b{\parallel }^2-\sigma {\lambda }_m\left(P\right)\parallel {\tilde{V}}^b{\parallel }^2+{\lambda }_M\left({\Gamma }_1\right)\overline{\delta }\parallel {\tilde{V}}^b\parallel \hfill \end{array}$$

(43)

with $0<\sigma <1$. Finally,

$$\begin{array}{ccc}\hfill \dot{\mathcal{V}}& \le & -(1-\sigma )\parallel {\lambda }_m\left(P\right){\tilde{V}}^b{\parallel }^2,\forall \hfill \end{array}\parallel {\tilde{V}}^b\parallel \ge \frac{{\lambda }_M\left({\Gamma }_1\right)\overline{\delta }}{\sigma {\lambda }_m\left(P\right)}$$

(44)

Hence, from Lemma 9.2 in [2], it follows that the observer error is ultimately bounded as observed in Figure 16. Note in (44), that the ultimate bound depends on the noise level modeled by $\overline{\delta }$.

A more challenging problem where the proposed observer design method may fail is the case where the noise enters the translational velocity dynamic as follows

$${\dot{V}}^b=g{y}_2^{\top }{e}_3+y_1-s\left(y_3\right)V^b+{\delta }_1\left(t\right)V^b+{\delta }_2\left(t\right)$$

(45)

with ${\delta }_1\left(t\right)\le {\overline{\delta }}_1$ and ${\delta }_2\left(t\right)\le {\overline{\delta }}_2$ bounded time variant disturbances.

4.6. Observer Gains

A nonlinear time-varying model describes the observer error dynamics (see Equation (30)). Thus, determining adequate observer gains is a complex task; however, with some mild assumptions, at least locally, it is possible to determine a suitable observer gains combination. For example, assume that $y_3\approx 0$; then, the observer error dynamics reduces to

$$\begin{array}{ccc}\hfill {\dot{\tilde{V}}}^b& =& {\displaystyle \left[\begin{array}{ccc}\frac{-{\gamma }_1({\alpha }_1\frac{\mu }{m}+{\alpha }_2\frac{f}{{\overline{y}}_5})}{{\alpha }_1+{\alpha }_2}& 0& 0\\ 0& \frac{-{\gamma }_2({\alpha }_1\frac{\mu }{m}+{\alpha }_2\frac{f}{{\overline{y}}_5})}{{\alpha }_1+{\alpha }_2}& 0\\ 0& 0& 2{\gamma }_3\end{array}\right]{\tilde{V}}^b}\hfill \end{array}$$

(46)

Hence, the following Eigenvalues locally shape the observer error dynamics

$$\begin{array}{ccc}\hfill {\lambda }_1& =& {\displaystyle \frac{-{\gamma }_1({\alpha }_1\frac{\mu }{m}+{\alpha }_2\frac{f}{{\overline{y}}_5})}{{\alpha }_1+{\alpha }_2}}\hfill \\ \hfill {\lambda }_2& =& {\displaystyle \frac{-{\gamma }_2({\alpha }_1\frac{\mu }{m}+{\alpha }_2\frac{f}{{\overline{y}}_5})}{{\alpha }_1+{\alpha }_2}}\hfill \\ \hfill {\lambda }_3& =& 2{\gamma }_3\hfill \end{array}$$

(47)

From Equation (47), the selection ${\gamma }_3$ follows trivially. Now, assuming that ${\gamma }_1={\gamma }_2$, the combination of ${\gamma }_1,{\alpha }_1$ and ${\alpha }_2$ was selected as follows. For a fixed value of ${\gamma }_1$, the Eigenvalue $\lambda ={\lambda }_1$ was computed considering intervals for ${\gamma }_1$ and ${\gamma }_2$ that give a negative value, as illustrated in Figure 18. It is important to remember that a negative Eigenvalue will not guarantee an adequate observation of the quadrotor velocity due to numerical problems in the simulation.

Figure 18. Eigenvalues for different combinations of gains ${\alpha }_1$ and ${\alpha }_2$.

In Figure 18, one can observe the optimal values region from the scalar constants, as well as the Eigenvalues, described previously, that were estimated with precision and accuracy. The displayed chromatic variety denotes the set of Eigenvalues $\lambda$, according to the different values that the scalar constants ${\alpha }_1,{\alpha }_2$ can take. The last result Figure 19 shows the estimation errors analyzed only with respect to the x-axis because analogically, the y-axis presents the same behavior. As expected, increasing the value of $\lambda$ decreases the observer error until the numerical errors appear at a specific value of $\lambda$, where the observer error diverges.

Figure 19. Eigenvalues vs. observer’s estimation error in the $0X^b$ axis.

5. Conclusions

This article proposed a novel sensor fusion observation algorithm to observe the translational velocity using available measurements from onboard sensors for a quadrotor. Besides the successful implementation of the designed observation algorithm, the main contributions are listed next:

• The designed observer can estimate the linear speeds of the aircraft for different trajectories with precision.
• The optical flow was obtained without using image features or patterns compared to other research works.
• The application of Lyapunov’s theory through a correct proposed Lyapunov function demonstrates asymptotic convergence to zero of the nonlinear observer error.
• As mentioned initially, this article was intended to compensate for the overall information loss in indoor flights by observing the vehicle’s translational velocities. The observed velocity is precise enough so that the main objective has been successfully fulfilled.
• As previously evoked in the introduction, this research project surpassed the performance of the observer proposed in [17], mainly in flight trajectories, in which, at certain moments, the MAV does not make any displacements, remaining in stationary or hover flight, thanks to the incorporation of the Optical Flow into the observation algorithm.