Fixed-Wing Unmanned Aerial Vehicle 3D-Model-Based Tracking for Autonomous Landing

Abstract

The vast increase in the available computational capability has allowed the application of Particle-Filter (PF)-based approaches for monocular 3D-model-based tracking. These filters depend on the computation of a likelihood function that is usually unavailable and can be approximated using a similarity metric. We can use temporal filtering techniques between filter iterations to achieve better results when dealing with this suboptimal approximation, which is particularly important when dealing with the Unmanned Aerial Vehicle (UAV) model symmetry. The similarity metric evaluation time is another critical concern since we usually want a real-time implementation. We explored, tested, and compared with the same dataset two different types of PFs, (i) an Unscented Bingham Filter (UBiF) and (ii) an Unscented Bingham–Gauss Filter (UBiGaF), using pose optimization in both implementations. Using optimization steps between iterations increases the convergence capability of the filter and decreases the obtained error. A new tree-based similarity metric approach is also explored based on the Distance Transform (DT), allowing a faster evaluation of the possibilities without losing accuracy. The results showed that the obtained pose estimation error is compatible with the automatic landing requirements.

1. Introduction

The Portuguese Exclusive Economic Zone (EEZ) incorporates three large areas associated with the continent (327,667 km${}^2$), Azores (953,633 km${}^2$), and Madeira (446,108 km${}^2$) [1,2]. As a coastal state, Portugal may exercise sovereign rights over managing natural resources and other economic activities (e.g., tidal power) and is also responsible for conserving living resources and fighting against pollution [3].

To assess compliance with the existing regulations and laws, many technologies can be used to perform surveillance, e.g., Synthetic Aperture Radar (SAR) [4], Satellite Automatic Identification System (S-AIS) [5], or Vessel Monitoring System (VMS) [6]. Using SAR images, we can detect non-AIS-transmitting vessels [7]. Nevertheless, the covered area depends on the satellite location, and the detection accuracy depends on the used algorithm [8]. Exploring and using complementary technologies are essential for constant surveillance in the largest area possible.

Currently, Fast Patrol Boats (FPBs) are essential to act on the field and are still extensively used for surveillance in Portuguese territorial waters. Extending the FPB surveillance capability using fixed-wing Unmanned Aerial Vehicles (UAVs) is essential. Still, the small ship dimensions add an extra challenge, mainly in the take-off and landing operations [9,10,11,12,13]. Small-sized fixed-wing UAVs (typically with a payload below 5 kg) can usually be launched by hand [10,14], bringing most of the operational risk to the landing stage. Landing safety is essential to culminate UAV-based missions successfully. There are three major causes of UAV accidents [15]: human, material, and environment (e.g., weather or illumination changes). The vast majority of UAV registered accidents are mainly due to human factors [16,17,18], and investing in the automation of the most-complex maneuvers is essential.

When operating UAVs at sea, we must adapt the system to the existing environment. It is essential to guarantee a waterproof platform, and the communication range and reliability must also be high [19,20]. The UAV system’s basic architecture is composed of a Ground Control Station (GCS), a communications data link, and a vehicle platform [21] (Figure 1).

Figure 1. UAV system’s basic architecture.

The UAV landing retention system also plays an essential part in the landing operation complexity, and it must be perfectly adapted to the vehicle’s size and weight. A net-based retention system (Figure 2) presents a good compromise between complexity and effectiveness [10,22], being able to be rapidly mounted on any ship without depending on structural changes. If we do not want to capture the UAV directly on the ship, we can also use an external retention system, for instance based on quadcopters [23] using a net [24] or a line [25].

Figure 2. Net-based UAV retention system.

We were trying to land a UAV on a moving platform (ship) in an outdoor environment where we are subject to different meteorological conditions, e.g., illumination, wind, and balance. The UAV is considered a cooperative rigid object with a simple autopilot that maintains a constant trajectory to the landing area. UAVs are also evolving with the development of new structural designs [26], new materials [27], new payloads [28,29], and optimized radar cross-sections [30]. The ship is also considered cooperative and performs maneuvering to adjust the relative speed of the UAV and, consequently, the relative wind. Depending on the ship’s superstructures, we can have wind vortices that can make the UAV adopt an erroneous trajectory.

An automated landing control system can be based on the onboard sensors [21], but we usually do not have enough processing power onboard a small UAV. As an option, we can also rely on the Global Positioning System (GPS), but this system can be affected by jamming or spoofing [31,32,33]. This article used a ground-based monocular Red, Green, and Blue (RGB) vision system with the camera on the ship deck [13,34]. Using a ground-based system, we can access more processing power and implement algorithms to perform the autonomous landing of a UAV with a simple autopilot. Since we used a ground-based Graphics Processing Unit (GPU) system [11,35], which only depend on the GPU processing capabilities, we can easily upgrade it and have access to power without any restrictions. On the other hand, if we use UAV onboard-based algorithms, we need high processing capabilities, which are not easily available in small-sized UAVs. To successfully perform an autonomous landing maneuver, knowing the UAV’s position and its orientation is crucial to predict and control its trajectory. Pose estimation and tracking are widely studied (and still) open problems in Computer Vision (CV), with new methods constantly emerging [36,37].

When we know the object we want to detect and have its 3D model or Computer-Aided Design (CAD) model, we can use it to retrieve knowledge from the captured image. We can use Sequential Monte Carlo (SMC) methods or Particle Filter (PF) [38] to establish the 2D/3D correspondence and try to estimate the UAV pose and perform tracking over time [39]. The PFs represents the pose distribution by a set of weighted hypotheses (particles) that explicitly test the object’s projection on the image with a given pose [40]. Despite it being desirable to have a large number of particles to have a fast convergence, particle evaluation is usually very computationally demanding [35], and we need to have a good compromise between speed and accuracy. Some examples of the obtained pose tracking results when using our method can be seen in Figure 3, where the obtained pose estimation is represented in red.

Figure 3. Pose tracking obtained results (examples). (a) Analyzed Frame A. (b) Pose estimate in Frame A (represented in red). (c) Analyzed Frame B. (d) Pose estimate in Frame B (represented in red).

The adopted tracking architecture is divided into: (i) Pose Boosting; (ii) Tracking; (iii) Pose optimization [41,42]. In the Pose Boosting stage, we used Machine Learning (ML) algorithms to detect the UAVs on the captured frame and used a pre-trained database to generate representative pose samples [12,43]. In the Tracking stage, we used a 3D-model-based tracking approach using the UAV CAD model based on a PF approach [42]. In the Pose optimization stage, since we used a sub-optimal similarity metric to approximate the likelihood function iteratively, we included optimization steps to improve the estimate [41].

This article presents the following main developments (innovations): (i) the analysis, test, and comparison on the same dataset of two different PF implementations using pose optimization: (a) an Unscented Bingham Filter (UBiF) [41,42,44] and (b) an Unscented Bingham–Gauss Filter (UBiGaF) [42,45]; (ii) the implementation of a new tree-based similarity metric approach to be able to obtain a faster and more accurate weight estimation; (iii) better analysis and evaluation of how the optimization steps can decrease the filter convergence time; (iv) a validation and comparison between methods using a realistic synthetic dataset. In this article, we did not perform the comparison between UBiF and UBiGaF and more traditional methods such as the Unscented Kalman Filter (UKF) for the orientation estimation since, in [41,42], we already made this comparison and showed that these implementations outperform the UKF. As far as we know, there are no publicly available datasets or other ground-based model-based UAV tracking approaches, making the comparison with other state-of-the-art methods impossible.

The main developments regarding our previous work are: (i) in [42], a tracking architecture using a UBiGaF without pose optimization implementation, which is explored in this article, was proposed; (ii) in [41], we applied optimization steps to the UBiF that is implemented in this article also to the UBiGaF; (iii) in [43], we proposed a distance transform similarity metric that is modified to a tree-based approach to be able to decrease the processing time without a loss in accuracy.

This article is organized as follows. Section 2 presents the related work concerning pose estimation, pose tracking, and UAV tracking. Section 3 details the overall implemented system, detailing the pose boosting, tracking, and optimization stages. In Section 4, we present the experimental results. Finally, in Section 5, we present the conclusions and explore additional ideas for future research work.

2. Related Work

This section describes the related work, namely the pose estimation (Section 2.1), pose tracking (Section 2.2), and UAV tracking methods (Section 2.3).

2.1. Pose Estimation

Pose estimation is a very challenging task, mainly if we are considering only the information coming from a single frame. Some pose estimation methods use Red, Green, Blue, and Depth (RGB-D) cameras to perform data fusion and obtain better estimation results [46,47,48], but those cameras are not usually suitable for outdoor applications. More simplistically, we can use the Efficient Perspective-n-Point (EPnP) algorithm [49] to estimate an object’s pose using a calibrated camera and the correspondence between known 3D object points in the 2D RGB captured frame [50]. We can also compare the captured RGB frame with pre-computed features from the object (template matching), usually obtaining a rough pose estimation [46]. Currently, most applied pose estimation methods are not based on hand-crafted features and algorithms, but rely on deep learning methods using a structure adapted to the application. Sometimes, this structure is poorly optimized, using very deep networks and many parameters without specific design criteria, slowing the training and prediction. Usually, some of the deep learning pose estimation methods start with the detection of a Region Of Interest (ROI) in the frame (object position on the captured frame) [51,52] and then perform pose estimation using additional methods [53,54]. Alternatively, the Deep Neural Networks (DNNs) that perform the detection and pose estimation directly in an encapsulated architecture are increasing and represent the vast majority of the currently existing applications [55,56,57].

2.2. Pose Tracking

The main difference between pose estimation and pose tracking relies on using temporal information to perform the estimation. This temporal information can be only from the last frame or even from other instants in time. Some methods, as described for pose estimation, also use RGB-D cameras to perform data fusion for tracking [58,59,60]. We can use a system designed for single- [61] or multi-object [62] tracking that can perform the tracking of unknown objects (without any a priori information) [63,64] or use the object information (e.g., size or color) [65,66] to develop our tracking algorithms. Currently, most applied pose tracking methods rely on deep learning methods to obtain better performances without having to hand-craft features that can discriminate the tracked object on the search space (image frame when using a RGB camera). Some applications use the transformer architecture, which can be combined with Convolutional Neural Networks (CNNs) [67,68] or used alone to perform tracking [69]. Other deep learning approaches use Siamese networks [70] or Energy-Based Models (EBMs) [71,72] to perform tracking.

2.3. UAV Tracking

According to the UAV’s intended application, we can change the used payload (e.g., RGB or Infrared Radiation (IR) cameras) to be able to perform the mission. One of the most-common UAV applications is surveillance, where we can also automatically track the detected frame objects [73,74,75]. We can also use data fusion techniques to perform Simultaneous Localization And Mapping (SLAM) [76,77], which can also be used for autonomous landing in unknown environments [78]. When we want to perform tracking for landing purposes, we have to consider whether we are considering a fixed-wing [79] or a multirotor [80] UAV application. Regarding the UAV-based applications, where the data acquisition and processing are performed in the UAV, we can integrate visual and inertial data [81] or integrate GPS, laser rangefinders, and visual and inertial data [82]. Usually, and especially if we are considering a small-sized UAV, the onboard processing computer does not have the necessary processing power to acquire and process the data in real-time. We can use a ground-based approach where the data can be acquired and processed using more advanced methods since we do not usually have tight payload restrictions. Regarding the ground-based applications, we can use stereo vision with a Pan–Tilt Unit (PTU) [83], multi-camera stereo vision with PTUs [84], multiple IR cameras [85], using passive Radio Frequency (RF) reflectors on the runaway [86], or using a monocular vision system [41,42]. In this article, we explored a ground-based approach for performing autonomous landing during the daytime on a ship (moving platform) using advanced algorithms, whose processing is impossible on low-power processors. To perform an autonomous landing during the nighttime, we must use a different sensor (e.g., IR camera) and adapt the algorithm. The outdoor environment of the intended application is another challenge since we cannot use algorithms that use, e.g., color information, since we were subject to several illumination changes.

3. Overall System Description

An autonomous landing system’s final objective is to use the sensor information to control the trajectory of the UAV. In this article, the main focus was on the use of the camera information (sensor) to perform pose tracking (measurement) over time (Figure 4).

Figure 4. Simplified control structure.

We used a ground-based monocular RGB camera system to sense the real world (capture image frames), using a frame rate of 30 Frames Per Second (FPS) and an image size of $1280\times 720$ pixels (width × height) [11,35]. Then, we combined that information with algorithms that use the UAV CAD model as a priori information. In this specific application, the camera should be located near the landing area to be able to estimate the UAV’s relative pose to the camera (translation and orientation) and perform tracking (Figure 5).

Figure 5. Simplified adopted landing approach: (i) landing area (red) and (ii) camera location (green).

Additionally, we only need to ensure communication with the UAV to send the needed trajectories to a simple onboard autopilot and to ensure also that the net-based retention system is installed and that it has the needed characteristics to ensure no damage to the UAV or ship (Figure 6). In a small-sized FPB, the expected landing area (net-based retention system available landing area) is about $5\times 6$ m, which is about 2.5-times larger than the used UAV model’s wingspan (Table 1).

Figure 6. Communication antenna (left) and net-based retention system (right).

Table 1. Used UAV characteristics’ description.

Wingspan:	1800 mm	Cruise Airspeed:	55 km/h ≅ 15.3 m/s
Length:	1200 mm	Autonomy:	≅2 h
Weight:	3 kg	Payload:	maximum 2 kg
Global Positioning System (GPS)
AHRS
Microcontroller-Based Autopilot

We used a small-sized fixed-wing UAV (Figure 7), with the characteristics described in Table 1. Due to its size and weight, the UAV’s take-off can be easily performed by hand, and all the focus will be on the landing maneuver, as described before. It is essential to develop autonomous methods that are more reliable than using a human-in-the-loop, being able to decrease the probability of an accident.

Figure 7. Used UAV model.

During the development of the adopted approach, we considered that (i) the UAV is a rigid body; (ii) the UAV’s mass and mass distribution remain constant during operation; (iii) the UAV’s reference frame origin is located in its Center Of Gravity (COG). The UAV’s state is represented according to the camera reference frame according to [41]:

$${\mathbf{x}}_t={[{\mathbf{u}}_t,{\mathbf{v}}_t,{\mathbf{q}}_t,{\mathbf{\omega }}_t]}^T$$

(1)

where ${\mathbf{u}}_t^T=[X,Y,Z]$ is the linear position, ${\mathbf{v}}_t^T=[x_x,v_y,v_z]$ is the linear velocity, ${\mathbf{q}}_t^T=[q_x,q_y.q_z,q_w]$ is the angular orientation (quaternion representation), and ${\mathbf{\omega }}_t^T=[{\omega }_x,{\omega }_y,{\omega }_z]$ is the angular velocity according to the camera reference frame represented in Figure 8.

Figure 8. UAV and camera reference frames [41].

As described initially in Section 1, the proposed tracking architecture is divided into (Figure 9) [41,42,43,87]:

Figure 9. Pose tracking architecture (simplified).

• Pose Boosting (Section 3.1)—In this stage, the UAV is detected in the captured frame using deep learning, and filter initialization is performed using a pre-trained database. In each iteration, information from the current frame (using the pre-trained database) using the most-current observation to add particle diversity is also used;
• Tracking (Section 3.2)—In this stage, we used a 3D-model-based approach based on a PF to perform tracking. By applying temporal filtering, we improved the accuracy by being able to minimize the estimation error;
• Pose Optimization (Section 3.3)—Since we used a similarity metric to obtain the particle weights, we added a refinement step that uses the current frame (current time instant) and the current estimate to search for a better solution.

3.1. Pose Boosting

This stage was initially inspired by the Boosted Particle Filter (BPF) [88], which extends the Mixture Particle Filter (MPF) application [89] by incorporating Adaptive Boosting (AdaBoost) [90]. We adopted the approach described in [41,42,43,87], characterized by the following two stages:

• Detection—In this stage, we detect the UAV in the captured frame using the You Only Look Once (YOLO) [91] object detector. Target detection is critical since we were operating in an outdoor environment and we can be in the presence of other objects that can affect the system’s reliability;
• Hypotheses generation—From the ROIs obtained in the detection stage, we cannot infer the UAV’s orientation, but only its 3D position. To deal with this, we obtained the UAV’s Oriented Bounding Box (OBB) and performed the comparison with a pre-trained database of synthetically generated poses to obtain the pose estimates (Figure 10).
  

  Figure 10. Best database matches (examples).

3.2. Tracking

We adopted a PF approach based on the Unscented Particle Filter (UPF) [92], whose standard scheme is divided into three stages: (i) initialization, (ii) importance sampling, and (iii) importance weighting and resampling. The initialization was performed in the pose boosting stage described in Section 3.1. The main difference, when compared with the standard UKF is in the importance sampling stage, where we used a UKF for the translational motion and a UBiF/UBiGaF for the rotational motion to be able to incorporate the current observation and generate a better proposal distribution. The adopted tracking stage is divided into the following two stages [41,42]:

• Proposal (Section 3.2.1)—In this stage, we generate the proposal distribution, which should be as close as possible to the true posterior distribution;
• Approximate weighting and resampling (Section 3.2.2)—In this stage, we approximate the particle weights (approximate weighting) by using a Distance-Transform-based similarity metric. After evaluating the particles, we apply a resampling scheme to replicate the high-weight ones and eliminate the low-weight ones.

3.2.1. Proposal

Motion filtering has the purpose of using measures affected by uncertainty over time and being able to generate results closer to reality. We adopted the motion model (translational and rotational model) described in [12,41,42], where a UKF is applied to the translational filtering and a UBiF/UBiGaF to the rotational filtering. The Bingham (Bi) distribution is an antipodally symmetric distribution [93] used by the UBiF to better quantify the existing uncertainty without correlation between angular velocity and attitude. On the other hand, the UBiGaF uses a Bingham–Gauss (BiGa) [45] distribution, which takes into account this uncertainty in the filtering structure. The UBiF update step is simple to implement since the product between two Bi distributions is closed after renormalization. Still, the same does not happen in the BiGa distribution, which needs to have an update step based on the Unscented Transform (UT) [42,45,94]. We already explored both implementations in [12,41,42], it being possible to state that we have a clear performance improvement when compared with traditional methods, e.g., the UKF.

3.2.2. Approximate Weighting and Resampling

Usually, the object characteristics and the search space complexity are the primary sources of complexity in tracking. We used a small-size fixed-wing UAV (Figure 11), which is very difficult to detect at long distances since it is designed for military surveillance operations. On the other hand, the vast search space is outdoors, making the problem very complex, since we wanted to start detecting the UAV at least 80 m from the landing area.

Figure 11. Used UAV CAD model.

The UAV model symmetries are another problem since we used a 3D-model-based approach, where we need clearly to discriminate in the captured frame the correct UAV pose. In Figure 12, we see that symmetric poses have almost the same appearance, affecting the obtained results since we used a pixel-based similarity metric. The $\alpha$ angle represents the rotation around the camera x-axis; the $\beta$ angle represents the rotation around the camera y-axis; the $\gamma$ angle represents the rotation around the camera z-axis, as described in Figure 8. In this article, we implemented the Distance Transform (DT) similarity metric described in [43,95]. This similarity metric computes the distance to the closest edge between the DT [96] of the captured frame and the edge map [97] of the pose hypothesis to compare. The DT similarity metric is obtained according to:

$$S={exp}^{-\sigma s}$$

(2)

where $\sigma$ is a fine-tuning parameter and s is given by

$$s=\frac{1}{k}\sum _{i=0}^{B}E\left(h\left(i\right)\right)·D\left(f\left(i\right)\right)$$

(3)

where k is the number of edge pixels of the hypothesis to compare, B is the total number of image pixels, h is the pose hypothesis image, f is the captured frame, $D\left(f\right)$ is the DT of the captured frame, and $E\left(h\right)$ is the edge map of the pose hypothesis image to compare.

Figure 12. Symmetry analysis (examples). (a) $\alpha =0,\beta =0,\gamma =180$ (degrees). (b) $\alpha =0,\beta =180,$$\gamma =180$ (degrees). (c) $\alpha =90,\beta =0,\gamma =0$ (degrees). (d) $\alpha =90,\beta =0,\gamma =180$ (degrees).

Some of the most time-demanding operations in 3D-model-based pose estimation are particle rendering (pose hypothesis rendering) and evaluation [35]. Considering that, at each instant (system iteration), we evaluated hundreds of particles (or even thousands), we realized that we needed to optimize these operations. A complex problem to be solved is normally divided into smaller ones, and the same approach was adopted here regarding the UAV CAD model. We divided the UAV CAD model into the following smaller parts (Figure 13): (i) wing, (ii) elevator, and (iii) body. Since the adopted similarity metric can be computed independently for each UAV part, we can adopt a particle evaluation strategy where each part is evaluated independently. If needed, discard the evaluation sooner, or combine the obtained weights.

Figure 13. CAD model divided into parts: (i) wing (left), (ii) elevator (center), and (iii) fuselage (right).

To optimize the particle evaluation, a tree-based similarity metric approach is used (Figure 14), where each part is evaluated independently and sequentially (wing, elevator, and then, fuselage) compared with a predefined threshold $\tau$ (which can be different for each part). If the obtained weight S is smaller than a predefined threshold, the analysis avoids rendering the complete UAV CAD model. This allows us to use the available processing power better, only performing a complete model rendering and analysis for the promising particles. The adopted tree-based similarity metric for each particle is given by

$$w=C_1{S}_1+C_2{S}_2+C_3{S}_3$$

(4)

where $C_1,C_2,C_3$ are fine-tuning parameters, $S_1$ is the DT similarity metric for the wing part, $S_2$ is the DT similarity metric for the elevator part, and $S_3$ is the DT similarity metric for the fuselage part. By adjusting the threshold levels $\tau$ and the fine-tuning parameters, we can guarantee that the sum of the evaluations is equivalent to the evaluation of the whole model alone. This method can speed up the particle evaluation without a loss in accuracy.

Figure 14. Tree-based similarity metric approach. (i) wing, (ii) elevator, and (iii) fuselage similarity metric evaluation.

As described in [13], the resampling strategy that obtained better results in the problem at hand was the resampling reallocation, which was the used resampling scheme in the experimental results presented in Section 4.

3.3. Pose Optimization

In this stage, we performed a local search (refinement steps) in the particle neighborhood to optimize the used similarity metric using the current frame information. The adopted similarity metric is multimodal with more than one peak and cannot be optimized using gradient-based methods [41,43]. We adopted the Particle Filter Optimization (PFO) approach described in [43], which is based on the PF theory, but uses the same input (image frame) in each iteration. Using a zero velocity model, the PFO method is similar to a PF applied repetitively in the same frame. By adjusting the added noise to decrease over time, we can obtain better estimates and reduce the needed convergence time.

4. Experimental Results

This section describes the experimental results, in particular the tests (Section 4.1), the analyzed synthetic sequence (Section 4.2), the adopted performance metrics (Section 4.3), and the obtained pose tracking results (Section 4.4).

The method was initially implemented on a 3.70 GHz Intel i7-8700K Central Processing Unit (CPU) and NVIDIA Quadro P5000 with a bandwidth of 288.5 GB/s and a pixel rate of 110.9 GPixel/s. If we need to render and process 100 particles using the proposed filters, we will obtain approximately 27 FPS [9,13], which is a performance suitable for a real-time application [9,13,34,41,42,87]. Since we used a ground-based system without any power limitations and easy access to high processing capabilities, the obtained processing time was expected to be lower than the one described here.

4.1. Tests’ Description

The experimental tests described in Section 4.4 aimed to quantify the system’s performance, considering the similar metric analysis and the obtained pose estimation error in a realistic synthetic dataset (Section 4.2). Taking this into account, three different tests were performed with the following objectives:

• Similarity metric analysis (Section 4.4.1)—In this test, we had the objective of evaluating the tree-based similarity metric approach and comparing it with the standard DT similarity metric;
• Pose tracking without pose optimization (Section 4.4.2)—In this test, we had the objective of comparing the two filtering approaches when changing the number of used particles;
• Pose tracking with pose optimization (Section 4.4.3)—In this test, we had the objective of comparing the two filtering approaches when changing the number of pose optimization iterations using a constant number of particles.

4.2. Synthetic Sequence

Since there is no publicly available dataset and we could not acquire an actual image dataset until now, we used a realistic synthetic dataset (Figure 15). Based on some obtained experimental data, a UAV trajectory (translation and rotation) was created to be able to test the developed algorithms, as described in Figure 16. This landing sequence represents the expected trajectory of a landing UAV on an FPB.

Figure 15. Synthetic sequence (examples).

Figure 16. Tested landing sequence.

4.3. Performance Metrics

The particle with the highest weight value gives the UAV’s pose on each iteration before resampling. The translational error is given by the Euclidean error, and the rotation error is obtained according to

$$\delta \left(R_g,R_r\right)=\sqrt{\frac{\Vert {log}_m\left(R_g^T{R}_r\right){\Vert }_F^2}{2}}\frac{180}{\pi }\left[degrees\right]$$

(5)

where $R_r$ is the ground truth rotation matrix and $R_g$ corresponds to the retrieved rotation matrices (possibilities). We also obtained the Mean Absolute Error (MAE), Root-Mean-Squared Error (RMSE), and Standard Deviation (SD) to be able to perform a better performance analysis.

4.4. Pose Tracking

Since the knowledge has to be cumulative, the adopted filter parameters were the same as described in [42], allowing a performance comparison with the obtained experimental results. It is essential to consider that, in the implementation described in [42], a different similarity metric was used, and no pose optimization scheme was not used.

4.4.1. Similarity Metric Analysis

To be able to assess the pose evaluation performance, we performed six different tests (Table 2) on three different similarity metric approaches: (i) DT, (ii) tree-based DT, and (iii) weighted-tree-based DT ($C_1=1/3$, $C_2=1/3$, and $C_3=0.5/3$).

Table 2. Similarity metric tests’ description.

Number	Initial Pose $(\mathit{X},\mathit{Y},\mathit{Z},\mathit{\alpha },\mathit{\beta },\mathit{\gamma })$	Test	Interval	Sampling Rate
1	$(0,0,5,0,0,0)$	Rotation around X	$[-180,\dots ,180]$	> $1deg$
2		Rotation around Y	$[-180,\dots ,180]$
3		Rotation around Z	$[-180,\dots ,180]$
4		Translation around X	$[3,\dots ,7]$	$0.01$ m
5		Translation around Y	$[-0.24,\dots ,0.24]$
6		Translation around Z	$[-0.24,\dots ,0.24]$

After analyzing Figure 17, it is possible to see that the standard DT transform similarity metric obtained almost the same results as the tree-based DT approach. With a small change in the tuning coefficients ($C_1, C_2, C_3$), it is possible to adapt the output better, as described in Figure 17. The tree-based DT approach generated results close to the standard DT similarity metric. It can be used without increasing the error obtained, decreasing the particle evaluation processing time.

Figure 17. Comparison of the DT, tree-based DT similarity, and weighted-tree-based DT similarity metrics—obtained translation (meters) and rotation (degrees) errors.

4.4.2. Pose Tracking without Pose Optimization

The translation estimate error decreased with the increase of the particle number (Table 3 and Table 4). Although we used the UKF in both implementations for the translation estimation (UBiF and UBiGaF) since the orientation also influences the similarity metric value, we obtained better translation results in the UBiGaF case.

Table 3. Translational error (meters) for the UBiF when changing the number of particles. The best results are in bold, and the worst in italics.

Number of Particles	50	100	250	500	1000	1500
Minimum	0.07	0.04	0.13	0.33	0.58	0.80
5th Percentile	0.23	0.24	0.55	0.70	0.98	1.72
25th Percentile	1.23	1.68	1.61	1.63	2.95	3.08
Median	4.53	4.73	3.42	3.49	4.84	5.34
75th Percentile	8.61	9.45	9.58	5.68	6.92	7.70
95th Percentile	15.37	19.66	15.65	12.87	10.24	10.25
Maximum	29.36	30.03	28.77	23.22	21.25	20.81
MAE	2.15	2.36	2.21	1.74	2.10	2.32
RMSE	2.04	2.20	1.90	0.84	0.63	1.02
SD	5.50	6.13	5.71	3.89	3.09	2.97

Table 4. Translational error (meters) for the UBiGaF when changing the number of particles. The best results are in bold, and the worst in italics.

Number of Particles	50	100	250	500	1000	1500
Minimum	0.02	0.02	0.12	0.23	0.55	0.86
5th Percentile	0.17	0.15	0.32	0.67	1.18	2.07
25th Percentile	0.96	0.92	1.17	1.80	2.71	3.05
Median	4.12	4.32	3.25	3.10	4.74	5.19
75th Percentile	7.76	8.20	8.25	5.31	6.52	7.34
95th Percentile	14.47	16.35	16.07	13.21	8.72	8.71
Maximum	31.41	29.69	28.80	23.01	20.14	18.14
MAE	1.93	2.08	2.06	1.68	1.99	2.21
RMSE	1.85	2.00	1.87	0.82	0.68	1.03
SD	5.00	5.52	5.57	3.77	2.84	2.46

Regarding rotation, the best compromise between speed and accuracy (particle number vs. obtained results) was obtained when using 100 particles (Table 5 and Table 6). The UBiF obtained better results when compared to the UBiGaF. Nevertheless, both filters obtained very good results given the realistic representation given by the used synthetic environment.

Table 5. Rotation error (degrees) for the UBiF when changing the number of particles. The best results are in bold, and the worst in italics.

Number of Particles	50	100	250	500	1000	1500
Minimum	0.63	0.35	1.12	0.94	0.47	1.90
5th Percentile	1.97	1.64	2.71	5.04	6.43	6.25
25th Percentile	4.02	3.80	5.00	9.34	11.15	12.37
Median	7.47	7.14	7.73	13.00	16.54	17.66
75th Percentile	12.75	12.60	12.37	17.12	20.82	22.87
95th Percentile	23.28	24.51	22.80	25.88	40.60	40.97
Maximum	50.46	39.89	50.56	52.06	51.77	51.45
MAE	4.35	4.33	4.58	6.76	8.72	9.29
RMSE	1.58	1.84	1.73	1.54	1.53	1.72
SD	7.12	7.12	6.49	6.95	9.42	9.44

Table 6. Rotation error (degrees) for the UBiGaF when changing the number of particles. The best results are in bold, and the worst in italics.

Number of particles	50	100	250	500	1000	1500
Minimum	0.51	0.14	1.04	1.50	1.68	2.16
5th Percentile	2.50	2.24	3.07	5.00	6.48	7.88
25th Percentile	5.30	4.83	6.11	9.63	12.38	13.76
Median	9.94	8.92	9.76	13.59	17.63	18.88
75th Percentile	15.75	13.83	14.42	18.31	22.55	24.83
95th Percentile	26.95	23.72	23.32	28.06	41.46	44.15
Maximum	69.48	51.83	41.18	41.84	50.94	57.29
MAE	5.37	4.78	5.22	7.09	9.28	10.26
RMSE	1.69	1.43	1.32	1.56	1.32	1.32
SD	8.25	7.19	6.35	6.96	9.58	10.27

The comparison between the Euclidean error and the rotation error for the UBiF and UBiGaF when using 100 particles can be seen in Figure 18. Both filter implementations had some associated error, with higher variations given by the UBiGaF implementation. This happened mainly due to the correlation between the orientation and the angular velocity, making the output noisier (Figure 19).

Figure 18. Obtained error using 100 particles: (i) translation (left) and (ii) rotation (right).

Figure 19. Obtained error using 100 particles—UBiF and UBiGaF.

4.4.3. Pose Tracking with Pose Optimization

In this test, we used 100 particles, which corresponded to the best-obtained results described in the previous subsection, and changed the number of pose optimization iterations. The translation estimate error decreased with the increase of the pose optimization iterations (Table 7 and Table 8). Although we used the UKF in both implementations for the translation estimation (UBiF and UBiGaF), we also obtained better results in the UBiGaF case.

Table 7. Translational error (meters) for the UBiF when changing the pose optimization iterations. The best results are in bold, and the worst in italics.

Iterations	0	1	2	3	4	5	6	7	8
Minimum	0.04	0.05	0.06	0.04	0.06	0.05	0.05	0.05	0.05
5th Percentile	0.24	0.24	0.25	0.23	0.31	0.22	0.22	0.22	0.25
25th Percentile	1.68	1.51	1.68	1.37	1.63	1.47	1.35	1.55	1.40
Median	4.73	4.50	4.70	4.57	4.45	4.62	4.45	3.91	4.37
75th Percentile	9.45	9.50	9.40	9.01	8.57	8.48	8.29	8.36	8.14
95th Percentile	19.66	18.92	17.32	17.29	16.71	17.07	15.71	16.47	15.54
Maximum	30.03	30.17	30.17	30.13	28.97	31.41	29.58	29.66	31.35
MAE	2.37	2.32	2.29	2.22	2.14	2.17	2.08	2.07	2.05
RMSE	2.20	2.15	2.13	2.09	1.98	2.04	1.95	1.93	1.91
SD	6.13	5.98	5.75	5.53	5.28	5.52	5.29	5.37	5.25

Table 8. Translational error (meters) for the UBiGaF when changing the pose optimization iterations. The best results are in bold, and the worst in italics.

Iterations	0	1	2	3	4	5	6	7	8
Minimum	0.02	0.05	0.03	0.03	0.05	0.02	0.04	0.05	0.05
5th Percentile	0.15	0.24	0.16	0.17	0.20	0.23	0.21	0.22	0.22
25th Percentile	0.92	0.98	0.95	0.97	0.99	0.90	0.99	1.06	0.91
Median	4.32	4.09	4.36	4.28	4.40	4.22	4.05	4.31	4.12
75th Percentile	8.20	7.82	8.44	8.05	7.97	8.12	7.93	7.82	7.82
95th Percentile	16.35	17.36	18.77	14.79	14.14	14.23	14.06	14.81	13.97
Maximum	29.69	31.35	31.35	31.35	31.41	31.35	31.35	31.41	31.35
MAE	2.08	2.05	2.15	2.01	1.98	1.99	1.96	1.97	1.94
RMSE	2.00	1.97	2.07	1.94	1.90	1.90	1.88	1.89	1.85
SD	5.52	5.69	5.93	5.29	5.07	5.20	5.13	5.10	5.10

Regarding rotation, the best compromise between speed and accuracy was obtained using two pose optimization iterations (Table 9 and Table 10). We can see some refinement in the obtained results, and we obtained better results when using pose optimization when compared with the particle number increase. We increased the algorithm’s accuracy by searching locally for a better solution. Combined with the tree-based DT similarity metric, this decreased the obtained error and the needed processing time.

Table 9. Rotation error (degrees) for the UBiF when changing the pose optimization iterations. The best results are in bold, and the worst in italics.

Iterations	0	1	2	3	4	5	6	7	8
Minimum	0.35	0.07	0.61	0.65	0.44	0.26	0.20	0.18	0.33
5th Percentile	1.64	1.83	1.43	1.58	1.56	1.80	1.60	1.43	1.63
25th Percentile	3.80	3.65	3.35	3.41	3.24	3.32	3.54	3.60	3.60
Median	7.14	7.06	6.44	6.46	6.67	7.09	6.80	7.30	6.93
75th Percentile	12.60	12.16	12.21	12.14	12.33	12.36	13.00	13.11	12.71
95th Percentile	24.51	23.78	23.30	24.10	23.93	24.29	24.15	23.45	23.67
Maximum	39.89	50.54	50.46	51.80	53.57	50.55	50.49	50.53	50.49
MAE	4.33	4.16	4.05	4.07	4.11	4.27	4.25	4.30	4.23
RMSE	1.84	1.80	1.73	1.82	1.78	1.84	1.75	1.75	1.78
SD	7.12	6.95	7.20	7.67	7.63	7.52	7.65	7.51	7.41

Table 10. Rotation error (degrees) for the UBiGaF when changing the pose optimization iterations. The best results are in bold, and the worst in italics.

Iterations	0	1	2	3	4	5	6	7	8
Minimum	0.14	0.17	0.47	0.16	0.64	0.10	0.18	0.37	0.77
5th Percentile	2.24	2.37	2.29	2.29	2.15	2.19	2.34	2.62	2.50
25th Percentile	4.83	5.33	5.21	5.50	5.01	5.00	5.30	5.54	5.12
Median	8.92	9.69	9.78	9.78	9.48	9.84	10.50	10.49	10.01
75th Percentile	13.83	14.67	14.49	14.88	14.43	15.51	16.61	16.71	16.14
95th Percentile	23.72	25.06	22.68	24.81	23.76	23.81	27.75	27.60	26.73
Maximum	51.83	52.06	46.06	60.02	52.23	68.93	64.86	63.52	60.76
MAE	4.78	5.23	4.96	5.14	5.02	5.08	5.59	5.66	5.44
RMSE	1.43	1.55	1.64	1.53	1.88	1.83	1.85	2.00	1.75
SD	7.79	7.53	6.73	7.34	7.42	7.67	8.87	8.58	8.53

The comparison between the Euclidean error and the rotation error for the UBiF and UBiGaF when using 100 particles and two pose optimization iterations can be seen in Figure 20. Both filter implementations had some associated errors, with some variations, but they were a bit smoother than those shown in the previous subsection.

Figure 20. Obtained error using 100 particles and two pose optimization iterations: (i) translation (left) and (ii) rotation (right).

Figure 21 describes in detail the obtained translation and rotation errors, and it is possible to see that the UBiGaF presented fewer jumps in the orientation estimation when compared with the UBiF when using pose optimization. This was a clear difference and improvement in performance when compared with the results obtained in the previous subsection. The results obtained when implementing these methods can be seen in Figure 22 and Figure 23, where the estimated pose obtained is represented in red.

Figure 21. Obtained error using 100 particles and two pose optimization iterations—UBiF and UBiGaF.

Figure 22. UBiF—100 particles and two pose optimization iterations (estimate represented in red).

Figure 23. UBiGaF—100 particles and two pose optimization iterations (estimate represented in red).

5. Conclusions and Future Work

It is expected to obtain better estimates and achieve real-time implementation by continuously improving and exploring the developed system. Directional statistics are critical to represent the orientation better since we can obtain lower estimation errors. In a world wholly taken over by deep learning approaches, it is essential to continue to explore more traditional techniques, continuously developing the knowledge in the field. The 3D-model-based tracking approach is not simple, especially outdoors, where we cannot control the environmental conditions. The similarity metric was not based on specific color information, but directly on the DT calculation to deal with that design constraint. The similarity metric decreased the processing time without a loss in the accuracy of the result. The obtained translation and rotation errors decreased with the UAV’s proximity to the camera since we had more pixel information. The obtained errors were compatible with the automatic landing requirements since we expected a minimum available landing area of 5 meters. This area is typically available in a small patrol boat that performs surveillance tasks or even conducts search and rescue operations. Future work will focus on improving the algorithm’s accuracy and obtaining a real dataset that can be used to improve the algorithm. This work will also contribute to developing other works in the field, since we plan to make it publicly available.