A Robust and Accurate Landing Methodology for Drones on Moving Targets

: This paper presents a framework for performing autonomous precise landing of unmanned aerial vehicles (UAVs) on dynamic targets. The main goal of this work is to design the methodology and the controlling algorithms that will allow multi-rotor drones to perform a robust and efﬁcient landing in dynamic conditions of changing wind, dynamic obstacles, and moving targets. Unlike existing GNSS-based vertical landing solutions, the suggested framework does not rely on global positioning and uses adaptive diagonal approaching angle visual landing. The framework was designed to work on existing camera-drone platforms, without any need for additional sensors, and it was implemented using DJI’s API on Android devices. The presented concept of visual sliding landing (VSL) was a wide range of commercial performing


Introduction
Over the last decade, in light of technological advances, the use of drones has increased significantly. Today, multi-rotor drones (with vertical take-off and landing capabilities) are becoming popular both in commercial and military sectors for several usages such as mapping, surveying, remote sensing, inspection, search and rescue, filming, recreational, and sports [1,2]. Most commercial drones are equipped with GNSS (Global Navigation Satellite System) receivers which allow them to perform RTH (return to home); in most cases, this operation is performed by flying at a fixed height to a point above the launch location and then performing a vertical landing. This paper presents a framework for performing autonomous VSL on dynamic targets (see Figure 1).
The main goal of this work is to design the methodology and the controlling algorithms that will allow unmanned aerial vehicles (UAVs) to perform a robust and efficient landing in dynamic conditions include wind, dynamic obstacles, and moving targets. The presented concept was implemented and tested on a wide range of commercial drones, performing hundreds of autonomous landings on moving targets (including boats, RC-boats, and RC-rovers).

Related Works
In the past two decades, many researchers and commercial companies have suggested different methods of approaching the problem of autonomous precision landing, using several sensors and tools to recognize the "landing area" and land on it.
• GNSS-based landing: This method is commonly used by commercial drones [3]. For improved accuracy, the use of differential GNSS [4] and RTK (real-time kinematics) [5] were suggested. In those methods, additional sensors such as INS (inertial navigation sensors), altimeter, or barometric pressure sensors are commonly used in conjunction with GNSS. • Visual processing landing: GNSS signals may not be available, therefore GNSS-based autonomous landing may not be possible in many remote regions. Optical flow seems to be a significant capability regarding autonomous flying. Therefore, researchers suggested vision-based solutions. Those solutions require a camera and an algorithm to detect the landing area, allowing the drone to track and maintain its position.
Research studies have suggested several methods of detecting the landing target: an algorithm that identifies an "H-shaped" landing target using invariant moments [6], a black and white pattern consisting of six squares of different sizes [7], or a target design with four white rings [8]. Recently, researchers have began to use ArUco markers as targets [9][10][11]. Alternatively, the use of April QR codes combined with simulated GNSS signals was suggested as a landing method for micro drones in windy conditions [12]. • Integrated systems: Up to the last decade, vision sensors were relatively expensive, heavy, and complicated; see [13]. The use of onboard GNSS sensor, fused with vision (or LiDAR) sensor, was suggested: e.g., [14,15].
• Ground-based system: Using a ground-based optical (or radar) system, the position and the velocity of the drone can be detected (with respect to the ground system). Thus, the ground station can direct (control) the drone on how and where to land [16,17]. • Drone in a box: The concept of a drone in a box [18] is designed to support an autonomous drone system where it allows a drone that performs remote automatic operations to nest inside an enclosure for charging and safe harbor. There are several "drone-in-a-box" systems available commercially, such as Airobotics and Percepto, and DJI.
Naturally, the emerging research field of "autonomous UAVs" is very wide and includes many related topics which are not covered in this paper. For a recent survey on safe landing techniques for UAVs, see [19]; for a general survey on challenges related to "urban air mobility", see [20].

Motivation
This research was motivated by the massive amount of commercial camera drones. Such drones have the ability to "return to home" (RTH), and can land vertically on a given position. Yet, such landings are restricted to vertical approach, and have limited accuracy, due to the GNSS inherent inaccuracy. Moreover, landing a drone vertically on a target implies that at the final stage of the landing the drone might not be able to track the target (as the target might be too close to the drone's camera). Yet, this is the most critical stage and with the additional complexity of the "ground-effect" [21], the drone may miss the target in the last 10 cm of the landing stage. In case of landing on a moving target, the outcome of such inaccuracy may be significant.
In this paper, we present a framework which allows "toy-grade" camera drones to perform a sliding (diagonal) accurate landing on moving targets. The suggested framework allows a drone to perform a diagonal landing (from vertical to almost horizontal) while maintaining a continence tracking of the reference target according to which it lands, as shown in Figure 1.
Our main motivation was to suggest a robust dynamic landing method which is applicable for most commercial drones without the need to add any additional hardware or sensors to the existing drones.

Our Contribution
In this paper, we present a new concept for autonomous precision landing for UAVs, designed for landing on dynamic helipads (landing plates that can move in all six DOF; heave, sway, surge, yaw, roll, and pitch). To make our solution accessible to the common user, we implemented and tested the suggested landing method on a wide range of commercial drones. The suggested method allows accurate landing in an indoor and outdoor environment, with a wide range of diagonal approaching angles; all the range from vertical to horizontal. Another innovation of the suggested method involves the control of the gimbal which allows a continence tracking of the target while maintaining an adjustable approaching angle to the helipad. To the best of our knowledge, this is the first paper suggesting a gimbal-controlled visual sliding (autonomous) landing, applicable for almost any commercial camera drones and using software-only solution.

Modeling a Commercial Camera Drone
In this section, we model a general multi-rotor camera drone, in terms of sensors, remote-control functionality, and existing landing capabilities. For the mathematical modeling of multi-rotor drones and their controlling mechanisms, see [22].

Drone Basic Sensors
Our main purpose in this article is to show the concept of a generic solution for common drones. For our method to work, the drone needs to have onboard optical flow sensor, camera, gimbal, IMU, and GNSS, which can be found on most common COTS drones.
The drone needs to be able to The gimbal pitch range needs to be from −90 • to +20 • (see Figure 2).

Remote Control API
A common remote controller (see Figure 3) allows manual flight, gimbal and camera control, and provides a robust wireless control link for the drone.
The remote controller has sticks, wheels, switches, and buttons that give control. It has a USB connector for mobile devices.
These days, most controllers can link to a mobile device and the mobile device can communicate with the drone and receive the live video stream from the camera or send it commands that replace the remote controller buttons.

Modeling the Landing Process Commercial Drones
We can divide the drone world into CTOL (conventional take-off and landing) and VTOL (vertical take-off and landing); each has its own advantage. In this section, we would like to describe how a VTOL drone lands.

Landing Using GNSS
As we describe in the Related Works section, landing using GNSS is a common landing method; most GNSS drones have a return to home (RTH) or return to lunch (RTL) functionality. In the RTH method, the UAV records the home coordinate points; when activated, the drone will plan the short path to the home coordinate points and fly to them while keeping a certain altitude from the landing area. When the drone arrives above the landing area, it concentrates in relation to the home coordinates and lands.
The main problem of this method is that the drone cannot return to the home point when the GNSS signal is weak or unavailable. It lacks the accuracy and the versatility to adjust if the target moves or the signal is lost.

Vision-Based Landing
In this section we present the main concepts of visual landing on a known target, in particular, approximating the distance, the relative angle, and the relative velocity between the drone and the target. We start by presenting the basic concept of vertical visual landing-as often implemented in commercial drones.

Target
Vision-based solutions have many benefits: they are more accurate and can work indoors and outdoors in a dynamic environment. Most solutions rely on object recognition and edge detection techniques, which are improving every day. In vision-based solutions we want to be able to recognize the target, be able to concentrate due to the target, and be able to measure the distance from the target. In most vision-based solutions, the target is well known (predefined), which allows the drone to recognize and extract important information from the target's known properties. For example, since we know the target's real size, the distance to the target can be estimated. For future work, we would like to generalize the suggested method to "unknown targets" (not just predefined targets or known helipads), allowing the drone to spot "landable regions" and perform the VSL method on such occurring targets.

Controller Mode
When using vision-based landing there are three ways to approach the target. Height : The height from the target. We will use parameters from downward vision system sensors.
Concentration: Centering the camera relative to the center of the helipad. Distance : The distance from the target. To estimate the distance from the target we will use image processing (see Figure 4). We need to set a PID controller (see Figure 5) for each mode in order to fix the drone's desired position as smoothly and accurately as possible.

Distance Calculation Method
We can use a simple method in order to calculate the distance between the camera to the helipad. First, we will have to calibrate our system and find the focal length of the camera.
where: F = is the focal length of the camera. W = is the known width of the marker. P = is the apparent width in pixels. D = is the distance from the camera to the object.
After finding the focal length we can use the same equation to find the distance between the camera to the helipad.
This method will solve the 2D problem. In order to solve the 3D problem, we need to define our coordinate system (see Figure 6). The next thing is to find the rotation matrix of the local system. In order to achieve this, we need to find the rotation matrix of the drone and the rotation matrix of the gimbal in relation to the target coordinate system.
The rotation matrix of the drone can be described by the following equation: The pitch rotation matrix of the gimbal can be described by the following equation: The matrix rotation of the local system will be Another way to estimate the distance while avoiding using the rotation matrix is to measure the distance to each corner from the center and divide by four, and we obtain a good approximation of P (P is the apparent width in pixels).

Camera Gimbal Position
There are two ways to set the camera gimbal angle; the first approach is to fix the gimbal angle and the second approach is to let the gimbal be more flexible in order to adjust the angle due to the drone position.

Angle of −90 •
The basic method for searching the target is to set the drone gimbal to be fixed to −90 • . The helipad target will lay on the ground. The general steps in this method will be as follows (see Figure 7): First we need to find the target. Next, we will centralize the drone on the helipad center and hover over the target. The final step is a safe landing. The main problem with this method is losing sight of the target when the drone is too close to it, which can cause problems if the drone lands on a dynamic target that can suddenly change its height, for example, a target on a boat. Angle of −45 • In this method, we will use one helipad target that will lie vertical to another helipad. The distance between the two helipads is fixed. The drone gimbal will be fixed to −45 degrees. The general steps in this method will be as follows (see Figure 8): First, we need to find the target. Next, we will reduce the distance to the target till we reach the "landing distance". Next, we will centralize the drone on the target center and hover over the helipad. The final step is a safe landing.
The main problem with this method is that it is very difficult to identify the target unless the drone is in front of it.

Adjustable Angle
In this method, we will set two helipad targets, one bigger than the other. The biggest one lies on a horizontal surface. The second helipad target lies on a plain at a slope. The distance between the the two helipads is fixed. The drone's gimbal pitch angle will be flexible and the angle can be adjusted from −90 deg to +20 deg. The general steps in this method will be as follows (see Figure 9): First, we will find the largest helipad target. Next, we will start to reduce the distance to the helipad target while changing the gimbal angle till we detect the second target.
We will reduce the distance to the second target till we reach the "landing distance". Next, we will centralize the drone on the second target center and hover over the helipad target.
The final step is to start the safe landing while changing the gimbal angle. That way we will not lose sight of the second target while we are landing. Figure 9. Scanning for first target, detecting first target, distance reducing and angle changing, detecting second target, distance reducing, centralizing, and landing.

Drone Landing Methodology
In this section we will present the overall VSL concept. We start by describing what we consider as a successful landing. The next step will be to describe the milestones in the basic landing algorithm and define the state machine.

Simulation of Autonomous Successful Landing
To illustrate a successful landing, several aspects need to be considered, such as the accuracy in landing and finding an efficient landing trajectory that allows reaching the goal in the shortest time and will be the most energy-efficient. And of course, will allow us to avoid objects that will interfere with the landing (see Figure 10).

Controlling Algorithms
This section covers the control state machine and algorithm for the drone's autonomous visual slide landing (VSL). We start by presenting the overall concept of VSL, including the landing platform. The landing platform includes two main targets: (i) helipad; (ii) guiding target (see Figure 11).        Informally the VSL use-case contains the following four steps: (a) At first, the drone tries to detect the helipad which is relatively horizontal and large and can be detected from a significant distance. (b) Then the drone approaches the helipad, while maintaining it in the center of the camera's FoV. While approaching, the drone gimbal camera is tilted with respect to the distance between the drone and the helipad. (c) Once the drone can robustly detect the guiding target (which is smaller than the helipad and is oriented in a slope), the drone position itself at a fixed distance and angle relative to the guiding target. (d) In the final stage, the drone descends on the helipad vertically, while maintaining a fixed distance to the guiding target.
Before presenting the algorithm, we define several possible modes for the drone state machine (see Figure 12).

VSL Algorithm
The visual sliding landing (VSL) algorithm is based on the states defined above and state machine (see Figure 12).

•
Disarmed: The drone is on the ground not armed. • Arm: The drone is on the ground armed (ready to take off). • Take Off: The drone starts flying upwards and reaches a predefined altitude (e.g., 1 m). • Mission: The drone initiates a predefined route (mission) which is composed of several waypoints. • Search Target: The drone is looking for a known target (e.g., QR code) (see Figure 13). • Leash Tracking: The drone maintains the distance and orientation from the target. The "leash" is relatively flexible, allowing a smooth tracking. Note that in this mode the relative velocity between the drone and the moving target is close to zero. • Centering: The drone maintains the detected target in the center of the image as captured by the drone gimbal camera. It can be achieved by using yaw roll and throttle; see Figure 14a-d. • Helipad Approach: Following the detection of the helipad target and executing centering process, the drone reaches a certain distance and angle to the helipad (i.e., approach funnel); see Figure 15a. • Guiding Target Tracking: This stage starts after the drone detects the guiding target. Then, it maintains "leash" and executes a centering process (according to the guiding target; see Figure 15b). • Gimbal Adjustment: The camera angle is moved according to the drop (e.g., distance 3 m, angle -45 • ).
• Final Approach: The drone performs a vertical landing while maintaining a fixed distance from the guiding target, and adjusting the gimbal according to the height and the required descent rate from the helipad. • Touchdown: On touching the helipad, the drone shuts down the motors, reports "landing", and moves to "arm" (or ready to fly) stage. • Fail-safe: The drone detects some kind of abnormality or risk, i.e., hardware malfunction, RC communication loss, or loss of sight of the target. The drone will return to a "safe-stage" according to the type of error and its fail-safe policy. After fixing the orientation we would like to shift the focus from the big target to the small target 366 and keep a "leash" of 1.5m from it while changing the gimbal angle to 0 • . The next step will be landing,

Fixed Camera Drones
Camera drones commonly have at least two axis gimbals, yet toy graded drones and FPV (first-person flight) drones often use a fixed (wide angle) camera for simplicity and reliability reasons. In order to adapt the suggested VSL method to drones with fixed (wide angle) camera, the following steps should be performed: (i) The wide-angle camera should be calibrated in order to improve the target detection and tracking. (ii) A virtual gimbal should be implemented on the camera image, by cropping the wide-angle frame into a standard FoV (field of view) image. (iii) In case the drone has standard FoV camera, the virtual gimbal can be implemented by changing the y-axis of the center point of the camera. In other words, the y-coordinate of the center of the image should be compensated according to the value as if the camera could be tilted by the gimbal.
The VSL algorithm can now be applied on the image as computed by the virtual gimbal. Thus, FPV drones and toy graded camera drones may also be using the suggested VSL method.

Safe Landing
As stated in the introduction, our main goal is to provide a landing method that could be robust in dynamic and challenging environments. Therefore, we would like to define safety envelope parameters and discuss how to proceed in "fail-safe" conditions, when the drone detects some kind of abnormality or risk. First, we define the safety envelope parameters (see Figure 16). The parameters can be divided into three groups: (i) Drone parameters: the drone working global behavioral parameters: e.g., max descent speed or max horizontal speed. (ii) Helipad parameters: e.g., type, size, or max slope (in our case, this should not be more than 20 • ). (iii) Drone to helipad parameters: e.g., drone to helipad relative speed, or marker tracking confidence (by the drone). The credibility factor is the overall combined parameter which takes into consideration all the parameters above and also checks for abnormality or risk. As mentioned, we also defined a fail-safe policy (see Figure 17).

Study Limitations
The presented framework was designed for commercial drones. Therefore, the following natural limitations should be dealt with. (i) Limited sensors: there are no airspeed sensors or lidar boards of most commercial drones. In fact, the suggested framework is applicable for any camera drone (even "toy-graded"), as the only sensors required onboard of the drone are IMU and camera. (ii) The autonomously landing controlling algorithms should be implemented on the remote control side, as most commercial drones do not have onboard SDK. This limitation introduces a controlling delay which should be compensated using the controlling algorithm. (iii) Safety: we limit the suggested framework to land-pads with predefined markers, and to operate within the safety envelope (as presented above).

Experimental Results
In this section, we present several landing experiments which were conducted in a few scenarios, including (i) static indoor landing (with no GNSS; see Figures 11b and 18a-e); (ii) dynamic landing on rovers and cars (see Figure 19); (iii) dynamic landing on sailing boat (see Figure 20    performing an adjustable tracking and approaching to a moving boat. This is an example of a case in which a diagonal sliding landing is needed as, due to obstacles such as the sails and wires, a vertical landing cannot be performed.

ArUco Tracking
In this work, ArUco markers were used as targets. Such targets can be detected using OpenCV library. Each detected target has both 2D and 3D parameters, including ID, center, corners, and orientation (normal). Given the camera's field of view (FoV) and the physical size of an ArUco marker, OpenCV allows computing the distance and the vector from the camera to the marker. Using the orientation of the camera (as given by the gimbal), one can compute the drone to marker relative position.
For our target, we use the ArUco markers "10" and "20" (see Figure 13). In order to estimate the accuracy of the ArUco target recognition by the drone's visual sensors, we used several popular commercial drones (including DJI's: Mavic Pro, Mavic Air, Mavic Air2, Mavic 2 Zoom, Mavic mini, and Phantom 4) and tested their landing performances using our framework. The suggested system was able to robustly detect and track a 160 × 160 mm ArUco markers within a range of 0.4-6.8 m, and an angular orientation of 0-60 degrees (between the target normal and the camera's center orientation). Using larger marker (500 × 500 mm) with VGA resolution, the tested drones were able to recognize the marker from a 20 m range (using 1080 p resolution, this range may be increased to 50 m). The detection remained reliable in relatively dark conditions (down to 20 lux); in low-light conditions (below 20 lux), a low power LED (0.1 watt) emitting the target was sufficient for performing safe landing in total darkness. It should be noted that few recent drones have a downwards LED to allow optical-flow sensors in low light conditions; such an LED is sufficient for marker tracing in a range of 5 m, yet it requires an almost vertical landing. Thus, equipping a drone with a smartphone LED flash should be sufficient for performing a VSL in any light conditions. Using VGA 640 × 480 resolution, the marker tracking process was able to recordin 20-30 fps (on a standard Android smartphone), while the overall latency of the system was commonly 60-100 milliseconds, which we found to be sufficient for performing landing on moving targets with low dynamics (such as boats). The average relative error was 1.07% and max relative error 4.8% (using uncalibrated Mavic mini drone). This leads to a 1-5 cm landing accuracy in the common case of landing within a 1 m from the target. For more insights regarding accurate tracking optimization of QR markers, see [23,24].

PID Calibration
As explained in the controller mode section, three important variables should be controlled: pitch, roll, and throttle. For each, we will have to find the right value of the PID parameters Kp, Kd, and Ki.
Kp is the proportional gain. Kd is the derivative gain.
Ki is the integral gain.
There are several methods for tuning a PID, such as the Ziegler-Nichols method, which is a heuristic technique and tuning software that uses a numerical optimization technique.
Since we do not have the mathematical model of the system, and common PID tuning method did not work for our drone, we find the right PID parameters manually for each mode. Our priority will be the following: 1. The drone needs to have smooth reactions. 2. The fastest settling time (the time our system will converge to steady state). 3. The max overshoot cannot be more than half of the distance from the target needed to land (the distance between the target to the helipad).
We increase the Kp value until it reaches the ultimate gain, at which the output of the control loop has stable and consistent oscillations, and define it as the upper bound. For the lower bound, we decrease Kp until the system does not reach the desired value. After analyzing the results of our calibration experiments (see Figure 18), the following parameters lead to the best results for distance mode: Kp = 0.23, Ki = 0.0001, and Kd = 0.01, while the roll and throttle modes were controlled mainly by a simple proportional controller: Kp = 0.135 (Ki = 0.0, Kd = 0.0). We found that the PID parameters that we gained were good enough for different environments, but for obtaining better results we need to perform fine-tuning.

Experiment System
We would like to try our method on both static and dynamic systems. Our experiment system requires an ArUco target and a helipad on wheels. We also need a smartphone with the app we created using Android Studio based on DJI SDK. Our API (see Figure 11) includes a live streaming area, data about the error output, and pitch, roll, and throttle input command. We are able to update the PID parameters for each pitch, roll, and throttle. The app also has three main buttons:

Hover and Landing
After fixing the orientation, we would like to shift the focus from the big target to the small target and keep a "leash" of 1.5 m from it while changing the gimbal angle to 0 • . The next step will be landing, so to accomplish this we close the distance to 1 m and change the gimbal angle to 20 • (see Figure 15).

Logging Results
Our API creates log data that log the drone flight parameters such as IMU, altitude, and the measurements gained from the image processing (see Figure 21).  There are two zones in the graph, "hover zone" and "land zone". The upper line represents the distance from the target, and the lower line represents the height from the surface. (c) Result of successful landing on static target.

Conclusions and Future Work
This paper presents the concept of diagonal autonomous visual landing on moving targets for commercial camera drones. The presented framework was designed and tested on a wide range of commercial drones in several scenarios, including (i) indoors: fixed targets, RC-rover moving targets; (ii) outdoors: boats, cars, and GNSS-denied urban cases.
There are several interesting directions for future work. We are currently working on improving the system to allow landing on high-speed targets. Another direction is to use the accurate landing capabilities in order to implement the concept of a "drone-in-a-box" for commercial off-the-shelf drones. Finally, we would like to generalize the suggested VSL solution for fix-wing VTOL drones.