Improving Up-Close Remote Sensing of Occluded Areas in Vineyards through Customized Multiple-Unmanned-Aerial-Vehicle Path Planning ^†

Abstract

This study presents a novel approach to address challenges regarding data acquisition for object detection and tracking purposes by enhancing UAV path planning specifically designed for fruit detection in woody crops trained on vertical trellises, considering the biophysical environment of the field. The proposed method implements the Ant Colony Optimization (ACO) algorithm and enables single and multiple UAVs to fly synchronously while ensuring a safe distance between platforms. The results highlight that ACO is able to generate optimal and safe routes, considering the vegetation and covering the whole agricultural area. Moreover, it shows potential to solve partial leaf occlusion for fruit identification.

1. Introduction

There is a current trend in precision agriculture which is focused on woody crops, such as vineyards [1] and fruit orchards [2], with a strong emphasis on using deep learning for fruit detection [3,4,5]. However, these studies primarily study object detection, computer vision methods, and their metrics without considering the method used for data acquisition or the most efficient path to collect it.

Path planning involves finding a suitable route from a starting point to a goal point while considering obstacles to avoid collisions [5]. In the context of aerial path planning for unmanned aerial vehicles (UAVs), optimization is critical due to limited autonomy and energy consumption constraints [6,7,8]. Unlike ground robots used in agricultural fields, UAVs can fly above obstacles and do not need to consider the topography [9], [10] since they are not affected by those constrains. Furthermore, aerial path planning can be executed using one or multiple UAVs working cooperatively. In Precision Agriculture, where large fields need to be covered efficiently, employing multiple UAVs can reduce the mission time and increase the area coverage. However, it is relevant to take into account during the whole mission the safety distance between platforms, which is influenced by flight speed and UAV size [11].

All in all, it is relevant to change the focus of attention from deep learning algorithms and their metrics to a step prior to that: the proper and efficient acquisition of the datasets to be employed. The importance of carefully planning missions and paths in real-world environments is emphasized, as many fruit detection algorithms are trained under artificial conditions, such as plants with leaf-removal [12,13], making them less robust in challenging and realistic settings. This study presents a novel approach for path planning that has the potential to improve fruit detection and assessment in vineyards trained in vertical trellis.

2. Materials and Methods

In order to customize multiple-UAV path planning, the first step is to identify the biophysical environment of the field through a nadir UAV flight, from which an orthomosaic will be computed to extract a Canopy Height Model (CHM). Once the specific characteristics of the field have been identified, the actual path planning can be designed, which will be executed during a second UAV mission.

2.1. Data Acquisition

The vineyard (Vitis vinifera cv. Loureiro) utilized for this study is located in ‘Tomiño, Pontevedra’, Galicia, Spain (X: 517186.7, Y: 4645072.3; ETRS89/UTM zone 29 N), and is owned by ‘Bodegas Terras Gauda, S.A’. The distance between plants and rows was 2.5 m × 3 m, respectively. The first flight, also called the survey flight, was carried out in 2021 at 30 meters above sea level. The platform used was a DJI Matrice 210 (DJI Sciences and Technologies Ltd., Shenzhen, Guangdong, China), equipped with a Micasense RedEdge 3 multispectral camera (AgEagle Sensor Systems Inc., Wichita, KS, USA).

2.2. Survey Flight

The survey flight’s purpose is to identify regions of interest, above which the UAV will acquire data, and obstacles or regions without interest, above which the UAV will not fly nor collect data. For that, the CHM was derived from the orthomosaic, and only areas between 0.5 and 2 meters were selected since those include the minimum and maximum heights of the vine plants. The rows which had missing plants or high trees surrounding them were selected as Forbidden, whereas the other areas were marked as regions of interest (ROI).

The next step was to design the optimal path, considering the ROIs and the Forbidden areas. An Ant Colony Optimization (ACO) algorithm [14] was selected for this study as the algorithm to identify the most optimal route that connects the ROIs without flying over the Forbidden areas since it has already been successfully applied to other agricultural field operations [15,16]. A requirement of the algorithm is to select the starting position and the number of platforms that will carry out the mission while simultaneously keeping a safety distance between UAVs during the whole trajectory.

3. Results

In order to collect data from a row, the UAV flew on top of the adjacent row and captured images from the left and right sides of the canopy of each vine plant. Figure 1 includes six rows of the vineyard above which the UAV would fly. However, since the canopy needs to be recorded from both sides, only the inner four rows are the ones which had images taken from the two laterals. Figure 1a shows the CHM of the vineyard, with the Forbidden areas marked in red. Those areas did undergo data collection, and hence, the UAV did not fly above both the top and bottom adjacent rows. For areas of agronomic interest, the UAV flew to the specific waypoint marked in Figure 1b, captured an image of that side of the canopy, and rotated 180° to collect images of the other side of the canopy. The path designed to capture data from three UAVs flying simultaneously can be observed in Figure 1b, where each UAV path is marked with a different color.

The length of the route designed implementing ACO was 36.89 m, and compared to the length proposed with path planning without optimization, the length obtained was 38.48 m, which represents a decrease in path length of 4.32%.

4. Discussion and Conclusions

The methodology studied in this project provides a solution to optimal data acquisition by considering the biophysical environment of the field in order to design an optimized path plan to boost object detection and tracking in vineyards. It requires two flights: (1) a survey mission to obtain insights into the specific characteristics of the field and (2) the actual path designed using ACO to enhance object detection and tracking purposes.

The proposed method, implementing Ant Colony Optimization, was able to improve fruit detection by selectively avoiding data collection from unnecessary areas, reducing the path length by up to 4.32% compared to traditional path planners without optimization algorithms. The optimization of the flight allows the flight time and the usage of the batteries to be minimized [17,18], which are two of the limiting factors of UAV missions [6,7], while improving the collection of crucial data from a vineyard, such as fruit images. This optimization might lead to higher fruit detection and tracking accuracy, which is a current issue that research carried out in commercial vineyards is facing when no leaf removal is executed [19].

Another strength of this method is that it only requires an RGB sensor to carry out both the survey and second flights. This is an advantage since these sensors are more affordable and might be more attractive to the final stakeholders in the study: technicians and farmers. Nevertheless, this methodology has not yet been tested on a field. To enable further research on the topic, the code written in MATLAB has been made available to the scientific community: https://github.com/saidlab-team/Drone-ACO-ACPP (accessed on 10 October 2023).

Future work should focus on executing these flights in several vineyards with multiple levels of difficulty regarding the number of Forbidden areas present in the field to validate the robustness of the proposed method.