Unmanned Aircraft System (UAS) Technology and Applications in Agriculture

Abstract

Numerous sensors have been developed over time for precision agriculture; though, only recently have these sensors been incorporated into the new realm of unmanned aircraft systems (UAS). This UAS technology has allowed for a more integrated and optimized approach to various farming tasks such as field mapping, plant stress detection, biomass estimation, weed management, inventory counting, and chemical spraying, among others. These systems can be highly specialized depending on the particular goals of the researcher or farmer, yet many aspects of UAS are similar. All systems require an underlying platform—or unmanned aerial vehicle (UAV)—and one or more peripherals and sensing equipment such as imaging devices (RGB, multispectral, hyperspectral, near infra-red, RGB depth), gripping tools, or spraying equipment. Along with these wide-ranging peripherals and sensing equipment comes a great deal of data processing. Common tools to aid in this processing include vegetation indices, point clouds, machine learning models, and statistical methods. With any emerging technology, there are also a few considerations that need to be analyzed like legal constraints, economic trade-offs, and ease of use. This review then concludes with a discussion on the pros and cons of this technology, along with a brief outlook into future areas of research regarding UAS technology in agriculture.

1. Introduction

A wide variety of sensors and other data-gathering equipment have been developed for agricultural purposes, such as underground sensors monitoring soil quality, aboveground sensors monitoring temperature and humidity, yield sensors, and weed sensors; yet, in the realm of precision agriculture, imaging sensors are perhaps the most important [1,2,3]. Traditionally, aerial images could only be gathered from satellites, such as those from the Landsat program or through aircraft [4,5]. Using multispectral and hyperspectral cameras equipped on a satellite or aircraft, images could be used to calculate various vegetation indices which can indicate variabilities in the field. Vegetation indices, such as the normalized difference vegetation index (NDVI), rely on comparing light intensities reflected from canopies in the visual and near infra-red (NIR) range [6,7]. Given the vast distance from a satellite to a field and the area an image can cover, resulting images often have high temporal resolution and low spatial resolution compared to other methods [8]. In order to produce more detailed images with high spatial resolution at a low cost, cameras mounted upon unmanned aerial vehicles (UAVs) were utilized with promising results [9]. As these UAVs began to incorporate more peripheral technologies and grew in complexity, a new term was developed to describe the whole system together—unmanned aircraft system (UAS) [10,11]. Along with this new technology, came many new challenges such as processing of geospatial data [12,13,14] and lower temporal resolutions when applied to large areas of land [7,15]. Therefore, it should be noted that UAS technology is not meant to replace satellite data, as there are trade-offs for using one over the other. In many situations, both UAS and satellite imagery are used in conjunction [8,16].

It should come as no surprise that this review has ambitious goals, as UAS technology is becoming increasingly widespread in the modern era. Thus, this article should serve as a comprehensive introduction to the topic as it is related to agriculture, rather than a complete summation of all the research in this area. This technology, at least in the realm of agriculture, is still in its infancy, with many possible uses yet to be explored. Therefore, some concepts presented have only been partially developed for drone usage or are still in the research phase of development and are not yet available for consumer use. The primary goals of this review are to cover the popular areas of use; physical components of these systems; data gathering and processing; a few considerations such as legal, economic, and integration factors; and to summarize with a brief discussion of the advantages, disadvantages, and future areas of research in agriculture.

2. Areas of Use

2.1. Field Mapping

As field mapping is often the first step in many other areas of use, this should prove the most appropriate place to begin. Field mapping has many uses from small farm improvement to large-scale governmental land surveys [4]. A few prominent studies involving field mapping in agriculture include: NDVI maps [7,17,18,19], assessing spatial variability and structure of vineyards [20], surveying and development of land cover maps [4,15], and estimation of tree locations and sizes [21,22].

These maps are often constructed using a series of independent images taken from an UAS and stitched together using various photogrammetric software packages [15,23]. Each individual image in the resulting mosaic are commonly georeferenced using ground control points (GCPs), overlaying Landsat satellite data, or a combination of the two [4,7,15,17,18,19,20,21,22]. Images will also undergo a process of data correction which can involve vignetting and orthorectification [18], among others, depending on the task.

2.2. Plant Stress Detection

Detection of various plant stresses is a widely used application of UAS in agriculture. The most common forms of stress being detected via UAS include water stress, nutrient stress, and plant diseases. Water stress can be fairly simple to detect and is generally detected through a combination of NDVI and crop water stress index (CWSI) values, the latter of which is generated using thermal imaging [24,25]. Nutrient stress can vary in detection difficulty, as there are many more causal factors associated. As for the detection of nitrogen deficiency, there have been a few successful studies. Zaman-Allah and colleagues showed how NDVI data from a multi-spectral camera aboard a UAS can be used to phenotype low-nitrogen stress tolerance in maize [26]. Other more specific vegetation indices have been developed for more accurate detection of nitrogen deficiency remotely, such as the nitrogen nutrition index (NNI), which can be calculated using the modified chlorophyll absorption ratio index (MCARI) and modified triangular vegetation index 2 (MTVI2) obtained through hyperspectral imaging [27]. Another index used in detection of nitrogen deficiency is the Nitrogen Balance Index (NBI), which calculates the ratio of chlorophyll to polyphenols. In a paper by Li and colleagues studying rice fields in China, NBI was predicted using standard RGB imaging converted to hue, saturation, and brightness which generates a dark green color index, and which proved representative of field nitrogen concentrations [28].

Automatic plant disease detection via UAS is a field that has emerged over the past couple decades with great future research potential, as individual diseases can have drastically different symptoms across many different host species. In one study looking at Huanglongbing (HLB) detection in citrus crops by Garcia-Ruiz and colleagues, it was found that the NIR-R index value and the average reflectance values at spectral bands of 560 nm and 710 nm were statistically different between healthy and HLB-infected trees [29]. Likewise, Flavescence dorée disease in vineyards was detected using a multi-spectral camera monitoring bands at approximately 530–600–650–690–730–750–800 nm, with an accuracy over 94% by Al-Saddik and colleagues [30]. In other studies, vegetation indices were more heavily used. For example, Lowe and colleagues showed how the presence of laurel wilt disease in avocado was found to be strongly correlated with excess red (ExR) and blue/green (B/G) vegetation indices [31]. However, for some UAS-based disease detection programs, a standard RGB camera is all that is needed for accurate detection. From a study looking at phymatotrichopsis root rot (Texas root rot) in alfalfa by Mattupalli and colleagues, it was shown how RGB images alone can provide enough data for accurate classification of the disease; with the extrapolation of RGB to HSV (hue, saturation, and value) and Hrot60SV (hue values rotated 60 units, saturation, value) having almost no effect on classification [32]. Yet, in the majority of cases, RGB imaging alone is not sufficient. Zhang and his colleagues showed how the detection of sheath blight in rice could be initially classified with high-resolution RGB cameras, but multi-spectral imaging gathering NDVI data was required in order to assess disease severity [33]. It should be noted, though, that not all research in this area targets purely automated disease detection. In a recent study by Kalischuk and her colleagues, it was found that UAV-assisted scouting could provide a 20% increase in early detection of gummy stem blight in watermelons than from traditional scouting alone [34].

Although many approaches have been developed for the detection of a particular disease affecting a particular plant with great success, much of this science has yet to be applied to large-scale monitoring across many crop lines. The shear amount of data involved in this detection is pushing many researchers to look into machine learning techniques for integrated disease management through early plant disease detection. Some of these techniques including classification trees, support vector machines, and deep learning which will be discussed further in a later section [35,36,37].

2.3. Biomass and Field Nutrient Estimation

There are many areas in which precision agriculture via UAS can be applied; however, biomass and nutrient estimation is certainly one of the most popular uses for this technology. There have been numerous studies using UAS which aim to predict biomass of crops and assess nutrient levels within soil and plants alike. In a few studies, biomass yields of grasses were estimated using point clouds and crop height models (CHMs) [14,38], though, the majority of studies primarily used some form of vegetation index to make predictions [6,7,23,39,40]. Geipel and his colleagues used vegetation indices to predict both biomass and nitrogen content within plants [41]. Tokekar and his colleagues introduced a new approach to detect nitrogen levels in the soil using vegetation indices, soil sampling, and other methods through a combination of both UAS and unmanned ground vehicles (UGVs) [2]. In 2016, Aldana-Jague and his colleagues were able to map soil organic carbon levels in fields using support vector machines with data provided by a multispectral camera aboard an UAS [42]. A few common indices used to estimate biomass and nutrient levels include soil plant analysis development, NDVI, normalized difference red edge index, and optimized soil-adjusted vegetation index (OSAVI) [7,43]. Aside from vegetation indices, the use of radiative transfer models such as PROSAIL has also shown great success in estimating biomass [44]. This method relies on estimating the biophysical variable called leaf area index (LAI) or green area index (GAI) which is generally performed through the use of lookup tables; though, iterative optimization techniques and artificial neural networks are also used [45,46].

2.4. Weed Management

Many weed scientists have seen the potential of UAS and are currently working on projects to build weed maps for various agricultural crops, such as wheat and cranberry [47,48]. In 2013, a research article was published that assessed the optimal flight conditions (e.g., altitude and quantity of GCPs) needed for weed detection in young wheat fields. Their research found that an altitude of 30–100 m and roughly one GCP per image would provide the best setting for weed detection [49]. Many following studies were able to build upon this work and begin to generate weed maps for their own crops of interest. For example, Mink and his colleagues equipped a drone with RGB and multispectral cameras to measure vegetation indices and plant height data in order to detect weeds in maize and sugar beet fields. They were able to develop a weed height model which they used to detect weed instances, based upon the vegetation index excess green red (ExGR) and CHM [50]. Shortly after, Pflanz and his colleagues published an article which demonstrated how they were able to use an image classifier called Bag of Visual Words along with a support vector machine (SVM) to map weeds within a field of wheat. Flying from a much lower altitude (1 to 6 m) than previously suggested and working with a dataset of over 25,000 sub-images, they were able to detect and map instances of weeds and label them by species [51]. In 2017, Bah and his colleagues were able to apply the Hough transform technique, along with the spatial relationship of pixels, to detect weeds in row crop fields with promising results [52]. In 2018, Rasmussen and his colleagues developed a procedure for weed detection called Thistle Tool which can be implemented with only RGB imaging taken from a UAS. This method involves a process of dividing the orthomosaic into subsections, calculation of excess green index values, a classifier step, and finally the resulting weed map with segmentation of weeds and crop [53].

2.5. Counting

Though not the largest area of use, many areas of agriculture require quick and efficient counting methods for inventory purposes [54]. This method of UAS-based counting proves especially useful in citrus tree production, where the majority of related research is being conducted. In 2014, She and her colleagues developed two different algorithms to be used for counting-related work with UAS imaging. The first, which applies to crops with uniform canopies like citrus trees, used vegetation index calculations to separate tree canopies from the background and then implemented a counting method based upon the average canopy area. The second algorithm, which applied to conical-tipped Christmas tree counting, involved locating and counting local reflectance maximums within particular windows which represent the cones of trees [55]. In 2018, the same group of researchers published another paper addressing issues regarding counting difficulty in densely packed areas of plants (e.g., for shipping purposes) and provided an updated methodology using SVMs for accurate counting in these settings [56].

From a research team at IBM (International Business Machines Corporation, Armonk, NY, USA), comes another study regarding citrus tree counting using machine learning techniques. They address the problem of accurate counting in high-density fields by incorporating convolutional neural networks, a form of machine learning based in deep learning. They were able to obtain accuracy scores (upwards of 94%) similar to She and her colleagues [54,55]. Aside from plant counting, there has been some effort in the realm of fruit counting. It has been shown by Rahnemoonfar and her colleagues that computer vision algorithms are able to count ripened tomatoes with a 91% accuracy from manually taken images of tomato plants [57]. This research was furthered in 2019 by applying these methods to UAS, as well as many other realms of agriculture including live animal counting [58].

2.6. Chemical Spraying

Modern agriculture has nearly become synonymous with chemical usage, as it is crucial to almost every sector and crop [59,60,61]. Therefore, the many disadvantages involved with chemical use should be constrained as much as possible. Unmanned aircraft systems equipped for chemical spraying have been proven to eliminate many of these disadvantages such as terrain limitations of ground-based sprayers [61]; chemical exposure risks for farmers/workers [59]; and with proper use, UAS sprayers can be more effective and economical than traditional methods [59,60,62]. For example, Mink and his colleagues were able to save 90% of herbicide in maize fields and 43% of herbicide in sugar beet fields by using site-specific weed control methods via UAS [50]. From the literature, it appears that Asia is leading the way in this area of research, with a great deal of research focus upon droplet distribution and drift [60,61,62,63,64]. According to Pederi and Cheporniuk, approximately half of all Japanese rice fields currently use UAS for spraying purposes [59]. Interestingly, this is the only prominent sector of drone usage in agriculture which is not solely reliant upon cameras and imaging, though many studies find imaging tools useful for site-specific spraying [65]. It should also be noted that depending on the country there might be legal issues surrounding the use of UAS for pesticide spraying, which will be covered in the Legal subsection (Section 5.1) of this review [63].

2.7. Miscellaneous

This subsection will primarily deal with novel, interesting uses of UAS in agriculture, which do not fall into neat categories, but is included to show some of the creative possibilities with this emerging technology.

Traditionally, phenotyping for breeding trials has been conducted through time-intensive and laborious means [66]; therefore, Chapman and his colleagues conducted a study using an UAS to help alleviate this time-consuming and costly procedure in 2014. They were able to autonomously detect variations within fields of crops using RGB, infrared, and thermal cameras mounted to a UAS, which can reduce the time involved in phenotyping new lines in plant breeding operations [67]. In a similar study, tomato spotted wilt disease resistance phenotyping was conducted through an UAS using multispectral imaging and vegetation indices, which improved detection time to as early as 93 days after germination [68]. Tripicchio and his colleagues set out to improve large-scale analysis of cultivation techniques in which they successfully implemented a method to remotely detect field plowing depths without the use of satellite imaging [69]. In an unusual cross-over between military security and agricultural research, from a research group in Thailand, UAS were fitted with custom-made gas sensors and used to detect volatile compounds such as those released from cattle or ethylene from ripened fruit [70]. In 2016, Rangel developed a UAS for carrying and dropping biological control agents on field environments [71]. Finally, from a study conducted in Germany, radiation use efficiency in maize fields was calculated remotely by a UAS equipped with low-cost imagery providing insight into photosynthetic activity of crops on a per field basis [72].

3. Platforms and Peripherals

This section describes the individual physical components that come together to form entire UAS currently used in agriculture. It will cover the underlying platforms (UAV), a few commonly used peripherals (cameras, sprayers, grippers, geospatial sensors, and sense-and-avoid sensors), and computing systems to tie everything together and process data.

3.1. Platforms

The term platform, in relation to UAS, refers to the underlying UAV structure onto which the extra tools and sensors are mounted. There are numerous types of available platforms to choose from including parachutes, blimps, gliders, rotorcrafts, and fixed-wing aircrafts [63,73]. However, the most commonly used platforms in agriculture are fixed-wing aircrafts and rotorcrafts. Fixed-wing aircrafts are generally able to fly faster and cover more distance than other platforms; though, they are not suited for tasks that require a great deal of maneuvering or hovering. Due to the fact of their ability to cover vast areas of land quickly and create datasets with relatively high temporal resolution, they can be primarily found in applications such as large-scale field mapping [15,74] and occasionally, spraying [59]. Most other applications are dominated by the rotorcrafts for their ease of use (no runway is required), lower cost, and high spatial resolution imaging due to the ability to hover as seen in Figure 1 [75]. Rotorcrafts come in many forms with any number of propellers, with the most common configurations including traditional helicopters [67]; quadcopters (4 propellers) [19,68]; hexacopters (6 propellers) [21,76]; and octocopters (8 propellers) [8,72]. When deciding which platform to build upon, it is important to check for max payload capability as well as any weather restrictions—such as whether or not the platform is waterproof—in order to fit the needs of the project.

3.2. Imaging

As aerial imaging is the most common use of UAS in agriculture, it follows that there have been many imaging sensors and cameras specially developed to operate on board these systems. Every UAV platform will have a particular payload which limits the sizes of imaging equipment that can be used. Likewise, as payload increases, the UAS will suffer from decreased speed, stability, and flight time [75]. Another major consideration when equipping a UAV is the velocity at which it will be flying when capturing images. Higher flight speeds can cause motion blur in images without the proper cameras and supporting algorithms; however, this becomes less of an issue in UAS with greater hovering capabilities such as rotorcrafts [15,67,77]. A few of the important sensors and cameras that will be covered in this subsection include: standard RGB cameras, multispectral and NIR cameras, hyperspectral cameras, thermal sensors, and depth sensors [73].

3.2.1. RGB Cameras

Red, green, and blue imaging is by far the most standard form of photography available to the average person. The concept is fairly intuitive; RGB cameras produce images with three values for each pixel—red, green, and blue—measuring the intensity of the respective color. This form of imagery does not provide the same spectral detail as other options; however, a few studies have shown how standard RGB imagery can be an effective tool for 3D modeling of agricultural crops [14,21,78] and biomass estimation [6,23,38]. For 3D modeling of crops, RGB images are used to generate point clouds (see Section 4.2) of canopies and trees using methods such as object-based image analysis [78] or structure from motion (SfM) [38] with relatively high accuracy. Another modeling method using RGB imaging and point clouds relies on finding local maxima in order to locate individual trees and associated heights [21]. The other main use of RGB imagery is for biomass estimation, where NIR and multispectral cameras are often used in conjunction with RGB cameras in order to improve accuracy [6,23]. Though in a recent paper, Grüner and her colleagues were able to accurately estimate biomass in grasslands from RGB imaging alone using SfM processing [38]. Finally, in a unique approach from Mattupalli and his colleagues using a maximum likelihood classification algorithm, phymatotrichopsis root rot disease was accurately detected in alfalfa fields using only RGB imaging [32].

3.2.2. Multispectral and NIR Cameras

The most common use of multispectral and NIR cameras in agriculture is for the creation of vegetation indices which relies on NIR data or specific bands of light [19,41,68,79,80]. Near infra-red cameras work similarly to RGB cameras with internal modifications or lens filters added to tweak the bands of light captured from the traditional RGB to incorporate other bands, usually from the NIR region [72,81]. On the other hand, multispectral cameras are specifically designed to capture multiple bands (5 to 12 are common) of light for every generated pixel, instead of just three for RGB images [80]. This can be useful as plants reflect light in a wider range of light than humans can detect with the naked eye. The majority of UAS-assisted aerial imaging for detection or estimation purposes uses multispectral imaging heavily to calculate vegetation indices such as NDVI and other variants involving NIR data [6,26,33,34,41,68]. A notable exception comes from an article by Al-Saddik and her colleagues which directly used multispectral data programmed to specific bands to detect Flavescence dorée disease in vineyards without using common indices [30].

3.2.3. Hyperspectral Cameras

Similar to multispectral imaging, hyperspectral imaging is able to collect light across a wider range than traditional methods; however, the difference is in the size of the bands—hyperspectral cameras can potentially monitor thousands of narrow bands of light for every pixel in the resulting images [80,82]. This method of imaging can be useful if very specific wavelengths of light need to be measured independently—often detecting light emitted from particular biomolecules such as chlorophyll [16,27]; xanthophyll [83]; mesophyll [82]; and carotenoids [16]. The downside of hyperspectral imaging is the generally high cost of cameras (especially lightweight versions to be mounted aboard a UAV) [66,80] and the generation of overwhelming amounts of useless data if not properly calibrated by experienced professionals [40,82,84,85,86]. Therefore, multispectral cameras should be the default imaging choice unless there is a special need for the detail found in hyperspectral images.

3.2.4. Thermal Cameras

Thermal sensors and cameras are able to capture infrared light radiation in a range of approximately 0.75 to 1000 μm (for reference, NIR ends at 2.5 μm), which can provide localized temperatures of objects within a thermal image [87]. While many NIR cameras can be cheaply modified from standard RGB cameras [81], thermal cameras are specially designed to cover most or all of the infrared region [87]. Thermal imaging is mainly used in agriculture to detect stomatal regulation and, thus, water stress in crops [24,25,66,87,88]. Unfortunately, thermal imaging tends to suffer from low resolution due to the environmental variables in an open system [87], especially in the case of small uncooled thermal cameras which are the dominant form used in UAS [89]. Thermal imaging is often used in conjunction with other imaging tools (such as RGB cameras) as a form of supplementary data [24,88].

3.2.5. Depth Sensors

A final, common class of peripherals aboard UAS in agricultural-related applications are depth sensors. The RGB-D sensors provide a relatively cheap and effective way of generating depth data by capturing an extra value in each RGB pixel—a value indicating distance from the sensor to that point in the image [90]. It is true that standard RGB imaging can predict depth values; however, Wang and Li demonstrated how RGB-D cameras are able to provide greater accuracy in volume estimation than standard RGB imaging in 2014 [91]. Another popular type of depth sensor uses LiDAR technology. In contrast to RGB-D sensors which primarily rely on light reflection intensities [90], LiDAR sensors use laser pulses to map distances [75]. Common reasons to incorporate depth sensors into a UAS for imaging purposes include: altitude monitoring during spraying [92]; phenotyping [90]; and 3D modeling [93,94].

3.3. Spraying Equipment

With the continual development of UAV and UAS technology, remote pesticide or fertilizer application through these tools has been a focus of much research. Compared to crop dusting performed by aircraft, UAS have the option of applying chemicals much closer to the field which results in less potential drift [95]. However, this comes with an increased payload (chemicals, tanks, pumps, nozzles) and, thus, shorter flight times [59,60]. To address this issue, Stark and his colleagues proposed a network of small UAS to autonomously spray chemicals to reduce temporal variance [95]. Other researchers have focused more on developing methods for site-specific spraying to reduce payload upon the UAS, all of which rely heavily upon image sensing technology and machine vision algorithms [47,50,60]. Regardless, the equipment needed for chemical spraying via UAS is fairly straight forward and standardized. The UAS platform needs to be equipped with three main components: a tank to hold the liquids, a nozzle to guide the spray pattern, and a pump to move pressurized fluid through the nozzle. Common tank sizes used aboard UAVs range from a couple liters to upwards of 10 L for larger rotorcrafts and fixed-wing platforms [59,62,76], with a few setups incorporating multiple smaller tanks capable of handling different chemicals [50]. There appears to be two dominant types of nozzles used: centrifugal for a more circular pattern and flat fan for dispersal in a straight line [65]. The type of pump should be decided depending on the intensity of spraying, the amount of power available aboard the UAS, and payload factors [65,76].

3.4. Gripping Tools

Robotic arms and grippers can be seen in many applications across agriculture; however, the technology currently comes with many limitations and has yet to be implemented on a wide scale due to the lack of infrastructure and supportive software available [96]. Though still largely in the research stage, many robotic tools have been designed for usage aboard UGVs. In 2016, Gealy and colleagues provided an example with the development of a robotic device enabling the automated tuning of emitters for irrigation purposes. In this project, they built a prototype for handheld use and mentioned that the design can also be applied to a UGV [97]. Other notable areas of application are in areas such as soil preparation and seeding [98], fruit and vegetable harvesting [99,100], and weed management [101,102].

In contrast, robotic arms and gripping tools aboard a UAS are much rarer and only used for specific cases given the limitations in carrying capacity, flight time, and system balancing. Therefore, most UAS equipped with grippers and other robotic attachments will be designed for specific niche purposes. A good example comes from Konam who proposed a design that would use a robotic arm to cut ripened mangos and collect them in a net located at the base of the tree, which is work that cannot be effectively done by a ground-based system given tree heights [103]. Addressing a similar situation, Varadaramanujan and his colleagues present another UAS equipped with a robotic end-effector which can be used to pick fruits and vegetables from hard to reach areas [104]. As this category of peripheral equipment is very much still in its infancy, future applications will be addressed in the last subsection of this paper, Future Areas of Research (see Section 6.2.).

3.5. Geospatial Technology

Incorporated into almost every UAS found in agriculture is some form of geospatial technology which allows for point-specific data gathering and treatment within fields. Generally, this will include a global navigation satellite system (GNSS) receiver which is in communication with satellites. A few common GNSS systems include: GPS from the United States, Global Navigation Satellite System (GLONASS) from Russia, Galileo from the European Union, and BeiDou from China [89,105]. A GNSS is extremely useful for georeferencing images, providing better accuracy than photogrammetric methods alone [9,18,20,40,89,106]. Many projects also incorporate an inertial measurement unit (IMU) to aid in the process of georeferencing where GNSS data are temporarily unavailable [6,41,107,108]. Aside from georeferencing, another popular use for geospatial technology is for geofencing, which can be defined as an autonomous method for creating restraints on the operating area the UAS is allowed to travel [109].

4. Data Processing

4.1. Vegetation

Potentially, the most important image analysis tools available in agriculture production are vegetation indices. Vegetation indices are numerical values generated through relationships between different wavelengths of light reflected from plants [82]. Depending on the index, these values can be associated with particular properties of plant health and growth including canopy greenness, pigment composition, water stress, nutrient stress, photosynthetic potential, and diseased tissue [26,33,68]. While many common vegetation indices rely upon multispectral imaging to provide a NIR band, a number of useful indices can be calculated using strictly RGB cameras as seen in the work done by Fuentes-Peailillo and his colleagues in 2019. They compared multispectral-based vegetation indices with RGB-based indices and found that the cheaper and easier to use RGB-based indices can still provide useful information for large scale applications and simple analysis [110].

The standard and most commonly used vegetation index is the NDVI and it has been applied to numerous studies across many areas of use [25]. The equation used to calculate NDVI is as follows:

$$NDVI= \frac{NIR - R}{NIR + R}$$

(1)

where NIR represents near infrared and R represents the red color band. Generally, the NIR value comes from reflected light around the 780–800 nm range and the R value comes from reflected light around the 670–700 nm range [23,25,41]. The intensities of these variables must be measured using a multispectral or hyperspectral camera and then formatted into an NDVI map. Therefore, in a resulting NDVI map, every pixel represents a particular NDVI value. Often, these maps are visualized by associating color maps with different levels of NDVI intensity to show variations in the field. Likewise, most other vegetation indices can be visualized in a similar fashion.

Aside from NDVI, there are numerous other indices used for agricultural purposes. For example, LAI which measures leaf density per unit area and can provide information regarding the spatial variability of agricultural crops in a field [7]. Optimized soil-adjusted vegetation index (OSAVI) attempts to correct for interfering soil reflectance in NDVI and can be useful for estimating biomass and growth [7,111]. If thermal imaging tools are available, CWSI values can be generated [87,89]. With only RGB imaging available, there are still a few indices that can provide useful information such as excess red (ExR), excess green (ExG), and excess blue (ExB), which are visualized in Figure 2 [112]. The equations are as follows:

$$ExR=1.4R-G$$

(2)

$$ExG=2G-R-B$$

(3)

$$ExB=1.4B-G$$

(4)

where R represents red, G represents green, and B represents blue. It should be noted that ExG is certainly the most common of these indices and used to determine greenness [47,56,113].

Vegetation indices are constantly being created and updated to serve particular research needs. In addition, novel indices can also be generated by combining pre-existing indices [8,32]; therefore, it would be impossible to provide a complete list of indices currently in use.

4.2. 3D Point Clouds

Along with vegetation indices, the generation of point clouds are another very common reason to use UAS in agricultural research. The concept is fairly intuitive; a 3D point cloud is a group of points in three-dimensional space which outlines the surface of an object [114]. These points are collected by various means. Honkavaara and her colleagues used next generation automated terrain extraction software (NGATE) to generate point clouds based upon NIR images taken from a UAS [39]. Torres-Sánchez and his colleagues used a combination of a few ground control points and standard RGB images taken from their UAS along with computer vision software to generate their point clouds [78]. Potena and his colleagues developed AgriColMap, which incorporates a vegetation index map and digital surface model (DSM) which were developed using point clouds, through a combination of UAS and UGV [115]. Other methods for the collection of point clouds include using SfM algorithms (a form of photogrammetric software) [21] and RGB-D imaging [69]. From a 3D point cloud, it is possible to generate many different types of 3D models, such as DSM which are commonly used in agricultural research [14,38,39,40,78]. Likewise, multiple 3D point clouds generated over a span of time to model growth or changes can be labeled as 4D point clouds. This model is becoming increasingly prevalent in agricultural applications; thus it can be possible to monitor the growth patterns of field crops over the course of a season [116,117]. Fortunately, the modeling process is made easier through photogrammetric software such as Agisoft PhotoScan Professional (Agisoft LLC, St. Petersburg, Russia) [14,38,78] or through the use of open source libraries such as Point Cloud Library (PCL) [69,118]. Point clouds and 3D modeling have primarily been used in agriculture for the purposes of biomass estimation, 3D tree and crop modeling, and weed detection.

4.3. Data Processing

4.3.1. Image Processing

The processes involved in image processing will be different for different research goals and from the use of differing imaging paradigms. For the generation of field orthomosaics, geo-referenced images will generally be captured in a RAW format and then corrected for various distortions, interferences, and vignetting [72]. Vignetting can be especially important, as this process corrects for the nature of photographs to have a higher level of brightness in the center compared to the edges, which can lead to inconsistencies in the resulting map. All of the individual georeferenced images will then be stitched together to create the orthomosaic [7]. For multispectral and hyperspectral imaging, image processing is more complex and involves calibration and data correction steps which have to consider aspects such as atmospheric weather and noise, though there is software available to help with this process [33,80]. Radiometric calibration is required for these sensors, which often involves utilizing the empirical line method on ground targets [119]. As these sensors are sensitive to varying levels of illumination due to the changing cloud cover and time of day, light sensors have been developed to correct for these discrepancies [23].

4.3.2. Data Processing

Unmanned aircraft systems are capable of generating very large amounts of data through their sensors, but these data are generally useless without proper data processing methods. Currently, machine learning methods have risen in popularity to help make sense of the vast datasets gathered by UAS. A good example of this can be seen in the paper by Pflanz and his colleagues which contained a dataset of 25,452 sub-images for their weed mapping and classification [51]. Popular machine learning models include random forest classifiers, support vector machines, bag of visual words, and convolutional neural networks (CNNs) [29,72,120]. The latter, in particular, have become an increasingly prominent tool for data analysis in agriculture as this is considered one of the best methods for effective image processing and classification [120]. This is a form of deep learning, which relies on multiple (sometimes hundreds or thousands) of neural network layers. There have been a wide variety of CNN models developed which can be selected at will to fit the needs of the project. Some boast very high accuracy, while some trade some accuracy for speed. As CNNs will require a very large training set (many popular models are trained on millions of images) to be effective and able to learn basic shapes in images, it is recommended to borrow an existing model and re-train it on a new dataset by freezing the lower levels responsible for basic abstraction of lines and shapes, and unfreezing the higher levels of abstraction (e.g., where the lines and shapes from lower levels begin to form objects and more complex structures) to be able to develop new classification filters for any unique dataset [35,121]. Traditional statistical analysis is also commonplace in some areas of research. Common methods in this category include the use of extremum estimators, spatial statistics, linear classifiers, image statistics, and focal statistics [4,9,21,31,62].

4.3.3. Multi-Sensor Processing

Since the system aspect of UAS is what separates the terminology from simple UAVs, it follows that many systems will be composed of multiple sensors, often working simultaneously. For processing all of this data, there will often be some form of on-board computer such as a Raspberry Pi [24] or ODOIRD [122], or a ground control station (GCS); though, Luxhøj points out that disruptions in transmission between a UAS and its GCS can cause many issues [123]. For some projects, relying solely on on-board computing is not feasible either financially or computationally. For example, if a project relies heavily on data mining or machine learning, an on-board computer will generally not have the required processing power. A GCS is also useful for integration with wireless sensor network technology as discussed in the Integration and Usability subsection (see Section 5.3).

4.3.4. Autonomy

Autonomy is an important aspect of UAS technology, not only for ease of use and efficiency purposes, but also for safety reasons [109]. The innovation of geofencing tools and sense-and-avoid technology in UAS have allowed for much safer flights [124]. Unmanned aircraft system autonomy cannot be accurately classified through a binary system, but instead lies upon a spectrum with multiple levels of magnitude. Different systems to classify UAS autonomy exist, such as the 10 levels of UAV autonomy described by the Air Force Research Laboratory (OH, USA) [125]; however, UAS used in agriculture are much less complex than those developed in military labs and will generally fall into two forms of autonomy. The first form of autonomy is where the pilot will plan out the flight ahead of time and the UAS will follow a set path. The majority of agricultural research using UAS autonomy will fall into this category [17,24]. These flight paths are generated through various flight planning software such as APM Planner (version 1.1.26) used by Koh and Wich [15], Michael Oborne’s Mission Planner used by Stefanakis and his colleagues [74], or through a custom-built program as done by Chapman and colleagues [67]. The second form of UAS autonomy emerging in agricultural research is complete autonomy capable of independent decision making and flight planning based upon sensor data. In 2017, Alsalam and his colleagues developed a system called On-board Decision Making Approach which uses ultrasonic sensors and a camera communicating with a ground station to automatically detect and spray weeds in a field [122].

5. Considerations

5.1. Legal

Both UAV and UAS laws and regulating bodies can be difficult to find in many areas, as the legality has not quite caught up to the emerging technology. Depending on the location, laws can be drastically different or non-existent. In the United States, the regulatory body is the Federal Aviation Administration (FAA). For recreational purposes, the owner of the UAS must register it and apply the registration number to the drone, fly only below 400 feet in class G airspace, and keep the drone in sight at all times, among a few other safety rules that can be found on their website [126]. For commercial usage, the owner/operator must also pass a knowledge test and become an FAA-certified drone pilot [127]. For the spraying of pesticide and other hazardous chemicals, the legality in the United States is still lacking fluidity and transparency. As of 2018, it is legally possible to spray pesticide via UAS; however, the operator must go through three exemption and waiver processes to receive permission [128]. Privacy concerns are also a pressing issue surrounding drone usage; though, for agriculture, this is less of an issue as there is generally not a need to leave one’s own property [129]. More legal information can be found online as there are many organizations and websites which aim to help provide guidance in this area based upon location of use [130,131,132].

5.2. Economic

Economic aspects are one of the primary motivations for agricultural research involving UAS. Many research projects in agriculture intend to eventually provide support for farmers and the food-generating community as a whole, so the economic impact of any advancement in technology or methodology should be evaluated. Unmanned aircraft systems aim to provide easier means for monitoring and treating fields, aligned with the precision agriculture paradigm [109], but there is a financial trade-off of which to be aware. Many of these systems for continuous usage can be expensive, costing tens of thousands of dollars and are not realistic for many farms [17,77]. While the UAS platform itself can be very affordable, the specialized equipment and cameras are not. For example, the DJI Mavic 2 Pro, a popular consumer UAV fitted with a high-resolution RGB camera, costs around (US) $1500, whereas, to add a hyperspectral camera built for aerial usage could cost over (US) $10,000 [80,133]. In terms of aerial sensing platforms, a study by Matese and his colleagues found that farms smaller than 5 hectares and larger than 50 hectares would not benefit from using UAS and that satellite or aircraft sensing would be more economical [9]. However, for routine scouting and monitoring of fields that cannot be done via satellite, UAS have been shown to be a potentially economically viable replacement for hired laborers [33,34]. Unfortunately, as most of this research is still in the proof-of-concept stage, there is little hard evidence of the cost trade-offs of using UAS within agriculture [24]. As the technology develops and extension and outreach centers are able to provide farmers with the knowledge to operate these tools, the cost-prohibiting nature of these systems is expected to decrease, and many researchers are optimistic about the future [24,77].

5.3. Integration and Usability

Unmanned aircraft system integration with other sensing systems is a key element in effective precision farming. Many modern farms and nurseries have begun to incorporate ground-based sensors in the field to monitor various aspects such as soil health, pests, irrigation, and environment [134,135]. These sensing systems are usually wireless and connected over a wireless sensor network (WSN) and communicate with a base station [3,136]. Therefore, it is possible for a UAS to be treated as another part of the existing WSN. In 2015, Kale and her colleagues applied this idea to pesticide spraying by developing a framework for UAS to communicate with ground-based sensors over the WSN to reduce chemical usage and drift [64]. In 2018, Moribe and his colleagues applied this idea by integrating infrared thermometer sensors on the ground and onboard a UAS, and developed a novel communication protocol between them for the WSN [137]. Furthermore, Uddin and his colleagues built a robust crop health monitoring system through integration of ground-based wireless sensors and UAVs in 2018 [138].

Another crucial aspect of this emerging technology is the ability to translate the theory into practice. Much of the topics covered previously in this review are of a technical nature, and it is not expected that many farmers will learn the nuances of, for example, equipping and calibrating hyperspectral imaging capabilities on a consumer-grade UAV for the purpose of disease detection in their fields. Therefore, streamlining these concepts into complete, easy-to-use packages with support for WSNs is of high importance to see any real application of this technology.

6. Discussion

The goal of this review article was to provide a concise, but thorough, survey of the current usage of UAVs within the domain of agriculture. Within the above text, there is information regarding specific applications, the individual physical components of the systems and how they work together, how data are gathered and processed through them, as well as a few minor considerations of which to be aware. With this knowledge, it will be beneficial to understand the advantages and limitations of this technology.

6.1. Advantages and Limitations

Unmanned aircraft system technology is able to provide us with very high spatial resolution at the cost of temporal resolution. It is capable of traversing difficult terrain; however, many platforms suffer from short flight times, especially with large payloads. Likewise, UAS efficacy can be very dependent upon weather conditions. A large advantage of UAS is the ability to fly in different trajectories and incorporate depth sensors in order to create accurate 3D models and maps; though, it should be noted that these photogrammetric processing times can be very extensive and require a great deal of computing time when using high-resolution images. As well, UAS are capable of physically interacting with the environment, performing tasks such as sample collection and chemical spraying. Similarly, the UAS can be connected into an existing WSN for an integrated approach to precision farming. However, much of this sensing technology and software comes with a large upfront cost. Therefore, due care must be taken in calculating the economic trade-off of using UAS over other options.

6.2. Future Areas of Research

Hopefully, it is clear that the potential of UAS in agriculture is very high. This review only brushes the surface of all of the possibilities of these systems. As stated previously, there are many applications where research is lacking or non-existent in this domain. For example, the development of agricultural image databases would provide a massive aid in the creation of machine vision algorithms to run from UAS, as many researchers find themselves having to create their own datasets which can be very time consuming and costly [120]. Likewise, a centralized source for the collection of which hyperspectral bands to monitor for particular diseases or stresses could save researchers time through derivative experiments. It would be worth investigating the potential of applying data mining methods to the collection of these hyperspectral readings of light reflected from plants and soil, as there would be far too much data for traditional inference. The field of deep learning is also currently a very hot topic in academia, and many models applied to agriculture are only in their infancy. Therefore, it should not be difficult to find novel areas of research within deep learning or machine learning as a whole. It should be obvious from living in the modern era that agriculture will see a more intimate relationship with computer science over the next few decades, and researchers should be encouraged to collaborate more between these disciplines. In fact, the authors of this article are currently involved in a cross-disciplinary project between computer science and agriculture which seeks to build a UAS with the capability of automatic biotic and abiotic stress detection of ornamental crops in a nursery setting, as well as the ability to create 3D models of trees to monitor growth patterns and a robotic arm for taking tissue samples. A final notable area of future research is within the discipline of animal science. From reviewing almost 200 papers and conference proceedings, a very small minority had any focus in applying this technology to livestock. It would seem that UAS with sensors such as thermal cameras could potentially help monitor the health of herds of cattle or even aid in discovering lost or trapped individuals. Regardless of particular applications, the end goal of the majority of this research will be the creation of a complete and integrated method for aerial monitoring and treatment of various anomalies in farm settings with limited human intervention. This is the future of precision farming and likely the most effective way to conduct agriculture in the 21st century.