Precision Chemical Weed Management Strategies: A Review and a Design of a New CNN-Based Modular Spot Sprayer

Abstract

Site-specific weed control offers a great potential for herbicide savings in agricultural crops without causing yield losses and additional weed management costs in the following years. Therefore, precision weed management is an efficient tool to meet the EU targets for pesticide reduction. This review summarizes different commercial technologies and prototypes for precision patch spraying and spot spraying. All the presented technologies have in common that they consist of three essential parts. (1) Sensors and classifiers for weed/crop detection, (2) Decision algorithms to decide whether weed control is needed and to determine a suitable type and rate of herbicide. Usually, decision algorithms are installed on a controller and (3) a precise sprayer with boom section control or single nozzle control. One point that differs between some of the techniques is the way the decision algorithms classify. They are based on different approaches. Green vegetation can be differentiated from soil and crop residues based on spectral information in the visible and near-infrared wavebands (“Green on Brown”). Those sensors can be applied for real-time on/off control of single nozzles to control weeds before sowing after conservation tillage and in the inter-row area of crops. More sophisticated imaging algorithms are used to classify weeds in crops (“Green on Green”). This paper will focus on Convolutional Neural Networks (CNN) for plant species identification. Alternatively, the position of each crop can be recorded during sowing/planting and afterward herbicides can be targeted to single weeds or larger patches of weeds if the economic weed threshold is exceeded. With a standardized protocol of data communication between sensor, controller and sprayer, the user can combine different sensors with different sprayers. In this review, an ISOBUS communication protocol is presented for a spot sprayer. Precision chemical weed control can be realized with tractor-mounted sprayers and autonomous robots. Commercial systems for both classes will be introduced and their economic and environmental benefits and limitations will be highlighted. Farmers ask for robust systems with less need for maintenance and flexible application in different crops.

1. Introduction

Weeds are distributed heterogeneously within agricultural fields [1]. They often occur in aggregated patches with different size and shape. The aggregation in the patches is higher for perennial weed species such as Cirisum arvense (L.) and Elymus repens (L.) than for annual weeds [2]. Furthermore, the weed species composition varies significantly within agricultural fields [3,4]. The occurring weed patches with their specific weed composition often remain stable over many years if weed management practices were applied across the entire field (flat spraying) [5]. Whether treatment with herbicides is useful depends on the specific case. In order to decide whether the use of herbicides is beneficial in a particular field, economic weed thresholds are used. When the weed infestation exceeds the economic weed threshold, the yield that is gained by the use of herbicides is higher than the costs of the chemicals and their application [6]. Herbicide applications and other direct methods of weed control should be targeted to areas with high weed infestation where the economic weed threshold is exceeded [5]. This method, the application of the economic weed threshold, is called site-specific weed control. Site-specific weed control resulted in 23–89% less area treated with herbicides, where weed populations were mapped and economic weed thresholds were applied with no additional costs for weed control in the following years [4,7]. Therefore, site-specific weed management can be considered as an efficient approach to achieve the targets of the EU commission to reduce the input of chemical synthetic pesticides by 50% by 2030 [8]. Another way to meet these goals is to use mechanical methods of weed control. However, these methods are not as effective under such variable environmental conditions as chemical weed control [9].

Modern sensor and information technologies have been included in application systems for herbicides leading to several commercial precise spraying systems for arable and vegetable crops [5]. With the use of those systems, the productivity of farms can be improved [10]. The spraying systems can be classified into detecting “Green on Brown” or “Green on Green”. Green on Brown (GoB), as shown in Figure 1, differentiates green vegetation from soil and crop residues based on spectral information in the visible and near-infrared wavebands [11,12]. Green on Green (GoG) differentiates between green crops and green weeds on the basis of more sophisticated imaging algorithms [11], as shown in Figure 2. With GoB the target is to identify the presence or absence of green plants, while GoG requires the classification of plant species or groups of species, such as crop, grass-weeds, broadleaved weeds, and perennial weeds. Those precise sprayers target high-density weed infestations (patch spraying), and single plants (spot spraying). Sensor-controlled spraying systems are composed of (1) sensors for the detection of weeds and crops, (2) expert systems to generate a decision on the need for weed control (on/off) and the optimum type and rate of herbicide and (3) application systems to spray the herbicides. One development within these technologies is the use of modular systems. Modular systems allow users to combine different sensors, expert systems and sprayers. In a modular system, standardized data communication should be used [13]. Although modular systems provide several benefits for the user, most commercial systems for sensor-based herbicide spraying are using proprietary communication systems that only operate with one specific sensor, expert system and sprayer. There are a lot of changes in the agricultural markets, especially with regard to artificial intelligence to comply with the new regulations. It is difficult to keep an overview of what is developing, how and where. The current literature still lacks a summary of what is happening with regard to chemical weed control. The focus of this paper is to summarize the technology related to chemical weed control. An overview of the innovations will be given, as well as an outlook on those that are already under development. This review paper is intended to help provide an overview of which innovations already exist and which will be developed in the future. The objective of this review is to summarize those technologies, highlight their benefits and limitations and give perspectives for new applications of site-specific weed management in agricultural crops.

2. Patch Spraying

Patch spraying, as shown in Figure 3, was mostly realized based on georeferenced weed maps. Herbicides were sprayed in areas with weed infestations higher than the economic weed threshold [5,14] and boom sections were turned off in areas with low weed infestations. This approach saved 23–89% of herbicides in cereals, maize, sugar beet and peas [7,15,16]. Savings were also realized by reducing the herbicide rate at locations with less problematic weeds or smaller weeds, which are more sensitive to herbicides [13]. Yields in the unsprayed areas were not lower than in the treated areas, and there was no cause of any additional costs for weed control in the following years [4,7]. If the heterogeneity of weed species distribution was considered and each weed species group was treated separately with different herbicides using a GPS-controlled multiple-tank sprayer, savings were even doubled [17]. Those results underline the enormous potential of patch spraying for herbicide saving. In addition, patch spraying reduced herbicide load into the environment and the risk of herbicide residues in the water and in the food chain. It also reduced the selection of herbicide-resistant weed populations [7,15,16,18,19]. Jensen and Lund applied patch spraying algorithms to protect endangered and rare weed species. The sprayer was turned off at locations where rare weed species were present/mapped [20]. Despite these benefits, adaptation of patch spraying to practical farming was low because of many technical constraints. Especially, spraying systems do not allow varying the herbicide mixture according to the weed distribution maps. Patch spraying also requires higher effort for data acquisition and documentation. Furthermore, the farmer will need to predict and organize the usage of two or more herbicide solutions, each targeting specific weeds or groups of weeds. With the right organizing and know-how, there is a high potential for herbicide savings, but the economic benefits may disappear due to maintenance costs or incorrect organization and programming [15,21].

Until the early 2000s, mostly offline patch spraying systems were used [22,23]. Unmanned Aerial Vehicles (UAV) for weed sensing facilitated weed mapping and patch spraying [21,24]. UAV-systems were capable to map and georeference the distribution of perennial weed species such as Cirsium arvense with lower costs and more efficiently than with near-ground-camera mapping [2]. However, weed mapping had to be performed prior to patch spraying and georeferenced application maps were loaded into the board computer of the sprayer [7,21]. Computer processors were not fast enough for real-time weed/crop classification in digital images using more sophisticated algorithms [25]. The first real-time patch spraying system based on weed coverage data from digital images was developed by Longchamps et al. in maize [26]. Another example is AgriCon [27], a German company that commercialized the H-Sensor for site-specific weed control in real time in cereals and maize. Both systems resulted in more than 50% herbicide savings.

Field sprayers were designed to apply equal rates of herbicides homogeneously across the entire field. Patch spraying, however, requires continuous opening and closing of boom sections and the adjustment of the spray rate. With improvements in sensor and information technologies for weed mapping and real-time weed detection, several patch spraying technologies were included in field boom sprayers. Multiple nozzle carriers, e.g., Amazone VarioSelect^® were developed for remote selection of nozzle size during the application [20]. This allowed a variation in spray rate in wide ranges. Several sprayer manufacturers developed pneumatic spray nozzle control systems for faster opening and closing of boom sections and individual spray nozzles [20,28]. Computing capacities and graphical interfaces of spray computers were improved to process georeferenced data of weed maps, to connect to online sensors, and to control the status of the boom sprayer [28]. Gerhards and Oebel [7] developed and tested a GPS-controlled three-tank sprayer based on GoG with a 21 m wide spray boom (Kverneland Cerberus). Each of the three tanks was controlled separately based on georeferenced spray maps. Spray maps were generated based on georeferenced images that were taken by bi-spectral cameras and the automatic weed classification was based on shape analysis. One tank was used for the control of annual broadleaved weeds, one tank contained herbicide against grass-weeds and one tank was used for the control of perennial weed species [7]. Patch-spraying with the Cerberus sprayer on average saved 50% of herbicides in cereals, oil-seed rape, sugar beet, and maize. [7]. Amazone has developed a commercial sensor-guided patch-sprayer, the AmaSelectSpot with DroneLink [29]. It uses an offline approach, where the field is scanned with an RGB-camera before the application. DroneLink creates an overview map from the captured images with the help of an integrated software. By the use of Artificial Intelligence, an application map is generated. A safety zone of 1 m is created around the detected treatment spots to ensure that all weeds have been sprayed. Depending on the weed density herbicide reduction can reach up to 80% [29].

Another example for real-time spraying of weed patches is Agrifac [30]. Three RGB cameras are mounted on the self-propelled sprayer. The cameras identify weed patches and nozzles/boom sections are turned on and off at a driving speed of 10–14 km h⁻¹. A different approach to vary the herbicide mixture during the application is a direct injection system [31], e.g., Danfoil, Denmark and Dammann/JKI, Germany [32]. Herbicides are mixed with water in the hydraulic system that is close to the nozzles. By the use of the system, no herbicide residues remain in the tank after the application [32].

Gonzales-de-Soto et al. [33] developed a robotized patch sprayer based on GoG. The direct-injection sprayer boom has 12 high-speed solenoid valves mounted at a range of 0.5 m. Each solenoid valve controls one nozzle separately. Weed detection can be performed offline by an external device that creates a work plan before the treatment or on-board and work in real-time. With the help of GPS and a base station, the position on the field is tracked and with the help of the laser obstacles in the front are recognized. The system speed was about 3 km h⁻¹. The system was tested in wheat and showed that the treatment is only applied in the desired area. Savings of herbicides were dependent on the location and the distribution of the weeds. The accuracy of detected weeds could reach up to about 99% [33].

A special form of path spraying is band-spraying. The aim of band spraying is to apply herbicides only on the intra-row area, where weed competition is high and mechanical weeding is more complicated. Usually, herbicides are only applied in strips of 10–15 cm [34]. If RTK-GPS and autosteering systems are used during seeding, then band-spraying with pre-emergence herbicides is also possible. In combination with post-emergence inter-row hoeing, very high weed control efficacy can be achieved, and 60–70% herbicide savings can still be realized in sugar beet, maize and soybean [34].

With the increasing amount of data processing on the sprayer, a need for standardized communication between sensor, controller and sprayer came up. So far, most of the current precision spraying systems represent proprietary systems with their own protocols for sensor, controller and sprayer [35]. Therefore, it is not possible to connect different sensors and actuators to those systems. Standard communication protocols such as ISOBUS allow the connection of different sensors to one controller and send decisions from the controller to different sprayers [35]. Systems using ISOBUS such as Agrointelli from Robotti in Denmark can process data from cameras, spectrometers and GPS receivers in one controller. Tasks generated from these sensor data can be transferred into precise seeding, fertilization and plant protection operations [36]. The standard protocol of ISOBUS can also be used for diagnostics and data exchange with farm management systems. By using ISOBUS the acceptance of patch spraying could increase.

3. Spot Spraying

In contrast to patch spraying, spot spraying systems have a closer/tighter “field of application” and target single plants [37], as shown in Figure 4. The fact that individual weed plants or smaller weed patches can be sprayed decreases the number of sprayed areas. In patch spraying, a threshold is set for spraying weed patches. Spot spraying can also spray smaller areas [37]. This means that invasive species already targeted from individual plants will spread less in the field. The two methods differ fundamentally in their approach and both types have their advantages and disadvantages.

Plants are usually classified into crop or weeds. With the development of more sophisticated hardware and detection algorithms, it was also possible to classify and protect rare/beneficial weeds. When spot-spraying is applied in emerged crops with non-selective herbicides with an GoG approach, classification accuracy must be almost 100% to ensure that no crops will be destroyed [37,38].

With the use of spot spraying, a binary decision (spray/not spray) for each single target is made. Therefore, the economic weed threshold as used for patch spraying or the decision algorithms for patch spraying is not used for spot spraying [39]. If non-selective herbicides were used for spot-spraying in crops, a minimum distance of weeds from crops needs to be defined in order to prevent crop damage. Herbicide savings for spot-spraying can be considerably high, up to 99% [5,37,40]. Spot spraying is realized in real-time systems with a camera directly mounted on the tractor or the robot. In the case of several mounted tanks, the system has the flexibility to adjust the herbicide application for specific weeds or groups of weeds. Multiple-tank sprayers or direct injection systems can be combined with spot spraying if each weed group is treated with a specific herbicide [41].

Artificial intelligence algorithms can be used for plant species detection. Since 2015, mostly Convolutional Neural Networks (CNNs) were applied for plant species classification [42]. Of course, it is possible to use a CNN also for patch spraying. Because spot spraying is aimed at smaller targets, the CNN is explained on the basis of spot spraying to emphasize the extreme learning ability of the neural networks. Deep Learning is the predecessor of Convolution Neural Networks (CNNs). CNNs were first introduced by LeCun et al. [43] and have shown a great potential for achieving high accuracy regarding tasks such as image classification and object detection. They can even be a valuable candidate for fine-grained decisions and classification [44]. CNNs consist of mainly three different layers, convolutional layers, pooling layers and fully connected layers [45]. The convolutional layer convolves the whole image and generates feature maps, the pooling layer reduces the dimensions and network parameters, and the connected layer converts the 2D feature map into a 1D feature vector for more feature representation [45]. The CNN uses feature extraction layers and classification layers, which makes the output highly reliant [46].

The powerful technique of Artificial Neural Networks and the CNNs that are a part of it enable the successful identification of plants and weeds. Generally, the basic structure of a neural network consists of a layer into which data are inserted, layers hidden in between and a layer outputting the data [47].

There are different kinds of CNNs. For example, the LeNet first appeared in the 1990s [48]. However, it was difficult to implement until 2010 due to limited computing power and storage capacity. LeCun managed to bring LeNet to the state of the art through a backpropagation algorithm and some experimentation. LeNet-5 [48], as it was established, consists of two convolutional layers, two subsampling layers, two fully connected layers and an output layer with a Gaussian connection [49]. As the performance of computer hardware improved, the use of CNNs became more popular [49].

In 2012, Alex Krizhevesky won the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) competition with AlexNet [50]. AlexNet is a deeper and broader CNN model that showed the best recognition accuracy compared to previous approaches. In the first convolutional layer, convolution and max-pooling with Local Response Normalization are performed [49]. A total of 96 different receptive filters are used, with a size of 11 × 11. In the next layer, the same steps are performed using 5 × 5 filters [49]. From the third to the fifth layer, the steps are performed again, this time using 3 × 3 filters [49]. AlexNet consists of two fully connected layers with dropout, a softmax layer at the end, three convolutional layers, and two fully connected layers [49]. To use this model, two networks are trained in parallel, which are similar in structure [49].

Another example is GoogLeNet, which won the ILSVRC 2014 [49]. The model, developed by Christian Szegedy and Google, was designed to reduce the computational complexity of a CNN [49]. So-called “InceptionLayers” with variable receptive fields were built into the network, which captured sparse correlation patterns in the new feature map stack [49]. This improved the recognition accuracy [49]. A 1 × 1 convolution kernel was added to the naive inception layer and the final inception layer, which enabled dimensionality reduction before computationally intensive layers [49]. With its 22 layers, it comprised more layers than previous networks [49].

Another network is the VGG16. The VGG 16 performed well in the 2014 ILSVRC [47]. It proved to be one of the best-performing networks with a Top-1 accuracy of 71.3% and a Top-5 accuracy of 90.1% [47]. Even with small image datasets, the VGG16 can achieve high performance and accuracy [51]. The input to the VGG16 is based on a three-channel RGB image of size 224 × 224 pixels [47]. The architecture selects 16 layers, of which 13 are convolutional layers and 3 are fully connected layers [47]. The activation function for each convolutional layer and the first two fully connected layers are Rectified Linear Units (ReLUs) [50]. Despite its high performance, it is less computationally intensive compared to other networks [47].

ResNet-50 has a similar architecture as VGG16, but with a trend toward increased layer depth [47]. At the center of ResNet-50 are 3 × 3 convolutional layers with a ReLU activation function. Before and after each of the 3 × 3 convolutional layers, 1 × 1 convolutional layers are generated [47]. In ResNet-50, only one pooling layer is used and batch normalization is performed. Compared to the previously mentioned VGG16, ResNet-50 has three times more layers and identity mapping capability [51]. An alternative shortcut for the gradient is created and, thus, the problem of the vanishing gradient is reduced [47]. This allows the ResNet-50 to be trained faster than the VGG-16, and although the network is deeper than the VGG16, it can bypass a CNN layer when it is not necessary [47]. A three-channel RGB image with a size of 224 × 224 pixels is also used as an input. The algorithm is one of the best to train new datasets due to the residual management used [47].

Xception stands for Extreme Version of Inception [52]. The introduction of Xception has fundamentally changed the way CNNs are designed [47]. While ResNet-50 increased the depth of the network to solve the image classification problem, Xception takes a different approach [47]. It is not the depth of the network that is increased, but the width. In a classical inception model, several different layers are to be computed in parallel over the input and merged in the output [47]. The increased width is created by three different convolutional layers and a max-pool layer that are activated in parallel [47]. Each output is combined in a single concatenation layer [47]. Therefore, in each layer, a 5 × 5, 3 × 3 and 1 × 1 convolutional transformation and an additional max-pool are performed [47]. How each layer can be used is determined by the concatenation layer of the model [47]. In the Xception network, the inception modules are replaced [47]. Depth wise separable convolutions are used, which compute the spatial correlations on each output channel independently from the others [47]. To capture the cross-channel correlation, a 1 × 1 convolution is performed at the end in depth [47]. Due to the 71 layers formed here, the depth is even deeper than the ResNet-50 [47]. The input is now different from the other networks [47]. The three-channel RGB image here has a fixed size of 299 × 299 pixels. By increasing the degrees of freedom through the width approach, the best detection scenarios are used for specific tasks [47].

The CNN uses feature extraction layers and classification layers, which makes the output highly reliant [46]. The use is influenced by the parameter selection, if the selection is changed, the performance changes [38]. However, CNNs need fewer parameters than other forms of neural networks, which makes them much faster and simplifies the training process [46], and because of that, the network is less vulnerable for overfitting [46]. However, under- and overfitting is still a problem, in combination with a small database or less training time [45]. Even the robustness of the weed classification can be improved by using images, taken under different conditions [53]. All in all, the use of CNN for large-scale network implementation is easier to handle than with different kinds of neural networks [46]. The performance reliability of the CNN is due to the hardware that is used [46]. Clearly, the accuracy of the detecting depends on the used CNN. If the classification is more difficult, for example, when the crop looks similar to the weed, a larger neural network is needed, combined with a high level of computational processing [54]. Alzubaidi et al. [46] came to the conclusion that are a few features that could be changed about the CNN in order to improve it, for example, to expand the dataset or to increase the training time [46].

CNNs have the advantage, that they can detect relevant features, without human interaction [46,55]. One problem with the use of CNNs is the quality of the pictures [56]. If the information given in the picture is less informative than expected, the labeling of weeds and crop is more difficult [56]. That means that if the information given is not enough, the recognition accuracy gets worse. Regardless, an accuracy of about 95% can be achieved [56]. Furthermore, the weeds on the field do not always look the same [14], and by that, the database could contain gaps, even if it is already a large database. Another point regarding pictures is that pictures are made under different conditions [14]. Different conditions generate different pictures of the same growing stage and sometimes are not comparable. All in all, the data acquisition is quite time consuming [14]. A large database is needed to train the CNN. By that, it is ensured that plants are recognized correctly [14]. A large database means a lot of work to deal with, before the work in the field can actually start.

CNNs, in general, start by learning local features from the bottom and gradually assembling them at the top to more complicated features [42]. Finally, they enable conclusions, decisions and classifications [57]. This enables CNN algorithms to “learn” very complex functions [57] by dividing the task by hand into more simple steps. Through repetition and supervised learning CNNs can achieve the desired outcome [57], building multi-layer learning models [46]. Although the use of CNNs is relatively new, recent publications have reached high classification accuracies of 99% for classifying segments as crop, soil, grass and broadleaf weeds [5,58]; 96% on a blob-wise crop/weed classification [59]; 97–98% for classifying between 12 different crop and weed species [47]; 94% between twelve different plant species [60]; and 95% of eight Australian weed species [61]. CNNs need fewer parameters than other forms of Neural Networks, which makes them much faster and simplifies the training process [46], because the Network is less vulnerable for overfitting [46]. Still, under- and overfitting is a problem in combination with a small database or not enough training epochs [45]. To improve the robustness of the weed classification, images should be taken under different conditions [53].

The decision making based on CNNs is by now mostly used in medical images with remarkable performance [62]. Another sector where CNNs are used for decision making is for robots. An adaptive anchoring module is presented that enables robots to improve their mobility and manipulation capabilities [63]. In agricultural sectors, the decision making by CNNs is not that widespread. To our knowledge, only Rautaray et al. have presented a model that can detect diseases in paddy plants by the use of CNNs [64].

There are different kinds of sensors that can be operated by a CNN. Mostly cameras are integrated into systems that use CNNs as, for example, in the Ecorobotix from ARA, a multi-camera vision system is used. CNNs have not only found favor in the agricultural sector. Different sectors tested different combinations of sensors with CNNs. Especially distance sensors have been used in research. Gao et al. used a CNN in combination with LIDAR in autonomous vehicles [65]. A problem that occurs here as well is disturbance variables that complicate the application. Autonomous vehicles in road traffic must also be able to detect objects quickly, precisely and accurately [65]. Various research groups have already carried out tests, for example, combining two heterogeneous neural networks and a support vector machine model for RGB-D-based object recognition and segmentation [66]. However, with the use of LIDAR and CNNs, the classification accuracy is superior in comparison to the use of only RGB or depth data [65]. Other sensors that have already been combined with CNNs are Ultrasonic Sensors. Kim et al. used reflected ultrasonic signals in combination with CNNs for road type identification [67]. The identification of the type of road surface is an important issue for automatic driving vehicles or robots. Ultrasonic sensors have the advantage that they are easy to handle, relatively cheap, and can adapt fast to changing conditions [67]. Other studies investigated the combination of CNNs with optical and electronic processing [68]. The idea behind the combination is to overcome the bottleneck in data transmission by electronically processing data in chips with parallel optical inputs and outputs. The functionality of detectors and emitters is combined in one device [68]. A totally different sector where CNNs were combined with different sensors is within aeroengine control systems [69]. This is thermal machinery, which has to be highly reliant. An intelligent fault diagnosis method was established by combining the continuous wavelength transform with a CNN [69]. In the agricultural sector, so far, spot sprayers have been designed for only specific applications; e.g., “See & Spray” from Blue River Technology [70], a brand acquired by John Deere, USA, was developed for spot spraying only in cotton and soybean. The AiCPlus camera system from Bilberry [71], France; can distinguish Rumex spp. and Taraxacum officinale in oil radish. A separate algorithm must be developed for each plant species, which greatly increases the programming effort [72]. Even though the technique is quite new and a huge programming effort is required, spot spraying has been integrated into several new systems (

Table 1. Overview of commercially available Spot Spraying systems with the currently used technology, the sensors, the access, the application and the possible herbicide reduction.

Product/Trade Mark	Company	Technology	Sensors	Access	Herbicide Reduce	Application
Robotti	Agrointelli	Combining Deep Learning and BigData	RTK-GPS, autonomous, Lidar, Camera	Close	40–60%	Robot
ARA	Ecorobotix	CNN-based weed detection in sugar beet and spot spraying	Multi-camera vision system	Open	Up to 95%	Tractor-mounted
Bilberry	Bilberry	AI-based weed detection and spot spraying	RGB camera	Open	More than 80%	Robot
Weedseeker	Trimble Agriculture	Infrared Sensors	High-resolution blue LED-spectrometer	Open	60–90%	Tractor-mounted
Weed-It	Weed-It	Detection of green vegetation	Blue LED-lighting and spectrometer	Open	95% (only in crop-free areas)	Tractor-mounted
FD20	Farmdroid	RTK-GPS recorded position of crop seeds and spot spraying	RTK-GPS	Open	unknown	Robot
H-Sensor	AgriCon	AI-based weed detection in cereals and maize	Bi-spectral camera	Close	50%	Tractor-mounted
Blue River’s see and spray	Blue-River Technologies	CNN-based weed detection in cotton and spot spraying	RGB-cameras	Close	Up to 90%	Tractor-mounted
EcoPatch	Dimensions Agri Technologies	AI-based weed detection and spot spraying	RGB-camera	Closed	unknown	Tractor-mounted
Kilter AX-1	Kilter Systems	RTK-based crop detection and selective spraying in vegetables	robot	Open	unknown	Robot
Greeneye	GreeneyeTechnology	AI-based weed detection and spot spraying	RGB-camera	Open	unknown	Tractor-mounted
Avirtech-MIMO	Avirtech	UAV-based weed mapping and patch spraying	4D Radar imaging	Close	unknown	Drone
Smart Spraying	BASF, Bosch, Amazone	Camera-based weed coverage measurement and spot spraying	Bi-spectral camera	Close	70%	Tractor-mounted

).

4. Integration of Precision Systems on Tractors

Agricultural spraying machines already separate the spray boom into sections, and with the use of the current technology, activation of each nozzle separately is possible [73,74,75]. With the help of the ISOBUS, tractors can achieve a certain level of precision. ISOBUS is a standard protocol for sprayers and tractors. It enables communication between different components, for example, between a sensor, the tractor and the implement [35]. One objective of the research is to develop a CNN-based modular spot spraying system combined with an ISOBUS-capable sprayer to detect and classify weeds in real time. [76]. Since each module is developed independently to operate with the ISOBUS standard, each new tool that provides precision applications can potentially be included in existing machines, and used in the system [76].

Figure 5 shows possibilities that could be implemented in the future when a modular system is introduced to the market. Different sensors are able to detect and thereby solve different parts of work in the field. RTK-GNSS is used for the positioning of the crop and thereby detection of the crop row [77]. The next sensor shown is the RGB-camera. With the use of RGB-cameras, it is possible to detect weeds on a large scale in agricultural fields [78]. Additionally, weeds and crops can be identified and the crop rows can be detected. Another approach is by the use of multispectral cameras, especially the utilization of Infra-Red wavelengths [79]. Plant material reflects the light in the near infrared, while it absorbs the majority of the light in the visible spectrum. Detection based on the spectral ranges in the red (610–690 nm) and near-infrared (above 700 nm) range can easily differentiate between plants and foreign objects, e.g., stones, mulch and straw. The high signal-to-noise ratio that this setup provides makes such systems reliable for variable light conditions and high working speeds [80]. Distance sensors can also be used, which are relatively low cost and customer friendly. Sensors such as Lidar or Ultrasonic sensors have been used for site-specific weed management [38,47,81,82]. By the use of this combination, the height detection of the crop is possible. This means that the system knows which height the crop should be. By that, smaller and bigger plants should be treated, as those plants are not the desired crop. The mentioned sensors can also have a secondary function, as they can regulate the appropriate height of the implement. Exact height guidance using a laser for a sprayer boom can reduce the herbicide drift by reducing greater distances between boom and canopy or by increasing the uniformity and efficacy of the treatment at lower distances. The sprayer is adapted to not harm the crop during the work, and the herbicide is applied where it is needed. Typically, the sprayer boom is adapted to a height of 40 cm above the canopy level. Concerning the hoe, the guidance and the choice of the respective hoeing sweep must be adapted to the soil, the crop, the weed population and the weed density for a sufficient control. By the use of a 3D camera, crops can be identified even if there are weeds on the field [83]. A GoG recognition is possible, which means that weeds and crops are differentiated. Through the images that are produced, the height of the plant can be detected and, again, the height guidance of the implement can be set up. Three-dimensional cameras are also able to detect the crop rows. Another sensor is a spectrometer. With the use of a spectrometer, the weeds and crops are differed by their spectral differences. The use of the spectrometer has already proven to have some commercial use [84]. Using a spectrometer is a safe and fast way to detect weeds in the field. The recognition is a GoB identification.

With the combination of sensors and applications, different machines can be operated. As shown in Figure 5, the Inter-row hoe works between the rows of the crop. Therefore, it only needs a sensor for crop row detection to operate. More sophisticated sensors are needed, for example for the Intra-Row-hoe. The Intra-Row-hoe works between the rows of the crop and inside the row, between the crops. If the crop row is detected, the weeds are identified as weeds and the height of the implement is adjusted, so that the Intra-Row-hoe can operate. Between the rows of the crop, which are identified, every plant is treated. By detecting weeds, the hoe can operate inside the row and treat every weed growing there [77]. For Band spraying, another combination of sensors is needed. As the band sprayer combines an Inter-Row-hoe and a sprayer, it needs similar sensors. Crop row detection is necessary for the hoe to operate between the rows. As the sprayer only applies herbicides inside the row and the crop row is already detected, it needs the height guidance of the implement to apply the herbicide precisely. The last sensor needed is height guidance, to detect how big the crops are and which plants do not fit in the pattern of the crop [19]. The next machine shown is the patch sprayer. Again, it needs more sensors than the machines before. The weeds have to be identified, in order to spray only the weeds. Therefore, the crop is also identified to not spray the crops. A height detection sensor is needed to detect how high the crop is, and the implement again needs height guidance [7]. For the last machine, the spot sprayer is shown. It needs the same sensors as the patch sprayer. It has a closer field of view, but the sensors need to stay the same. A spot sprayer needs more sophisticated algorithms, which are not shown in the present figure [11].

A project that aims to realize these possibilities is DACWEED (Detection and ACtuation for WEED management), shown in Figure 6 and Figure 7 [76], a project funded by EIT FOOD. Existing technologies, artificial intelligent processes and smart tools controlled by ISOBUS should be developed and improved. By working with existing modules site-specific weed management with commercially available components should be made possible. It will be possible to “plug-in” an independent sensor system and turn any section-controlled sprayer into a site-specific sprayer. This enables the farmer to use the sprayer and tractor he already owns, and connect it with the new technology of spot spraying [76]. A CNN is the basis of the weed detection. The CNN differentiates between weeds or crop, decides if a specific nozzle should open, and sends the information through the ISOBUS to the implement [76]. Patch spraying is conducted in maize, potato and sunflower and a CNN was trained for nine weed species including Alopecurus myosuroides Huds, Amaranthus retroflexus L., Avena fatua L., Chenopodium album L., Lamium purpureum L., Matricaria chamomila L., Setaria spp., Solanum nigrum L. and Stellaria media Vill [47]. For horticultural crops, DACWEED was trained for tomato in Spain with the main weed species Solanum nigrum L., Setaria viridis (L.) Beauv, Cynodon dactilon L. and Cyperus rotundus L [85]. A total of 105,000 images, taken with a Jai camera as shown in Figure 6, of the species were used for training the VGG16, ResNet-50, and Xception networks with learning transfer methods. After the training, the detection system was connected to the ISOBUS terminal of a tractor. The results of the support system decisions could be transferred to the sprayer via the standard ISOBUS protocol. The system could achieve identification accuracies between 77% and 98% (VGG16/Xception) for the classification and categorization of weed species [85]. Due to the speed of activation and deactivation of the valves, it was not possible to fragment the crop into smaller areas. In typical fields with moderate weed infestation, the system can achieve 40–60% herbicide savings. While these percentages can increase in cases of spare weed presence [85]. The aims of the project are to reduce the amount of herbicide used, and increase the precision of the application by opening the nozzles only where and when it is needed. Those steps can lead to more sustainable agriculture for consumers and the environment [76].

Another spot sprayer that uses ISOBUS is the system of Bilberry [71]. It can be combined with any kind of sprayer. By the use of Artificial Intelligence, it detects the weeds and opens the nozzles only at the position of the weed. It is possible to use the system for GoG and for GoB. Another advantage of the system is that it can be used for different growing stages of the weed [71]. Weedseeker, another system able to use the ISOBUS connection, is capable of linking different machines and implements [86]. The different approaches make it possible for the farmer to work independently. However, the majority of spot sprayers do not use ISOBUS to connect between different implements. For their operation, each system relies on its specific technique, for the sprayer or the camera or most typically both. DACWEED aims to improve and facilitate standardized communication between sensors and implement [76].

5. Discussion

The different technologies for patch- and spot spraying highlight the potential of herbicide savings, which are up to 90% [22]. The herbicide reduction was significant regardless of which patch/spot spraying technology was used. Besides the herbicide reduction, the environmental risks associated with herbicide use were reduced, including herbicide leaching into the groundwater, herbicide resistance development in weeds, and herbicide residues in the food chain and drift [7]. By reducing the herbicide load into the environment, the diversity of weed species and insects can also be increased and rare weed species can be protected [87,88].

The amount of saved herbicide differs due to different locations and different management strategies, for example, different crop rotations. Another changing variable is the field in which the system was tested. A lot of systems are tested in cereal crops, with monocot weeds [41]. In each field and crop, there are heterogeneous weed distribution and weed infestation levels. The spatial variable application could be less efficient under different conditions [41]. To address the previously mentioned heterogeneous conditions in a field, there are several construction-related options. One option is to use small robots that can individually drive to individual plants and spray them precisely [89]. In cases where this is not practical, for example, due to the size of the field, it makes sense to adjust the spray boom. The height of the spray boom can be adjusted and the individual nozzles can be individually controlled and directed to the plant [73]. Certainly, new nozzles that can provide the appropriate dosage in the predefined flow will need to be developed. Which of the above-mentioned options is more appropriate must be decided on a case-by-case basis, as the same option is not best for every field and every type of crop. Nevertheless, all tests on all different fields show that a reduction could be achieved and, thus, herbicides and costs could be saved. It needs to be mentioned that the risk of weed escape can increase with patch spraying as more weeds are left untreated in the field [22]. Even though Wiles [22] proved that the weed escape even decreased by 0.04–0.11 plants m⁻² when using a patch sprayer. The outcomes of site-specific weed management in total are specific for every field and cannot be predicted overall [22]. If the characteristics of weed populations are included in the spraying decision, site-specific weed management might be more beneficial [22]. The control of weeds can be more successful when multiple herbicides are used because of the lower chance of weed escape [22]. That can save both more herbicides and more costs in the long term. Still, there is a need for the support of farmers regarding which decision to make on which specific field [22].

However, when comparing the different sprayers, the cost of the sprayer must be considered. If the costs for patch sprayers are compared to the costs for flat sprayers, the sprayer cannot be recommended [22]. The additional management costs cannot be compensated with the additional return [22]. However, by the use of sprayers with Artificial Intelligence, the herbicides and labor can be used more cost efficiently [54]. In the calculation should be included: the size of the farm, so as to decide the size of the sprayer that should be chosen; the training of the technology; and the refilling of the sprayer [90]. Further investigation should be made into choosing different systems and cheaper components for farmers with small fields [90]. Whether the sprayer based on Artificial Intelligence is more economic than the labor costs is dependent on each specific case. Examples show that the costs where a robotic platform would be profitable must be immensely cheaper than the available industry solution, which makes it unrealistic [91]. It is clear that in the beginning, the system has to be established on the specific farms. That means that it has to be trained to the actual conditions and crops. It takes time to explain the new machines to the farmers and for them to become used to the technique, which might take more time than with a conventional sprayer. Compared to the time that is gained later and the herbicides that are saved, the advantages predominate overall. The current market is mainly dominated by patch sprayers. Although some spot spraying systems are already commercially available, they do not yet dominate the market. This is due to the fact that their price currently limits their commercial availability [92]. When considering the economics of robotic platforms, there are other points that need to be mentioned. For a tractor must work efficiently, the size of the boom in relation to the size of the fields on a farm must be considered [90]. On smaller farms with smaller fields, a smaller boom is more economical. On larger farms with larger fields, a larger boom is more economical [90]. If the size of the boom changes, many parameters change. For example, looking at a robot with movable nozzles, the conditions are different from those of a tractor with a mounted sprayer [89]. The robot is able to drive directly over individual plants, stop there, and with its movable nozzles, spray the plant with precision [89]. This is different from a tractor with a mounted sprayer up to 36 m wide. The tractor would have to stop over the individual plants and readjust the spray boom each time to achieve the same precision. However, if this were the norm, the system would lose efficiency, which would negate the economy through lost time and increased fuel consumption. Larger farms need to be time efficient to complete the work at the right time, and in the right weather conditions [90]. The type of farm, as well as the size of the farm, must be considered when determining the economic efficacy [90]. The efficacy of a spot sprayer, moreover, depends on the crop in which the sprayer is used. Not all crops were tested. This means that further investigation on different crops is necessary. Another point is the difference between the crop and the weed. The bigger the difference in their optical appearance, the better it can be differed. To use a spot sprayer for different kinds of crops that are really similar to weeds, such as, for example, for wheat, which is sown in narrow rows less than 20 cm apart, further investigation is necessary [11]. It is easier to use a spot sprayer in crops with wider row spacing, for example, maize with up to 75 cm row spacing, than in crops with narrow row spacing, for example, cereals with 12.5–15 cm row spacing [19]. More space in between the rows offers more space for the sensor to detect the weeds. When looking at the shape of the weeds compared to the crop, problems can occur. For example, looking at Lolium perenne L. in a wheat field [93], it is more difficult to detect the differences between the two plants than, for example, Veronica persica P. in a maize field. The choice of crop, as well as the weeds present, must be included in the evaluation of the accuracy of a machine. Another important point relates to the basis of detection. If it is a GoG analysis in which green plants are to be distinguished from green weeds, a more complex algorithm is necessary [94]. In addition, it cannot always be assumed that the crop and weeds are in the same growth phase [94]. The growth phases of the plants can be similar in some cases, making classification more difficult [94]. When looking at the definition of weeds, problems arise. If maize is growing in a winter wheat field, the wheat, which is otherwise considered a crop, might be classified as a weed. The algorithm must therefore recognize which field it is and whether the crop in this case is possibly a weed.

Even if huge technical improvements are made, there is still space for further investigation [45]. The systems are still unstable under different conditions [45]. Systems still have problems with different biotic and abiotic disturbances such as overlapping of the crops or influences such as changing light conditions [3]. The systems still have to be tested under a wider range of conditions [3]. There are still problems with the computational speed, which is limiting when using real-time approaches [94].

Many of the techniques mentioned above are based on the use of CNNs. Even though the use of CNNs for large-scale network implementation is easier to handle than with different kinds of Neural Networks [46], the performance reliability of CNNs is due to the hardware that is used [46]. Clearly, the accuracy of the detection depends overall on the CNN used. If the classification is more difficult, for example, when the crop looks similar to the weeds, a larger Neural Network is needed combined with a high level of computational processing [54]. Alzubaidi et al. [46] showed that there are a few features that could be changed about CNNs in order to improve them. For example, to expand the dataset or to increase the training time [46]. A problem with the use of CNNs is the quality of the images [56]. If the information given in the picture is less informative than expected, the labeling of weeds and crop is more difficult [56]. This means that if there is not enough information given, the recognition accuracy gets worse. Regardless, an accuracy of about 95% could be achieved [56]. Furthermore, the weeds on the field do not always look the same [14], and therefore, the database could contain gaps, even if it is already a large database. A CNN needs a huge database to be trained to ensure that plants are recognized correctly [14]. A large database means a lot of programming effort before the work on the field can start. Another point regarding images is that they must be taken under different conditions presenting a higher diversity of the plant [14]. Moreover, the data acquisition is quite time consuming [14]. One solution to address the problem of the huge database required is a method based on RICAP (Random Image Cropping and Patching). This method is extended, which allows it to be used for data augmentation of semantic segmentation tasks. Based on experimental evaluations, it was found that this method is suitable for achieving an increase in performance compared to the original form [95].

Most of the solutions for site-specific weed management are isolated solutions based on a specific technique, a point that the project DACWEED is dealing with. Machines and solutions that are already available should be combined to enable the use of the given technology. Sensors and implements of different companies should be comparable, increasing the flexibility of the farmer [76,96]. For example, if instead of an RGB camera, an RTK-GNSS is used, less computational power will be needed [47]. With the use of the RTK-GNSS while seeding, the position of the crop is known. With the position of the crop, a row guidance is possible. A band sprayer needs only row guidance to operate, together with the sensors to adapt the implement. Between the rows, the hoe is operating, and inside the row, the sprayer is applying the herbicide. By not spraying between the rows, the herbicide reduction can achieve up to around 80% without the need of more sensors [97,98]. With these connections, a lot of possibilities arise without the need for many different sensors or different machines. Even though the systems are still not flawless, there are promising approaches to bring the robotic systems forward [3]. It could be possible in the future to use such systems not only for the application of herbicides, but also for fertilization and several specific tasks in the agricultural landscape [33].

6. Conclusions

Patch spraying and spot spraying are potential alternatives for flat-rate applications of herbicides in agricultural crops. They both guarantee high weed control efficacy with much less herbicide use. Patch spraying probably fits better in large-scale arable crops such as cereals, maize and soybean. It can be implemented on large boom sprayers and realized with conventional driving speeds. Spot spraying is more attractive for high-value crops, such as vegetable crops and sugar beets. Due to complex weed/crop classification with CNNs, driving speed is lower. Therefore, spot spraying has been realized in several commercial robot systems. The attractiveness of those robot systems can be increased if they can be operated in many different crops. So far, they are still limited to a few specific applications. Besides significant herbicide savings, patch spraying and spot spraying allow the protection of rare and endangered weed species, which would increase weed biodiversity in agricultural fields. Therefore, patch spraying and spot spraying can contribute to the EU-Green Deal target for reducing herbicide use and increasing biodiversity.