Multi-Temporal Site-Specific Weed Control of Cirsium arvense (L.) Scop. and Rumex crispus L. in Maize and Sugar Beet Using Unmanned Aerial Vehicle Based Mapping

Abstract

Sensor-based weed mapping in arable fields is a key element for site-specific herbicide management strategies. In this study, we investigated the generation of application maps based on Unmanned Aerial Vehicle imagery and present a site-specific herbicide application using those maps. Field trials for site-specific herbicide applications and multi-temporal image flights were carried out in maize (Zea mays L.) and sugar beet (Beta vulgaris L.) in southern Germany. Real-time kinematic Global Positioning System precision planting information provided the input for determining plant rows in the geocoded aerial images. Vegetation indices combined with generated plant height data were used to detect the patches containing creeping thistle (Cirsium arvense (L.) Scop.) and curled dock (Rumex crispus L.). The computed weed maps showed the presence or absence of the aforementioned weeds on the fields, clustered to 9 m × 9 m grid cells. The precision of the correct classification varied from 96% in maize to 80% in the last sugar beet treatment. The computational underestimation of manual mapped C. arvense and R. cripus patches varied from 1% to 10% respectively. Overall, the developed algorithm performed well, identifying tall perennial weeds for the computation of large-scale herbicide application maps.

1. Introduction

One of the major milestones in weed remote sensing technology research has been the implementation of Unmanned Aerial Vehicles (UAVs) as sensor carriers. Rasmussen et al. [1] presented an estimation of plant soil cover from small and inexpensive aircraft systems evaluating the efficacy of mechanical weed harrowing in barley (Hordeum vulgare L.) and chemical weed control in oilseed rape (Brassica napus L. subsp. napus). Early season site-specific weed management in sunflowers based on UAV imagery is described in Torres-Sánchez et al. [2]. Both authors conclude that UAVs are useful to map weed pressure for site-specific weed management. Several UAV imaging sensors (e.g., Red, Green and Blue (RGB) and multispectral cameras), spatial resolutions and data analysis algorithms (e.g., Object-Based Image Analysis (OBIA)) are discussed in the literature [1,2,3,4,5,6,7,8,9].

Airborne imagery enables vegetation detection using vegetation indices, as reviewed in Salamí et al. [10]. Based on the Structure from Motion technique, 3D Models and Digital Elevation Models (DEM) can be created out of UAV imagery [10,11]. Combining this information can provide biomass estimations on barley [12] or yield predictions in maize, when combined with multispectral images [13]. Further, weed identification has been performed by analysing vegetation height differences in a maize field using ground-based ultrasonic sensors [14].

Christensen et al. [15] discussed the complexity of various weed detection procedures, along with the generally low economic weed threshold levels (e.g., <5 weeds m⁻² or <0.1% weed cover). The authors remarked that the field area to be treated with herbicide highly depends on the used economic weed thresholds as also presented in Hamouz et al., Keller et al. and Longchamps et al. [16,17,18]. The information about local weed infestations becomes even more important in less competitive crops (e.g., sugar beet) or perennial weeds with a development stage dependent herbicide compatibility. Thus, high-resolution weed detection and multi-temporal mapping can play a major role in weed management.

Considering these possibilities and the well-known heterogeneous nature of weeds [19,20], site-specific herbicide applications as reported by Gerhards et al. [21] and Gerhards and Oebel [22], should already be state of the art in today’s farming practise. Yet, only a few site-specific herbicide application techniques have been commercially used [23]. Fewer systems support the use of UAV weed mapping based on application maps.

It is crucial to fuse information from the different sensor systems and computing capabilities, used in modern precision farming, to crosslink all gathered field data for weed classification. Nevertheless, weed differentiation systems working in multiple crops by combining comprehensive field information have so far not been described in literature.

Consequently, the hypothesis was that the UAV mapping of herbicide-relevant weeds like creeping thistle (Cirsium arvense (L.) Scop.) can be accomplished by combining multiple sensor-based sources of precision farming information. Information on vegetation coverage and height derived from UAV imagery was connected to crop planting information (e.g., geo-coordinates of crop row locations and row space). By concatenating these data inputs, we propose a new methodology for improving the UAV weed mapping in arable fields. It is of particular interest to separate perennials like C. arvense and curly dock (Rumex crispus L.) from the rest of the fields’ vegetation cover.

The objectives of the present study were (a) the development of an algorithm for the computation of herbicide application maps based on UAV weed mapping, (b) the subsequent use of these maps for weed spot spraying. (c) Realise the above objectives in row crops like maize at the three-leaf stage and in sugar beet between the cotyledon stage and the five-leaf stage. The computed application maps were transferred to a multiple tank spot sprayer for the site-specific treatment of C. arvense and R. crispus. This additional herbicide treatment was then applied along with a uniform herbicide application against annual grasses and broadleaf weed species.

2. Materials and Methods

2.1. Trial Sites and Precision Sowing

For the current study, four experiments were chosen at the Ihinger Hof research station (48.74° N, 8.92° E, 478 m a.s.l.) of the Hohenheim University, in southwest Germany during 2016 and 2017. The average 30-year annual temperature and precipitation are 8.4 °C and 738 mm. The fields were rotated within the two years of this study and they differed in their slope as shown in

Table 1. Experimental details of the UAV—weed mapping trials at Ihinger Hof, Germany.

Year	Crop	Trial Size (ha)	Terrain Slope (%)	Row Distance (cm)	Sowing Date	Seeds ha−1
2016	sugar beet	2.07	2	50	21 April 2016	107,000
2016	maize	1.72	5	75	18 May 2016	94,000
2017	sugar beet	2.09	3	50	13 April 2017	107,000
2017	maize	2.07	2	75	11 May 2017	89,000

. The soil types differed from Terra fusca, brown soil and pseudogley.

Sowing was realised using a Real-Time Kinematic (RTK)-corrected, Global Navigation Satellite System (GNSS)-assisted steering system on the tractor with a steady-state reference station located at the research station. The RTK-GPS enabled the pneumatic precision spaced planter (KUHN, MAXIMA 2TI, KUHN Maschinen-Vertrieb GmbH, Schopsdorf, Germany) to follow previously defined geographical coordinates for seeding with a precision of ≤2 cm.

2.2. Ground-Truth Weed Mapping

The weed plants were counted for each weed species at each centroid of a 9 m × 9 m grid square, overlaid on the trial field. The totals were 173 grid squares in maize (2017) and sugar beet (2016); and 201 in maize (2016) and 220 in sugar beet (2017). Manual weed counting was performed one to three days before the site-specific weed control measurements. The collected weed scouting data of 0.4 m² per grid square was entered directly into a database, along with weed and crop cover and a photograph of the respective counting area. To this end weed species individuals were counted in a 0.1 m² counting frame with four repetitions around each grid square centroid, as tested and discussed in the literature [24,25,26]. These counting results were used to get a general overview about the field weed infestation. The database directly computed the plant density per m² for each species, using the single mapping repetitions per counting frame. The presence or absence of the later UAV-mapped C. arvense and R. crispus patches was additionally mapped outside the predefined counting points over the entire mapping area.

2.3. UAV and Sensor Setup

The weed mapping flights were performed using a hexacopter, model XR6 (geo-konzept, Adelschlag, Germany). The hexacopter was equipped with a Sony α 6000 (Sony Corporation, Minato, Japan) RGB camera and a Tetracam µMCA 4 Snap (Tetracam Inc., Chatsworth, CA, USA) multispectral camera including an electronic Incident Light Sensor (eILS).

The RGB image sensor lens focal length was 19 mm, with an image resolution of 6000 × 4000 pixels. Therefore, a Ground Sample Distance (GSD) of approximately 0.3 cm at a flight altitude of 15 m was achieved. The shutter was set to 1/1250 s and ISO between 200–1600 depending on actual lighting conditions prior to flight, resulting in an automatic aperture of (f − x) with x ∈ [7–10].

Multispectral images were taken at four narrowband spectral wavelengths of 670, 700, 740 and 780 nm using optical band-pass filters, each mounted in front of one of the camera’s four image sensors. In addition to this, the uplooking eILS was mounted on the UAV. The eILS was capturing the downwelling radiation at the same wavelengths as those used in the aforementioned image sensors below. The camera was configured with a predefined focal length (9.6 mm), aperture (f/3.2), and automatic exposure time. The multispectral image sensor gathered 8-Bit, RAW file images with a resolution of 1280 × 1024 pixels, resulting in a GSD of approximately 1.51 cm at a flight altitude of 30 m. This flight altitude was chosen due to sensor limitations (e.g., image interval, battery power), to avoid the separation of flight plans in more than two parts.

Image triggering was software-controlled via a USB connection to the copter’s flight-control-unit. At predefined points, image triggering was set as a flight plan action. Images were captured with a minimum overlap of 60% in- and cross-track to enable photogrammetric calculations. The copter’s flight altitude for use with the RGB sensor was set to 15 m and for the multispectral sensor to 30 m. If necessary, the flight plan was split into two parts for changing the two 24 V, 3700 mAh flight batteries. To overcome the decrease in the ground resolution, flight plans were adjusted to the ground morphology with a digital terrain model (DTM), using data from a previous flight. Therefore, the DTM was used to create the adjusted flight paths for the following flights.

2.4. Image Processing

The camera weed sampling flights were done at different crop growth stages, and flight dates, before and after the herbicide treatment. Weed mapping and application dates, along with the crop plant development stage, are given in

Table 2. Plant development stages at weed mapping and herbicide application dates.

Application Date	Herbicide Application ‡	Crop	Crop Development Stage	Mapping Date †
20 May 2016	POST I	sugar beet	cotyledon stage	18 May 2016
10 June 2016	POST II	sugar beet	5 leaf stage	9 June 2016
22 June 2016	POST I	maize	3 leaf stage	20 June 2016
12 May 2017	POST I	sugar beet	cotyledon stage	11 May 2017
24 May 2017	POST II	sugar beet	4 leaf stage	23 May 2017
1 June 2017	POST I	maize	3 leaf stage	31 May 2017

^‡ Herbicide applications were performed after crop emergence with specific herbicide doses per weed group as shown in Table 3 and Table 4 for each crop respectively; ^† Mapping date includes UAV flight and manual weed mapping.

. Prior to image alignment for the orthomosaic image creation, multispectral images had to pass through a pre-processing step. The alignment of the four single-channel images to the same field of view was carried out using the software PixelWrench2 (Version 1.2.3.1, Tetracam Inc., Chatsworth, CA, USA). In the same step, the images were corrected to the downwelling radiation, according to the data gathered by the eILS.

For generating georeferenced DEMs and orthomosaic images, the 3D reconstruction software PhotoScan Professional Edition (Version 1.3.4, Agisoft LLC, St. Petersburg, Russia) was used. Beside the airborne images, the copters flight information and positions of the RTK-GPS-referenced Ground Control Points (GCPs) located inside the trial and around the borders [2,13] were passed to the software.

2.5. Generating Weed Control Maps Based on UAV Imagery

To compute large area site-specific herbicide application maps, a robust and relatively fast to compute algorithm was designed. The targeted output was an application map to provide the respective geo-coordinates and decisions for one tank of a three tank site-specific sprayer. For further processing, the multispectral and RGB orthomosaic images were loaded as raster objects in the statistical computation software R [27], using the packages raster, version: 2.6-7 [28] and rgdal version: 1.2-18 [29].

The crop row locations were known from the RTK-GNSS-assisted seeding and therefore a predefined AB-line was available from the beginning of the cultivation. Based on this AB-line, all crop row positions were calculated and used to calculate a polygon layer. The crop rows were deleted out of the orthomosaic images by overlaying the crop row polygon layer. The width of the crop row polygon layer was adjusted, based on the actual crop growth stages.

For further discrimination of weeds and soil, the Normalised Difference Vegetation Index (NDVI) was calculated based on the multispectral images, adapted from Rouse et al. [30] using the wavelengths 670 and 780 nm, as shown in Equation (1). The resulting NDVI raster object was used as a mask layer as proposed by Liebisch et al. [31] and demonstrated in Roth and Streit [32] on the relative RGB image. All parts of the raster object with an NDVI > 0.2 were considered as vegetation. If multispectral data were not available, weed maps were calculated only using the RGB images. The Excessive Green Red index (ExGR, Equation (2)) was calculated out of the three layers of the processed RGB orthomosaic image (Ortho), representing the crop fields inter-row space. The ExGR combines two colour indices: the Excessive Red index (Ortho_ExR, Equation (3)) as shown in Meyer et al. [33] and the Excessive Green index (Ortho_ExG, Equation (4)) as shown in Woebbecke et. al. [34]. The result derived from the subtraction of the two aforementioned indices is shown in Equation (2). Thereby the background noise in the Excessive Green grayscale image was reduced and a zero threshold could be applied for the creation of a binary image. The ExGR binary orthomosaic image (Ortho_ExGR) was generated using the ExGR zero threshold method as introduced by Meyer et al. [35] and suggested by Hamuda et al. [36]. Prior to the index calculation, all single layers had to be normalized by the max value of each RGB channel.

$$\mathrm{NDVI}=\frac{780\mathrm{nm}-670\mathrm{nm}}{780\mathrm{nm}+670\mathrm{nm}}$$

(1)

$${\mathrm{Ortho}}_{\mathrm{ExGR}}={\mathrm{Ortho}}_{\mathrm{ExG}}-{\mathrm{Ortho}}_{\mathrm{ExR}}$$

(2)

$${\mathrm{Ortho}}_{\mathrm{ExR}}=\frac{1.4\times {\mathrm{Ortho}}_{\mathrm{red}}-{\mathrm{Ortho}}_{\mathrm{blue}}}{{\mathrm{Ortho}}_{\mathrm{red}}+{\mathrm{Ortho}}_{\mathrm{blue}}}$$

(3)

$${\mathrm{Ortho}}_{\mathrm{ExG}}=\frac{2\times {\mathrm{Ortho}}_{\mathrm{green}}-{\mathrm{Ortho}}_{\mathrm{red}}-{\mathrm{Ortho}}_{\mathrm{blue}}}{{\mathrm{Ortho}}_{\mathrm{green}}+{\mathrm{Ortho}}_{\mathrm{red}}+{\mathrm{Ortho}}_{\mathrm{blue}}}$$

(4)

We considered that each pixel in the classified Ortho_ExGR raster object can be attributed to vegetation, or to secondary substances, like soil or organic matter, based on the result of the binary map, as shown in Equation (5). The classified pixels in vegetation or soil and organic matter had the geographic coordinates north (n) and east (e).

$$f:X_{\left(n,e\right)}=\{\begin{array}{c}1,\forall {\mathrm{Ortho}}_{{\mathrm{ExGR}}^{\left(n,e\right)}}\in \mathrm{vegetation} \\ 0,\forall {\mathrm{Ortho}}_{{\mathrm{ExGR}}^{\left(n,e\right)}}\notin \mathrm{vegetation}\end{array}$$

(5)

Since the classification was conducted with 0 as the threshold value, the Ortho_ExGR image was binarized as shown in Equation (6). The result is the classified Orthoimage of Equation (5).

$$f:X_{\left(n,e\right)}=\{\begin{array}{c}1,\forall {\mathrm{Ortho}}_{{\mathrm{ExGR}}^{\left(n,e\right)}}\in \left(-\infty , 0\right]\\ 0,\forall {\mathrm{Ortho}}_{{\mathrm{ExGR}}^{\left(n,e\right)}}\in \left(0, +\infty \right)\end{array}$$

(6)

Apart from the vegetation information, height information was also used for the discrimination algorithm. The separation of the inter-row vegetation into lower and taller weeds was carried out based on the Canopy Height Model (CHM). The CHM could be calculated by subtracting the Digital Terrain Model (DTM) raster object from the Digital Surface Model (DSM) raster object [11,13] as shown in Equation (7). The CHM presented the height for every single pixel in the corresponding raster object.

$$\mathrm{CHM}=\mathrm{DSM}-\mathrm{DTM}$$

(7)

The DTM was computed by UAV imagery gathered directly after seeding. For calculating the DSM, UAV images from the respective mapping dates were used. The general computation of this digital elevation model was performed as a prior step during the orthomosaic computation from the 2D images. The CHM threshold for separating lower and higher vegetation of the inter-row space was set to a height of 6 cm above ground as shown in Equation (8).

$$f:C_{\left(n,e\right)}=\{\begin{array}{c} 1, \forall {\mathrm{CHM}}_{\left(n,e\right)}\in \left[0.06,+\infty \right)\\ 0, \forall {\mathrm{CHM}}_{\left(n,e\right)}\in \left(0, 0.06\right)\end{array}$$

(8)

All Vegetation pixels above the used threshold were regrouped in a new class of the raster layer. To verify CHM raster cells as vegetation, we overlaid the crop row excluding ExGR raster to the CHM raster (Equation (9)).

$$f:{\mathrm{WHM}}_{\left(n,e\right)}=X•\left(X+C\right)$$

(9)

The resulting Weed Height Model (WHM) raster object, therefore, combined field and seed planning precision framing data with models calculated from UAV based airborne images. It provided information about the field weed infestation in two classes and the extracted information was used in the site-specific herbicide sprayer on-board computer database.

2.6. Site-Specific Herbicide Application in Field Trials

Site-specific herbicide applications were realised with a trailed multiple-tank spot-sprayer, with three separate hydraulic systems. The sprayer was equipped with RTK-GPS and section control to adjust the application of different herbicides at specific field locations [22,26]. Using three separated hydraulic systems on the sprayer, three different herbicide mixtures were stored and applied wherever it was designated from the application maps. Each hydraulic circuit had its own tank and boom. The booms had a total width of 21 m and were divided into seven sections each of 3 m width.

The sprayer’s section control turned on and off the sections used for the site-specific application via solenoid valves. The same terminal also regulated the herbicide doses for the two other hydraulic systems, applying herbicides on the complete field.

Herbicide application maps of sugar beet and maize trials were used to apply products as shown in

Table 3. Weed-specific herbicides applied with a three-tank spot-sprayer in sugar beet trials.

Weed	Herbicide a.i.	Product	Formulation	Product	Herbicide Rate †
Group		Name		Concentration	g a.i. ha−1
Dicotyledon	phenmedipham	Betasana SC®	160 g L−1	0.52 L 100 L−1	208.0
	ethofumesat	Ethosat 500®	250 g L−1	0.18 L 100 L−1	112.0
	metamitron	Goltix Gold®	700 g L−1	0.72 L 100 L−1	1254.5
Cirsium arvense L.	clopyralid	Lontrel 720 SG® ‡	720 g kg−1	32 g 100 L−1	79.9
Rumex crispus L.
Monocotyledon	fluazifop-P-butyl	Fusilade max®	125 g L−1	0.16 L 100 L−1	50.8

^† All herbicides applied at 250 L ha⁻¹; ^‡ Lontrel 720 SG was applied with 0.5 L ha⁻¹ liquid paraffin.

(sugar beet) and

Table 4. Weed specific herbicides applied with three tank sprayer in maize trials.

Weed	Herbicide a.i.	Product	Formulation	Product	Herbicide Rate †
Group		Name		Concentration	g a.i. ha−1
Dicotyledon	bromoxynil	Bromotril®	225 g L−1	0.17 L 100 L−1	112.5
	mesomitrone	Callisto®	100 g L−1	0.23 L 100 L−1	66.5
Monocotyledon	foramsulfuron	Maister®	30 g L−1	0.20 L 100 L−1	17.7
	+ iodosulfuron-methyl-natrium		1 g L−1		0.6
	+ isoxadifen-ethyl (safener)		30 g L−1		17.7
Cirsium arvense L.	clopyralid	Efigo®	267 g L−1	0.08 L 100 L−1	59.3
	+ picloram		67 g L−1		14.9

^† All herbicides applied at 290 L ha⁻¹.

(maize) for the mapped group of weeds respectively. The composition of the herbicide mixtures was the same in both study years. The herbicide selection for the post-emergence treatments of the mapped species was based on regional guidelines in Germany and is given in Table 3 and Table 4. As shown in Table 2, herbicide applications, weed mapping and flight dates were synchronized for each treatment within a three-day period. Inside each field, two of the grid squares were established as an untreated control by manually adjusting the herbicide application maps prior to spraying.

In 2016, a hexacopter with an RGB and a four channel multispectral camera was used for weed mapping. The GSD of the gathered images during these flights was insufficient for computing valuable weed maps when following the presented protocol. Therefore, site-specific weed control application was completely based on manual weed maps in 2016. The complete workflow, including the different computation steps is presented in the flowchart of Figure 1. The procedure described therein was applied to every XR6 flight dates without changes in the main algorithm.

3. Results

The automatic flight altitude adjustment by following a digital elevation model resulted in a steady GSD of the orthomosaic images and DEMs.

A binary raster was created using the workflow as described in Figure 1 on RGB and multispectral images gathered by an UAV. Figure 2 shows the different processing stages of the binary creation. The raster containing the height information is shown along with the raster containing the vegetation index and the crop row polygons, all overlaid on the starting RGB raster, showing a C. arvense patch in sugar beet at the 4 leaves crop development.

Although, overlaying different raster files from different flights, e.g., DTM and DSM for CHM calculation worked as expected, the image resolution slightly decreased during the, resampling of the thereby adjusted raster files.

In this study, the weed species distribution was heterogeneous within the experiments. In the 2017 maize trial the main weed species was corn sowthistle (Sonchus arvensis L.) with a mean plant density of 15 plants m⁻² and bearbind (Fallopia convolvulus (L.) Á. Löve) with a mean plant density of 12 plants m⁻². In the maize trial of 2016 it was common lambsquarters (Chenopodium album L.) with 55 plants m⁻² and annual bluegrass (Poa annua L.) with 24 plants m⁻². In some distinct places of the maize trial in 2016 the weed density was very high, with up to 375 P. annua and 150 C. album plants m⁻². In the sugar beet trials the weed density also changed between the application dates. While the first herbicide application of 2017 had to treat weed densities of up to 450 black-grass (Alopecurus myosuroides Huds.) individuals, the number decreased to 280 plants m⁻² at the second treatment. Another major weed in all trials was chamomile (Matricaria chamomilla L.) with plant densities between seven plants m⁻² in the sugar beet trial of 2016 and 108 plants m⁻² in the sugar beet trial 2017. In general the weed species composition was linked closer with the trial field, than with the planted crop. In 2017 patches containing C. arvense or R. crispus were manually mapped only in 16 distinct places in maize. On the other hand, up to 114 grid points with the abovementioned weeds were found in the sugar beet field of 2017. Therefore, herbicide savings concerning these patches was 90% in the maize POST I treatment, 45% in the sugar beet POST I and 43% in the sugar beet POST II treatment. The spatial distribution of the manual and UAV classified weed patches are given in Figure 3.

Weed threshold levels for herbicide application against all mapped weeds were set at 0 plants m⁻² in sugar beet and maize. Therefore, site-specific treatment of the UAV classified weeds (C. arvense and R. crispus) was only possible using a GPS-controlled patch sprayer with multiple separated hydraulic circuits. Using this sprayer in the field, herbicide application against monocotyledon and dicotyledon weeds was realised using two of the three hydraulic systems of the sprayer, while the site-specific treatment of the UAV mapped weeds (C. arvense and R. crispus) was applied with the third hydraulic system.

Compared with the manual weed maps, the UAV categorized maps showed an accuracy of 96% in the maize POST I treatment, 90% in the sugar beet POST I and 80% in the POST II treatment, compared with the manual weed maps. The overall manual and UAV weed patch identification is presented in confusion matrices in

Table 5. UAV and manual mapping prior to maize POST I treatment.

		Manual Mapping
		Weed-Free	Weed
UAV mapping	Weed-free	16	2
	Weed	5	150

Weed refers to C. arvense and R crispus > 6 cm, n = 173.

,

Table 6. UAV and manual mapping prior to sugar beet POST I treatment.

		Manual Mapping
		Weed-free	Weed
UAV mapping	Weed-free	144	7
	Weed	14	85

Weed refers to C. arvense and R crispus > 6 cm, n = 220.

and

Table 7. UAV and manual mapping prior to sugar beet POST II treatment.

		Manual Mapping
		Weed-free	Weed
UAV mapping	Weed-free	103	23
	Weed	21	73

Weed refers to C. arvense and R crispus > 6 cm, n = 220.

.

Compared to the respective manual weed map, the UAV identification overestimated weed patches of 3% in maize POST I and 10% in sugar beet POST II, shown in Table 5, Table 6 and Table 7. However, the number of UAV underestimated patches was 50% lower. The only exception to this statement was the sugar beet POST II treatment.

4. Discussion

The large-scale precision farming field experiments conducted in this study proved that the herbicide output can be reduced, when combining UAV imagery and plant seeding information, for computation of detailed application maps.

The presented workflow could be used in maize and sugar beet with generally low weed threshold levels [22,37]. Thus, the sprayer was only turned off at locations where none of the respective weeds was found. For weed recognition, similar results were found using tractor mounted, camera based, online systems [21,22,38]. Although all these studies integrated camera mapped weeds in the site-specific management, the results in the present study clearly stated the possibility of weed categorization and herbicide savings, even though the UAV data were only computed for two weed species.

While the spatial resolution of the UAV images for weed recognition was lower compared to tractor-mounted cameras, the UAV mapping has its advantages. The UAV weed mapping can be conducted, before the application and without soil compaction. Therefore, farmers can calculate the amount of herbicide needed on the field beforehand. Also, a multi-temporal, weekly surveillance can be realised for surveying actual weed infestation in order to meet the economic threshold levels for herbicide applications.

Using polygons to exclude crop rows at a known position speeds up the data analysis and reduces the total data size that not only needs to be computed, but also transferred, and stored. Although the intra-row space was not yet investigated, there was no considerable loss of data concerning the investigated weed patches due to the intra-row deletion. The generally equivalent weed coverage between inter row and crop row areas was also stated in Longchamps et al. [39], although they found a partially higher weed infestation in the crop rows. Furthermore, the total size of the dataset was reduced by up to 30% before entering the main algorithm computation. Extracting plants from the RGB orthomosaic images with the ExGR vegetation index and the use of a steady zero threshold level for binary image calculation was also possible for the UAV imagery. This complies with the proposal of Hamuda et al. and Mayer et al. [35,36]. The fixed threshold level for separating vegetation from soil is a key element towards automation of image-based weed recognition. Especially the differences in soil and vegetation appearance, e.g., wet soil, dense vegetation or the changing light conditions between flights, can cause deviations from predefined thresholds. Thus, the combination of the ExG and ExR vegetation indices in the ExGR provides an option to circumvent this problem.

Even though this robust and fast computable vegetation index for C. arvense and R. crispus worked, the loss of information in our data set did not allow the analysis of small broad-leaved and grass weeds. To realise the separation of smaller weeds at the cotyledon stage, for example by using the OBIA algorithm, higher resolution images will be needed. If this can be achieved, UAV weed mapping of different species and herbicide treatments at early weed plant stages, as needed in sugar beets, can be realised. This well-known coherence between image sensor resolution, flight altitude and vegetation index thresholds is also discussed by López-Granados et al. and Mesas-Carrascosa et al. [6,40,41]. Again, the need of high resolution image sensors becomes apparent. This was also shown in our multispectral data sets providing an almost four times lower GSD. The NDVI orthoimages that we calculated from our multispectral images could be used for pre-categorisation of our RGB orthoimages. However, single weed detection was not possible. On the other hand, the multispectral images could be corrected to the downwelling sunlight using the eILS data. Therefore, the result of the NDVI >0.2 [31] was relatively consistent.

Combining information by overlaying the RGB calculated ExGR vegetation index and CHM already provided information about the field weed infestation. It enabled the separation of single weeds in different herbicide application relevant categories. Compared to simple colour-based algorithms, a secondary information source was added. A comparable approach is also described by de Castro et al. [8] for weed detection in common sunflower (Helianthus annuus L.) and cotton (Gossypium spp.). With this additional pixel height information, plants can not only be identified by their colour but also by their height. In our case, this provided already enough information for separating C. arvense and R. crispus from other crop and weed plants, found in the crops inter-row space.

The detection of C. arvense plants at 6 cm enables the control at the plant’s compensationpoint, where the acropedal allocation of carbohydrates turns into a basipedal movement of photosynthates [42]. At this point the plant is highly susceptible to control measures, since the root system has the lowest regenerative capacity [43]. The typical appearance in patches of C. arvense, which is caused by its mainly vegetative reproduction inside arable fields [44], can also provide information of patches appearing in the near future or the next vegetation period.

From a practical point of view, it is better to overestimate the presence of C. arvense, than underestimate it. Even in this case, herbicide savings can be achieved, especially at low C. arvense infestations. Using this raster layer for OBIA analysis is not yet compatible, since no distinct shape structures of the weeds are available anymore, as shown in Figure 2.

The generation of orthomosaic images and CHMs with a steady spatial resolution is essential to follow the weed mapping workflow presented in this paper. The DTM following flight plan was a useful adjustment in our fields to follow the slope of up to 10 m. The unintended increase of the flight altitude caused by the terrain would have caused a loss in the spatial resolution. When flying the UAV at low altitudes of 15 m, a slope of 5 m, as tested in our study fields, would reduce the spatial resolution in the lowest parts of the field from a GSD of 0.3 cm to 0.4 cm, if the flight altitude is not adjusted. Furthermore, the precise overlaying of rasters was also supported by georeferencing the images using the RTK-GPS corrected GCPs, which helped to improve the accuracy of the coordinate system on the orthoimages.

5. Conclusions

(a) We can conclude that the presented aerial imagery computation procedure is able to identify the presence or absence of weeds above a defined height threshold level in row crops. The use of a field CHM in combination with the Vegetation Index ExGR and crop row geo-coordinates enabled the separation of C. arvense and R. crispus from the rest of the vegetation by their height with an overall accuracy of 96% in the maize POST I, 90% in the sugar beet POST I and 80% in the sugar beet POST II treatments. (b) The output data enables the control of site-specific weed management strategies. Resulting in herbicide savings of 90% in the maize POST I treatment and 43% in the sugar beet POST II treatment. (c) The same algorithm was able to work in two row crops. Thereby the aggregation of different precision farming data sources lead also to a reduction in data size and needed computation power, already in early processing steps. This can be a key element in agricultural fields where fast Internet connections for transferring data to high-performance computing clouds are not yet guaranteed. In order to mainstream the processing chain into a simple fly and treat application, further investigations are needed, specifically orthomosaic image georeferencing and overlaying of multi-temporal UAV orthoimage rasters, which are labour-intensive.