Next Article in Journal
Mobile Laser Scanned Point-Clouds for Road Object Detection and Extraction: A Review
Previous Article in Journal
Tomographic Imaging of Ionospheric Plasma Bubbles Based on GNSS and Radio Occultation Measurements

Remote Sens. 2018, 10(10), 1530; https://doi.org/10.3390/rs10101530

Article
Weed Mapping with UAS Imagery and a Bag of Visual Words Based Image Classifier
1
Julius Kühn-Institut - Federal Research Centre for Cultivated Plants, Institute for Plant Protection in Field Crops and Grassland, Messeweg 11-12, 38104 Braunschweig, Germany
2
Leibniz Institute for Agricultural Engineering and Bioeconomy e.V., Potsdam-Bornim, Max-Eyth-Allee 100, 14469 Potsdam, Germany
*
Author to whom correspondence should be addressed.
Received: 20 August 2018 / Accepted: 19 September 2018 / Published: 24 September 2018

Abstract

:
Weed detection with aerial images is a great challenge to generate field maps for site-specific plant protection application. The requirements might be met with low altitude flights of unmanned aerial vehicles (UAV), to provide adequate ground resolutions for differentiating even single weeds accurately. The following study proposed and tested an image classifier based on a Bag of Visual Words (BoVW) framework for mapping weed species, using a small unmanned aircraft system (UAS) with a commercial camera on board, at low flying altitudes. The image classifier was trained with support vector machines after building a visual dictionary of local features from many collected UAS images. A window-based processing of the models was used for mapping the weed occurrences in the UAS imagery. The UAS flight campaign was carried out over a weed infested wheat field, and images were acquired between a 1 and 6 m flight altitude. From the UAS images, 25,452 weed plants were annotated on species level, along with wheat and soil as background classes for training and validation of the models. The results showed that the BoVW model allowed the discrimination of single plants with high accuracy for Matricaria recutita L. (88.60%), Papaver rhoeas L. (89.08%), Viola arvensis M. (87.93%), and winter wheat (94.09%), within the generated maps. Regarding site specific weed control, the classified UAS images would enable the selection of the right herbicide based on the distribution of the predicted weed species.
Keywords:
low altitude UAS flights; weed mapping; bag of visual words; object based image classification

1. Introduction

Weeds are in competition with crops for light, water, space, and nutrients [1,2,3], and can cause substantial crop yield losses [4,5,6]. Since wheat accounts for the second largest food supply to the world’s population (i.e., 527 kcal per capita per day, [7]), weed induced yield losses have a serious impact on the global economy and food security. Consequently, in a world of steady rises in food demand, sustainable agriculture requires the implementation of adequate strategies for weed control. Given its effectiveness, the use of herbicides is still increasing worldwide [8], despite possible negative impacts on the environment, e.g., leaching into ground water, risk to soil biology, or impact on biodiversity [9,10]. However, the social awareness of the proper use of herbicides is also increasing [11].
One strategy to reduce herbicide use and the related environmental impact is the use of site specific weed control (SSWM). The aim of SSWM is to confine the use of herbicides only to those areas where weeds are present in the fields, whilst avoiding reducing the efficacy of the overall crop protection application. Since weeds are spatially aggregated in wheat fields, it has been shown that SSWM can reduce herbicide use effectively without negative impacts on wheat yields [12]. Three elements are needed for SSWM: A detection and monitoring system for determining the weed plants, a decision algorithm for herbicide selection and the application rate at specific locations, as well as field sprayers that are able to vary application rates according to the detected or mapped weed distribution [13,14]. Our research study is primarily focused on the first part, i.e., the monitoring, detection, and localization of weeds in agricultural fields.
To monitor agricultural fields, unmanned aerial systems (UAS) have become suitable platforms due to the moderate costs for acquisition, flight support, and high flexibility regarding the choice of flying altitude [15,16,17,18,19]. The altitude of the remote sensing platform has the most influence on image structure and size. From the projected size of one pixel on the ground (objects) follows the spatial resolution (Figure 1), which depends mainly on the distance between the object and camera sensor, its dimensions, and the focal length. Although low flight altitudes significantly reduce ground coverage compared to satellite-based remote sensing [20], it decreases the ground sample distance (GSD) and thus increases the level of detail in the images [21]. It has been shown that UAS imagery collected from an altitude higher than 50 m can broadly monitor the coverage of plants, biomass, as well as water and nutrient requirements for entire fields [15,19,22,23,24]. Low altitude flights can provide detailed UAS imagery that allows the extraction of relevant plant traits from local image characteristics. This would enable the exact localization and classification of different plant species for weed monitoring, which is a fundamental prerequisite to site-specific and selective herbicide application [19,25].
Regarding these preliminary considerations, the aim of this study was to show an approach for mapping emergence of weeds in winter wheat using low altitude UAS imagery. Our intention was to develop an approach that could recognize weed species in images taken from RGB camera systems, which can be installed on UAVs. Since human perception can differentiate and annotate weed species in the super high-resolution imagery, it is plausible to try and recognize weed plants using specific learning approaches that partially mimic human brain functions. We follow an object and form-based approach relying on local-invariant image features. The predictive model, we chose, is based on a learned image classifier using a Bag of Visual Words (BoVW) framework. The classifier was calibrated on a large weed image data set annotated from natural cluttered UAS images, which were not segmented before modeling. Overall accuracy for separating weed and crop, as well as separating different weed species, were tested on an independent image data set. We further tested the influence of altitude (1–6 m), filter intensity, and parameterization of the BoVW model.

Background

In general, weed detection has been studied for decades. Hyperspectral measurements have been used to characterize the reflectance of leaves to differentiate weed and crop plants under laboratory conditions [26,27]. To some extent, these results could be corroborated under semi-controlled or outdoor conditions, under ambient lighting [26]. This would enable a pixel-wise acknowledgement of weed occurrences and may even detect weed leaves that only appear below crop plants, depending on the resolution of the images. However, different lighting conditions strongly affect reflectance spectra and this has led to inacceptable classification rates [28]. Furthermore, the data-handling of large hyperspectral images is complicated and the equipment is expensive. In contrast, commercial cameras are cost-effective and can be easily integrated onto low-cost aerial platforms such as UAVs. However, the broad spectral bandwidths of RGB cameras limits the pixel-wise detection of weeds to conditions where there is a good discrimination between crop and weed, by color or RGB indices [29,30,31]. Thus, it makes sense to further incorporate information about leaf shape, texture, and size in the classification.
Consequently, earlier research targeted the global morphology of the leaves, so that features such as contour, convexity, or first invariant moments were used for classification [32,33,34]. However, in cluttered images, global features have been used to train support vector machines (SVM) with only decent results. This is because overlapping plants often result in complex objects and must be handled with a more specific and complex shape extraction method [35]. In particular cases, active shape models (which use deformable templates for matching leaves) and texture-based models had some success in detecting and discriminating plants from images with overlapping leaves [36,37]. More recently, local invariant features have been used to discriminate weeds from crops. Local invariant features are distinctive points of interest in an image, which are representative of its structure as found by a keypoint detector and bear a unique description to its immediate neighborhood. Kazmi et al. [38] could show a very high discrimination of creeping thistle from sugar beet using local invariant features, with a BoVW classifier and a maximally stable extremal regions (MSER) detector. A recent study by Suh et al. used BoVW to differentiate sugar beet from volunteer potato [39]. As local image descriptors they used a scale-invariant feature transform (SIFT) and speeded-up robust features (SURF). They trained models for weed detection with different machine learning classifiers. They concluded that the SVM approach showed better classification performance than random forest or neural networks when used with a BoVW. However, their study was based on a rather small annotation data set of 400 images.
For more differentiated plant detection, convolution neural networks (CNN) are a further approach that has recently gained attention. Unlike BoVW, image features will be directly learned from a neural network using convolution hidden layers [40]. Dyrmann et al. [41] used more than 10,000 RGB images depicting single weed plants to train their own CNNs. Images were taken under controlled light conditions, as well as under field conditions. The background was removed from the plants by color segmentation before modeling. They showed medium to high classification rates for 22 weed species.
To extract weed information from high resolution UAS imagery, most studies use a user-generated approach, following an object-based image analysis (OBIA) workflow. Peña et al. [42] successfully classified weeds in a maize field from UAS imagery collected from a 30 m altitude, using the relative position of the plants with reference to the crop row structure. They used a rule-set algorithm, which comprised spectral, contextual, and morphological information, for image segmentation in different working tasks using the software “Ecognition”. In a later study, the workflow was further improved to differentiate Sorghum halepense from maize crops. Tamouridou et al. [43] were able to map Silybum marianum out of a fallow field with a high variety of different weeds. For this purpose, UAS imagery was used with a ground resolution of 0.04 m and a maximum likelihood classification with green, red, near-infrared, and texture filter information. Thus far, there are few studies that use local invariant features for the classification of weeds in UAV imagery. Hung et al. [44] identified three weed types in Australia from UAS imagery by learning a filter bank with a sparse autoencoder. Recently, Huang, et al. [45] have shown the generation of crop-weed maps in a rice field from UAS imagery with different types of CNNs. Nonetheless, specifically for UAS imagery, BoVW approaches have not been tried for weed detection until now. We postulate that the use of local invariant image features, such as SIFT, might be beneficial because they are both translation and rotation invariant, and are thus robust to changes of flight altitude, flight orientation, or slight deviations from the Nadir camera perspective.

2. Materials and Methods

2.1. Test Site and UAS Flight Mission

The study was conducted in a field in the Northern part of Germany near Braunschweig (52°12′54.6″ N 10°37′25.7″ E). Winter wheat (TRZAW, cv. ‘Julius’) was grown in this field with a row distance of 12.5 cm and a seed density of 380 seeds∙m−2. The soils in this field were gleyic Cambisols with sporadic occurrences of stones. The flight mission was completed during one day in March 2014. At this date, all the weed and crop plants were abundant, and winter wheat sown in November 2013 had reached the development stage of BBCH 23 (Table 1). No weed control measures were performed before the measurements.
The flight mission was conducted with a hexa-copter system (Hexa XL, HiSystems GmbH Moormerland, Deutschland) using modified flight control software based on the Paparazzi UAV project ([46]; CiS GmbH, Rostock-Bentwisch, Deutschland). For image acquisition, a Sony NEX 5N (Sony corporation, Tokyo, Japan) point and shoot camera was used, which was able to resolve 4912 × 3264 image points on a 23.5 × 15.6 mm sensor surface (APS-C Sensor). The lens attached had a fixed focal length of 60 mm (Sigma 2.8 DN, Sigma Corp., Kawasaki City, Japan). The camera was mounted onto a gimbal underneath the copter. The copter was navigated along two routes at altitudes between 1 and 6 m. In total, over 100 waypoints each located at the centers of a 24 × 24 m pattern were passed (Figure 1). At each waypoint, an image was shot from a Nadir perspective. Camera parameters and flight altitudes resulted in a GSD between 0.1 and 0.5 mm. The images spanned an area between 0.1 and 3.7 m2 on the ground. We subdivided the field in training and test set areas, where we allocated the taken images in the corresponding data sets for the later use in calibrating and testing the image classifier (Figure 2).

2.2. Plant Annotation, Image Preprocessing, Sub-Image Extraction

All UAS images were visually examined and the plants were annotated. Plant annotations were located by setting a point in the center of each plant and described by using relative image coordinates with a Matlab GUI script (ImageObjectLocator v0.35). In total, 25,452 ground truthing locations were annotated in the UAS images and referenced to an annotation data base. Around the midpoint of each ground truthing location, a 200 × 200 px quadratic frame was drawn for extracting the sub-images for later model building and validation. Each sub-image showed either a specific plant or soil.
The investigation area was subdivided in two training areas and one test area, as given in Figure 2. The exact allocation of the sub-images into train and test sets for each plant species and soil background is shown in Table 2. Sample images are given in Figure 3. For model optimization, we divided the train set further into calibration and validation sets. Samples were selected randomly from the train set with 75% for calibration and 25% for validation.

2.3. Calibration of An Image Classifier for Weed Recognition

The image classifier was generated with the BoVW concept, as given in Figure 4. The idea was to breakdown the content of an image in relation to a generalized set of features that was a priori derived from a large set of images. The generalized set of features or code words was referred to as the visual dictionary. The relations between the features of an image to this visual dictionary can be expressed as a frequency vector counting the numbers of unique relations between image features and code words (generalized features of the visual dictionary). This frequency vector is called the “bag of visual words” [47]. Combined with known labels of the images, these frequency vectors are then used for calibrating the image classifier.
We used only the sub-images (Figure 3) extracted from the UAS images located in the training area for calibrating (Figure 2). The sub-images were processed in three different color spaces before presenting them to the model for comparison, i.e., unprocessed RGB, a plain gray-scale transformation using the Matlab built-in function rgb2gray, and a transformation to HSV color space by rgb2hsv, which applies the algorithm of Smith [48].
The visual dictionaries were built by a modified framework, originating from the computer vision feature library (VLFEAT, Version 0.9.21) of Matlab [49]. As a standard algorithm for key point detection and description, this toolbox uses a blob detector based on SIFT [50]. To find out the optimal descriptor scale, spatial histograms were calculated for different bins, which varied by pixel widths of 2, 4, and 8. The visual dictionary was built using K-means clustering. This meant that the code words, which constitute the visual dictionary, represented the average Euclidean center of key point descriptors mapped to one of n clusters that were found by the average Euclidean dissimilarity from cluster to cluster. This mapping of new input data into one of the clusters by looking for the closest center was tested for vector quantization (VQ) and the k-dimensional binary tree (KDTREE). In contrast to the linear approximating process of VQ, KDTREE uses a hierarchal data structure to find nearest neighbors by partitioning the source data recursively along the dimension of the maximum variance [49].
The dictionary size, e.g., the number of code words (or cluster centers), were systematically varied from 200, 500, and 1000. Key point descriptors of the new images were related with an approximate nearest neighbor search to construct the BoVW vectors. Support vector machines were used to calibrate the image classifier with linear kernel and different solvers. While the Stochastic Dual Coordinate Ascent solver (SDCA) maximizes the dual SVM objective, the Stochastic Gradient Descent (SGD) minimizes the primal SVM objective. We also tested the open-source Large Linear Classification library (liblinear), which implements linear SVMs and logistic regression models using a coordinate descent algorithm [51]. All hyper parameters of the BoVW framework were systematically permuted, which produced a number of n = 486 trained BoVW models in total (Table 3). All analyses were done using MATLAB (Version 2016b, The Mathworks, Natick, MA, USA).

2.4. Mapping the Models on the UAS Field Scenes

A sliding window approach (Figure 5a) was used for mapping the BoVW models, on a total of n = 34 UAS images of the test zone [52,53]. To accomplish this, the UAS images were segmented into numerous smaller sub-images with different block sizes (BS) of 50 and 200 px and overlaps (shift) of 5 and 20 px, respectively. The lower the value of the shift is, the higher the overlap of the sub-images and the finer the sampling rate. Then, the BoVW model was evaluated over all rectangular sub-images, and a value of 1 was added to each pixel position of BS in the corresponding category layer. Each image region was subsequently classified using the BoVW model, and its result was summarized in the corresponding layer. Finally, a quality function mapped the category number into a winner matrix considering the object’s location and the maximum value across the categories of layers (Figure 5b). The mapping results were validated by n = 8686 ground truthing points within the UAS field scenes from the test set area (see Figure 2).

2.5. Metrics for Evaluation

The testing of the BoVW models and the generated maps was done using the sub-images and the annotations made in the UAS images both taken from the test zone. The weed classification approach shown here considered not only a global crop–weed classification, but it discriminated between four different weed species, soil, and winter wheat. Nevertheless, this task of detection needs a metric to evaluate the quality of decisions. True Positive (TP), True Negative (TN), False Negative (FN), and False Positive (FP), were calculated from 6 × 6 confusion matrices for each class. For example, in case of MATCH, the correct predictions of the category MATCH are called TP, whilst TN is the sum of all other correct predictions of not MATCH categories. FP summarizes cases in which MATCH is falsely predicted as MATCH, whilst in reality it belongs to another category; whereas FN describes cases in which another category is falsely predicted as MATCH. TP, TN, FP, and FN were used to estimate the performance of the BoVW models with the following metrics:
P r e c i s i o n = T P T P + F P
R e c a l l = T P T P + F N
A c c u r a c y   o v e r a l l = T P + T N T P + T N + F P + F N
Precision represents the predicted positives that are truly real positives (Equation (1)). That refers to how well a given model can deliver only correctly classified species in the result set. In contrast, recall (also known as sensitivity) is the probability of predicting positives being real positives (Equation (2)). It reflects how many plants, predicted in a specific category, are truly part of this category. The overall accuracy was calculated by Equation (3). All metrics calculations were done in R [54] using the ‘caret’ package [55].

3. Results

3.1. Image Classification

The classification results of the BoVW models with different vocabulary sizes of the visual codebook and transformations of the input images are summarized in Table 4. In short, only the results from the SDCA solver and KD-TREE quantization were shown here because results from LIBLINEAR and VQ did not provide better accuracies and computing performance was around two times slower (see Table S1). Generally, there was a slight increase of the overall classification accuracy when increasing the vocabulary size from 200 to 500, whereas the increase in accuracy from the 500 to 1000 vocabulary size was negligible. When transforming the images to HSV color space before subjecting them to the BoVW model, the overall classification accuracies were improved.
In more detail, the identification of soil and wheat images had a higher precision (>95%), in comparison to the identification of weed images (<70%). However, to some extent, there was the chance that weed images were falsely classified as soil or wheat, as can be seen by the recall reaching only 80–85% (Figure 6). Among the weed species, MATCH was most precisely identified in the test image set, whilst VERHE and VIOAR the had worst precision. This is corroborated by the confusion matrix of model 578 (Table 5, vocabulary size 500 and transformation HSV), which shows in more detail that VERHE and VIOAR had a high misclassification rate between each other. Obviously, the BoVW model confused both weed species because of its high similarity in visual appearance. However, the HSV transformation positively influenced the precision and recall of the VERHE and VIOAR models, as well as the identification of the soil background; whereas it had almost no effect or even an unfavorable effect on the MATCH and PAPRH models.

3.2. BoVW Mapping of the UAS Field Scenes

The BoVW models were used to predict categories continuously over the UAS field scenes to map the occurrences of soil, wheat, and weed species. We compared the model performances of two different window block sizes (50 and 200 px) and two step sizes (5 and 20 px) of the sliding window computing (Table 5 and Table 6 and Table S2). Generally, with finer filtering and scanning, the overall accuracy increased about 20% for the pixel-exact validation. The precision of the individual categories showed a similar pattern as the models of the image test set. Soil and wheat were identified with the highest precision, greater than 90%, for the models with finer filtering. This showed that generally a good differentiation between wheat, background, and weeds was possible using the mapping approach. However, whilst the soil category had a high precision for both fine and coarse filtering, the recall for soil in the case of coarse filtering was quite low. This meant that a lot of wheat and weed images were wrongly classified into the soil category. According to the confusion matrices shown in Table 5 and Table 6, this was particularly the case for VERHE and VIOAR. Nevertheless, with finer filtering, the recall for SOIL strongly increased from 38 to 77%, whilst the rate of misclassifications of VERHE and VIOAR remained almost constant.
In terms of the four different altitudes of the unmanned aerial vehicles (UAV) platform, the overall accuracy increased with choosing a lower altitude for the UAV platform. This gradient was stronger with coarser filtering. However, it is noteworthy that in most cases, it was not the lowest altitude that yielded the best accuracy, but it was the altitude between 2.5 and 4.0 m. Amongst the weed categories, except for PAPRH, the precision increased with finer filtering. MATCH had the highest mapping precision with 87%, whilst the performance of the other weed species was mainly underwhelming, except at the lowest altitude, at which VIOAR and PAPRH could obtain high precision values. However, with increasing altitude, the mapping precision of VIOAR and PAPRH strongly decreased and was far below 50%.
It is rather hard to match the ground truthing location exactly with one pixel, because of its small dimensions in reality. When allowing a small tolerance as a buffer around the prediction location that corresponds to the average plant dimensions, the overall accuracy strongly increased (Table 7) and was greater than 90% for all altitudes of the finer filtering (Table 8), with the highest accuracy at altitudes between 2.5–4.0 m (compare Table 9 and Table 10). This meant that the models predicted well, the right category in the immediate neighborhood of the ground truthing location. This was further supported by the prediction maps given in Figure 7 and Figure S3, for one of the UAS images. It can be seen quite strikingly, that the category soil shines through the plants similar to a background matrix, whereas the wheat plants assembled regular, long stripes within the map because of the seed rows. The different weed species, however, were spatially irregularly located in between the wheat rows. The white markings of the maps signified the ground truthing location of the predicted categories, and the red circles delineated the tolerance buffers allowed around the prediction points. In almost all cases, the correct category lay within the tolerance buffer and showed the high validity of the map.

4. Discussion

The BoVW model proposed in this study was able to differentiate weeds from crops in the cluttered UAS imagery with similar accuracy (>90%), compared to research studies using OBIA, supervised classification, or image recognition, for weed classification. For example, Peña et al. [42] could generate weed–crop maps from UAS imagery using an OBIA approach with an 86% overall accuracy for discriminating three categories of weed coverage in maize crops. Huang et al. [45] were able to differentiate rice from unspecified weeds using a patch based convolutional neural network, with an accuracy of 97%. However, our study focused on differentiating various weed species from winter wheat. Most similar to our study would be the study of Dyrmann et al. [41], when solely focusing on the image set alone. For wheat (Triticum aestivum), MATCH, PAPRH, VERHE, and VIOAR obtained mixed classification accuracies with 92%, 90%, 69%, 45%, and 33%, respectively. In comparison, we achieved similar results with 98%, 66%, 65%, 48%, and 51%, respectively, for the same plant species. However, results cannot be directly compared with each other, because in their study they tested 21 plant species and images, which were recorded from a constant altitude, whereas in this study only 5 plant species were tested, but images were acquired from an unstable UAV platform with an altitude range between 1.0 and 6.0 m.
The use of UAS imagery for model calibration could also have influenced the accuracy of the BoVW model. However, we found it more logical to use a realistic image data set than to rely solely on a data set acquired under laboratory conditions where illumination, shading, image sharpness, and resolution were held constant. It might be interesting to improve the calibration data set with accurate image data acquired under controlled conditions. The altitude of the UAV platform had an influence on the mapping of VIOAR and VERHE, which might be explained by their small plant sizes. Since the BoVW model significantly depends on a unique collection of key point descriptor information, then with increasing altitude this information becomes increasingly homogeneous with other categories. Specifically, due to the high resemblance of VIOAR and VERHE, the distinctive form specific information might get lost in the descriptors with higher altitude, due to lower spatial resolution of the images. Still, VIOAR reached a 79% mapping accuracy when allowing a plant specific tolerance buffer.
It is difficult to match exactly the midpoint of the ground truthing locations because of the small dimensions of one pixel in reality. Allowing a small, plant-specific buffer was more realistic than the pixel-exact testing, because it integrated the classification results in its nearest vicinity and considered the multiple testing during the scanning process whilst mapping. The visual evaluation of the weed maps corroborated this approach. Furthermore, it might be discussed that the model calibration may have been influenced by the rather unbalanced distribution of the annotations. This was partly unavoidable owing to the general abundance of weeds in the UAS imagery. However, we had at least 1000 annotations in each category. Contrary to neural network approaches, we spared data augmentation because the SIFT features used were rotation and translation invariant. We also did not segment the images before presenting them to the model because we encountered in the cluttered UAS images dry old leaves, stones, or soil with different illumination due to micro relief, which might pose a problem for simple segmentation procedures, such as image thresholding. We argue that the implementation of a background category will give better results in the end, especially if more data will be integrated subsequently.

5. Conclusions

With high resolution UAS imagery, we were able to map weeds on a species level within a winter wheat field. For this research, we annotated a large set of images taken from a UAV platform at different altitudes, and calibrated bag of visual words models based on SIFT image features and support vector machine classification. The validation with an independent image set showed a producer’s accuracy of 98%, for differentiating wheat crops from images with weeds and images with soil background. Among the weed species, Matricaria recutita L. showed the highest classification accuracy, whereas for Viola arvensis M. and Veronica hederifolia L., the models were weaker and showed signs of confusing both species with each other. The choice of the hyper-parameters for the BoVW models had only little influence on the outcomes. The weed maps generated by the BoVW models were able to distinguish weeds from crops and single weed species, with an accuracy of nearly 90% or greater. Regarding this kind of mapping, high resolution images with the capability of resolving even subtle details of crop and weed plants are an important prerequisite. Therefore, scanning resolution and altitude had a strong effect on the mapping accuracy of the UAS images and generally the better resolution, the better the map quality. Relating to SSWM, our approach would enable the selection of the right herbicide based on the distribution of the weed species in the classified UAS images. Further research will be directed to improve the generalization capabilities of the models by enhancing the annotation database by implementing more image data from different crop–weed associations and regions.

Supplementary Materials

The following are available online at https://www.mdpi.com/2072-4292/10/10/1530/s1, Table S1: Precision and recall values of all the models tested with an independent image set. KDTREE and VQ were used as quantizer, and SDCA, SVM, and liblinear as solver; Table S2: Precision and recall values of all the models tested on UAS field images using the sliding window approach. Figure S3: Category map of the spatial distribution of weeds, winter wheat, and soil.

Author Contributions

Conceptualization, M.P., M.S., and H.N.; Methodology, M.S. and M.P.; Software, M.P.; Validation, M.P., M.S., and H.N.; Formal Analysis, M.P.; Investigation, M.P.; Resources, H.N.; Data Curation, M.S.; Writing—Original Draft Preparation, M.S., M.P., and H.N.; Writing—Review and Editing, H.N.; Visualization, M.P. and M.S.; Supervision, H.N.; Project Administration, H.N. and M.P.; Funding Acquisition, H.N.

Funding

This research received no external funding.

Acknowledgments

We thank the staff of the JKI for their technical assistance, specifically Annika Behme and Werner Löhr, for their precious assistance in annotating weeds and ground truthing in numerous field trials. We thank Dominik Feistkorn and Arno Littman for conducting the flight missions and we thank Prof. Dr. Peter Zwerger for having made possible our investigations at his institute.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Spitters, C.J.T.; Van Den Bergh, J.P. Competition between crop and weeds: A system approach. In Biology and Ecology of Weeds; Holzner, W., Numata, M., Eds.; Springer Netherlands: Dordrecht, The Netherlands, 1982; pp. 137–148. ISBN 978-90-481-8519-1. [Google Scholar][Green Version]
  2. Zimdahl, R.L. Fundamentals of Weed Science, 3rd ed.; Elsevier/Academic Press: Amsterdam, The Netherlands; Boston, MA, USA, 2007; ISBN 978-0-12-372518-9. [Google Scholar]
  3. Munier-Jolain, N.M.; Guyot, S.H.M.; Colbach, N. A 3D model for light interception in heterogeneous crop:weed canopies: Model structure and evaluation. Ecol. Model. 2013, 250, 101–110. [Google Scholar] [CrossRef]
  4. Hume, L. Yield losses in wheat due to weed communities dominated by green foxtail [Setaria viridis (L.) Beauv.]: A multispecies approach. Can. J. Plant Sci. 1989, 69, 521–529. [Google Scholar] [CrossRef]
  5. Milberg, P.; Hallgren, E. Yield loss due to weeds in cereals and its large-scale variability in Sweden. Field Crops Res. 2004, 86, 199–209. [Google Scholar] [CrossRef]
  6. Fahad, S.; Hussain, S.; Chauhan, B.S.; Saud, S.; Wu, C.; Hassan, S.; Tanveer, M.; Jan, A.; Huang, J. Weed growth and crop yield loss in wheat as influenced by row spacing and weed emergence times. Crop Prot. 2015, 71, 101–108. [Google Scholar] [CrossRef]
  7. Food and Agriculture Organization of the United Nations. FAOSTAT Statistics Database; Food and Agriculture Organization of the United Nations: Rome, Italy, 2013. [Google Scholar]
  8. Kniss, A.R. Long-term trends in the intensity and relative toxicity of herbicide use. Nat. Commun. 2017, 8, 14865. [Google Scholar] [CrossRef] [PubMed][Green Version]
  9. Rendon-von Osten, J.; Dzul-Caamal, R. Glyphosate residues in groundwater, drinking water and urine of subsistence farmers from intensive agriculture localities: A survey in Hopelchén, Campeche, Mexico. Int. J. Environ. Res. Public Health 2017, 14, 595. [Google Scholar] [CrossRef] [PubMed]
  10. Rose, M.T.; Cavagnaro, T.R.; Scanlan, C.A.; Rose, T.J.; Vancov, T.; Kimber, S.; Kennedy, I.R.; Kookana, R.S.; Zwieten, L.V. Impact of herbicides on soil biology and function. Adv. Agron. 2016, 136, 133–220. [Google Scholar]
  11. European Food Safety Authority. The 2015 European Union report on pesticide residues in food. EFSA J. 2017, 15. [Google Scholar] [CrossRef]
  12. Christensen, S.; Heisel, T.; Walter, A.M.; Graglia, E. A decision algorithm for patch spraying. Weed Res. 2003, 43, 276–284. [Google Scholar] [CrossRef]
  13. Christensen, S.; SøGaard, H.T.; Kudsk, P.; NøRremark, M.; Lund, I.; Nadimi, E.S.; JøRgensen, R. Site-specific weed control technologies. Weed Res. 2009, 49, 233–241. [Google Scholar] [CrossRef]
  14. Jensen, P.K. Target precision and biological efficacy of two nozzles used for precision weed control. Precis. Agric. 2015, 16, 705–717. [Google Scholar] [CrossRef]
  15. Lelong, C.C.D.; Burger, P.; Jubelin, G.; Roux, B.; Labbé, S.; Baret, F. Assessment of unmanned aerial vehicles imagery for quantitative monitoring of wheat crop in small plots. Sensors 2008, 8, 3557–3585. [Google Scholar] [CrossRef] [PubMed]
  16. Schmale, D.G.; Dingus, B.R.; Reinholtz, C. Development and application of an autonomous unmanned aerial vehicle for precise aerobiological sampling above agricultural fields. J. Field Robot. 2008, 25, 133–147. [Google Scholar] [CrossRef]
  17. Xiang, H.T.; Tian, L. An automated stand-alone in-field remote sensing system (SIRSS) for in-season crop monitoring. Comput. Electron. Agric. 2011, 78, 1–8. [Google Scholar] [CrossRef]
  18. Lebourgeois, V.; Begue, A.; Labbe, S.; Houles, M.; Martine, J.F. A light-weight multi-spectral aerial imaging system for nitrogen crop monitoring. Precis. Agric. 2012, 13, 525–541. [Google Scholar] [CrossRef]
  19. Rasmussen, J.; Nielsen, J.; Garcia-Ruiz, F.; Christensen, S.; Streibig, J.C. Potential uses of small unmanned aircraft systems (UAS) in weed research. Weed Res. 2013, 53, 242–248. [Google Scholar] [CrossRef]
  20. Pflanz, M.; Feistkorn, D.; Nordmeyer, H. Unkrauterkennung mit Hilfe unbemannter Luftfahrzeuge. Julius-Kühn-Arch. 2014, 443, 396–403. [Google Scholar]
  21. Marceau, D.J.; Hay, G.J. Remote sensing contributions to the scale issue. Can. J. Remote Sens. 1999, 25, 357–366. [Google Scholar] [CrossRef]
  22. Zhang, J.-H.; Ke, W.; Bailey, J.; Ren-Chao, W. Predicting nitrogen status of rice using multispectral data at canopy scale. Pedosphere 2006, 16, 108–117. [Google Scholar] [CrossRef]
  23. Rama Rao, N.; Garg, P.; Ghosh, S.; Dadhwal, V. Estimation of leaf total chlorophyll and nitrogen concentrations using hyperspectral satellite imagery. J. Agric. Sci. 2008, 146, 65–75. [Google Scholar] [CrossRef]
  24. Schirrmann, M.; Giebel, A.; Gleiniger, F.; Pflanz, M.; Lentschke, J.; Dammer, K.-H. Monitoring agronomic parameters of winter wheat crops with low-cost UAV imagery. Remote Sens. 2016, 8, 706. [Google Scholar] [CrossRef]
  25. Pflanz, M.; Nordmeyer, H. Automatisierte Unkrauterkennung auf dem Acker–Möglichkeiten und Grenzen. Julius-Kühn-Arch. 2016, 452, 241–248. [Google Scholar]
  26. Borregaard, T.; Nielsen, H.; Nørgaard, L.; Have, H. Crop–weed discrimination by line imaging spectroscopy. J. Agric. Eng. Res. 2000, 75, 389–400. [Google Scholar] [CrossRef]
  27. Rao, N.R.; Garg, P.; Ghosh, S. Development of an agricultural crops spectral library and classification of crops at cultivar level using hyperspectral data. Precis. Agric. 2007, 8, 173–185. [Google Scholar] [CrossRef]
  28. Vrindts, E.; De Baerdemaeker, J.; Ramon, H. Weed detection using canopy reflection. Precis. Agric. 2002, 3, 63–80. [Google Scholar] [CrossRef]
  29. López-Granados, F.; Peña-Barragán, J.M.; Jurado-Expósito, M.; Francisco-Fernández, M.; Cao, R.; Alonso-Betanzos, A.; Fontenla-Romero, O. Multispectral classification of grass weeds and wheat (Triticum durum) using linear and nonparametric functional discriminant analysis and neural networks: Multispectral classification of grass weeds in wheat. Weed Res. 2008, 48, 28–37. [Google Scholar] [CrossRef]
  30. Lamb, D.W.; Weedon, M.M.; Rew, L.J. Evaluating the accuracy of mapping weeds in seedling crops using airborne digital imaging: Avena spp. in seedling triticale. Weed Res. 1999, 39, 481–492. [Google Scholar] [CrossRef]
  31. Langner, H.R.; Bottger, H.; Schmidt, H. A special vegetation index for the weed detection in sensor based precision agriculture. Environ. Monit. Assess. 2006, 117, 505–518. [Google Scholar] [CrossRef] [PubMed]
  32. Franz, E.; Gebhardt, M.R.; Unklesbay, K.B. Shape description of completely visible and partially occluded leaves for identifying plants in digital image. Trans. ASABE 1991, 34, 673–681. [Google Scholar] [CrossRef]
  33. Woebbecke, D.M.; Meyer, G.E.; Von Bargen, K.; Mortensen, D.A. Shape features for identifying young weeds using image analysis. Trans. ASAE 1995, 38, 271–281. [Google Scholar] [CrossRef]
  34. Zulkifli, Z.; Saad, P.; Mohtar, I.A. Plant leaf identification using moment invariants & general regression neural network. In Proceedings of the 2011 11th International Conference on Hybrid Intelligent Systems (HIS), Melaka, Malaysia, 5–8 December 2011; pp. 430–435. [Google Scholar]
  35. Rumpf, T.; Römer, C.; Weis, M.; Sökefeld, M.; Gerhards, R.; Plümer, L. Sequential support vector machine classification for small-grain weed species discrimination with special regard to Cirsium arvense and Galium aparine. Comput. Electron. Agric. 2012, 80, 89–96. [Google Scholar] [CrossRef]
  36. Persson, M.; Åstrand, B. Classification of crops and weeds extracted by active shape models. Biosyst. Eng. 2008, 100, 484–497. [Google Scholar] [CrossRef]
  37. Pahikkala, T.; Kari, K.; Mattila, H.; Lepistö, A.; Teuhola, J.; Nevalainen, O.S.; Tyystjärvi, E. Classification of plant species from images of overlapping leaves. Comput. Electron. Agric. 2015, 118, 186–192. [Google Scholar] [CrossRef]
  38. Kazmi, W.; Garcia-Ruiz, F.J.; Nielsen, J.; Rasmussen, J.; Jørgen Andersen, H. Detecting creeping thistle in sugar beet fields using vegetation indices. Comput. Electron. Agric. 2015, 112, 10–19. [Google Scholar] [CrossRef][Green Version]
  39. Suh, H.K.; Hofstee, J.W.; IJsselmuiden, J.; van Henten, E.J. Sugar beet and volunteer potato classification using Bag-of-Visual-Words model, Scale-Invariant Feature Transform, or Speeded Up Robust Feature descriptors and crop row information. Biosyst. Eng. 2018, 166, 210–226. [Google Scholar] [CrossRef]
  40. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems; MIT Press: Cambridge, MA, USA, 2012; pp. 1097–1105. [Google Scholar]
  41. Dyrmann, M.; Karstoft, H.; Midtiby, H.S. Plant species classification using deep convolutional neural network. Biosyst. Eng. 2016, 151, 72–80. [Google Scholar] [CrossRef]
  42. Peña, J.M.; Torres-Sánchez, J.; de Castro, A.I.; Kelly, M.; López-Granados, F. Weed mapping in early-season maize fields using object-based analysis of unmanned aerial vehicle (UAV) Images. PLoS ONE 2013, 8, e77151. [Google Scholar] [CrossRef] [PubMed]
  43. Tamouridou, A.A.; Alexandridis, T.K.; Pantazi, X.E.; Lagopodi, A.L.; Kashefi, J.; Moshou, D. Evaluation of UAV imagery for mapping Silybum marianum weed patches. Int. J. Remote Sens. 2016, 1–14. [Google Scholar] [CrossRef]
  44. Hung, C.; Xu, Z.; Sukkarieh, S. Feature learning based approach for weed classification using high resolution aerial images from a digital camera mounted on a UAV. Remote Sens. 2014, 6, 12037–12054. [Google Scholar] [CrossRef]
  45. Huang, H.; Deng, J.; Lan, Y.; Yang, A.; Deng, X.; Zhang, L. A fully convolutional network for weed mapping of unmanned aerial vehicle (UAV) imagery. PLoS ONE 2018, 13, e0196302. [Google Scholar] [CrossRef] [PubMed]
  46. Brisset, P.; Drouin, A.; Gorraz, M.; Huard, P.-S.; Tyler, J. The paparazzi solution. In Proceedings of the 2nd US-European Competition and Workshop on Micro Air Vehicles, MAV, Sandestin, MA, USA, October 2006. [Google Scholar]
  47. Csurka, G.; Dance, C.R.; Fan, L.; Willamowski, J.; Bray, C. Visual Categorization with Bags of Keypoints. In Proceedings of the Workshop on Statistical Learning in Computer Vision, Prague, Czech Republic, 11–14 May 2004. [Google Scholar]
  48. Smith, A.R. Color gamut transform pairs. ACM Siggr. Comput. Graph. 1978, 12, 12–19. [Google Scholar] [CrossRef]
  49. Vedaldi, A.; Fulkerson, B. VLFeat: An open and portable library of computer vision algorithms. In Proceedings of the 18th ACM international conference on Multimedia, Firenze, Italy, 25–29 October 2010; pp. 1469–1472. [Google Scholar]
  50. Lowe, D.G. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 2004, 60, 91–110. [Google Scholar] [CrossRef]
  51. Fan, R.-E.; Chang, K.-W.; Hsieh, C.-J.; Wang, X.-R.; Lin, C.-J. LIBLINEAR: A library for large linear classification. J. Mach. Learn. Res. 2008, 9, 1871–1874. [Google Scholar]
  52. Lampert, C.H.; Blaschko, M.B.; Hofmann, T. Beyond sliding windows: Object localization by efficient subwindow search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA, 23–28 June 2008; pp. 1–8. [Google Scholar]
  53. Forsyth, D.A.; Ponce, J. Computer Vision: A Modern Approach; Prentice Hall Professional Technical Reference: Englewood, NJ, USA, 2002; ISBN 0-13-085198-1. [Google Scholar]
  54. R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2017. [Google Scholar]
  55. Wing, M.K.C.J.; Weston, S.; Williams, A.; Keefer, C.; Engelhardt, A.; Cooper, T.; Mayer, Z.; Kenkel, B.; Team, R.C.; Benesty, M.; et al. Caret: Classification and Regression Training; Astrophysics Source Code Library: Houghton, MI, USA, 2017. [Google Scholar]
Figure 1. Capturing high resolution images at more than 100 waypoints (+) from Nadir. At a flight altitude of 5 m the ground sample distance (GSD) was 0.4 mm with a lens having a focal length of 60 mm and a camera equipped with an APS-C sensor.
Figure 1. Capturing high resolution images at more than 100 waypoints (+) from Nadir. At a flight altitude of 5 m the ground sample distance (GSD) was 0.4 mm with a lens having a focal length of 60 mm and a camera equipped with an APS-C sensor.
Remotesensing 10 01530 g001
Figure 2. Test field of approx. 6.2 ha, unmanned aircraft system (UAS) flight routes (FL), image positions and allocation of images in test and training data sets. Coordinates shown are in UTM zone 32 N, ETRS89 (EPSG code: 5972).
Figure 2. Test field of approx. 6.2 ha, unmanned aircraft system (UAS) flight routes (FL), image positions and allocation of images in test and training data sets. Coordinates shown are in UTM zone 32 N, ETRS89 (EPSG code: 5972).
Remotesensing 10 01530 g002
Figure 3. Examples of sub-images extracted from the UAS imagery showing the different weed species and wheat plant. The average dimension (in pixel) was manually estimated.
Figure 3. Examples of sub-images extracted from the UAS imagery showing the different weed species and wheat plant. The average dimension (in pixel) was manually estimated.
Remotesensing 10 01530 g003
Figure 4. Bag of Visual Words (BoVW) framework: To train an image classifier using the BoVW concept, three steps need to be performed: First, a visual dictionary must be generated from many images. These images may be related to a specific theme, such as unspecific images from plants. From these images, the most relevant information will be extracted using key point extractors and descriptors. Depending on the type of the key point extractor, it finds key points at edges, corners, homogeneous regions, or blobs in the images, and the key point descriptor describes the immediate local neighborhood of the key points invariant to rotation or scale. The key point descriptions are collected over the entire image set and are generalized to a smaller set of code words by finding the centroids with a vector quantization method, such as k-means, to generate a visual dictionary. The second step involves a new labelled set of images. With the same key point descriptor as the visual dictionary that was built, the features of the new image will be extracted and described. The new key point descriptions are then referenced to the code words of the visual dictionary based on its similarity. The frequencies of the key point allocations are recorded over the code words to generate the BoVW vector. In the last step, the labelled BoVW vectors are used to train the image classifier using in our case a support vector machine (SVM). In case of a prediction given a new image, a new BoVW vector must be produced in relation to the codebook, and using the calibrated image classifier, the new BoVW vector will be decided to belong to a specific category, such as a specific weed species.
Figure 4. Bag of Visual Words (BoVW) framework: To train an image classifier using the BoVW concept, three steps need to be performed: First, a visual dictionary must be generated from many images. These images may be related to a specific theme, such as unspecific images from plants. From these images, the most relevant information will be extracted using key point extractors and descriptors. Depending on the type of the key point extractor, it finds key points at edges, corners, homogeneous regions, or blobs in the images, and the key point descriptor describes the immediate local neighborhood of the key points invariant to rotation or scale. The key point descriptions are collected over the entire image set and are generalized to a smaller set of code words by finding the centroids with a vector quantization method, such as k-means, to generate a visual dictionary. The second step involves a new labelled set of images. With the same key point descriptor as the visual dictionary that was built, the features of the new image will be extracted and described. The new key point descriptions are then referenced to the code words of the visual dictionary based on its similarity. The frequencies of the key point allocations are recorded over the code words to generate the BoVW vector. In the last step, the labelled BoVW vectors are used to train the image classifier using in our case a support vector machine (SVM). In case of a prediction given a new image, a new BoVW vector must be produced in relation to the codebook, and using the calibrated image classifier, the new BoVW vector will be decided to belong to a specific category, such as a specific weed species.
Remotesensing 10 01530 g004
Figure 5. (a) Principle of the sliding window approach used; (b) One layer for each category was pre-allocated with the size m × n of the original image.
Figure 5. (a) Principle of the sliding window approach used; (b) One layer for each category was pre-allocated with the size m × n of the original image.
Remotesensing 10 01530 g005
Figure 6. Precision-recall-plots of the different BoVW models shown in Table 4 for the classes (a) MATCH, (b) TRZAW, (c) PAPRH, (d) SOIL, (e) VERHE, and (f) VIOAR.
Figure 6. Precision-recall-plots of the different BoVW models shown in Table 4 for the classes (a) MATCH, (b) TRZAW, (c) PAPRH, (d) SOIL, (e) VERHE, and (f) VIOAR.
Remotesensing 10 01530 g006
Figure 7. Weed classification results of the model 578 shown for the UAS image taken at field position 42. The sliding window has a block size of 50 px and a shift of 5 px. A category map (bottom left) draws the spatial distribution of weeds, winter wheat, and soil. Ground truthing points for MATCH (x), TRZAW (∆), and soil (□) illustrate the accuracies.
Figure 7. Weed classification results of the model 578 shown for the UAS image taken at field position 42. The sliding window has a block size of 50 px and a shift of 5 px. A category map (bottom left) draws the spatial distribution of weeds, winter wheat, and soil. Ground truthing points for MATCH (x), TRZAW (∆), and soil (□) illustrate the accuracies.
Remotesensing 10 01530 g007
Table 1. Summary of categories from weed species and crop plants, their EPPO-Codes, plant development stage, and specific plant dimensions. Category #4 comprises images from soil only and is not shown here.
Table 1. Summary of categories from weed species and crop plants, their EPPO-Codes, plant development stage, and specific plant dimensions. Category #4 comprises images from soil only and is not shown here.
Species#EPPO-CodeBBCHAvg. Radius (px)
Matricaria recutita L. (Wild Chamomille)1MATCH16–1855
Papaver rhoeas L. (Common Poppy)3PAPRH17–1967
Veronica hederifolia L. (Ivyleaf Speedwell)5VERHE2129
Viola arvensis M. (Field Pansy)6VIOAR1531
Triticum aestivum L. (Winter Wheat)2TRZAW2310
Table 2. Allocation of annotated sub-images into train set and test set for wheat, soil, and the four weed species.
Table 2. Allocation of annotated sub-images into train set and test set for wheat, soil, and the four weed species.
nMATCHVERHEPAPRHVIOARTRZAWSOILTotal
Train Set63971728159817012164407717,665
Test Set20868917111107103819547787
84832619230928083202603125,452
Table 3. Permuted repetition of input parameters for generating n = 486 BoVW-models.
Table 3. Permuted repetition of input parameters for generating n = 486 BoVW-models.
InputVariation
Vocabulary size{200…500…1000}
Color space{gray…rgb…hsv}
Number of spatial bins{2…[2 4]…[2 4 8]}
Quantizer{KDTREE…VQ}
SVM.C{1…5…10}
SVM solver{sdca… sgd…liblinear}
Table 4. Precision and recall values of the model testing with an independent image set. K-dimensional binary tree (KDTREE) was used as quantizer and Dual Coordinate Ascent solver (SDCA) as the support vector machines (SVM) solver.
Table 4. Precision and recall values of the model testing with an independent image set. K-dimensional binary tree (KDTREE) was used as quantizer and Dual Coordinate Ascent solver (SDCA) as the support vector machines (SVM) solver.
IDWordsColorPrecision in %Accuracy Overall in %
MATCHPAPRHVERHEVIOARTRZAWSOIL
414200gray67.1647.1244.5631.7195.2879.4364.53
415200rgb66.3545.8544.3334.6094.9979.3864.53
416200hsv63.1461.7447.1449.5997.2197.4472.40
576500gray68.6050.6345.7934.9696.1582.2466.66
577500rgb69.9950.0748.0434.4296.1582.6067.25
578500hsv65.8764.9847.9251.3197.9898.3174.09
7381000gray68.5551.3447.0335.5996.5383.9867.43
7391000rgb71.2848.2447.9235.3296.6383.5767.86
7401000hsv65.4467.5147.5950.6898.6598.5274.21
Recall in %
MATCHPAPRHVERHEVIOARTRZAWSOIL
414200gray85.6444.0237.0742.6078.6269.3864.53
415200rgb87.4342.7835.4941.1879.4571.8764.53
416200hsv88.1557.9237.9458.0380.6685.3472.40
576500gray87.5249.1839.9246.8078.3469.9666.66
577500rgb88.6549.2438.7748.7878.2771.5167.25
578500hsv90.2861.6841.7860.6280.7183.6374.09
7381000gray88.6554.1540.3348.8878.2269.1267.43
7391000rgb88.7251.1240.4449.6877.5170.9167.86
7401000hsv91.1860.7642.2762.7579.7583.0174.21
Table 5. Confusion matrix for the low resolution prediction (200/20) of model 578 on the UAS field scenes. The accuracy overall was 48%.
Table 5. Confusion matrix for the low resolution prediction (200/20) of model 578 on the UAS field scenes. The accuracy overall was 48%.
Actual/Predicted123456TotalRecall
1MATCH9215052113231149250679.06
2PAPRH513098510323571177.44
3VERHE161651724456790128.65
4VIOAR1351849582911025157669.05
5TRZAW92102913102103848.18
6SOIL33481211887195438.01
Total116539917884189549658686
Precision36.7543.465.663.6887.9696.57
Table 6. Confusion matrix for the high resolution prediction (50/5) of model 578 on the UAS field scenes. The accuracy overall was 67%.
Table 6. Confusion matrix for the high resolution prediction (50/5) of model 578 on the UAS field scenes. The accuracy overall was 67%.
Actual/Predicted123456TotalRecall
1MATCH218502734155105250668.47
2PAPRH28189351131128171171.20
3VERHE2521910621114117290134.53
4VIOAR35616122743209130157666.10
5TRZAW171161693652103859.77
6SOIL100017131833195477.24
Total31911253071124156623738686
Precision87.1912.5211.7647.1490.1793.81
Table 7. Confusion matrix for the low resolution prediction (200/20) of model 578 on the UAS field scenes. When allowing a ground truthing tolerance considering plant dimensions, the accuracy overall was 80%.
Table 7. Confusion matrix for the low resolution prediction (200/20) of model 578 on the UAS field scenes. When allowing a ground truthing tolerance considering plant dimensions, the accuracy overall was 80%.
Actual/Predicted123456TotalRecall
1MATCH199818203112355250694.47
2PAPRH1656040438871196.05
3VERHE6270924813490192.08
4VIOAR71126769185524157699.23
5TRZAW205098051103870.86
6SOIL22261151908195462.35
Total2115583770775138330608686
Precision79.7378.7678.6948.7994.4197.65
Table 8. Confusion matrix for the high resolution prediction (50/5) of model 578 on the UAS field scenes. When allowing a ground truthing tolerance considering plant dimensions, the accuracy overall was 95%.
Table 8. Confusion matrix for the high resolution prediction (50/5) of model 578 on the UAS field scenes. When allowing a ground truthing tolerance considering plant dimensions, the accuracy overall was 95%.
Actual/Predicted123456TotalRecall
1MATCH2478020188250692.50
2PAPRH62573823301571198.79
3VERHE142864113790195.47
4VIOAR9452913595336157697.42
5TRZAW102110268103890.48
6SOIL3000141919195496.29
Total26795809051395113419938686
Precision98.8880.5995.8986.2398.8498.21
Table 9. Correlation between precision, recall, altitude (ALT), and window filtering (block size, BS, and shift), where n is the number of ground truthing points from manual annotations of the images.
Table 9. Correlation between precision, recall, altitude (ALT), and window filtering (block size, BS, and shift), where n is the number of ground truthing points from manual annotations of the images.
IDBS [px]Shift [px]ALT [m]nPrecision in %Accuracy Overall in %
MATCHPAPRHVERHEVIOARTRZAWSOIL
578200201.0–2.5117758.5589.080.0024.1493.8497.7773.24
578200202.5–4.0127541.3033.330.003.7789.0699.4655.53
578200204.0–5.5414437.3928.872.880.8389.4693.1440.15
578200205.5–6.0209014.730.009.230.2982.6498.0143.30
5785051.0–2.5117783.2213.790.0087.9382.8886.5973.15
5785052.5–4.0127584.420.001.7978.7793.7594.0780.24
5785054.0–5.5414488.4012.2610.8637.8794.0993.5963.88
5785055.5–6.0209088.600.0015.3829.8686.5098.5665.12
Recall in %
MATCHPAPRHVERHEVIOARTRZAWSOIL
578200201.0–2.5117788.1281.150.0082.3574.8663.9973.24
578200202.5–4.0127592.4410.000.0057.1464.5345.7855.53
578200204.0–5.5414473.3177.2710.0046.6737.2228.4940.15
578200205.5–6.0209078.48NA54.5525.0050.2038.2943.30
5785051.0–2.5117773.5592.310.0067.4060.8084.4773.15
5785052.5–4.0127587.370.0016.6769.2969.7790.4180.24
5785054.0–5.5414465.5776.4722.5266.6753.9072.4363.88
5785055.5–6.0209062.900.0053.8558.5262.5672.5165.12
Table 10. Correlation between precision, recall, altitude (ALT), and window filtering (block size, BS, and shift), with buffered accuracies (tolerance threshold was chosen according to the average plant size of each species).
Table 10. Correlation between precision, recall, altitude (ALT), and window filtering (block size, BS, and shift), with buffered accuracies (tolerance threshold was chosen according to the average plant size of each species).
IDBS [px]Shift [px]ALT [m]nPrecisionAccuracy Overall in %
MATCHPAPRHVERHEVIOARTRZAWSOIL
578200201.0–2.5117792.1199.4385.71100.0098.63100.0097.45
578200202.5–4.0127591.69100.0083.0466.9895.3199.4689.65
578200204.0–5.5414477.4471.8967.7335.1594.3494.9371.79
578200205.5–6.0209067.4675.0084.8445.2291.9698.1979.38
5785051.0–2.5117798.6887.3695.24100.0099.3297.2196.77
5785052.5–4.01275100.00100.0099.1195.2898.4499.4698.75
5785054.0–5.5414498.4278.1193.6184.1499.2397.3292.45
5785055.5–6.0209099.52100.0096.7078.8498.3999.1095.22
Recall
MATCHPAPRHVERHEVIOARTRZAWSOIL
578200201.0–2.51177100.0098.3094.74100.0097.9693.9697.45
578200202.5–4.0127598.06100.0098.9498.6186.7379.7089.65
578200204.0–5.5414491.6995.0185.4898.6756.6446.6071.79
578200205.5–6.0209095.95100.0094.38100.0075.8664.0879.38
5785051.0–2.5117796.15100.0086.9695.6094.1698.3196.77
5785052.5–4.0127599.2375.0098.2399.0296.9299.4698.75
5785054.0–5.5414490.0499.0493.0296.7384.1094.5092.45
5785055.5–6.0209092.4966.6796.9299.2793.8795.1595.22

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Back to TopTop