RGB Image-Derived Indicators for Spatial Assessment of the Impact of Broadleaf Weeds on Wheat Biomass

: In precision agriculture, the development of proximal imaging systems embedded in autonomousvehiclesallowstoexplorenewweedmanagementstrategiesforsite-specificplantapplication. Accurate monitoring of weeds while controlling wheat growth requires indirect measurements of leaf area index (LAI) and above-ground dry matter biomass (BM) at early growth stages. This article explores the potential of RGB images to assess crop-weed competition in a wheat ( Triticum aestivum L.) crop by generating two new indicators, the weed pressure (WP) and the local wheat biomass production ( δ BMc). The fractional vegetation cover (FVC) of the crop and the weeds was automatically determined from the images with a SVM-RBF classiﬁer, using bag of visual word vectors as inputs. It is based on a new vegetation index called MetaIndex, deﬁned as a vote of six indices widely used in the literature. Beyond a simple map of weed infestation, the map of WP describes the crop-weed competition. The map of δ BMc, meanwhile, evaluates the local wheat above-ground biomass production and informs us about a potential stress. It is generated from the wheat FVC because it is highly correlated with LAI (r 2 = 0.99) and BM (r 2 = 0.93) obtained by destructive methods. By combining these two indicators, we aim at determining whether the origin of the wheat stress is due to weeds or not. This approach opens up new perspectives for the monitoring of weeds and the monitoring of their competition during crop growth with non-destructive and proximal sensing technologies in the early stages of development.


Introduction
The emergence of proximal sensing technologies in precision agriculture provides new opportunities to drastically reduce chemical herbicides from site-specific weed management (SSWM) while maintaining production yield, quality, and commercial value [1][2][3]. Although weed management in field crops is a long story [4,5], it is still topical [6][7][8]. The development of affordable and easy to use unmanned aerial vehicles (UAV) allows accurate weed monitoring thanks to high resolution images [9][10][11]. Imaging systems are mostly based on multi or hyperspectral optical sensors. They require complex image processing algorithms to discriminate between crops and weeds and to generate weed maps [11][12][13]. The first step consists of extracting vegetation pixels in the image. The remote sensing community developed spectral indices to quantify greenness in a multispectral image [14][15][16]. The segmentation is generally performed using vegetation indices built as a combination of spectral bands. Their choice depends on the application: for RGB images, the most common is the excess green The destructive measurements served as a reference. The aerial parts of the wheat and the weeds were collected separately and were packed into paper bags. We measure the leaf area index (LAI expressed in m 2 .m −2 ) using a planimeter. Then, for each sampling date, the above-ground dry matter biomass (BM, g.m −2 ) was determined by weighing plants, after being oven dried at 80 • C for 48 h.
In parallel, we built a movable sensing platform made of PVC pipes, on which a Canon EOS 450D (Canon Inc., Tokyo, Japan) commercial digital camera was mounted at a height of 1 m (Figure 1b). Capturing vertical images allows comparing the results with LAI at early growth stages. The main parameters of the camera are described in Table A1 (Appendix A). The camera shutter was controlled remotely. Each experiment was conducted in clear weather under stable light conditions. The exposure time and shutter speed were optimized for the light conditions. The RGB images have a spatial resolution of 0.2 mm/pixel. Images were acquired before sampling the plants. For each measure, two images are shot: one before plant sampling and containing both the wheat and weed populations, and another after the wheat plants were removed, therefore, containing only weeds. Outside the sampling and measurement areas, six images of wheat were taken this way. These images labelled "weeds" and "wheat" will be used, after being transformed into thousands of thumbnail images to feed the classifier during the training phase. In total, 18 images were recorded at several plant growth stages listed in Table 1.
On the first date only, the whole surface of the plot was imaged. As for a UAV platform, we built an orthomosaic photo of the entire plot with a~60% overlap between successive images and a~40% overlap between passes. We used Image Composite Editor (Version 2.0.3.0, 2015, Microsoft Corporation, Redmond, WA, USA), an image stitcher software, to create a panoramic image.  Figure 2 illustrates the flowchart of the methodology used to generate output maps to assess crop-weed competition. After the data collection (step A), the image processing (step B) determines the fractional vegetation cover (FVC) of both the wheat (FVCc) and the weeds (FVCw). It is defined as the ratio of the number of pixels of vegetation (crop vs. weeds) to the total number of pixels in the image. Then, a correlation is established between FVCc and BMc to map wheat biomass. In step C, information about the effect of weed on wheat crop is provided by two non-destructive image-derived indicators: the weed pressure (WP) and the wheat biomass production, δBMc. The image processing (step B) pipeline can be summarized in four steps: (1) segmentation of the initial RGB image to create a vegetation image from a MetaIndex; (2) feature extraction from bag of visual word (BoVW) descriptors using superpixels of vegetation; and (3) supervised classification from support vector machine (SVM) to discriminate crop from weeds.
All the image pre-processing and processing algorithms were implemented in Matlab (Version 2016b, The Mathworks, Natick, MA, USA).
Greenness identification from MetaIndex (step B-1). We propose the new vegetation index called MetaIndex, which combines the advantages of six vegetation indices (Table 2) commonly used in the literature and recently reported in Baniaich et al. [45], Meyer et al. [16], Guo et al. [46], and Yang et al. [47]. It consists in assigning a pixel to a class (vegetation or other) by a majority vote performed on four indices over the six listed in Table 2. This method is completed by a geodesic segmentation to refine the results and obtain a B&W vegetation image also called B&W vegetation mask (white pixel for the vegetation, black for the background). Feature extraction (step B-2). The input data of the two-class SVM-RBF classifier is not the RGB image but a vector of main features specific to each class (crop vs. weed). Extraction of these features is quite complex and requires several image-processing steps. To reduce computation time, superpixels (128 × 128 px) are then created with the SLIC (Simple Linear Iterative Clustering) algorithm commonly used in the literature [26]. Combining a RGB image with its vegetation mask, only superpixels of vegetation are kept in the thumbnail images. We used the labelled ones to create the crop and weed training dataset ( Figure 3). The SURF (speeded-up robust features) descriptor algorithm [51] allows extracting features (~1000) from these images. Thousands of variables are then extracted in each stand to construct a 500-dimensional BoVW vector, containing the most influential features (~500) owing the Bag of Visual World method [27,52,53]. This approach is very popular in agriculture to discriminate between crop and weeds [8,27,51]. With the labelled image database, the BoVW vector of each thumbnail is associated to a class, wheat or weeds. From this technique, we increase easily the number of labelled images of the training dataset.
Supervised classification and evaluation (step B-3). The classifier learns how to distinguish the crop from the weed flora (training phase). It is based on a support vector machine (SVM) algorithm [54] with a radial basis function (Gaussian type). Among all supervised techniques used for crop/weed discrimination, SVM-RBF is one of the most powerful classification algorithms based on machine learning [8,11,21,23,25,55].
The learning dataset is made of 7860 thumbnails labelled into two classes: crop and weed. The classifier algorithm is calibrated from the learning set divided into a training set and a validation set. Labelled thumbnail images were randomly selected from the learning set with 85% for training and 15% for validation. Once the classification algorithm is validated, we test it with a new RGB image. Then a new BoVW vector is produced in relation to the codebook, and using the calibrated image classifier, the new BoVW vector is labelled as crop or weed. Finally we can build crop and weed maps, and calculate the respective fractional vegetation cover (FVC), FVCc and FVCw ( Figure 2). The calculation of specific statistical metrics derived from the confusion matrix [56] allows assessing the SVM-learning algorithm. Three metrics were computed for performance evaluation: Recall (Equation (1)) reflects the ability to reveal the needed information; Precision (Equation (2)) indicates the correctness of the detected results; F-score (Equation (3)) indicates the balance between Precision and Recall.
with TP being the number of true positives, FP and FN are the number of false positives and false negatives, respectively [8,56].

Two Non-Destructive Indicators for a Crop-Weed Competition
Accurate monitoring of weeds while controlling wheat growth requires indirect measurements of leaf area index (LAI) and dry matter biomass (BM) at early wheat growth stage. Therefore, it is necessary to measure LAI and BM destructively to correlate them to FVCc. We developed two indicators deduced from the image. The weed pressure (WP) characterizes the relationship between crop and weed. For their growth, plants are in direct competition for water, nutrients and for light, main parameter at the end of winter. However, this indicator alone is not sufficient to conclude on the negative weed impact on wheat crop. The second indicator is a local above-ground biomass production, δBMc. It provides information about stress but not about its cause. To check whether the stress originates from weeds or not, it has to be compared with the WP.

•
The weed pressure (WP) is expressed as a percentage and defined as with FVCw, the fractional weed vegetation cover, and FVCc, the fractional wheat vegetation cover. It provides information on the resource competition between the crop and the weeds: light, nutrients, and nitrogen. WP can be viewed as a substitute to the BMw/BMc ratio, which results from a destructive approach until the tillering stage by late winter.
• Evaluation of the local wheat above-ground biomass production: δBMc The objective of this second indicator is to observe locally the wheat biomass (δBM obs ) and to compare it to a reference value (BM re f ) from the image parameter, FVCc. It is defined as with BM re f being a reference value of above-ground wheat biomass considered as the average value of wheat above-ground biomass observed in the entire plot and BM obs the observed value of BMc. This new indicator assessing the crop health is therefore deduced from FVCc at each location in the plot. It can take three values: 1/ δBMc < 0 may indicate an excess of biomass. 2/ δBMc = 0 indicates no health problem. 3/ δBMc > 0 may indicate a stress (i.e., pests, weeds or diseases) in wheat growth.

Crop/Weed Map from SVM Classifier and Classification Performance
The input data of this supervised classification are the BoVW vectors that contain 500 features for each class. These vectors are built during the training phase with labelled thumbnail images automatically assigned as crop or weeds ( Figure 3). The classification accuracy is quantified using a classical metric deduced from the confusion matrix. At the end, more than 7682-labelled thumbnails have been obtained, 3841 for the 'Wheat' class and 3841 for the 'Weeds' class (Table 3). Then the labelled dataset was randomly split into 85% training dataset and 15% testing dataset. The SVM-RBF classifier is assessed from the confusion matrix (Table 4). The overall classification accuracy was 93%, the Recall (also known as sensitivity) 94%, and the Precision (also known as selectivity) 92%. The F-score helps to measure Recall and Precision at the same time, and its value of 93%, confirms that our classifier performed globally well in the crop/weed discrimination. Moreover, the high value of the Kappa coefficient (κ = 0.86) [57,58] also indicates that the SVM-RBF classifier correctly classified most of the objects. It encourages us to substitute image-derived parameters (FVC) for destructive measurements (LAI and BM).  Figure 4 presents the results of image stitching over the whole plot (18 m 2 ) on date 1. The first map ( Figure 4a) is a mosaic of 254 RGB images acquired in the field (Table 1). Then a black and white map of vegetation is obtained using the MetaIndex. Depending of the pixel position in the image, the method selects certain indices with a majority vote according to the illumination of the objects, which increases the robustness and sensitivity of this segmentation ( Figure 5). At the end of the vote, a thresholding is carried out to provide a binary map of vegetation hereafter called the MetaIndex map ( Figure 4b). The third map (Figure 4c) is generated from the classification results using the SVM supervised learning classifier, which discriminates between the wheat and the weeds. In this plot, the weed infestation rate is 7.5%. Finally, the WP map characteristic of the crop-weed competition is produced. It is obtained by a simple linear interpolation of the values at neighboring grid points is performed. This grid divides the plot into three subplots composed of 84, 84, and 86 rows respectively. A color map with three colors illustrates the in-field differences associated with high, medium, and low WP levels (Figure 4d). On date 1, WP ranges from 3.37% to 20.63%. Figure 4c shows that low pressure corresponds to 73.5% of the data ( Figure A1). A few high intensity spots are observed, especially near the edge of the plot immediately adjacent to headlands (bottom, top and left side), except on the right side of the plot. The median equals 7.5% and the standard deviation 3%. Figure A1 displays outliers that may originate from contamination by the unploughed headlands.  At this point, it is difficult to quantify the effect of WP on wheat health. One needs to interpret it with regard to the weed species diversity, abundance, and growth stage that may disturb crop growth [59,60]. The literature also shows that the early growth stages are also crucial in determining the intensity and outcome of subsequent crop-weed competition. In this experience, a high diversity of weed communities is observed at seedling stage of wheat (see Section 2.1). However, the dicot species observed are unusual but, according to the literature, they are little or moderately harmful to winter wheat [61][62][63]. Therefore, the WP values (Table A2 in

A Non-Destructive Indicator of Wheat Crop Growth: δBMc
A non-destructive indicator of the local wheat crop growth (δBMc) is produced based on indirect measurements of above-ground BM. Figure 6a presents the relation between destructive measurements of LAI and BM with FVCc. The prediction accuracy using a linear regression model is high. At the early stage of wheat growth, BM = 176.86 × FVCc with r 2 = 0.93 and LAI = 1.06 × FVCc with r 2 =0.99. Our results obtained with the cultivar Apache are consistent with those obtained by Jeuffroy and Recous on the cultivar Soissons [64]. In their case, LAI was calculated daily from the total above-ground biomass finding that the ratio of leaf area to plant biomass (LA/BM) is constant (6.10 −3 m 2 .g −1 ) only for the beginning of the growth cycle and until LAI reached the value of four. Our value of 0.006 (LAI/BM = 1.06/176.86) is consistent with this study and with others [65][66][67][68]. We demonstrate the relevance of the machine-learning algorithm (SVM-RBF classifier) to estimate the fractional wheat vegetation cover (FVCc) and the use of visible images to estimate the LAI and BM at early growth stages of wheat.
Concerning weed stand, the situation is different. The variability between the three replicates (Q1, Q2, and Q3) at each date is high for all the variables (high standard deviation in Figure 6b). These results reveal the strong spatial heterogeneity of the weeds, even in a micro-plot. FVCw is positively correlated with BM (r 2 = 0.93). However, the linear correlation with LAI is fair (r 2 = 0.44). The underestimation of LAI may be caused by the destructive approach (planimeter resolution, quantification errors) that is not adapted to small plants.   (Figure 7c): the lower values (δBMc < 0) that represent 25% of the pixels indicate that the crop biomass is higher than the reference value (BM re f ), the intermediate values (δBMc~0) that represent 61% of the pixels correspond to a normal crop growth. Finally, the highest values (δBMc > 0) that represent 14% of the pixels indicate a problem of crop growth probably caused by stress condition (i.e., weed, disease, pest, . . . ). However, with this map deduced from visible images, we clearly identify the local problem of wheat growth but it is not possible to conclude about the origin of the stress. Combining the two δBMc with WP, it allows understanding the role played by weeds in the decrease of wheat growth.
the intermediate values (δBMc~0) that represent 61% of the pixels correspond to a normal crop growth. Finally, the highest values (δBMc > 0) that represent 14% of the pixels indicate a problem of crop growth probably caused by stress condition (i.e., weed, disease, pest, ...). However, with this map deduced from visible images, we clearly identify the local problem of wheat growth but it is not possible to conclude about the origin of the stress. Combining the two δBMc with WP, it allows understanding the role played by weeds in the decrease of wheat growth.

Comparison of δBMc and WP Maps
The two maps are compared on date 1 (Figure 8). The overlay is presented to help interpret the causes of a wheat stress related to weeds. One can note that the crop grow is generally good (δBMc < 0 or close to zero) and WP is low to medium. This indicates an unstressed field crop. Under stress (δBMc > 0, red spots in Figure 8a), we observe various situations depending on the location. We divided the plot into three regions associated to the three sampling dates. In the upper region, high δBMc values are located where WP is high or medium. For high WP values, that seems to indicate that the weeds are the main source of stress and that they compete with the crop. Nevertheless, for

Comparison of δBMc and WP Maps
The two maps are compared on date 1 (Figure 8). The overlay is presented to help interpret the causes of a wheat stress related to weeds. One can note that the crop grow is generally good (δBMc < 0 or close to zero) and WP is low to medium. This indicates an unstressed field crop. Under stress (δBMc > 0, red spots in Figure 8a), we observe various situations depending on the location. We divided the plot into three regions associated to the three sampling dates. In the upper region, high δBMc values are located where WP is high or medium. For high WP values, that seems to indicate that the weeds are the main source of stress and that they compete with the crop. Nevertheless, for medium WP values, it is very likely that the crop stress originates from a combination of several stresses. In the central region, δBMc is high while WP is medium. Furthermore, when WP is low or medium, wheat growth does not seem to be affected. This is consistent with the literature [69] that mentions that some weed species observed on date 2 have low to moderate negative impact on crops [70]. Therefore, the critical problems on wheat growth in this region are due to weeds but also to other stressors, which have been not characterized. The lower region is similar to the upper region. The only weed observed on date 1 is Silphium perfoliatum L. (Asteraceae), a tall perennial plant that can negatively affect wheat growth depending on its size [71].
To sum up, the combination of the results of the two non-destructive indicators (WP and δBMc), allows evaluating the crop-weed competition at specific date and determining when weeds have a dominant effect compared to other stressors. To go further, it would be necessary to look at the overall evolution of the field on different dates. A more effective method will be to explore in the near future using a UAV.

Relationship between Wheat Stress and Weed Pressure
The combination of the δBMc and WP maps provides useful information for crop and weed management. Visible images can be used to monitor weed competition during crop growth with non-destructive and proximal sensing technologies in the early growth stages. However, proposing weed decision rules that address the evaluation of a crop agronomic risk remains a challenge at this stage of work. A more detailed analysis about weed species is required. These results must be treated with caution and experiments need to be carried out on a larger scale, looking at the yield loss depending on weed species. These experiments can be related to the ecophysiological model that predicts the wheat growth (e.g., AZODYN, [64]) and calculates the dry biomass of the aerial plant organs at a daily time-step during the vegetative phase under no stress conditions. Updates can then be made regularly during wheat growth simulation with these remote or proximal sensing data to optimize the site-specific weed management. To go further in long-term agro-ecological weed management, different combinations of cropping techniques should be explored and their long-term effects be assessed. One solution is the modeling approach. FlorSys is to date the only model that quantifies cropping system effects in interaction with pedoclimate on a multi-specific weed flora. It is a mechanistic "virtual field" model simulating daily weed and crop growth and reproduction over the years, on which arable cropping systems can be experimented in temperate climates [72]. However, like any model, it requires experimental data that will serve either as input variables or to validate the Remote Sens. 2020, 12, 2982 13 of 19 model predictions. Thus, our field trials will not only help the farmer in the daily management of his plots, but also the modelers.

Temporal Evolution of Weed Harmfulness
Up to now, weed competition in crop field has been addressed through weed density (plants/m 2 ) considered as one of the most important factors [59,71]. However, in the 1990s, some authors [73] suggested to study other relevant variables based on the contribution of weed species to the total leaf area index to describe the competition between crop and weeds. They were named the relative leaf area of the weed, the fractional vertical cover, or the weed coverage [73][74][75][76]. They can be deduced from imagery [37,38]. To investigate the substitution of destructive measurements to image-derived parameter, Figure 9 compares WP to the BMw/BMc ratio for accurate weed and crop monitoring. As far as the biomass [36,77] is concerned, BMw/BMc ratio is one of the closest indicators to the concept of direct primary harmfulness [78][79][80]. In this figure, the time evolution of these two ratios is compared over the three dates. Therefore, on each date, the BMw/BMc ratio behaves like WP on average and both ratios decrease over time. However, some differences are locally observed, especially for the quadrats Q 3 and Q 6 where high BMw/BMc ratio (and particularly high BMw value) is observed while WP is low (Table A2,

Conclusions
The development of proximal sensing techniques allows exploring new strategies of weed management for sustainable agriculture practices. High-resolution imaging systems help to discriminate between crop and weeds by generating weed cover maps for a site-specific herbicide application. The next challenge of precision farming is to move towards the use of no herbicides in agriculture, which requires a better understanding of the crop-weed competition.
This article focused on the implementation of automatic weed detection using RGB images in order to generate maps of two indicators, the weed pressure and the wheat biomass production. Thanks to the performance of the SVM-RBF classification, using a bag of visual word vectors as inputs, the fractional vegetation cover (FVC) of both plants was determined. Beyond a simple location map, the weed pressure map described the competition between the crop and the weeds. Concerning wheat, the fractional vegetation cover (FVC) deduced from visible images provided a reliable proxy for LAI and BM measurements We also generated a map of δBMc that estimates the local wheat above-ground biomass production, informing about a possible stress. The combination of these two indicators shows that wheat stress is not always correlated to a high weed pressure. Although these results were obtained on a small plot, they are very promising. They provide a useful basis for accurate weed monitoring but they need to be confirmed in agricultural fields using UGV or UAV platforms, for example. In the future, we will develop a decision support tool for the monitoring of weeds while controlling wheat growth from indirect measurements of LAI and BM at early growth stages. Acknowledgments: Many thanks to the technicians, Vincent Durey and Annick Matéjicek, who were involved in this project, one on PAR sensor control and the other for plant identification.

Conflicts of Interest:
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Abbreviations Abbreviation
Description RGB image Red Green Blue image: visible image.

LAI
The Leaf Area Index (LAI expressed in m 2 .m −2 ) is defined as the total area of the upper surfaces of the leaves contained in a volume above a square metre of soil area. It is determined destructively using a planimeter. It is a key variable used for physiological and functional plant models and by remote sensing models at large scale δBMc indicator It is defined as the difference between BM re f , the mean value of wheat above-ground biomass in the field, and BM obs . It is an evaluation of the local wheat above-ground biomass production. A local excess of wheat above-ground biomass is observed when BM obs > BM re f whereas a stress is observed when BM obs < BM re f

SVM-RBF classifier
Support Vector Machine with a Radial Basis Function kernel. It allows classifying data that is not at all linearly separable. A two-class classification (crop and weeds) is used and classifier input data are the BoVW vectors containing the main features for each observable (i.e., crop, weed).

SLIC algorithm
Simple Linear Iterative Clustering algorithm. It is a fast and robust algorithm to segment image by clustering pixels based on their color similarity and proximity in the image. Thus, it generates superpixels that are more meaningful and easier to analyze. In our study, the superpixels of vegetation (128px x 128px) are then called 'thumbnails' and used to create the training data set (5000 labelled thumbnails per class). From this technique, we increase the number of labelled images of training set.