Assessment of Weed Classification Using Hyperspectral Reflectance and Optimal Multispectral UAV Imagery

Weeds compete with crops and are hard to differentiate and identify due to their similarities in color, shape, and size. In this study, the weed species present in sorghum (sorghum bicolor (L.) Moench) fields, such as amaranth (Amaranthus macrocarpus), pigweed (Portulaca oleracea), mallow weed (Malva sp.), nutgrass (Cyperus rotundus), liver seed grass (Urochoa panicoides), and Bellive (Ipomea plebeian), were discriminated using hyperspectral data and were detected and analyzed using multispectral images. Discriminant analysis (DA) was used to identify the most significant spectral bands in order to discriminate weeds from sorghum using hyperspectral data. The results demonstrated good separation accuracy for Amaranthus macrocarpus, Urochoa panicoides, Malva sp., Cyperus rotundus, and Sorghum bicolor (L.) Moench at 440, 560, 680, 710, 720, and 850 nm. Later, the multispectral images of these six bands were collected to detect weeds in the sorghum crop fields using object-based image analysis (OBIA). The results showed that the differences between sorghum and weed species were detectable using the six selected bands, with data collected using an unmanned aerial vehicle. Here, the highest spatial resolution had the highest accuracy for weed detection. It was concluded that each weed was successfully discriminated using hyperspectral data and was detectable using multispectral data with higher spatial resolution.


Introduction
Weeds are unwanted plants that can damage and reduce crop yields, becoming worse over time if no action is taken to control the growth rate of the weeds. The competition for moisture, nutrients, and sunlight between the weeds and crops affects crop yields, which can be reduced by 45-95% [1][2][3]. The use of chemicals is widespread because of their effectiveness.; however, because of the diversity of weed and due to herbicide resistance [4], this is not a long-term solution. The excessive use of herbicides harms crops and increases the end product price [3], in addition to jeopardizing environmental protection efforts [4] and leading to soil contamination [5], unless natural herbicides made of organic acids or essential oils are employed to mitigate these negative impacts; however, organic herbicides can be expensive.
As such, spot spraying can reduce the amounts of herbicides to which the crops are exposed. This method is used to prevent weeds from affecting the crops [6]. This is better than blanket application using the site-specific weed management (SSWM) approach, in which the herbicide is sprayed on only a few spots in the field according to the patch area at an exact location and exact time [7]. This method is used to identify the locations of the weed patches and the weed species [8].
The SSWM method uses statistical analysis to differentiate weeds from crops [9]. With the SSWM method, information is collected using remote sensing, whereby the spectral This study was conducted at the Gatton Campus, Queensland (27 • [14]. The experimental design used in this study involved a 6 × 4 randomized block design. The plots of those treated with control (untreated) and pre-emergence herbicide were separated. Two pre-emergence herbicides were used, namely Gesaprim ® 600 SC at a rate of 2 L/ha (with 600 g/L atrazine active constituent) and DualGold ® at a rate of 2 L/ha (with 960 g/L S-metolachlor active constituent).
Sorghum species were placed in each plot. In this study, one type of sorghum species was placed in each quadrat. The four replicates were placed randomly. In this condition, we focused on the common weed species and those highlighted in Table 1.

Spectral Separability Procedures
The hyperspectral data in the range of 325-1075 nm were collected using the ASD FieldSpec ® HandHeld 2 spectroradiometer (Malvern Panalytival, Cambridge, United Kingdom (U.K)). This instrument is a minimal, convenient, visible near-infrared (VNIR) alternative to the ASD FieldSpec 4 [17]. The ASD FieldSpec ® HandHeld 2 spectroradiometer is a robust and sturdy device that uses the ASD FieldSpec 4 VNIR spectrometer for efficient measurement [17]. The data for each weed species and sorghum crop were collected in 2013 and 2014. Data collection started from week two and continued until week four at different growth stages of the weeds and crops. The optical panel was calibrated using white references. Prior to measurement, the ASD FieldSpec ® HandHeld 2 spectroradiometer was calibrated with a Spectralon white reference panel provided by the manufacturer (Malvern Panalytival, Cambridge, United Kingdom (U.K))., as indicated in the operation guide [18]. ASD FieldSpec ® HandHeld 2 instrument was aimed so that the optical input was shown but did not darken the white reference panel, while being close enough for the panel to fill the field of view. It was important to conduct a calibration process before and after hyperspectral data were collected for optimization and dark current collection. Each spectral signature of the leaves was collected. Then, the data were transferred to a computer and saved in MasterFile format. The MasterFile data were binned and run using the first derivative (FD) analysis.

Classification Procedures
The data were split into calibration data (group 1, 70%) and validation data (group 2, 30%) ( Figure 2) [14]. The calibration and validation data were randomly split into two groups. Stepwise linear discriminant analysis (SLDA) was run to identify 20 significant bands in the calibration data via the STEPDISC procedure using SAS (SAS Institute, North Carolina State University, United States of America (USA)) software. The linear discriminant analysis was constructed using the "rule set" from the 20 significant bands. Then, the validation data were run with the same "rule set" to verify the accuracy.

Band Identification
The identified bands were combined into six bands because the multispectral sensor MCA 6 (Tetracam Inc., California (USA)) can only be fitted with six bands; therefore, in this paper, only six bands are shown. The best six-band combinations were chosen from the eight bands (from the currently available MCA 6 (Tetracam Inc., California (USA)), with another six bands from the 20 significant bands ( Table 2). The MCA 6 multispectral camera was used to collect the aerial images in the sorghum field using UAV. For the MCA 6 camera, only six bands were used during the flight (because the sensor could only be fitted with six bands). The combination of the eight bands was used because produced more combinations; thus, to find the best six bands, the band combinations were run using discriminant analysis to obtain the highest classification accuracy. Almost all spectral regions resembled those from MCA 6 and the six bands from the 20 significant bands (using linear discriminant analysis (LDA)); however, for the red-edge region, MCA 6 highlighted bands at 750, 730, 720, and 710 nm, while LDA only highlighted bands at 720 and 710 nm.

Multispectral Imagery Data Collection
The MikroKopter JR11X UAV and multiple camera array 6 (MCA 6) sensor (Tetracam Inc., California (USA)) were used to collect the aerial images at week 3 (17 December 2014). The MCA 6 camera has six sensors that can be fitted in the UAV. The six bands were chosen based on the results in Section 2.3. The multispectral data were collected for different spatial resolutions, which were 5.43 mm (flight height: 10 m), 10.83 mm (flight height: 20 m), and 20.31 mm (flight height: 37.5 m). The RGB imagery was collected at a height of 1.6 m (0.87 mm spatial resolution). The field of view (FOV) for MCA 6 was 38.3 • × 31.0 • . The image size was 1280 × 1024. The advantages of MCA 6 are that it can provide 8 or 10 bit data and its weight of 790 g enables it to be attached to UAVs.

Object Based Image Analysis (OBIA) Procedures
The images were segmented and classified using the nearest neighbor (NN) method with vegetation, soil, and shadow samples. The OBIA workflow is shown in Figure 3. This shows the hierarchy level for the whole image, which is then narrowed down to the vegetation, before being dividing it into two classes in level 3. In level 4, there are four types of weed.
The image segmentation started at the scale of 40, using the "multiresolution segmentation" algorithm, based on the color (spectral) and shape or the spatial criterion ( Figure 4). In this study, the spectral and shape values were weighted based on their homogeneity. The shape value was 0.1 because the overall shape of the weeds and sorghum was only ten percent of the total, which led to inaccurate segmentation. The color value (w colour ) was 0.5 because the spectral bands of the weeds and sorghum were similar, with only slight differences. The compactness values indicated different shapes and smoothness weights, showing variability in terms of features. The parameters values were flexible because they were based on trial and error.  The objects were classified according to soil (SO) and vegetation, then the vegetation was segmented on a scale of 1 to 10. In fact, the vegetation was divided into sorghum and weeds. Then, both sorghum and weeds were reclassified. Using the "enclosed by class" function, sorghum was merged while weeds were split into different classes. This rule set can be slightly modified to run other images and data.
The confusion matrix was used to assess the accuracy before its adoption as a normal procedure [19][20][21][22]. The KHAT statistic was computed based on [23] as: where r is the number of rows in the matrix; x ii is the number of observations of row i and column i; x i+ and x +i are the marginal total of row i and column i, respectively; N is the total number of observations.

Classification of Weeds Species
Weeds can be discriminated using stepwise linear discriminant analysis (SLDA), and the classification of the validation data in 2013 was 85-90% accurate, as illustrated in Table 3.
Stepwise linear discriminant analysis was used in lettuce varieties, showing a classification accuracy of 81.04% [15]. These results can be improved using high-dimensional analysis methods, such as a support vector machine (SVM) [12,24]. Table 3. Results for the classification of 20 significant bands in 2013. Adapted from [14].

Species
Calibration  Table 4 shows that the validation data in 2014 were 97-100% accurate. This means that the accuracy increased from the previous year and the results shown were consistent.

Classification and Validation
The significant bands for the classification of weeds and sorghum were successfully analyzed using linear discriminant analysis (LDA). In 2013, the significant bands were tested for their classification accuracy. The accuracy of Malva sp. (MW) decreased to 90%, while sorghum reached more than 90% accuracy in week two (Table 3, calibration section). This shows that the classification accuracy was high (>80%). Meanwhile, the sorghum accuracy increased to 92% in week three, then finally all weeds showed 100% classification accuracy during week four. Here, the larger the size of the weeds, the more accurate the classification.
The weed classification was tested using independent data (Table 3, validation section). In week two, the classification accuracy for sorghum increased to 70%, reaching 75% in week three; however, it decreased to 4% during week four in 2013. The same was true for Malva sp. (MW), where the classification was 75% accurate in week two but increased to 100% in weeks three and four. The accuracy for Amaranthus macrocarpus (AM) decreased to 80% in week three, although for Urochoa panicoides (LS) remained at 100% from week two to week four. Although the accuracy of Cyperus rotundus (NG) decreased from 100% to 67% in week three, it improved by up to 100% in week four. The classification accuracy decreased by 3-4% in week four for sorghum. Overall, the results proved that the classification improved from week two to week four in 2013. In 2014, the classification was 100% successful for most of the significant bands for each week (both calibration and validation data), except for the 89% classification accuracy for Urochoa panicoides (LS) in week two.
The independent data tested in 2013 using linear discriminant analysis (LDA) produced accurate results, as highlighted in Table 3. Meanwhile, stepwise linear discriminant analysis (SLDA) was used to test the dependent data and the classification accuracy was found to be slightly lower. The classification accuracy increased to 100% in 2014 using SLDA, as shown in Table 4. The accuracy of the classification for the weed discrimination using LDA and support vector machine (SVM) has been increased at 15 bands compared to the eight bands at more than 90% and 85%, respectively [24]. Similar results were found for the separation between weed species using LDA, which has a high degree of separation [25]. A previous study found that discriminant analysis (DA) can detect grass and broadleaf weed species with 70 to 100% accuracy [26].
The differences in the classification between weeds and sorghum in 2013 and 2014 were insignificant. The statistical analysis showed that most of the classification was at 100% for both years. Additionally, the different growth stages between week two and week four were not prominent. Weed discrimination with other species was investigated during pre-planting and found the overall accuracy above 95% [25]. It is of excellent significance to increase weed identification accuracy for pre-planting applications. The differences in results seen here were influenced by the growth stages and locations of the plants, as well as abiotic and biotic parameters [27]. Additionally, the spectral reflectance of plants depends on the weather conditions and the amount of sunlight received, due to the atmospheric conditions and azimuth [28]. The plants' morphology also affects their spectral reflectance, such as the shape and color of the leaves. These similar findings suggest that the environmental factors, microclimates, plant leaves, solar angles, and changing morphological and spectral properties are different at every growth stage, which will influence weed detection [29]. Humans are unable to differentiate between these plants features using the naked eye; thus, by using spectral signatures, the plants can be differentiated and classified using statistical analysis and machine learning [30][31][32].
A previous study showed that linear discriminant analysis (LDA) is more accurate compared to principal component analysis (PCA) in classifying Sesbania herbacea (Mill.) McVaugh (hemp sesbania), Ipomoea wrightii (palm leaf morning glory), Ipomoea lacunosa (pitted morning glory), Sida spinosa (prickly sida), Senna obtusifolia (sicklepod), and Jacquemontia tamnifolia (small flower morning glory) species in soybeans [33]. A similar study found that weeds and wheat were successfully classified with 100% accuracy using partial least squares (PLS) analysis and linear discriminant analysis (LDA) [34]. The short-wave infrared (SWIR) wavelengths between 1445 and 2135 nm were used in the previous study. Compared to our study, wavelengths of up to 1000 nm were used; thus, in the future, it is recommended to use SWIR wavelengths for weed classification. In a study in New Zealand, hyperspectral imaging was used to detect weeds using machine learning (ML). This method was another way of detecting weeds quickly and accurately using the multilayer perceptron method at 89% accuracy [26]; thus, hyperspectral imaging can be used in weed detection using ML as compared to object-based image analysis.

Band Identification
The results showed that the highest accuracy for the six combinations was 93%, involving the combination of the bands at 440, 560, 680, 710, 720, and 850 nm. The linear discriminant analysis (LDA) results from the hyperspectral data were significant for the six combinations. Table 5 shows the results from the classification of band combinations based on Table 2. There were 28 combinations (please refer to [14]), although only three are highlighted in in Table 5. The band combinations were narrowed down into five-, four-, and three-band combinations. The combination results can be found in [14]. The spectral signatures showed that each plant had significant bands. Based on the spectral signatures, each weed species in the sorghum field was identified. The band combinations helped us to more accurately identify the weeds in the sorghum field. The 20 significant bands were narrowed down to eight bands, which were used in the multispectral sensor. This method was similar to the previous research, in which the bands underwent further analysis [35]. The discrimination accuracy between sugar beets and maize was 90% when using specific spectral bands [36].
A similar study used spectral signatures (500-550, 650-750, 1300-1450, and 1800-1900 nm) to successfully discriminate weeds from sugarcane [30,32]. The shortwave infrared bands (SWIR; 1660, 1890, and 2000 nm) matched the primary features of the pure cellulose and lignin spectra of the plants. Changes in cellulose and leaf water content determine the levels of these characteristics, which can be leveraged to classify plant species [30]. Several wavelengths, including 491, 541, 641, 722, 772, 852, 942, 1047, 1132, 1443, and 2475 nm, were selected and calculated for the purpose of identifying plants species using an artificial neural network (ANN) and support vector machine (SVM) [31]. We found that ANN and SVM provided high levels of accuracy of up to 86%.
Mediterranean plants were found by using the high spectral resolution of the midinfrared (MIR) radiation, enabling accurate species discrimination [39]. Since this study focused on the visible to near-infrared (325 to 1075 nm) range, we suggest that future studies should consider spectral reflectance from 1055 up to 2000 nm for further analysis of weed discrimination, because MIR reflectance is also affected by the moisture content of plants [39].

Results of Image Processing
The same six spectral bands were used for image collection in the field. The spectral, physical, and contextual characteristics of the weeds and sorghum affect the development and efficiency of the segmentation and classification rule sets in image processing procedures. The rule sets in the object-based analysis (OBIA) can be modified to identify weeds at different growth stages so that the results are more accurate. In addition, the detection of weeds will be more accurate when using a high spatial resolution.
The growth stages of the weeds from week one to week four after being planted were assessed via multispectral images. Based on the results, weeds and sorghum were accurately classified in week three (Table 6), because the sizes of both were bigger in comparison to weeks one and two; however, the size of the weeds was still small and they were easy to separate from the sorghum plants. The same was true for Cyperus rotundus (NG) and Urochoa panicoides (LG), which were successfully classified in week three after planting. The shapes of leaves such as broadleaf Portulaca oleracea (PG) and Ipomea plebeian (B) were easier to distinguish. Both were successfully classified in week three. The literature showed how grass weed seedlings were discriminated from monocotyledonous crops, with seedlings of broadleaf weeds in dicotyledonous crops proving more challenging to identify at later stages [40] because they have similar spectral reflectance and due to the spread of weeds in small patches and the effects of soil reflectance at the early stages of growth [41]. Table 6. Weed species at a ground sampling distances (GSDs) of 10.83 mm (altitude: 20 m) and 20.31 mm (altitude: 37.5 m) for week three. Adapted from [14]. The weeds started to mix with the crop at week four; thus, the broadleaf weeds were detected at an early stage. Weed detection earlier stages of growth is consistent with weed control strategies that are designed to minimize competition with sorghum plants, as weeds become difficult to detect at the later stages of growth. Additionally, the background reflectance in images also affects weed detection in the field. For example, the background reflectance of sorghum can be used in object-based image analysis (OBIA) to identify the shapes of weed leaves more accurately as compared to the sorghum background; therefore, the background was extracted as a black shade (with a low digital number) in the image analysis. This is consistent with the study by [42], who normalized red-green-blue (RGB) data into the hue, saturation, and intensity (HSI) image space to segment plant images against background regions. The HSI transformation is a standard technique used to numerically describe color in the image domain using spherical coordinates, which are roughly analogous to the conventional attributes of hue, saturation, and lightness (I) [43], whereby images are collected under artificial light. In contrast with our study, the images were collected under field illumination conditions. Table 6 shows the object-based image analysis (OBIA) results for week three after planting via different spatial resolutions, also known as the ground sampling distance (GSD) technique. In week three, the weeds were neither too small nor too large and they were individually identifiable. Overall, most of the weed species were successfully classified; all species were correctly classified at 10.83 mm, while at 20.31 mm, Cyperus rotundus (Q1NG) was misclassified but the rest were correctly classified.

Weed Map Analysis
The results show that the spatial resolutions of the aerial images affect the classification accuracy. The higher the spatial resolution (5.42 mm), the higher the accuracy ( Figure 5). Most of the weeds were detected with a 5.42 mm resolution. Nevertheless, a spatial resolution of 20.31 mm can still be used to detect weeds in sorghum fields. It was concluded that weeds can be detected in a sorghum field at altitudes of 10 to 40 m. In this study, image processing was carried out using rule sets constructed using object-based image analysis (OBIA). The rule sets are flexible and can be modified to obtain optimal results. The rule sets have to be modified to accommodate plant growth stages and image environments [20]. In this study, similar rule sets were used, although the scale, shape, and compactness settings were modified based on the spatial resolution of the images. The second group of segmentation settings depend on how much segmentation is achieved from the first-stage segmentation settings and how many more images need to be segmented.
Multiresolution segmentation generates objects by merging several pixels together based on their homogeneity [44]; however, other algorithm techniques can be used to investigate the effectiveness of the segmentation. Since the classification process was focused on identifying weeds in the sorghum field, the soil (SO) classification was only of minor interest. In the future, the shadow objects could be separated to obtain more accurate results [45]. This also highlights the desirability of image collection when there are fewer shadows (before 12:00 p.m.).
Image analysis could be enhanced using the object-based image analysis (OBIA) automatic processing technique. The geographic object-based image analysis (GEOBIA) automatic classification technique has the potential for large-area processing and might be useful in the automatic processing of multiple images [20,46]. This could be an alternative for detecting crop rows before identifying weeds within the crops [47]. This is possible because crop rows are planted at fixed locations and weeds tend to grow randomly within the fields.
It is important to look at how objects are sampled to obtain unbiased and accurate results [48]. Since this study only focused on three main classes of objects, the use of KHAT statistics was acceptable. Kappa coefficients are frequently used to indicate classification accuracy and are more accurate than using percentages [49].
Six spectral bands were used in this analysis. It might be useful to choose fewer bands or a combination of bands in future studies. For example, six bands were used in a previous study for segmentation analysis of weed detection [50]; however, these were narrowed down to calculate the normalized difference vegetation index (NDVI) bands, so as to discriminate between vegetation and soil.
The detection accuracy can be improved by using hyperspectral imaging, as proposed by [41]. Our study used only six bands, while hyperspectral imaging involves of up to 1000 much narrower bands (depending on the sensor). Nevertheless, the cost is higher when hyperspectral imaging is used compared to using a multispectral camera; thus, for the multispectral bands, the use of three to six bands to detect weeds in fields is suitable and affordable.

Accuracy Assessment Confusion Matrix Accuracy
Three resolutions were validated using the confusion matrix (Table 7). In fact, the higher spatial resolution (5.42 mm) gave a higher accuracy level of 92%. The weed detection was underestimated at 20.31 mm spatial resolution when the producer accuracy (PA) was lower than the user accuracy (UA). Overall, KHAT accuracies were high for all spatial resolutions, indicating accurate classification, except for the 20.31 mm resolution (KHAT <80%). A KHAT accuracy of more than 70% is adequate for classification [44]. Weed mapping was performed using the object-based image analysis (OBIA) method at different spatial resolutions. Kappa statistics in the range of 0.61-0.80 indicate good classification [44]. They are deemed comparable to 61-80% KHAT values. In this study, the classification accuracy remained higher than 80% for the overall and KHAT accuracies, except for Cyperus rotundus (NG) at 10.83 mm spatial resolution. Meanwhile, at 5.42 mm spatial resolution, the classification accuracy increased to 92% and 97% for overall and KHAT accuracies, respectively, as shown in Table 7. This shows that higher spatial resolutions will provide higher accuracy. A previous study recommended the use of very high spatial resolution for weed detection [51]. Larger crop rows can help in detecting weeds using UAV. At least four pixels should be used for 15 cm crop rows to detect small objects via aerial images [52,53]. The same approach was used in another study, where 2 cm spatial resolution was used to detect weeds in maize fields [50]. The researchers obtained 90% accuracy for weed detection using a combination of normalized difference vegetation indexes (NDVI) and OBIA. Fewer band combinations could be tested in the future. There may also be a trade-off between the number of bands and the spatial resolution of the images.

Conclusions
In this paper, we identified the significant spectral bands for weed discrimination using hyperspectral data, which can be used in multispectral sensors for weed mapping. We used discriminate analysis to produce six significant spectral bands for the weed classification process. We collected multispectral imagery of the study area using an MCA 6 camera and analyzed these images using object-based image analysis (OBIA). The hierarchical model of multiscale weed detection was used successfully to simplify the OBIA rule sets for weed mapping. We determined that the use of fewer bands and high spatial resolution improves weed classification. This demonstrates the importance of having optimum spectral and spatial resolutions. We also found that the growth stages during play an important role data collection for weed detection in sorghum fields. These findings will be significant in overcoming difficulties in weed detection and mapping for sorghum crops. The spectral signatures of weed species can be used as inputs in spectral libraries and as guidelines for accurate weed mapping. The findings can also be used as inputs to develop automated technology for accurate weed detection applications. This approach can be integrated with artificial intelligence (A.I) systems, tractors, unmanned aerial vehicles (UAV), or drones for weed detection and automated targeted spraying of herbicides. This approach could have a big impact on the agroecology environment and provide a cost-effective method for weed control management. Currently, the results of this study are limited to the analysis procedures used for the multispectral images of the weeds and sorghum plants collected in the study area. In future studies, we recommend using hyperspectral imaging and machine learning techniques at different growth stages, with different herbicide applications and over diverse areas.