Monitoring Wheat Fusarium Head Blight Using Unmanned Aerial Vehicle Hyperspectral Imagery

: The monitoring of winter wheat Fusarium head blight via rapid and non-destructive measures is important for agricultural production and disease control. Images of unmanned aerial vehicles (UAVs) are particularly suitable for the monitoring of wheat diseases because they feature high spatial resolution and ﬂexible acquisition time. This study evaluated the potential to monitor Fusarium head blight via UAV hyperspectral imagery. The ﬁeld site investigated by this study is located in Lujiang County, Anhui Province, China. The hyperspectral UAV images were acquired on 3 and 8 May 2019, when wheat was at the grain ﬁlling stage. Several features, including original spectral bands, vegetation indexes, and texture features, were extracted from these hyperspectral images. Based on these extracted features, univariate Fusarium monitoring models were developed, and backward feature selection was applied to ﬁlter these features. The backpropagation (BP) neural network was improved by integrating a simulated annealing algorithm in the experiment. A multivariate Fusarium head blight monitoring model was developed using the improved BP neural network. The results showed that bands in the red region provide important information for discriminating between wheat canopies that are either slightly or severely Fusarium -head-blight-infected. The modiﬁed chlorophyll absorption reﬂectance index performed best among all features, with an area under the curve and standard deviation of 1.0 and 0.0, respectively. Five commonly used methods were compared with this improved BP neural network. The results showed that the developed Fusarium head blight monitoring model achieved the highest overall accuracy of 98%. In addition, the di ﬀ erence between the producer accuracy and user accuracy of the improved BP neural network was smallest among all models, indicating that this model achieved better stability. These results demonstrate that hyperspectral images of UAVs can be used to monitor Fusarium head blight in winter wheat. a backward feature selection was applied before model development. Five commonly used methods (i.e., PLSR, FLDA, LR, RFs, and SVM) were used to establish the monitoring model, and the results were compared with those of the improved BP neural network. This comparison showed that the improved BP neural network performed best among all tested models, with overall accuracy of 98% and kappa of 0.96. This study provides a reference for wheat FHB monitoring via hyperspectral UAV images. The approach proposed in this study extends the previous models in terms of feature types and monitoring area. However, the limited plots have some e ﬀ ects on model development and validation. Future research could attempt to use data augmentation, unsupervised training, and transfer learning to overcome the problem. Moreover, future research should explore the performance of this method in ﬁelds with di ﬀ erent wheat cultivars, in ﬁelds with multiple diseases, and in ﬁelds with wheat at other growth stages. In addition, more features (such as soil type, cultivation procedures, and management practices) should be considered in future research to develop a more robust and more reliable wheat FHB monitoring model.


Introduction
As the dominant staple in most regions of North Africa as well as West and Central Asia, wheat (Triticum aestivum L.) is consumed by 2.5 billion people in 89 countries, and, annually, a total of 215 million hectares are used to grow wheat [1]. Wheat Fusarium head blight (FHB), or wheat scab, is an intrinsic infection by Fusarium graminearum (Gibberella zeae) [2]. The normal physiological function of FHB-infected wheat is destroyed, and both its internal physiological structure and external morphology change [3]. In addition, the disease produces a number of mycotoxins, of which deoxynivalenol (DON) textural characteristics of wheat, and select the features that are sensitive to wheat FHB; (ii) to develop a multivariate FHB monitoring model based on these sensitive features; and (iii) to apply a multivariate FHB monitoring model to map wheat FHB infection in the study area.

Study Area and Dataset
The experimental field investigated in this study is located in Lujiang County, Anhui Province, China. Its coordinates are 117 • 13 12"E and 31 • 29 0"N. Winter wheat is the main crop produced in Lujiang County, and the main variety is Yangmai 25, which is susceptible to wheat FHB. This location has a north subtropical humid monsoon climate, with an average annual temperature of 15.8 • C; the highest temperature is in July and the lowest temperature is in January. The average annual precipitation is 1188 mm and the rainy season lasts from June to July every year [29]. In this region, the soil belongs to alfisols and FHB is a major wheat disease [30]. Figure 1 shows the location of the field site. Hyperspectral UAV images were acquired on 3 and 8 May 2019, when the wheat was at the grain filling stage. The UAV used in the experiment was an M600 Pro aircraft of Daijang Innovations (DJI), and a Cubert S185 FireflEYE SE hyperspectral imaging camera was used. The spectral range of the S185 was from 450 to 950 nm, and the average raw sampling width was 4 nm. The radiance calibration of the hyperspectral imaging camera was carried out before capturing hyperspectral images. The camera exposure time was automatically matched to the environmental condition. The height above ground of the UAV was 60 m, the flight speed was 3 m/s, camera triggering frequency was 0.8 s, forward overlap was 80%, and side overlap was 65%. The spatial resolution of hyperspectral images was 4 cm, and the raw number of bands in the imagery was 125. The area of the experimental field was 5000 m 2 , and uniform wheat cultivars, cultivation procedures, and management practices were applied to this field. All wheat plants in this field were infected with FHB because no pesticide was used at the early stage. Fifty plots, each with an area of 1 m 2 , were selected in the experimental field. To accurately locate the plots in hyperspectral images, a flag was placed next to each plot. The severity of the FHB of the plot was calculated according to the rules for monitoring and forecasting wheat head blight suggested by the National Plant Protection Department of China (Chinese Standard: GB/T 15796-2011) [31,32]. When inspecting the disease severity, 50 individual plants were randomly selected in every plot, and the symptoms of wheat FHB were identified by visual inspection. Then, the number of plants with infected spikelets was recorded, the ratio of infected plants in each plot was calculated, and all plots were classified into five classes according to the calculated ratio of infected plants: 0.1-10% (class 1), 10-20% (class 2), 20-30% (class 3), 30-40% (class 4), and 40-100% (class 5). In practice, wheat fields with more than 30% infected wheat ears would be destroyed because of the excessive harmful substances produced. Consequently, plots were quantitatively classified into two classes for subsequent analysis: Plots with a ratio of infected wheat ears to healthy wheat ears below 30% were labeled as slightly diseased, and plots with a ratio exceeding 30% were labeled as severely diseased. Figure 2 shows slightly diseased and severely diseased wheat canopies in red-green-blue (RGB) imagery captured by the UAV.

Data Preprocessing
When acquiring UAV hyperspectral images, the drone was equipped with a position and orientation system to record the real-time position and attitude of the sensors. After the mission was completed, the images obtained from the UAV imaging spectrometer were a high-spatial-resolution panchromatic image (JPG format) and a low-spatial-resolution hyperspectral cube image (CUE format). Then, the fusion and stitching operations were carried out in CubertPolot and Photoscan, respectively. During the acquisition of hyperspectral images, noise would be added to the radiated energy output of the sensor due to the systematic errors in the sensor, and this resulted in a discrepancy between the radiation values of hyperspectral images and the true radiation values. Thus, a radiometric calibration was carried out using the following equation: where L is the radiation value, M is the gain, A is the bias, and DN is the pixel value. After that, atmospheric correction was used to remove the effects of the atmosphere on the reflectance values of hyperspectral images. Figure 3 shows the main steps of the methodological framework developed in this research. These steps are described below. When inspecting the quality of hyperspectral images, the first nine bands and the last 15 bands of each image were excluded because they were found to have been affected by noise during the post-process. Then, a number of features, including original spectral bands, vegetation indexes, and texture features, were extracted from these hyperspectral images.
After removal of all noise-affected bands, 101 bands remained in each hyperspectral image. Since the spectral sampling interval of hyperspectral images was small, two adjacent bands are highly correlated. In this study, an original spectral band was selected as a feature in every five adjacent bands to decrease the correlation between features, and the selection was carried out in a regular, consistent way, leading to a 20 nm sampling interval.
The vegetation indexes used in this study are commonly used in the monitoring of crop pests and diseases. Table 1 presents detailed descriptions of these vegetation indexes.  [42] Several texture features were also extracted from these hyperspectral images. The texture of an image is a feature that reflects the homogeneity of the image. When the wheat canopy is infected by FHB, the ears of wheat will show brown spots, which gradually expand to the whole spikelet and cause the spikelet to shrivel [43]. Wheat canopies with different FHB-infection severities will have different texture features. The local binary pattern (LBP) was used to describe texture features in this study. The LBP is a commonly used local binary descriptor to describe the texture characteristics of an image [44]. The mathematical expression of the LBP is presented in the following: where P represents the number of neighbors, R represents the radius, i represents one of the neighbors, x i represents the pixel value of i, and x c represents the pixel value of the central pixel. The proposed work used three combinations of P and R: P = 8 and R = 1 (LBP (8,1) ), P = 8 and R = 2 (LBP (8,2) ), and P = 16 and R = 2 (LBP (16,2) ). Figure 4 shows the relationship between a pixel and its neighbors for three combinations of P and R. When original spectral bands, vegetation indexes, and texture features were collected, the pixels located in experimental plots were extracted. Firstly, a support vector machine (SVM) was used to classify the pixels in the hyperspectral images into five categories: 1, bare soil; 2, road; 3, wheat canopy; 4, other plants; 5, other non-plants. The pixel-based classification approach was applied because the pixels in the wheat canopy were well-defined and well-separated, and mixed pixels in the wheat canopy were abandoned in the end. The user accuracies and producer accuracies of all classes were higher than 80%, and the overall accuracy was 87.14%. In this experimental field, wheat is rarely mixed with other vegetation, and weeds are mainly distributed in trails between wheat fields. Thus, the errors between the "wheat canopy" and the "other plants" classes are small. Since the pixel-based classification result contained noise, majority/minority analysis was used in the post-processing of the classification to eliminate noise [45]. After identifying the pixels that belong to the wheat canopy, the wheat canopies close to a flag identifying the experimental plot were selected, and the pixels located in experimental plots were extracted. Considering the spatial resolution of hyperspectral images of 4 cm and the size of each plot of 60 cm × 60 cm, the pixels in the range of 15 pixels × 15 pixels at the center of the selected wheat canopies were considered as being located in experimental plots. For each plot, the pixels located in the plot were chosen (except for the central pixel), and the values of these pixels were averaged. Eighty percent of these mean values and corresponding disease severities were used as the training set and 20% were used as the validation set to train and evaluate the FHB monitoring model. These mean values and corresponding disease severities were sorted randomly. The classification was repeated 100 times, and the average accuracy of the FHB monitoring model was calculated. Moreover, the values of central pixels in experimental plots and the corresponding disease severities were used as the test set.

Fusarium Head Blight Detection Using a Univariate Classification Approach
Firstly, a univariate classification approach was used to develop the FHB monitoring model. The receiver operating characteristic (ROC) curve, which is widely used in remote sensing applications, was applied to evaluate the performance of the model to discriminate two types of plots. The construction of an ROC curve, as required for the identification of slightly diseased wheat and severely diseased wheat at the pixel level, was considered a binary classification problem, which results from the thresholding of a variable (original spectral bands, vegetation indexes, and texture features). For each pixel, the possible classification result is listed in Table 2. Two indicators of classification performance, sensitivity and specificity, were calculated using the following equations: The ROC curve was drawn with sensitivity as the ordinate and 1-specificity as abscissa [46]. When evaluating the performance of a univariate FHB monitoring model, the values of the variable were given based on an initial value and a step. The corresponding values of specificity and sensitivity were calculated, and the ROC curve was obtained. Theoretically, the optimal value of the variable can be located in the ROC curve where the sum of sensitivity and specificity is at its maximum. In addition, the area under the ROC curve (AUC) was used to evaluate the overall classification performance of the univariate model. The values of AUC range from 0.0 to 1.0, and the higher the value, the better the classification performance of the univariate model; Figure 5 shows the general overview of the ROC curve and AUC.

Fusarium Head Blight Detection Using a Multivariate Classification Approach
After the univariate FHB monitoring models (that were developed using all features) were evaluated, a multivariate classification approach was used to develop the FHB monitoring model. In this study, a backpropagation (BP) neural network was used to develop the multivariate FHB monitoring model. The BP neural network is a multilayer feedforward neural network. In general, it has three layers: an input layer, a hidden layer, and an output layer [47]. The training process of the BP neural network follows two steps: in the first step, variables are assigned to the input layer, and the outcome is calculated through the weighted sum of the hidden layer; in the second step, the error of the outcome is transferred from the output layer to the input layer, and the weights and biases of variables are adjusted to decrease the error of the outcome. When this error is reduced to 0.1, the training of the neural network is completed.
However, the training process of the BP neural network is a local search mechanism, and it may lead to trapping of the BP neural network in the local optimum. To avoid this problem, the simulated annealing algorithm was integrated into the BP neural network. The simulated annealing algorithm is a greedy algorithm, which lets the algorithm escape from the local optimum by accepting values that increase the error [48]. Figure 6 shows the structure of the improved BP neural network. In addition, the Akaike information criterion (AIC) was used to evaluate the complexity and accuracy of the multivariate FHB monitoring model. The AIC is a model evaluation index that comprehensively considers the number of model parameters, the number of samples, and the monitoring accuracy. The larger the AIC, the better the model [49]. In addition, five commonly used methods, including partial least square regression (PLSR), Fisher's linear discriminant analysis (FLDA), logistic regression (LR), random forests (RFs), and SVM, were compared with the improved BP neural network. The existing methods were widely utilized to monitor disease outbreaks, and Table 3 presents detailed descriptions for these.  Table 4 shows the AUC and corresponding standard deviation (Std) of the univariate FHB monitoring model developed using features. Table 4 shows the first five original spectral bands with better performances, and the sensitivity and specificity of the optimal threshold of each feature were also included. In general, the modified chlorophyll absorption reflectance index (MCARI) performed best among all features with AUC and Std values of 1.0 and 0.0, respectively. Band 50 (650 nm) and Band 60 (690 nm) had the same AUC and Std, whereas the specificity of Band 50 (650 nm) was higher than that of Band 60 (690 nm). Texture features had moderate performances compared with other features, and they had the highest Std. Figure 7 shows the optimal thresholds and values of samples of the univariate FHB monitoring model. The yellow dots represent slightly diseased samples, red dots represent severely diseased samples, and the black dotted line indicates the optimal threshold. For features with higher AUC, sensitivity, and specificity (i.e., MCARI), two types of samples were more separable. In contrast, for features with lower AUC, sensitivity, and specificity (i.e., LBP (8,1) ), two types of samples were difficult to separate. Moreover, a number of features had similar optimal thresholds, such as Band 50 (650 nm) and Band 55 (670 nm). This may be because of the similar spectral reflectance of the wheat canopy in 650 and 670 nm.  (8,2) 0.47 0.06 0.22 0.92 LBP (16,2) 0.40 0.06 0.12 0.94

Evaluation of the Multivariate Monitoring Model
When developing the multivariate FHB monitoring model using an improved BP neural network, all features in Section 3.1 were used as input first. These include five original spectral bands, 11 vegetation indexes, and three texture features. Then, backward feature selection was used to filter these features. Table 5 shows the features and AIC values of the multivariate FHB monitoring model before and after backward feature selection. Only seven of 19 original features remained after backward feature selection, and the AIC of the multivariate FHB monitoring model decreased from −362.3 to −500.64, indicating systematic improvement of the power of explanation. It should be noted that the modified simple ratio (MSR) performed worse in Section 3.1 (with AUC of 0.07) while it was contained in the feature set after backward feature selection. The reason was that the MSR contained useful information for FHB monitoring and realized a complementing effect with other features. Table 5. Feature variables and Akaike information criterion (AIC) of the models before and after backward feature selection.

Number of variables 7
Gain (% AIC reduction) 38.1 The test set was used to evaluate the performance of models developed using the improved BP neural network, PLSR, FLDA, LR, RFs and SVM, and the test results were exhibited in the form of a confusion matrix (Table 6). This confusion matrix, also known as an error matrix, is a matrix with two rows and two columns. Values of producer accuracy, user accuracy, overall accuracy, and Kappa coefficient were applied to the confusion matrix. The overall accuracy indicated the general classification performance of the classifier on the test set. The Kappa coefficient was used to quantify the consistency between the real classes and classification results of samples. The producer accuracy represents the number of classified reference samples that were accurately divided by the total number of reference samples for that class. The user accuracy was calculated by dividing the total number of correct classifications for a particular class and by the row total. Table 6 shows that the FHB monitoring model that was developed using an improved BP neural network achieved the highest overall accuracy of 98%. Moreover, the difference of producer accuracy and user accuracy of the improved BP neural network was smallest among all models, indicating that this model had superior stability. With regard to the models that were developed using five commonly used methods, FLDA, RFs, and SVM achieved the same overall accuracies. However, the difference of producer accuracy and user accuracy in FLDA was larger than in the other two methods. The higher producer accuracy of severely diseased samples of FLDA indicated that it tended to misclassify slightly diseased samples as severely diseased samples. LR had the lowest overall accuracy, producer accuracy, and user accuracy, indicating that it performed worst in distinguishing between slightly and severely diseased wheat canopies. Based on the overall verification results shown in Table 6, the model developed using the improved BP neural network was chosen to monitor the severity of wheat canopy effects in the experimental field. The monitoring result is shown in Figure 8.

Discussion
Hyperspectral images from UAVs have been used to monitor vegetation pests and diseases in many studies. Some people tried to detect pests in vineyards by combining UAV hyperspectral images with ground data. The vegetation indexes that were sensitive to the pest were calculated, and the pest monitoring model was developed based on these indexes [57]. The approach proposed in this study also combined the UAV hyperspectral imagery with ground survey data. Considering that wheat FHB could change the structure and shape of a wheat canopy, this study added texture features to the spectral features as inputs for the monitoring model. There has also been some research attempting to apply deep learning to monitor pests and diseases using UAV hyperspectral imagery [26,58,59]. The monitoring accuracies achieved in these studies were generally high, but the poor interpretability of the extracted features makes it difficult to apply the model to other regions. This study combined plant pathology and remote sensing disease monitoring mechanisms to select the features, and the proposed model can be easily applied to areas that are similar to the experimental field.
Early attempts were made to apply remote sensing techniques to the monitoring of wheat FHB [60]. However, given the demands of production, most of the research only applied remote sensing technology to detect FHB in wheat kernels [61,62]. These studies attempted to extract the spectral characteristics of diseased wheat kernels through spectral indices, principal component transformations, and other methods, and thus constructed monitoring models. Consistently with these studies, different vegetation indices exhibited different traits when monitoring FHB in this paper. In recent years, some studies attempted to identify diseased areas of spikelet through multispectral or hyperspectral images taken in wheat fields [8,63,64]. They found that the wavelengths near 650 nm were sensitive to wheat FHB, which is consistent with the findings of this study. However, the target areas of these studies were small, and the models constructed did not meet the need for accurate monitoring of wheat FHB in a wheat field. This study combined spectral and textural features to construct a model for the monitoring of FHB at field scale, extending the previous models in terms of feature types and monitoring area. It is worth noting that, in this study, univariate monitoring models developed using texture features had moderate mean AUCs; this may be largely due to the lower spatial resolution of images, and this also indicated that utilizing texture features alone for FHB monitoring at the field scale is not sufficient.
The combination of the simulated annealing algorithm and BP neural network has been used in many remote sensing applications, such as air quality prediction, traffic flow forecasting, rock mass parameter prediction, etc. [65][66][67][68]. These studies found that the model constructed by this method was able to achieve higher monitoring accuracy than the BP neural network. In this study, the improved BP neural network performed better than five commonly used algorithms. The producer accuracy and user accuracy of the improved BP neural network in slightly and severely diseased samples exceeded 95%, and kappa was 0.96. This superior performance of the improved BP neural network was likely the result of two features. The relationship between features (i.e., original spectral bands, vegetation indices, and texture features) and the severity of FHB was complex, and the improved BP neural network has the ability to generate complex decision boundaries in the feature space. In addition, the improved BP neural network was proposed by integrating a simulated annealing algorithm into the BP neural network. Therefore, it could avoid becoming trapped in the local optimum, and offers the advantages of the BP neural network to effectively avoid overfitting of the data.
In Section 3.2, the improved BP neural network was used to monitor the severity of FHB infection of wheat canopies in the experimental field, and the monitoring results of 3 and 8 May are shown in Figure 5. It was obvious that the area of severely diseased wheat canopies increased rapidly from 3-8 May. Considering that no measures had been taken to prevent FHB in the experimental field, the conclusion can be drawn that the spreading of Fusarium graminearum would be rapid if no preventive measures were taken at the grain filling stage.
Although this study yielded satisfactory results for wheat FHB monitoring, there are still some weaknesses that need to be improved in future research. First, the number of plots was small in this study due to the high cost in the process of plot data collection. The small plot size led to the lack of a model validation process based on real independent data. Limited training data are a common problem in remote sensing applications [56], and many approaches have been used to mitigate small training samples, including data augmentation, unsupervised training, and transfer learning [69]. Future research could attempt to use these methods to overcome the problem of small plots and thus develop a more stable and efficient monitoring model. Second, the monitoring model proposed in this study is a binary classification approach based on multiple features. In order to make the monitoring results more instructive, future studies could assess the uncertainty of classification and improve the model based on the characteristics of the pixels with greater uncertainty.
In addition, a number of limitations and challenges still remain when applying the proposed approach in monitoring wheat FHB. Firstly, the crop cultivars, cultivation procedures, and management practices of the experimental field were uniform. When the proposed approach is applied to wheat fields with different wheat cultivars, cultivation procedures, or management practices, the features that performed well in this study may not be applicable. However, the idea of wheat FHB monitoring based on spectral and textural features is still valid. Future research could validate the proposed approach in wheat fields with different wheat cultivars, cultivation procedures, or management practices. Secondly, FHB was the only disease of wheat in this experimental field; thus, it remains unclear whether the features used in this study (i.e., original spectral bands, vegetation indexes, and texture features) are effective for monitoring the severity of FHB in cases where this disease is accompanied by other wheat diseases. Thirdly, it is not clear whether this approach will perform well when distinguishing slightly and severely infected wheat canopies from other vegetation and non-vegetation classes. Future research could evaluate the ability of this approach in distinguishing wheat from other classes. In addition, the hyperspectral images from the UAV and field plots were acquired at the grain filling stage; thus, the performance of the multivariate FHB monitoring model at other stages of the FHB infection needs to be evaluated by future research.
In this study, the hyperspectral images from a UAV were used to monitor the severity of FHB. Many studies have attempted to use RGB images for detection of wheat FHB because UAV RGB imagery has advantages in terms of cost and coverage [70][71][72]. However, Dammer et al. (2011) found that bands other than red, green, and blue contain some useful information for FHB monitoring [73]. Moreover, when the symptoms are completely exhibited, the observation of FHB is too late for preventative measurement. Therefore, it is important to detect FHB in the early stages. The symptoms are hard to observe in the RGB images in the early stages, though acquiring rich spectral information on the wheat canopy is more important for FHB monitoring. Future research could compare the performance of UAV RGB images and UAV hyperspectral images on the monitoring of FHB in different stages. Currently, UAV imagery is suitable for FHB monitoring in a smaller area, while satellite imagery would be more appropriate when carrying out FHB monitoring in large areas. However, when monitoring FHB in large areas, due to the lower spatial resolution of satellite imagery and large regional variation, some natural and economic factors, such as temperature, humidity, and management practices, do have a significant impact on the occurrence of the disease. It could be useful to extract these factors when monitoring the occurrence of the disease in large area.

Conclusions
Hyperspectral images from UAVs offer valuable and reliable data for wheat FHB monitoring in the field. This study achieved accurate monitoring of wheat FHB by utilizing spectral features and texture features of UAV hyperspectral images. After obtaining hyperspectral images, three types of features (including original spectral bands, vegetation indexes, and texture features) were extracted. Based on these features, univariate FHB monitoring models were developed to evaluate the ability of each feature to identify different levels of severity of wheat FHB. Then, the multivariate FHB monitoring model using an improved BP neural network was developed based on the most sensitive features. To decrease the complexity and avoid overfitting of the model, a backward feature selection was applied before model development. Five commonly used methods (i.e., PLSR, FLDA, LR, RFs, and SVM) were used to establish the monitoring model, and the results were compared with those of the improved BP neural network. This comparison showed that the improved BP neural network performed best among all tested models, with overall accuracy of 98% and kappa of 0.96. This study provides a reference for wheat FHB monitoring via hyperspectral UAV images. The approach proposed in this study extends the previous models in terms of feature types and monitoring area. However, the limited plots have some effects on model development and validation. Future research could attempt to use data augmentation, unsupervised training, and transfer learning to overcome the problem. Moreover, future research should explore the performance of this method in fields with different wheat cultivars, in fields with multiple diseases, and in fields with wheat at other growth stages. In addition, more features (such as soil type, cultivation procedures, and management practices) should be considered in future research to develop a more robust and more reliable wheat FHB monitoring model.