Early Detection of Excess Nitrogen Consumption in Cucumber Plants Using Hyperspectral Imaging Based on Hybrid Neural Networks and the Imperialist Competitive Algorithm

: To achieve healthy and optimal yields of agricultural products, the principles of nutrition must be observed and appropriate fertilizers must be applied. Nutritional deﬁciencies or overabun-dance reduce the quality and yield of the products. Thus, their early detection prevents physiological disorders and associated diseases. Most research efforts have focused on spectroscopy, which extracts only spectral data from a single point of the product. The present study aims to detect early excess nitrogen in cucumber plants by using a new hyperspectral imaging technique based on a hybrid of artiﬁcial neural networks and the imperialist competitive algorithm (ANN-ICA), which can provide spectral and spatial information on the leaves at the same time. First, cucumber seeds were planted in 18 pots. The same inputs were applied to all the pots until the plants grew; after that, 30% excess nitrogen was applied to nine pots with irrigation water, while it remained constant in the other nine pots. Each day, six leaves were collected from each pot, and their images were captured using a hyperspectral camera (in the range of 400–1100 nm). The wavelengths of 715, 783 and 821 nm were determined as the most effective for early detection of excess nitrogen using a hybrid of artiﬁcial neural networks and the artiﬁcial bee colony algorithm (ANN-ABC). The parameter of days of treatment was classiﬁed using ANN-ICA. The performance of the classiﬁer was evaluated using different criteria, namely recall, accuracy, speciﬁcity, precision and the F-measure. The results indicate that the differences between different days were statistically signiﬁcant. This means that the hyperspectral imaging technique was able to detect plants with excess nitrogen in the near-infrared range (NIR), with a correct classiﬁcation rate of 96.11%. and J.P.; visualization, S.S. and R.P.; supervision, G.G.-M., J.P. and J.M.M.-M.; funding acquisition, S.S., R.P., G.G.-M. and J.M.M.-M. All authors have read the


Introduction
In recent decades, the demand for higher quality agricultural products has increased. Mechanical, bacterial and contamination damage are factors that may shorten the shelf life of fruits and vegetables and reduce their marketability. Therefore, research is being conducted to inspect these products to ensure their quality and health, using non-destructive techniques such as imaging and spectroscopy. However, traditional spectroscopy only provides spectral information from a given location of the products, determining locationdependent quantities. On the other hand, external features of the agricultural products (1) extraction of images of the leaves at different wavelengths in the range from 400 to 1100 nm to find the most effective hyperspectral images for classification; (2) selection of the key wavelengths using hybrid artificial neural networks and the artificial bee colony algorithm (ANN-ABC) to obtain the most useful spectral information; (3) classification using a hybrid artificial neural network (ANN) with an optimal structure that guarantees the high accuracy of the method; and (4) training the system at different iterations to examine the reliability of the proposed system. Figure 1 shows the main steps of the research process carried out in the present study, from the collection of the cucumber leaves used in the experiments to the comparison of the proposed method with a traditional laboratory analysis of the nitrogen content. These steps are described in detail in the following subsections.

Data Collection
Initially, 18 pots were prepared and cucumber seeds (Super Arshiya-F1 cultivar) were planted in each pot and kept in the research greenhouse of the University of Mohaghegh Ardabili, Iran, as shown in Figure 2. All the pots consumed the same inputs until the plants grew. After that, 30% excess nitrogen was applied with irrigation water to 9 pots. Next, 6 leaves were picked from each pot every day, and hyperspectral images of these leaves were captured with a hyperspectral camera. The sampling process continued until the leaves turned yellow and the symptoms of fertilizer overapplication were clearly visible. This occurred on the fourth day, so the images go from Day 0 (just before applying excess nitrogen) until Day 3.
The increase of 30% in the application of nitrogen was selected by considering some previous works related to the use of fertilizers. For example, Ma et al. [31] presented a method to detect deficiencies and excess of nitrogen in soybean leaves using RGB color images, with levels of nitrogen of 0%, 50%, 100% and 150%. Yang et al. [32] analyzed the effect of different levels of nitrogen in cucumber yield and fertilization efficiency, with application levels of 250, 300 and 350 kg/hm 2 , i.e., 20% and 40% increases, respectively. Thus, our selection of a 30% increase can be considered as an average value, trying to detect the excess nitrogen in relatively low increments.

Capturing Hyperspectral Images and Extracting the Spectral Properties of the Leaves
To extract spectral images from the leaves, a hardware system was configured. The components of this system were: (1) a laptop with Intel Corei5CFI, 330 M at 2.13 GHz, 4 GB of RAM, and Windows 10; (2) a hyperspectral camera (FSR, Fanavaran Physics Noor Co., Tehran, Iran) with a spectral range from 400 to 1100 nm; (3) a light source of tungsten halogen with a power of 10 W (StellarNet, Tampa, FL, USA) and (4) an illumination chamber to prevent ambient light. The hyperspectral camera was located on the left side of the chamber, and the samples were located horizontally in front of the camera at a distance of 1 m. A pair of lamps illuminated the sample from the corners, as depicted in Figure 3.  In total, 327 images of wavelengths in the range of 400-1100 nm were captured for each cucumber leaf. As can be seen, at the beginning and the end of this range, the images have poor quality due to noise and lack of illumination. These images were also included in further processing but were automatically removed since they were not effective. Moreover, although the images contain a part of the background, as shown in Figure 4, these background pixels were removed from the analysis.

Selection of the Key Wavelengths for Early Detection of Excess Nitrogen
After imaging the leaves from pots with normal nitrogen levels and with excess nitrogen, a hybrid approach of artificial neural networks and the artificial bee colony algorithm (ANN-ABC) was used to find the most interesting wavelengths for the subsequent processes. ABC is a heuristic optimization process proposed by Pham et al. [33]. This algorithm is inspired by the behavior of bees in their search for food sources [34]. The steps of the ABC algorithm are as follows:

1.
First, the initial responses (or candidate sites) are generated and evaluated.

2.
The better responses are selected and the scout bees are sent to those sites.

3.
The scout bees return to the hive with a waggle dance (producing a neighboring response).

4.
All the answers are compared and the best one is selected. 5.
The position of the best answer is saved. 6.
If the ideal conditions are not met, return to Step 2.
Following this scheme, in this study, different vectors of wavelengths were sent as the input to an artificial neural network, and the output of the network was the corresponding nitrogen content of the leaf (i.e., the number of days of application of excess nitrogen). For example, consider that the selected vector of wavelengths is (700, 800, 900 nm); this means that the ANN receives tuples of 3 values, corresponding to the light intensities in these spectral frequencies for a given pixel of the images. For each vector of wavelengths, a complete process of training the ANN and testing the trained network on the test set was done. Each vector was recorded together with the answer to this input, which was given by the mean squared error (MSE) produced by the ANN in the classification of the test set. The vector with the least MSE was then selected as the optimal input, and the related wavelengths were selected as the key wavelengths. The structure of the ANN used in this procedure is presented in Table 1. Table 1. Structure of hidden layers of the artificial neural network (ANN) used to select the key wavelengths in the artificial neural networks and the artificial bee colony algorithm (ANN-ABC) process.

Property Value
Number

Pre-Processing of the Spectral Data
As depicted in Figure 4, the original data may be affected by noise due to the light source, the effect of the ambient light, the camera and so on. Therefore, the data must be pre-processed [35]. In this study, reflective spectral data were first converted to absorption using the following equation: The light scattering was then corrected by a standard normal distribution (SNV) with wavelet detrending. Finally, the smoothing operation was performed with the Savitzki-Golay filter using ParLeS software, version 3.1 [36]. ParLeS is a specialized chemometrics software package that is used for multivariate modeling and forecasting, as shown in Figure 5.

Destructive Extraction of Nitrogen in the Laboratory
The process for extracting and measuring nitrogen in the leaves was a destructive laboratory procedure, using the classical Kjeldahl method [37]. Figure 6 depicts the equipment used in this process. The method consists of five steps:

1.
Powdering. The samples (leaves) must first be powdered. For this purpose, an oven (model BC OVEN 70, Behdad Co., Tehran, Iran) was used. It is made of stainless steel, resistant to high heat and has an electrical protection system, as shown in Figure 6a.

2.
Kjeldahl device. In order to measure the total leaf nitrogen, a Kjeldahl device (model VAP20, Gerhardt GmbH & Co., Königswinter, Germany) was used, as shown in Figure 6b. The Kjeldahl method proceeds with the steps of digestion, distillation and titration to determine total nitrogen. First, the sample is digested with sulfuric acid; next, the nitrogen of the sample is converted to ammonium sulfate. The nitrogen of ammonium sulfate is then released in the form of ammonia and converted to ammonium borate with boric acid, titrated using normal sulfuric acid 1% and, finally, the total nitrogen content of the sample is obtained by calculating the consumed acid.

3.
Digestion. The first step in the standard procedure to determine total nitrogen of leaf is digestion. In this study, a digester of model VAP20 (Gerhardt GmbH & Co., Königswinter, Germany) was used, as can be seen in Figure 6c. It has 12 digestion stands. The main characteristics of this device include safety, processing of fatty and inhomogeneous samples, the possibility of digestion of samples with very low volume, an automatic digestion system with temperature control, and an automatic digestion system with time control.

4.
Distillation. After the digestion step, a refrigerant was used to distill the sample. Distillation was performed in the shortest possible time by adding distilled water and NaOH at 32%. The heat required for distillation was supplied by constant-pressure steam, presented in Figure 6d.

5.
Titration. The last step is titration, which was carried out with a burette made of glass with a stopper and a valve at the tip to control the flow of the chemical solution.
After the titration step, the total nitrogen was obtained by calculating the acid consumption as: where Vs is the consumed volume of the sample, Vb is the consumed volume of the control treatment, md is the dry weight of the sample and N H2SO4 is the normality of sulfuric acid with 0.014 mEq.

Classification of the Days of Application of Nitrogen to the Pots Using ANN-ICA
The objective of the present study was to detect excess nitrogen consumption using hyperspectral images in the key wavelengths selected. For this purpose, the images of the leaves were classified using an ANN, whose output was the number of days of application of excess nitrogen, as previously described. In this way, the system was able to predict overapplication of nitrogen in the early stages. Various parameters had to be set in the configuration of the ANN, which greatly affected the accuracy achieved in the classification. These included the number of hidden layers, the number of neurons, transfer functions, the back-propagation network training function, and back-propagation weight/bias learning function. In our case, the imperialist competitive algorithm (ICA) algorithm was used to optimally adjust the parameters of the ANN. ICA is a metaheuristic algorithm based on cultural, social and political evolution. In this algorithm, all countries are looking for the general optimal point to solve an optimization problem [38,39]. As in the hybrid ANN-ABC approach, ICA controls the execution of the ANN under different configurations, combining the aforementioned parameters of the ANN and selecting the optimal configuration of the network. Specifically, the number of hidden layers can be between 1 and 3. The number of neurons in each hidden layer can be between 0 (no hidden layer) and 25. The transfer function for each layer was selected from a set of 8 transfer functions available in the neural network toolkit of MATLAB (MathWorks, Natick, MA, USA) such as tangential sigmoid, radial basis and softmax. The back-propagation training function was selected from a set of 6 functions, and the back-propagation weight/bias learning function was selected from a set of 15 different functions available in the same toolkit.
During the ANN-ICA process, the method considers a vector with a selection of the parameters, representing a specific configuration of the ANN. For example, the vector x = [13, logsig, traingdx, learnos] indicates that the ANN has 1 hidden layer with 13 neurons, a log-sigmoid transfer function, gradient descent with momentum back-propagation, and the Outstar weight/bias learning function, respectively. The MSE of each vector obtained in the training/test process of the ANN was measured and the vectors with the lowest MSE were selected as the optimal configuration of the network. Table 2 shows this selection of the parameters obtained by ANN-ICA. After selecting the optimal structure of the ANN, 200 iterations were executed to evaluate the validity of the classifier. In all the iterations, 60% of the total data were randomly selected for training, 10% for validation and the remaining 30% for testing.

Evaluation of the Performance of the ANN-ICA Classifier
Different criteria are commonly applied to evaluate the performance of a classifier. Two groups can be distinguished. The first category includes the performance parameters extracted from the confusion matrix, i.e., recall, accuracy, specificity, precision and the F-measure. The second category is given by the graphical criteria associated with the receiver operating characteristic curve (ROC) and the area under the ROC curve (AUC) [2]. Table 3 contains the definitions of the parameters in the first group.

Name Equation Description
Recall Harmonic weighted average of recall and precision

Key Wavelengths for Classifying Leaves from Different Days
The key wavelengths for early and non-destructive detection of leaves with excess nitrogen were obtained using the ANN-ABC approach described in Section 2.3. The selected wavelengths were 715, 783 and 821 nm. Figure 7 presents some samples of the hyperspectral images of the selected wavelengths for different days.

Performance of ANN-ICA for the Classification of Leaves with Excess Nitrogen
As described in Section 2.6, the hybrid ANN-ICA method was used to predict the number of days when the leaves began to receive excess nitrogen using the images of the selected wavelengths. Table 4 presents the performance of the classifier using the confusion matrix, the accuracy or correct classification rate (CCR) and the error rate for the 200 iterations of the training/test process. This makes an equivalent of 155,000 test samples. Only 6024 of these samples were incorrectly classified in the wrong class. This number of misclassifications resulted in a total accuracy of 96.11%.  Table 5 evaluates the same results using the five performance criteria, namely recall, accuracy, specificity, precision and F-measure. A recall of 100% means that none of the samples was incorrectly classified in the given class; thus, the low recall obtained for D1 (94.18%) means that most of the samples were incorrectly classified in class D1. The reason is that in the early hours, it is more difficult to identify excess nitrogen than in the following days. According to the table, it is obvious that more samples were classified correctly in class D3 than in other classes, since the accuracy of class D3 was the highest (98.87%). In fact, on the third day, the leaves reacted clearly to the excess nitrogen and it was easy to detect them visually. The value of specificity was 99.14% for class D2, which was higher than the others. It means that class D2 had the least error. The accuracy was 94.76% for class D3, which was the lowest value among all classes, indicating that many samples were incorrectly classified in other classes. Finally, since the F-measure is the weighted harmonic mean of recall and accuracy, it can be understood that the value for class D0 was higher than the others.  Figure 8 represents the performance of the classifier using box plots of CCR and AUC for the 200 repetitions of the process. According to the results in Figure 8b, the box plot diagrams of AUCs are compact for all classes, indicating the high performance and stability of this classifier. It is also evident from the box plots that although the excess nitrogen could be detected on the first day after applying it (D1), from the second day onwards (D2 and D3), the AUC charts increased. This indicates that the detection algorithm had higher performance on D2 and D3 than on D1, as expected. However, contrary to expectations, the AUC of D2 was higher than that of D3. This could be due to newly sprouted leaves on the third day, which, compared with the older leaves, produced errors. However, these errors were not large enough to cause statistically significant differences.

Performance of the Classifier in the Best Training Case
From the 200 repetitions of the training/test process, we considered it interesting to extract the results of the best case. The performance measures for this specific case are presented in Tables 6 and 7 and in Figure 10.   Table 8 compares the performance of the ANN-ICA classifier using the mean and standard deviation of the CCR and AUC for the 200 repetitions and the best training case. As shown, the CCR and AUCs in the best training case indicate that the proposed algorithm was able to detect excess nitrogen early.

Statistical Analysis of the Key Wavelengths Using ANOVA and Tukey's Test
To ensure a reliable classification of the proposed algorithm, a statistical analysis was performed on the effective spectrum data using ANOVA and the Tukey test. The null hypothesis (H 0 ) was that the spectral images of the key wavelengths were equal for all the days of treatment, and the alternative hypothesis (H a ) was that they were different. While the ANOVA tests only inform whether there are some differences in the means, the Tukey test analyzes which means are different from the others. For conciseness, the results of these tests are shown in Tables A1-A6 in Appendix A.
These statistical tests also included an analysis of the values of the nitrogen content of the leaves obtained in the laboratory using the destructive Kjeldahl method. In this case, H 0 was that the nitrogen content was equal in all the treatments, while H a was that they were different. The results are presented in Tables A7 and A8.
According to the obtained results, it is evident that there was a statistically significant difference between all the classes (days of treatment), both in the spectral images and in the nitrogen content, which means that it was possible to identify the plants with excess nitrogen consumption immediately after 1 day of applying it. These results are consistent with the results of the proposed classification algorithm; thus, the conclusions of the experimental results can be trusted.

Sample Images and Comparison with Other Works
Some examples of the hyperspectral images of two classes of leaves with normal nitrogen fertilizer (D0) and leaves with excess nitrogen (D1) at two spectral ranges of visible (Vis) and near-infrared (NIR) light are shown in Figure 11. As can be seen, on the first day after applying excess nitrogen, the contaminated pixels were not detectable in the visible range. However, at the NIR range there were white pixels clearly visible at the edges of the leaf that represented excess nitrogen consumption. In other words, if classical spectral analysis were applied to the entire leaf, the detection of overapplication would be done at a later stage, since the negative effects start at the tips of the leaves. By using spectral images, the problems can be detected earlier.
After assessing the performance of the proposed algorithm, the results were compared with other methods conducted by different researchers. Table 9 compares the results of these methods using the reported values of classification accuracy (or CCR). It must be observed that the problems addressed in these papers and the product types are different, since no comparable works have been found in the literature regarding the detection of excess nitrogen in cucumber leaves. Therefore, this information is offered to compare our results with other state-of-the-art work in HSI processing. The accuracy of the proposed method, above 96%, is comparable with or better than most of these previous techniques. Another important advantage of HSI with respect to traditional spectral analysis is that it enables automation of the image acquisition process. Although in the current experiment, the images were obtained in a capture chamber, it is possible to mount such cameras on drones or satellites. In this way, after obtaining the images, a leaf detector could be applied to find the leaves and then analyze the nitrogen content on each one of them. This is not possible if spatial information is not available. However, the problems derived from natural lighting and uncontrolled conditions should be addressed, which pose a challenge in NIR spectral analysis.

Conclusions
In recent years, agricultural producers have increased the use of chemical fertilizers per unit of area, instead of using modern agricultural knowledge to produce more. The illusion of increasing yield due to the use of more water and chemical fertilizers has led to the excessive use of water and fertilizer resources. The continuation of these practices has led, in some countries, to situations of soil and water contamination, in addition to additional expenses.
Hyperspectral imaging (HSI) has been recognized as a non-destructive and rapid analytical tool for evaluating product quality and diagnosing diseases using new technological advances in data measurement. In the present study, a new HSI methodology was proposed to evaluate early detection of excess nitrogen consumption in cucumber plants, complemented with computational intelligence methods using hybrid approaches of artificial neural networks (ANN) and metaheuristic algorithms (ABC and ICA). First, using the hybrid ANN-ABC algorithm, the wavelengths of 715, 773 and 821 nm were determined as the key wavelengths to analyze nitrogen in the leaves. The hybrid approach ANN-ICA was then applied to predict the number of days of treatment using the HSI data in the selected wavelengths.
The performance of the proposed classifier was evaluated using the confusion matrix and the ROC curves. The total correct classification rate achieved was 96.11%, indicating a good ability to detect excess nitrogen in cucumber plants. The statistical analysis showed that the differences between the images of the selected wavelengths for the different days of treatment were statistically significant.
By early detection of excess fertilizer, nitrogen-rich soil can be improved. One of the ways to improve the soil with excess nitrogen is by applying mulch. The nitrogen-rich soil of mulch decomposes large amounts of soil nitrogen. The proposed technique can be used to adjust the use of nitrogen to the optimum values, thus reducing the use of fertilizers and the risk of contamination.
One of the limitations of the proposed method is that it requires specialized imaging equipment, although the processing can be done on an average computer. The high cost of this equipment would not be affordable for most farmers. In the future, with the development of specialized cameras in the selected wavelengths, it will be possible to estimate the nitrogen content of the plants using images taken in the field, thus maintaining it at optimal levels. For this purpose, new challenges derived from the use of natural lighting conditions should be addressed. Another future research line would be to study different levels of nitrogen lower than 30% so that the system would be more sensitive to detect small excesses in the use of fertilizers.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author. The data are not publicly available since they are partially owned by the laboratory where the analyzes were carried out.

Conflicts of Interest:
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analysis, or interpretation of data; in the writing of the manuscript or in the decision to publish the results.

Appendix A
This appendix contains the results of the statistical tests used to analyze the differences in the images under the different treatments. The null hypothesis (H 0 ) was that the images of cucumber leaves of all the treatments were equal, considering the key wavelengths selected in the ANN-ABC process: 715, 783 and 821 nm. The treatment was the number of days of application of excess nitrogen (0, 1, 2 and 3). The alternative hypothesis (H a ) was that they were different. Two tests were applied for each wavelength: the ANOVA test to analyze if all the treatments were equal and Tukey's honestly significant difference (HSD) test to analyze the statistical significance of the differences between each pair of treatments. Table A1. Statistical analysis of ANOVA for the key wavelength of 715 nm for the four excess nitrogen treatments.   The same statistical tests (ANOVA and Tukey's HSD) were also applied for the nitrogen content of the cucumber leaves measured in laboratory using the destructive Kjeldahl method for the four treatments. Again, the null hypothesis (H 0 ) was that the nitrogen content of the leaves was the same for all the treatments, and the alternative hypothesis (H a ) was that they were different. Table A7. Statistical analysis of ANOVA for the nitrogen content of the leaves for the four treatments.