Assessment of Injury by Four Major Pests in Soybean Plants Using Hyperspectral Proximal Imaging

Abstract

Arthropod pests are among the major problems in soybean production and regular field sampling is required as a basis for decision-making for control. However, traditional sampling methods are laborious and time-consuming. Therefore, our goal is to evaluate hyperspectral remote sensing as a tool to establish reflectance patterns from soybean plants infested by various densities of two species of stinkbugs (Euschistus heros and Diceraeus melacanthus (Hemiptera: Pentatomidae)) and two species of caterpillars (Spodoptera eridania and Chrysodeixis includens (Lepidoptera: Noctuidae)). Bioassays were carried out in greenhouses with potted plants placed in cages with 5 plants infested with 0, 2, 5, and 10 insects. Plants were classified according to their reflectance, based on the acquisition of spectral data before and after infestation, using a hyperspectral push-broom spectral camera. Infestation by stinkbugs did not cause significative differences in the reflectance patterns of infested or non-infested plants. In contrast, caterpillars caused changes in the reflectance patterns, which were classified using a deep-learning approach based on a multilayer perceptron artificial neural network. High accuracies were achieved when the models classified low (0 + 2) or high (5 + 10) infestation and presence or absence of insects. This study provides an initial assessment to apply a non-invasive detection method to monitor caterpillars in soybean before causing economic damage.

1. Introduction

Soybean (Glycine max (L.) Merrill) is one of the most important crops worldwide and the primary source of protein and vegetable oil demanded by the world’s population. Since the 1970s, no other crop has increased in area as large as soybean, which now occupies around 6% of the world’s agricultural land [1]. In the 2020/21 growing season, soybean grown in South and North America was responsible for 80% of the 361 million tons produced worldwide. Nowadays, Brazil is the leading global soybean producer, with 138 million tons produced in 39 million hectares, followed by the United States [2,3].

Due to the severe attack of pests, such as weeds, plant pathogens, nematodes, insects, and mites, Brazilian agriculture faces systematic and high economic losses [4]. It has been estimated that insect pests alone cause annual losses of USD 12 billion [5]. This potential damage by insects is boosted in crops grown in tropical and subtropical climates because of the weather conditions favorable to insect development. In addition, the cropping system used in Brazil and other South American countries is based on the rotation and succession of plant species on a large scale, increasing the availability of food resources for insect pests [6].

A complex of defoliating caterpillars and stinkbugs are the major economic pests that damage the soybean in the Neotropical region [7]. In Brazil, the main defoliating caterpillar is the soybean looper Chrysodeixis includens (Walker) (Lepidoptera: Noctuidae) [8]. Other caterpillars of the genus Spodoptera (Guenée), such as southern armyworm Spodoptera eridania (Stoll) (Lepidoptera: Noctuidae), may also cause significant damage to the soybean due to their high density in fields, in both Bt (genetically modified crops with genes of Bacillus thuringiensis) and non-Bt soybean fields, and injury potential on leaves, flowers, and pods [9].

Regarding stink bugs, the brown stink bug Euschistus heros (Fabricius) (Hemiptera: Pentatomidae) is the most abundant species in Brazil [10]. Moreover, more recently, other sucking pests within the Pentatomidae have come to be considered important soybean pests. The polyphagous green-belly stink bug Diceraeus melacanthus (Dallas) (Hemiptera: Pentatomidae) has become more abundant and significant as a key pest, mainly in the soybean–maize production system [11].

Insect control in soybean fields is mainly based on chemical control; on a calendar basis, the pesticide sprays are planned with no field inspection to monitor pest population and control thresholds [12]. This means that no Integrated Pest Management (IPM) concepts are considered in the process of decision-making for control. Consequently, control failures and reapplication of pesticides are constant, leading to many problems, such as pesticide residuals in the food, field workers being intoxicated, selection of resistant populations of insect pests, and an imbalance in the populations of beneficial insects [13,14,15].

IPM has proven to be an efficient solution to this problematic scenario. It is based on frequent field inspections to monitor pest populations and uses various control tools only when economic thresholds are reached. However, monitoring becomes challenging, expensive, and time-consuming in large fields when using traditional sampling methods and tools. The rapid and unpredictable growth of insect pest populations, such as caterpillars and stink bugs, require frequent sampling as longer sampling intervals can lead to misinformation. For example, a population at a safe and controlled level initially may increase in density in a short period and reach a control threshold sooner than expected. In addition, factors such as the increase in the size of plots, difficulty in hiring personnel, and the high cost of training and maintaining teams that monitor the fields represent some of the reasons for the lack of willingness of producers to adopt IPM [16,17].

The development of advanced electronics, Global Navigation Satellite System (GNSS), and Geographic Information System (GIS), combined with the science of remote sensing, allowed significant advances in the adoption of precision agriculture. Irrigation, fertilization, yield mapping, and weed detection represent some agricultural practices transformed by remote sensing [18]. Furthermore, the recent technological advances in imaging and sensors have shown new opportunities for use in the automated monitoring of insect pests and/or the damages associated with them, optimizing the use of pesticides and reducing yield losses in large-scale agriculture [19,20]. The remote detection of insect-pest by imaging sensors is possible because the stress caused by the herbivory interferes with photosynthesis and the physical structures of plants, hampering the absorption of luminous energy and altering the plant’s reflectance spectrum [21]; this spectrum is impacted by the type, moment, and intensity of herbivory, as well as on the plant tissues and organs injured [22]. Hence, to obtain a reliable measurement of a soybean plant’s reflectance spectra, it is imperative to study how the plants in different phenological stages respond to damage caused by various important pest species, such as caterpillars (leaf, flower, and pod chewers) and stinkbugs (pod and seed suckers).

The use of hyperspectral images to detect insect pests is a recent approach, showing high potential to identify and classify insect infestation levels [23]. Hyperspectral images contain various spatial and spectral information from the plant tissue, allowing precise and efficient detection of spectral alterations caused by insect damage. However, each sampling with hyperspectral sensors results in a large amount of data information. This makes the data collection, storage, and transferring not trivial and, more importantly, the analysis process to mine valuable information from the samples is challenging [19]. In this sense, machine learning techniques have been successfully used to analyze hyperspectral images [24] and deep learning models recently became popular to identify and classify pest and disease levels in agricultural fields [25,26,27,28,29]. For example, multilayer perceptron neural networks can be used with these same goals as they represent a powerful deep learning tool for high-performance modeling of complex problems and have been proven efficient in hyperspectral pest image classification [30].

The fundamental goal of this study was to examine the spectral patterns of soybean plants infested by major soybean lepidopteran (C. includens and S. eridania) and hemipteran (D. melacanthus and E. heros) pests using hyperspectral imaging. Our study also explored the application of advanced deep learning models, such as multilayer perceptron neural networks, to discriminate soybean plants under different situations of stress, which included a combination of (i) vegetative and reproductive soybean stages; (ii) pest infestation levels; and (iii) time of feeding (days after infestation). To the best of our knowledge, this is the first attempt to shed light on the spectral responses of soybean infested by its major pests using hyperspectral imaging and deep learning approaches.

2. Material and Methods

2.1. Insects

Following the methodology described by Bueno et al. [31], caterpillars of C. includens and S. eridania were kept individualized in glass tubes (8 × 2 cm) and fed with an artificial diet developed by Greene et al. [32], based on beans, brewer’s yeast, casein, soy protein, and wheat germ. Upon reaching adulthood, the insects were transferred to cages made of PVC tubes (21.5 cm in height and 10 cm in diameter) to obtain the eggs.

Adults were fed with a 10% aqueous solution of honey. The eggs were collected daily and placed in plastic cups (16.5 × 5 cm) until the caterpillars hatched. From the second instar, the caterpillars were individualized and maintained the same way described in the last paragraph.

The populations were kept at the Insect Biology Laboratory of the Department of Entomology and Acarology at USP/ESALQ, in a room with standard rearing conditions at 25 ± 2 °C, 70 ± 10% RH, and a 14:10 h light:dark photoperiod.

The stink bugs E. heros and D. melachantus were reared with a natural diet, based on green beans (Phaseolus vulgaris), dry peanut beans (Arachis hypogaea), dry soybean beans (Glycine max), sunflower seeds (Helianthus annuum), and water provided in hydrophilic cotton. The insects were maintained in cages (36.5 cm in length, 25.5 cm in width, and 14.4 cm in height) with a mesh cover of the “organza”-type fabric. Eggs were collected every three days and transferred to Petri dishes (4.5 cm radius). After the nymphs hatched, they were maintained in the Petri dishes until the second instar, when they were transferred to the cages previously described. The cages were cleaned, and the diet was replaced with the same frequency as the egg collection.

The populations were kept at the Integrated Pest Management Laboratory of the Department of Entomology and Acarology at USP/ESALQ, in a room with standard rearing conditions at 25 ± 2 °C, 70 ± 10% RH, and a 14:10 h light:dark photoperiod.

2.2. Bioassay

In a greenhouse, soybean plants of the variety “BMX Potência RR” were cultivated in 10 L pots containing soil as a substrate, with five plants per pot being maintained. Nutrients necessary for plant development were supplied via soil seven days before sowing, and the plants were irrigated daily.

For the exposure of plants to pest attack, cages suitable for infestation were used in a metallic structure, covered with a cylindrical-shaped fabric (120 cm high and 25 cm in diameter), in which one pot with five soy plants was placed, as well as the insects that were the target of this study (Figure 1A,B).

The experimental design was completely randomized, with 15 repetitions of 4 treatments. Four infestation levels were evaluated for each insect species: zero, two, five, and ten insects (introduced in the second instar for the caterpillars and as adults for the stinkbugs) per pot with five plants, totaling 60 experimental units. The infestation was maintained for 10 days, after which the insects were removed manually.

For the assays with the caterpillars C. includens and S. eridania, the infestation was tested at two different times during the development of the crop (in different trials): one time during the vegetative period, starting at the phenological stage V3 stage; and one time at the reproductive period, starting at the phenological stage R3 stage [33] (Figure 1C). For the assays with the stinkbugs E. heros and D. melacanthus, the infestation was only tested during the reproductive period, starting at the R3 stage [33] (Figure 1D).

2.3. Data Collection

Based on the acquisition of spectral data before and after infestation, plants were classified according to their reflectance using a hyperspectral pushbroom spectral camera (PIKA L, Resonon Inc., Bozeman, MT, USA) with a 23 mm objective lens. This sensor collects spectral data in the range from 400 to 1000 nm (Spectral Range), 281 bands (Spectral Channels), 3 nm (Spectral Resolution), 2.1 nm (Spectral Bandwidth), and 900 spatial pixels per line (Figure 1E). Hyperspectral data were collected in a dark room with artificial lighting mounted on a tower from 15 W, 12 V LED light bulbs mounted in two angled rows, one on either side of the lens, with two bulbs in each row. A voltage stabilizer (Type PR-7b, Tripp-Lite, Chicago, IL, USA) was used to stabilize the power supply to the lighting system. A polyethylene plastic board (Type 822, Spectronon Pro, Resonon, Bozeman, MT, USA) was used for white calibration, and the lens had its cover for dark calibration. Then, the relative reflectance of the samples could be calculated based on white and dark calibration.

Data prior to the infestation were acquired moments before the release of the insects, and data after infestation were collected five and ten days after the release of the insects into the cages. Data were obtained from leaves collected in the greenhouse and taken to the laboratory within a few minutes. From each cage, one trifoliated leaf was collected from the top layer of the plants. To collect the spectral data, each leaflet was removed from the trifoliated leaves and placed on a linear platform below the sensor, which moves automatically, performing the data acquisition controlled by the Spectronon software (Resonon Inc., Bozeman, MT, USA). Each leaflet was positioned on the platform with its adaxial face facing the sensor (Figure 1F).

2.4. Data Analysis

Because of the large amount of information collected in hyperspectral sampling, multivariate statistics are the best tools to analyze data from vegetation under stress [18]. In this study, hyperspectral data were manually extracted from each data cube referring to the spectral sample of each leaflet. These data correspond to the mean reflectance value of the area of interest (total leaf area) for each of the 281 bands in each of the leaflets. Data were then submitted to a principal component analysis (PCA) that has been reported in several studies regarding hyperspectral data because of the high autocorrelation of the adjacent bands [17,34]. The PCA indicated which bands had greater relevance in the classification of the samples according to an ordinal ranking of the contribution values for the PCA axes.

For each sampling date, the 28 highest ranked bands among the 281 bands were selected and subjected to a new set of analyses. These 28 bands represent about 10% of the total bands. Then, an analysis of variance (two-way ANOVA) was performed for each of the 28 bands, comparing the mean values of reflectance for each level of infestation for all the pest species tested.

For the data acquired in the assays with caterpillars, a deep-learning approach based on multilayer perceptron artificial neural network (MLP-ANN) was used for classifying the datasets regarding the following parameters (situation of stress): plant phenology, infestation period, infestation level, and insect species. The neural networks used optimized parameters to maximize the classification accuracy (

Table 1. Overview of the hyperparameter space used in the optimization of the architecture.

State	Hyperparameters	Values
Optimized	Activation functions	{Tanh, Relu}
	Optimizer	{Adam, SGD}
	Learning rate	Adaptative
	Number of hidden layer	{1, 2}
	Hidden layer size	{50, 100}

). The dataset was randomly divided into two parts: 70% was used for training, and the remaining 30% was used for testing. Machine learning approaches have been largely used to analyze hyperspectral data because of their capacity to process high-dimensional data and adapt the models’ behavior to the specific characteristics of each dataset [35]. All the analyses were done using the software R and Python [36].

3. Results

3.1. Defoliation

Significant differences were found in soybean defoliation rates under progressive levels of caterpillar infestations in the vegetative and reproductive soybean stages (

Table 2. Soybean leaf consumption by Chrysodeixis includens and Spodoptera eridania in different population densities.

Caterpillars (n Five Plants−1)	Leaf Area in cm2 (Leaf Area Index)
	C.i.	S.e.	df; F; p; CV% #
Soybean vegetative stage
0	34.7 ± 2.0 † a,A (6.14)	34.6 ± 1.9 a,A (6.12)	1; 0.0002; 0.98; 22.16
2	24.7 ± 2.2 b,A (4.38)	29.5 ± 1.9 b,A (5.23)	1; 2.64; 0.11; 29.84
5	15.4 ± 2.6 c,B (2.74)	25.1 ± 1.2 b,A (4.45)	1; 11.38; 0.002; 38.63
10	4.2 ± 1.9 d,B (0.75)	19.1 ± 0.9 c,A (3.39)	1; 47.41; <0.001; 50.63
df; F; p; CV% *	3; 34.62; <0.001; 43.22	3; 17.35; <0.001; 22.58	--
Soybean reproductive stage
0	35.2 ± 3.8 a,A (6.23)	34.4 ± 2.1 a,A (6.09)	1; 0.03; 0.86; 34.22
2	23.00 ± 2.1 b,A (4.07)	27.5 ± 1.6 b,A (4.87)	1; 2.89; 0.09; 29.05
5	10.2 ± 2.1 c,B (1.81)	22.2 ± 1.2 b,A (3.93)	1; 22.81; <0.001; 42.43
10	1.4 ± 0.9 d,B (0.25)	12.4 ± 0.5 c,A (2.19)	1; 108.16; <0.001; 42.01
df; F; p; CV% *	3; 35.83; <0.001; 34.83	3; 37.96; <0.001; 24.25	--

C.i. = Chrysodeixis includens. S.e. = Spodoptera eridania. * Mean values followed by the same lowercase letter in the columns * and by the same uppercase letter in the rows ^# do not differ significantly, as per the ^* Scott-Knott or ^# t-tests (p < 0.05). ^† Sum of leaves individual area/soil area occupied by one plant.

). In the bioassay with C. includens, the average leaf areas differed significantly from the control treatment (no infestation) under densities of two, five, and ten caterpillars per cage with five plants, both at the vegetative (Veg.: df = 3, F = 34.62, p ≤ 0.001) and reproductive (Rep.: df = 3, F = 35.83, p ≤ 0.001) soybean stages (Table 1). The highest reduction in the leaf area occurred at the level of ten caterpillars per cage (Veg.: 4.25 cm²; Rep.: 1.39 cm²), in which leaf area was significantly lower than leaf area at levels of 2 (Veg.: 24.75 cm²; Rep.: 22.99 cm²) and 5 (Veg.: 15.48 cm²; Rep.: 10.22 cm²) caterpillars per cage. Leaf Area Index (LAI) also was affected by levels of C. includens infestations; LAI was reduced from 6.14 to 0.75 and 6.23 to 0.25 at the maximum infestation level compared to the non-infested treatment, respectively, in the vegetative and reproductive soybean stages (Table 2).

Similar results were obtained in the bioassay with S. eridania. The average leaf areas in all the infested treatments differed significantly from the control, both the vegetative (Veg.: df = 3, F = 17.35, p ≤ 0.001) and reproductive (Rep.: df = 3, F = 37.96, p ≤ 0.001) soybean stages (Table 2). The most significant reduction in the leaf area was observed at the level of 10 caterpillars per cage (Veg.: 19.15 cm²; Rep.: 12.37 cm²), which differed from leaf areas of infestation levels two and five caterpillars per cage. The average leaf areas in levels of two (Veg.: 29.56 cm²; Rep.: 27.55 cm²) and five (Veg.: 25.15 cm²; Rep.: 22.24 cm²) caterpillars per cage did not differ significantly from each other. LAI varied among treatments and was reduced from 6.12 to 3.39 and 6.09 to 2.19 at the maximum infestation level compared to the non-infested treatment, respectively, in the vegetative and reproductive soybean stages (Table 2).

In addition, significant differences were found comparing the defoliation rates between the caterpillar species at the same infestation level in each soybean stage (Table 2). Although the average leaf areas did not differ significantly among the caterpillars at the level of two caterpillars per cage with five plants, the defoliation rates from C. includens were higher than from S. eridania at the levels of 5 (Veg.: df = 1, F = 11.38, p = 0.002; Rep.: df = 1, F = 22.81, p ≤ 0.001) and 10 (Veg.: df = 1, F = 47.41, p ≤ 0.001; Rep.: df = 1, F = 108.16, p ≤ 0.001) caterpillars per cage, regardless of the soybean phenological stage. At the level of ten caterpillars per cage, C. includens reduced the leaf area up to nine times more than S. eridania (Table 2).

3.2. Reflectance Patterns

3.2.1. Stinkbugs

Regarding infestation by stinkbugs, E. heros and D. melacanthus, no visual difference was observed in the reflectance patterns of infested or non-infested soybean plants, even at high infestation levels of ten insects per five plants for ten days (Figure 2). These results show that the damage caused by stinkbugs feeding in soybean plants does not change the physiological or physical features in the leaf enough to cause differences in the amount of energy reflected in the visible or near-infrared (NIR) regions.

3.2.2. Caterpillars

The damage caused by C. includens led to differences in the reflectance patterns of soybean leaves, in the vegetative and reproductive periods. On day 0, when the plants were not infested yet, the spectral curves from the four infestation levels were similar, as expected (Figure 3).

On day 5, it is possible to observe a similar reflectance pattern in the two periods. In the visible region, no difference can be observed between the treatments. On the other hand, in part of the NIR region (900–1000 nm), two groups were created: one composed of the curves from plants with zero and two insects per cage with five plants with lower reflectance, and one composed of the curves form plants with five and ten insects per cage, with higher reflectance.

On day 10, a reflectance pattern was also observed in the two periods. The spectral curves from plants with ten insects per cage were separated from the other curves in parts of the visible (550–650 nm) and NIR (950–1000 nm) regions, with higher reflectance; also, at the vegetative period, the spectral curve from plants with ten insects per cage has lower reflectance in the red-edge region (680–730 nm) inflection than the other curves. The lower reflectance at the red edge, combined with higher reflection at the red region, is indicative that possibly the index made of the combination of some bands in the spectral region could be used to monitor this stress, such as the Normalized Difference Vegetative Index (NDVI) that has been used for other monitoring other stressors.

The damage caused by S. eridania in the vegetative stage of soybean plants was lower than that caused by C. includens. Analyzing soybean foliar area data at this stage, it can be estimated that the highest infestation level of S. eridania reduced about 45% of the foliar area while C. includens reduced about 90% of the area. This difference in leaf consumption can also be observed in the reflectance patterns of soybean plants infested with each species. Both at five and ten days after infestation, the visual difference in the reflectance curves of each infestation level was less intense in S. eridania than in C. includens.

After five days of infestation, a slight difference between the curves can be seen at the final portion of the infrared region (950–1000 nm). Then, after ten days of infestation, the difference in intensity in the same spectral region is more intense (Figure 4).

The leaf consumption by S. eridania was higher in the reproductive stage than in the vegetative stage of soybean. This was also observed in the spectral curves. Visually analyzing the curves at five and ten days after infestation in the reproductive stage, the differences in the curves from each infestation level are more intense than that in the vegetative stage.

3.3. Principal Component Analysis and Analysis of Variance

3.3.1. Stinkbugs

The data collected in the assays with E. heros were submitted to PCA, where the individual contribution information of the bands was calculated, and the bands ranked. On day 0, the regions with the highest contribution were 738–774 nm and 921–945 nm. After five days of infestation, the regions were 525–562 nm and 703–720 nm. Finally, after ten days of infestation, the regions were 502–511 nm, 610–646 nm, and 746–755. ANOVA was performed on 28 bands from these regions with the highest contribution, and no significant difference was observed between the mean values of reflectance from each infestation level in any of these bands (p > 0.05) (Table S1, Supplementary Material).

The same results were observed for D. melacanthus. For the three days of data collection, the bands of greatest contribution were in the range of 735 to 872 nm. Again, no significant difference was observed when comparing the mean reflectance values of each infestation level in any of these bands (p > 0.05) (Table S2, Supplementary Material).

3.3.2. Caterpillars

Regarding the soybean reflectance data collected in the assays with the S. eridania and C. includens caterpillars, significant differences were observed when comparing the mean reflectance values of each infestation level at both of the soybean’s development stages.

In all the assays, at day 0, no significant difference was observed between the infestation levels at any band (p > 0.05) (Tables S3 and S4, Supplementary file). Then, on days 5 and 10, for each species, different groups were formed (p< 0.05) (Tables S3 and S4, Supplementary Material). The variability in grouping results is due to the different amounts of damage caused by each species.

Analyzing the data collected in the assays with C. includens, after five days of infestation in the vegetative stage, the spectral region with the greatest contribution was 892–952 nm, where ten of the 28 bands showed a significant difference between the reflectance of the infestation levels (p < 0.05), separating two groups: the first composed by plants with zero and two caterpillars per cage, and the second by plants with five and ten caterpillars per cage (Table S3, Supplementary file). Then, after ten days of infestation, the most significant regions were 579–629 nm and 695–699 nm. There were differences between the reflectance values in all bands (p < 0.05), being formed three groups: one with plants infested by zero and two caterpillars per cage, one with plants with five caterpillars per cage, and the last with plants with ten caterpillars per cage. This can be translated in practical terms as low, medium, and high infestation.

The assays with C. includens in the reproductive stage showed a very similar pattern as the one in the vegetative stage. On day 5, the most important spectral regions were 848–910 nm, but no difference was observed between the reflectance values of each infestation level (p > 0.05) (Table S4, Supplementary Material). However, four bands in the visible region (511–517 nm) showed a significant difference (p < 0.05), and two groups were formed: one with plants with zero and two caterpillars per cage and the other with plants with five and ten caterpillars cage. On day 10, all highest contributing bands were in the visible region (445–500 nm) and had significant differences between the reflectance values (p < 0.05). Despite four bands (490–496 nm) that showed a group with plants with zero and five insects per cage, in all other bands, four distinct groups were created, corresponding to each infestation level.

Spodoptera eridania causes more damage in the reproductive stage than in the vegetative stage of soybean. Hence, by analyzing the spectral behavior of soybean plants after five days of infestation, it is possible to see different patterns in the two development stages. In the vegetative stage, two spectral regions were more significant: the first at 700–815 nm, where no difference between the average reflectance in each infestation level was observed (p > 0.05) (Table S5, Supplementary Material). In the second region, 879–901 nm, two groups were formed regarding the average reflectance (p < 0.05): one group formed by plants with no insects and one group formed by plants with insects (two, five, and ten insects per cage). On the other hand, in the reproductive stage, all the principal bands in three regions (642–690 nm, 733–742 nm, and 899–910 nm) showed a significant difference in the average reflectance (p < 0.05), separating plants with a high infestation (ten insects per cage) from the other plants (zero, two and five insects per cage) (Table S6, Supplementary file).

On day 10, in the vegetative stage, there were two main regions, 716–742 nm and 919–952 nm, and all the bands in the last region showed a significant difference in the average reflectance (p < 0.05), separating plants with no insects from plants with insects (2, 5, and 10 insects per cage). The same groups were observed in the reproductive stage (p < 0.05), and all the most significant bands were in the spectral region of 792–850 nm.

3.4. Results of Multilayer Perceptron Artificial Neural Network

The MLP-ANN was used to automatically discriminate the health conditions of soybean plants regarding the following situations of stress: soybean phenology (vegetative (V) and (+) or (;) reproductive (R) stages), infestation period (5 or (;) 10 days after caterpillar infestations (DAI)), and combinations of different caterpillar densities (0, 2, 5, and 10 caterpillars (C. includens and/or S. eridania) per cage with five plants), which generated eight situations of stress (

Table 3. Results for accuracy, Kappa coefficient, precision, recall, and F1 score for the testing and training datasets for discrimination of soybean plants under different situations of stress caused by C. includens and S. eridania based on multilayer perceptron artificial neural networks.

Situation of Stress d			Accuracy (%)	Kappa Coefficient	Precision	Recall (Sensitivity)	F1 Score
Phenology a	DAI b	Caterpillars (n) c
Testing dataset
1- V; R	5; 10	0; 2; 5; 10	47.0	0.46	0.48	0.48	0.48
2- V; R	5; 10	0; 2; 5 + 10	59.0	0.58	0.59	0.56	0.56
3- V; R	5; 10	0 + 2; 5 + 10	70.0	0.67	0.69	0.71	0.70
4- V; R	5; 10	0; 2 + 5 + 10	74.0	0.73	0.70	0.72	0.71
5- V + R	5; 10	0; 2; 5; 10	50.0	0.48	0.50	0.51	0.51
6- V + R	5; 10	0; 2; 5 + 10	56.0	0.53	0.51	0.50	0.50
7- V + R	5; 10	0 + 2; 5 + 10	66.0	0.63	0.66	0.66	0.66
8- V + R	5; 10	0; 2 + 5 + 10	71.0	0.67	0.65	0.65	0.65
Training dataset
1- V; R	5; 10	0; 2; 5; 10	49.8	0.49	0.51	0.50	0.49
2- V; R	5; 10	0; 2; 5 + 10	61.4	0.60	0.59	0.57	0.57
3- V; R	5; 10	0 + 2; 5 + 10	69.3	0.68	0.70	0.69	0.69
4- V; R	5; 10	0; 2 + 5 + 10	76.9	0.76	0.73	0.69	0.70
5- V + R	5; 10	0; 2; 5; 10	49.5	0.47	0.50	0.50	0.49
6- V + R	5; 10	0; 2; 5 + 10	59.0	0.56	0.56	0.55	0.55
7- V + R	5; 10	0 + 2; 5 + 10	68.2	0.65	0.68	0.68	0.68
8- V + R	5; 10	0; 2 + 5 + 10	75.1	0.72	0.71	0.66	0.67

(;) Indicates that it is one factor or the other. (+) Indicates that it is a combination of the factors. ^a Soybean vegetative (V) and reproductive (R) stages. ^b Days after caterpillar infestations (DAI). ^c Density (n) of caterpillars per cage with five plants. ^d Situation of stress = Phenology + DAI + Caterpillars (n).

,

Table 4. Results for accuracy, Kappa coefficient, precision, recall, and F1 score for the testing and training datasets for discrimination of soybean plants under different situations of stress caused by C. includens based on multilayer perceptron artificial neural networks.

Situation of Stress d			Accuracy (%)	Kappa Coefficient	Precision	Recall (Sensitivity)	F1 Score
Phenology a	DAI b	Caterpillars (n) c
Testing dataset
1- V; R	5; 10	0; 2; 5; 10	50.0	0.47	0.47	0.52	0.52
2- V; R	5; 10	0; 2; 5 + 10	64.0	0.60	0.59	0.58	0.58
3- V; R	5; 10	0 + 2; 5 + 10	72.0	0.70	0.73	0.72	0.72
4- V; R	5; 10	0; 2 + 5 + 10	75.0	0.70	0.70	0.73	0.72
5- V + R	5; 10	0; 2; 5; 10	33.0	0.23	0.32	0.33	0.33
6- V + R	5; 10	0; 2; 5 + 10	63.0	0.54	0.59	0.58	0.58
7- V + R	5; 10	0 + 2; 5 + 10	73.0	0.71	0.70	0.71	0.71
8- V + R	5; 10	0; 2 + 5 + 10	75.0	0.70	0.72	0.71	0.72
Training dataset
1- V; R	5; 10	0; 2; 5; 10	67.9	0.66	0.69	0.68	0.67
2- V; R	5; 10	0; 2; 5 + 10	76.4	0.74	0.76	0.74	0.73
3- V; R	5; 10	0 + 2; 5 + 10	82.8	0.80	0.84	0.83	0.83
4- V; R	5; 10	0; 2 + 5 + 10	86.1	0.84	0.83	0.82	0.82
5- V + R	5; 10	0; 2; 5; 10	67.8	0.63	0.69	0.68	0.68
6- V + R	5; 10	0; 2; 5 + 10	73.1	0.67	0.71	0.70	0.70
7- V + R	5; 10	0 + 2; 5 + 10	82.1	0.76	0.83	0.82	0.82
8- V + R	5; 10	0; 2 + 5 + 10	85.7	0.75	0.78	0.76	0.77

(;) Indicates that it is one factor or the other. (+) Indicates that it is a combination of the factors. ^a Soybean vegetative (V) and reproductive (R) stages. ^b Days after caterpillar infestations (DAI). ^c Density (n) of caterpillars per cage with five plants. ^d Situation of stress = Phenology + DAI + Caterpillars (n).

and

Table 5. Results for accuracy, Kappa coefficient, precision, recall, and F1 score for the testing and training datasets for discrimination of soybean plants under different situations of stress caused by S. eridania based on multilayer perceptron artificial neural networks.

Situation of Stress d			Accuracy (%)	Kappa Coefficient	Precision	Recall (Sensitivity)	F1 Score
Phenology a	DAI b	Caterpillars (n) c
Testing dataset
1- V; R	5; 10	0; 2; 5; 10	54.0	0.51	0.55	0.54	0.54
2- V; R	5; 10	0; 2; 5 + 10	61.0	0.57	0.62	0.58	0.58
3- V; R	5; 10	0 + 2; 5 + 10	66.0	0.57	0.62	0.65	0.64
4- V; R	5; 10	0; 2 + 5 + 10	74.0	0.69	0.70	0.70	0.70
5- V + R	5; 10	0; 2; 5; 10	40.0	0.31	0.38	0.40	0.39
6- V + R	5; 10	0; 2; 5 + 10	57.0	0.47	0.57	0.53	0.53
7- V + R	5; 10	0 + 2; 5 + 10	63.0	0.51	0.63	0.65	0.64
8- V + R	5; 10	0; 2 + 5 + 10	74.0	0.62	0.70	0.67	0.70
Training dataset
1- V; R	5; 10	0; 2; 5; 10	57.0	0.54	0.59	0.57	0.56
2- V; R	5; 10	0; 2; 5 + 10	70.1	0.67	0.68	0.68	0.67
3- V; R	5; 10	0 + 2; 5 + 10	77.3	0.74	0.78	0.77	0.77
4- V; R	5; 10	0; 2 + 5 + 10	82.0	0.79	0.78	0.76	0.76
5- V + R	5; 10	0; 2; 5; 10	57.8	0.52	0.59	0.58	0.58
6- V + R	5; 10	0; 2; 5 + 10	65.5	0.58	0.64	0.63	0.63
7- V + R	5; 10	0 + 2; 5 + 10	75.0	0.67	0.76	0.75	0.75
8- V + R	5; 10	0; 2 + 5 + 10	82.1	0.73	0.77	0.75	0.75

(;) Indicates that it is one factor or the other. (+) Indicates that it is a combination of the factors. ^a Soybean vegetative (V) and reproductive (R) stages. ^b Days after caterpillar infestations (DAI). ^c Density (n) of caterpillars per cage with five plants. ^d Situation of stress = Phenology + DAI + Caterpillars (n).

). The MLP-ANN model was built to calibrate the model performance in the discriminate analysis of soybean plants with the training dataset, and the testing dataset was used to validate the neural model. This study classified the health condition of soybean plants in vegetative and/or reproductive phenological stages into three groups. The first was to discriminate soybean plants injured by any of the caterpillar species (C. includens and S. eridania) (Table 3). The second was to discriminate soybean plants injured by only C. includens (Table 4). The third was to discriminate soybean plants injured by only S. eridania (Table 5).

In general, the highest values of the average overall accuracy and the corresponding metrics (Kappa coefficient, precision, recall, and F1 score) were found in the situations of stress “3”, “4”, “7”, and “8” for the three groups (Table 3, Table 4 and Table 5). For the discrimination of soybean plants in the situation of stress “3” (infestation of C. includens and/or S. eridania in the vegetative (V) or (;) reproductive (R) stages, at 5 or (;) 10 days after infestation, under two different caterpillar densities (low infestation = 0 and (+) 2) or (;) high infestation = 5 and (+) 10 (caterpillars per cage with five plants)), the overall accuracy and the corresponding metrics of the model in the testing dataset (Kappa coefficient, precision, recall, and F1 score) were 70%, 0.67, 0.69, 0.71, and 0.70, respectively (Table 3). In the situation of stress “4”, the discrimination of soybean plants grouped in “absence” (0) or “presence” (2 + 5 + 10) of C. includens and/or S. eridania was obtained with an overall accuracy and corresponding metrics (Kappa coefficient, precision, recall, and F1 scores) of 74%, 0.73, 0.70, 0.72, and 0.71, respectively (Table 3). The situations of stress “7” and “8” simulated the same density of C. includens and/or S. eridania as situations “3”and “4”; however, with the phenological stages grouped (vegetative (V) and (+) reproductive (R)). The overall accuracy (66–71%) and other metrics in the testing datasets were lower in these cases.

The best prediction performance of the MLP-ANN model for discrimination of soybean plants was achieved in the C. includens-infested group (Table 4). In the situation of stress ”3” and “7”, corresponding to “low” (0 and (+) 2) and “high” (5 and (+) 10) infestation of C. includens, respectively, the overall accuracy reached values between 72% and 73%; the Kappa coefficient and other model-related metrics (precision, recall, and F1 score) reached values of 0.70 in the testing set. For the situations of “absence” (0) or “presence” (2 + 5 + 10) of C. includens (“4” and “8”), the MLP-ANN model achieved the highest classification accuracy and lowest false positive and false negative rates, with a 75% and 86% overall accuracy and values higher than 0.70 and 0.75 for all related metrics in the testing and training sets, respectively (Table 4).

For the plant group S. eridania infested, the greater results of classification were observed in the situation of stress of “absence” (0) or “presence” (2 + 5 + 10) of S. eridania (“4” and “8”), with an overall accuracy of 74 and 82% for the testing and training sets, respectively; the Kappa coefficient varied from 0.62 to 0.79, precision from 0.70 to 0.78, recall from 0.67 to 0.76, and F1 scores from 0.70 to 0.76 for the testing and training sets (Table 5). For the situations of “low” (0 and (+) 2) and “high” (5 and (+) 10) infestation of S. eridania (“3” and “7”), the MLP-ANN model had a slightly lower overall classification performance than situations “4” and “8”. The overall accuracy varied from 63 to 66% for the testing set, and from 75 to 77% for the training set. The Kappa coefficient and other related metrics varied from 0.51 to 0.65 for the testing set, and from 0.67 to 0.78% for the training set (Table 5).

Confusion matrices were utilized to summarize the classification performance for each stress scenario; the confusion matrices for the species combined (Group 1) were not shown; the accuracy obatined in the confusion matrices calculated for the C. includens-infested (Group 2) and S. eridania-infested (Group 3) plant classification is shown in Figure 5.

4. Discussion

Outbreaks of the main soybean pests occur in nonuniform spatial distributions [20]. The spatial distribution is characterized as aggregating for species of the complex of defoliating caterpillars [37,38] and stink bugs [39,40]. In addition, pest inspection using beat-cloth is still currently being used as a traditional sampling method in soybean crops. This method renders pest sampling in large soybean fields a laborious and time-consuming task, discouraging IPM adoption [12]. In view of these facts, digital agriculture and remote sensing technologies can offer important opportunities for IPM implementation [41,42]. From this perspective, the present study investigated the spectral patterns of soybean plants infested by major soybean caterpillar and stinkbug pests by hyperspectral imaging.

In recent years, hyperspectral remote sensing, a technology that combines the simultaneous advantages of imaging and spectroscopy, has been applied in agricultural entomology [43,44]. Previous studies using spectral information by hyperspectral imaging were highly effective in detecting phytophagous infestations of distinct feeding behavior [22,45,46,47,48,49,50]. The results obtained herein showed that there was a great difference in the reflectance patterns between infested soybean plants by the chewing and piercing–sucking insects evaluated. We demonstrated that the injuries caused by the chewing species C. includens and S. eridania led to differences in the reflectance patterns of soybean leaves, in the vegetative and reproductive phases. On the other hand, the injuries caused by the piercing–sucking species E. heros and D. melacanthus did not change the leaf’s physiological or physical features to such an extent to cause significant differences in the amount of energy reflected in the visible or NIR regions. For this reason, we used the MLP-ANN approach only for classifying the caterpillars’ datasets (C. includens and S. eridania).

Overall, chewing insects, such as caterpillars, can induce higher injury levels in vegetable tissues than piercing–sucking insects, such as stink bugs [50]. Chewing insects tear off the tissue and deposit small amounts of saliva on the disrupted tissue. The plant response to this feeding behavior is associated with the response to wounding and the active components of saliva (“insect-associated elicitors”). Piercing–sucking insects do not remove solid tissue, and the response is not related to a wound response from the plant; its saliva contains components that directly affect cellular processes in plants [51].

The degree of response to the attack is singular in each scenario. For instance, Furuya et al. [50] reported that it was easier to classify stink bug (D. melacanthus) attacks than caterpillar (Spodoptera frugiperda (J. E. Smith)) attacks from non-injured maize plants using hyperspectral data. Stink bugs injury to maize causes brown spots, death of the youngest leaves (“dead-heart”), leaf twisting, and tillering. These symptoms will alter the leaf’s color and texture, which will lead to changes in the spectral reflectance of the leaves. In contrast, in soybean, although stink bugs can feed on all the aboveground parts of the plant, they prefer pods and developing seeds. Thus, the higher injury levels in soybean tissues by herbivory of the caterpillars compared to stink bugs, as well as the marked feeding preference of stink bugs for pods and seeds, support the better discrimination of soybean plants injured by caterpillar species obtained in the present study.

Chrysodeixis includens is among the major soybean pests causing damage in fields from the world’s biggest producers, such as Brazil, USA, and Argentina [52,53,54,55]. In Brazilian fields, there was a pest replacement scenario. Before 2000, this species was a secondary pest in soybean fields, but over the last 20 years, outbreaks have increased, and the problems regarding their management have worsened [54].

The genus Spodoptera has become more frequent in Brazilian soybean fields for a few reasons, including their low susceptibility to transgenic Bt plants’ toxins [55]; changes in agricultural scenarios, with cotton, maize, and soybean being cultivated in succession; and the use of broad-spectrum pesticides to manage other arthropod pests and plant pathogens, leading to a decrease in the populations of natural enemies and natural entomopathogens [56]. Along with S. eridania, other common species from this genus found in soybean are Spodoptera frugiperda (J. E. Smith), Spodoptera albula (Walker), and Spodoptera cosmioides (Walker). Although these species can feed on soybean leaves, flowers, and pods, larval viability is higher when the caterpillars feed both on leaves and reproductive structures (flowers and pods) than when they feed exclusively on reproductive structures [9].

At the third instar, the consumption by one caterpillar of S. eridania or C. includens can be considered equivalent and is estimated at 1.5 cm² of leaf area in half a day in attractive soybean cultivars [57]. Even though the insect-injury equivalent can be considered the same for C. includens and S. eridania [55], we observed higher leaf consumption by C. includens, in both the vegetative and reproductive stages of soybean. This observation can be related to the fact that over 90% of leaf consumption happens at the final larval stages, which were reached around 10 days after infestation in our assays, and there might be differences in development time for C. includens for S. eridania [58,59].

Generally, abiotic or biotic stresses in cultivated plants cause higher reflectance in parts of the visible portion of the spectrum (400–700 nm), mainly the regions related to the absorption of photosynthetic pigments approximately centered in the blue (450 nm–chlorophyll b) and red (650 nm–chlorophyll a) regions [60]. In the assays with C. includens, in both development stages, we observed an increase in reflection in the visible portion (specifically in the 500–700 nm range), as the caterpillars density increased, indicating that infested plants were less healthy. The damage caused by S. eridania did not result in expressive changes in the visible portion of the spectrum but an increase in the reflectance was observed in the NIR region registered by the sensor used (700–1000 nm). Although this region does not have a direct association with the absorption of photosynthetic pigments, it has been reported to have an indirect association with photosynthetic performance and, therefore, plant health [61].

Considering future perspectives of using remote sensing as a tool to monitor pests in agricultural fields, it is necessary to analyze narrow-band data from the NIR region, as was done in this study, to indicate specific bands to be registered in new lightweight sensors that can be carried by drones, for instance, and provide rapidly available information regarding pest infestation. Several studies have shown the importance of the NIR region in classifying infested and non-infested plants, most regarding Hemipteran pests [18,19]. However, some of them reported promising results regarding Lepidopteran pests, specifically defoliators, in row crops, such as peanuts [22], cotton [62], rice [63], and soybean [20,64].

Collecting spectral data from cultivated plants is a viable tool for monitoring plant health. This approach has become more accurate as the technology has improved, with sensors registering more detailed information with higher frequency. On the other hand, collecting more data has increased the need to develop new data analysis techniques. In this sense, machine learning and deep learning techniques have been increasingly used to describe patterns of plant health data, such as pathogen analysis [23,30,65,66], nutritional and water deficiency [67,68,69], pest infestations [45,66,70], and the presence of weeds [71].

In this study, we applied a deep learning approach based on MLP-ANN to discriminate the health conditions of soybean plants under distinct densities of C. includens and/or S. eridania. For the discrimination results combining soybean phenology (vegetative and reproductive), infestation period (5 and 10 DAI), and caterpillar densities (zero, two, five, and ten caterpillars per cage with five plants), we obtained satisfactory classification results for discrimination of soybean plants for the situations of “low” (0 + 2) and “high” (5 + 10) infestation, and of “absence” (0) or “presence” (2 + 5 + 10) of caterpillars in plants.

Other recent studies used machine or deep learning methods to detect pests based on plant responses, achieving high classification accuracy [17,70,72,73,74]. However, these studies could not classify the level of infestation, which is the great challenge. To date, no studies have elucidated the discrimination of chewing caterpillar infestation in soybean plants by combining phenology and infestation period based on hyperspectral data. Our study made it possible to discriminate infested soybean plants under close-spaced interval infestation. The MLP-ANN model had the best performance for discriminating C. includens-infested plants (overall accuracy of 75%). Although the overall accuracy for discriminating C. includens-infested plants had been highest, the recall (sensitivity) also deserves attention for soybean managers because it represents the missed detection rate of infested plants (“negative false”) [72]. In these cases, the average recall was 72%. It is important to highlight that the maximum infestation level adopted in this study (10 caterpillars per five plants) corresponds to the current economic threshold level for C. includens control in Brazil (20 caterpillars per 1-m soybean line (~10 plants)) [6]. In this sense, our study provides an initial assessment with potential to apply a C. includens non-invasive detection method in soybean before causing economic damage.

We evaluated the spectral behavior of soybean plants under different types of pests and population levels separately. Despite the encouraging prospects, it is necessary to study this technology in depth before taking it to the field at the level of accurately monitoring pest populations on crops. Some of the aspects that should be considered in future studies involve the analysis of the response of plants under multiple stressors, for example, attack by more than one type of pest at the same time; attack by pests on plants with and without water/nutrient deficiency; or attack by pests in different varieties of the same cultivated plant, among other combinations. To do so, operational challenges must be overcome, such as the correction of responses in different atmospheric conditions, the possibility of boarding hyperspectral sensors in unmanned aircraft, sensor calibration, and the collection, storage, analysis, and interpretation of data.

5. Conclusions

Our objective to examine the spectral pattern of soybean plants infested by the main soybean pests in Brazil was achieved. It can be concluded that caterpillar attacks by populations lower or close to injury thresholds already established in the field, for periods of five to ten days, cause changes in the plant’s spectral behavior that can be observed through hyperspectral sensors. However, under the same conditions, the stink bug attack does not cause significant changes in the spectral pattern of these plants.

The large amount of data collected favored the use of advanced analysis techniques based on deep learning. Regarding this approach, we can conclude that this technique allowed the creation of predictive models with accuracy levels higher than 70% for very similar stress situations, as was the case in this study.