Crop Classification for Agricultural Applications in Hyperspectral Remote Sensing Images

Abstract

Hyperspectral imaging (HSI), measuring the reflectance over visible (VIS), near-infrared (NIR), and shortwave infrared wavelengths (SWIR), has empowered the task of classification and can be useful in a variety of application areas like agriculture, even at a minor level. Band selection (BS) refers to the process of selecting the most relevant bands from a hyperspectral image, which is a necessary and important step for classification in HSI. Though numerous successful methods are available for selecting informative bands, reflectance properties are not taken into account, which is crucial for application-specific BS. The present paper aims at crop mapping for agriculture, where physical properties of light and biological conditions of plants are considered for BS. Initially, bands were partitioned according to their wavelength boundaries in visible, near-infrared, and shortwave infrared regions. Then, bands were quantized and selected via metrics like entropy, Normalized Difference Vegetation Index (NDVI), and Modified Normalized Difference Water Index (MNDWI) from each region, respectively. A Convolutional Neural Network was designed with the finer generated sub-cube to map the selective crops. Experiments were conducted on two standard HSI datasets, Indian Pines and Salinas, to classify different types of crops from Corn, Soya, Fallow, and Romaine Lettuce classes. Quantitatively, overall accuracy between 95.97% and 99.35% was achieved for Corn and Soya classes from Indian Pines; between 94.53% and 100% was achieved for Fallow and Romaine Lettuce classes from Salinas. The effectiveness of the proposed band selection with Convolutional Neural Network (CNN) can be seen from the resulted classification maps and ablation study.

1. Introduction

Due to advancements in remote sensing image acquisition mechanisms and the growing availability of rich spectral and spatial information by using a variety of sensors, hyperspectral imaging has gained importance. In particular, Hyperspectral Image (HSI) classification has become a prominent source for practical applications in fields like agriculture, environment, forestry, mineral mapping, etc. [1,2,3,4,5].

The present paper focuses on analyzing and using HSI in the agriculture field. Accurate information about growing crops with different climate conditions and agricultural resources and with different timestamps (before, during, and after cultivation) is extremely important and useful for agricultural development. Traditional methods, like field surveys and other statistical-based analyses, are very time-consuming. Advanced remote sensing technology, including HSI, provides a suitable solution and can fill the gap [6,7,8,9,10] with solutions like crop classification.

The problem of crop classification using hyperspectral images has been addressed by researchers with various methods [11,12]. A method based on regression analysis was used to classify the variety of sugarcane crops in Brazil. This HSI data was captured using the EO-1 satellite [13]. The method, proposed in [14], is a combination of Support Vector Machine (SVM) and linear spectral models and was used successfully on the data captured from the Hyperion satellite. This method was also used to classify litchi crops in Guangzhou. Crops in the Karnataka area were classified using the Spectral Angular Mapper (SAM) classifier method for the Hyperion data [15].

The HSI sensor called Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) has recently become important in the remote sensing community. AVIRIS has high spectral bands (224 bands) and spatial resolution (20 m for Indian Pines and 3.7 m for Salinas datasets) with a wavelength range of 380–2500 nm covering VIS, NIR, and SWIR regions and hence is known to be crucial for agricultural applications [16].

Some of the crop classification methods in this connection are as follows. Combined linear and nonlinear SVM algorithms were used to classify corn crops on AVIRIS data. This method obtained moderate accuracy. Soybeans and wheat crops were classified using SVM and Markov Random Field with good accuracy for the AVIRIS HSI data [17]. Unmanned Aerial Vehicle (UAV) datasets also experimented with classifying crops like cabbage, cotton, and strawberry. High accuracy was noticed using Conditional Random fields [18,19]. Further, the Salinas data set was tested to classify different crops using a support vector machine. This method achieved a moderate level of accuracy [20]. Other methods, including spatial context support vector machines, had reasonable accuracy [21].

The above methods are insufficient to extract the required information, and it is difficult to obtain commendable results [22,23,24]. Several successful band selection methods have been introduced (to perform before classification) in the literature over the last few decades, including ranking-based approaches, clustering-based approaches, searching-based approaches, relative entropy, and information entropy-based approaches [25,26,27,28]. The optimal clustering framework was introduced in [29] for band selection and successfully applied the novel objective function with several constraints. In these methods, bands are clustered initially and then ranked according to different measures to select the representative bands from the image. All these methods work well for selecting the informative bands and hence produce high classification accuracy. However, these methods may not be suitable for application-specific classification problems. For example, in the present context, we consider crop classification as our target. It is important to adopt the band section strategy in the view of agricultural phenology, where biophysical properties of plants are also taken into consideration. This is shown in Figure 1 (Source for Figure 1a: Vegetation analysis: using vegetation indices in ENVI [16]).

The details of the measures used for the band selection, mathematical equations, range of the measures, and BS procedure are discussed in the Section 2.

The contributions of the paper are shown below:

• There are different successful methods in the literature for HSI classification. However, not all methods are suitable for all the available application areas to perform classification. In the present paper, we designed a band selection model for crop classification based on the physical and biological properties of plants.
• A new framework for informative band selection is proposed by partitioning the original hyperspectral cube based on the reflectance nature in the visible, near-infrared, and short wave infrared regions of the electromagnetic spectrum. This further uses measures such as entropy, NDVI, and NDWI, respectively, for band quantization.
• A two-dimensional convolution network for hyperspectral image classification is designed and implemented for the accurate classification of agriculture crops with the selected bands. Detailed analysis of the results in crop classification is showcased.

The following is a breakdown of the paper’s structure: materials and methods are presented in Section 2, which consists of a technical description of the proposed method. Dataset description is described in Section 3, followed by Section 4, which includes HSI classification, experimental results, and analysis. Finally, the conclusions are presented in Section 5.

2. Materials and Methods

Crop classification using HSI consists of two steps. The first is band selection, and the second is classification. Partition-based band selection is proposed in this paper. Initially, the partition is performed based on properties of the vegetation spectrum, and three partitions are created. These are termed VIS, NIR, and SWIR partitions (Figure 1). Further bands are selected from these three partitions based on three relevant metrics, entropy, NDVI, and MNDWI, respectively. In the second step to perform classification, the concatenated bands are given as input to the designed CNN model. The CNN model consists of a series of convolution and fully connected layers. Finally, crop classification can be achieved for the selected input data. The proposed architecture for the classification of crops using HSI data is shown in Figure 2.

Let H be hyperspectral data with h rows, w columns and d represents number of spectral bands of H. Each denotes one band in the dataset. According to spectral reflectance properties in various regions of the electromagnetic spectrum, HSI data can be separated into n partitions Partition#1, Partition#2 …Partition#n with bands d₁, d₂,…, d_n respectively. Here:

$${\mathrm{d}}_1+{\mathrm{d}}_2+\cdots {\mathrm{d}}_{\mathrm{n}}=\mathrm{d}$$

(1)

In the present context, n value is considered as 3 with Partition#1 denoting bands in the VIS region, Partition#2 denotes bands in the NIR region, and Partition#3 denotes bands in the SWIR region. Three metrics are then chosen to select bands from each partition.

Information Entropy (IE) is a criterion to measure spatial information in the HSI bands. For a particular band ${\mathrm{X}}_{\mathrm{j}}$, Information Entropy is defined as:

$$\mathrm{IE}\left({\mathrm{X}}_{\mathrm{j}}\right)=-{{\displaystyle \sum }}_{\mathrm{x}\in {\mathrm{X}}_{\mathrm{j}}}\mathrm{P}(\mathrm{x})\ast \log \mathrm{P}\left(\mathrm{x}\right)$$

(2)

where $\mathrm{P}\left(\mathrm{x}\right)$ is the probability of number of grey level of the histogram of the band x in the image.

Based on IE, each band in the VIS region is quantified, and top ${\mathrm{m}}_1(<{\mathrm{d}}_1)$ bands are selected from Partition#1 based on the threshold limit value of δ1.

Normalized Difference Vegetation Index (NDVI) measures plant health in terms of greenness density, as shown in Equation (3). This is a widely used vegetation index in the remote sensing community. The NDVI ranges from +1 to −1. Dead plants have −1 as NDVI value, and healthy plants have values between 0.65 and 1.

$$\mathrm{NDVI}=\frac{\left(\mathrm{NIR}-\mathrm{RED}\right)}{\left(\mathrm{NIR}+\mathrm{RED}\right)}$$

(3)

Based on NDVI, each band in the NIR region is quantified, and top ${\mathrm{m}}_2(<{\mathrm{d}}_2)$ bands are selected from Partition#2 based on the threshold limit value of δ2.

Modified Normalized Difference Water Index (MNDWI) measures the open water enhanced identification and is computed using Equation (4). This will suppress noise generated by vegetation and soil and, at the same time, improve the open water features. MNDWI ranges from +1 to −1.

$$\mathrm{MNDWI}=\frac{\left(\mathrm{GREEN}-\mathrm{SWIR}\right)}{\left(\mathrm{GREEN}+\mathrm{SWIR}\right)}$$

(4)

Based on MNDWI, each band in the SWIR region is quantified, and top ${\mathrm{m}}_3(<{\mathrm{d}}_3)$ bands are selected from Partition#3 based on the threshold limit value of δ3.

Here:

$${\mathrm{m}}_1+{\mathrm{m}}_2+{\mathrm{m}}_3=\mathrm{m}\mathrm{and}\mathrm{strictly} \mathrm{m}<\mathrm{d}.$$

(5)

The proposed band selection algorithm is presented as Algorithm 1 below.

Algorithm 1: Proposed Band Selection Approach for Crop Classification

Input: $\mathrm{H}\in {\mathrm{R}}^{\mathrm{h}\times \mathrm{w}\times \mathrm{d}}$ be the Hyperspectral image Data, R: Red band, G: Green band, thresholds: δ1, δ2, and δ3

Output: ${\mathrm{H}}_{\mathrm{BS}}$, finer sub cube with informative and selected bands

Step 1: Partition the image H with d bands into three sub cubes based on light properties and biological conditions

Step 2: Let the number of bands in each of the sub cubes be ${\mathrm{d}}_1$, ${\mathrm{d}}_2$ and ${\mathrm{d}}_3$ bands respectively from visible, near Infrared, and shortwave infrared regions

Step 3: for i:1 to ${\mathrm{d}}_1$, Compute the entropy,${\mathrm{E}}_{\mathrm{i}}$, of each band using Equation (2)

Step 4: Generate finer sub cube ${\mathrm{H}}_{\mathrm{vis}}$ with m1 bands for those bands from ${\mathrm{d}}_1$ whose ${\mathrm{E}}_{\mathrm{i}}>$ δ1

Step 5: for i = 1 to ${\mathrm{d}}_2$, Compute the Normalized Difference Vegetation Index, ${\mathrm{NDVI}}_{\mathrm{i}},$ using Equation (3)

Step 6: Generate finer sub cube ${\mathrm{H}}_{\mathrm{NIR}}$ with ${\mathrm{m}}_2$ bands for those bands from ${\mathrm{d}}_2$ whose ${\mathrm{NDVI}}_{\mathrm{i}}>$ δ2

Step 7: for i = 1 to ${\mathrm{d}}_3$, Compute the Modified Normalized Difference Water Index, ${\mathrm{MNDWI}}_{\mathrm{i}},$ using Equation (4)

Step 8: Generate finer sub cube ${\mathrm{H}}_{\mathrm{SWIR}}$ with ${\mathrm{m}}_3$ bands for those bands from ${\mathrm{d}}_2$ whose ${\mathrm{MNDwI}}_{\mathrm{i}}>$ δ3

Step 9: Combine the sub cubes ${\mathrm{H}}_{\mathrm{vis}}$, ${\mathrm{H}}_{\mathrm{NIR}}$ and ${\mathrm{H}}_{\mathrm{SWIR}}$ as ${\mathrm{H}}_{\mathrm{BS}}={\mathrm{H}}_{\mathrm{vis}}{\displaystyle \cup }{\mathrm{H}}_{\mathrm{NIR}}{\displaystyle \cup }{\mathrm{H}}_{\mathrm{SWIR}}$ which satisfies Equation (5)

There are two modules in the proposed methodology for the band selection task known as Partition and Ranking. The partition module is focused on the Biophysical properties of plants in Visible, NIR, and SWIR regions. These regions are named Partition#1, Partition#2, and Partition#3. In the ranking module, agricultural phenology metrics such as Entropy, NDVI, and MNDWI are computed using Equations (2)–(4) for the three partitions, respectively. Then, the computed values and selected representative bands are ranked using an adaptive threshold for each of the measures. The parameter tunning is shown in the experimental section.

Let ${\mathrm{H}}_{\mathrm{BS}}\in {\mathrm{R}}^{\mathrm{h}\times \mathrm{w}\times \mathrm{m}}$ denote hyperspectral data cube after the selection of m spectral bands [30,31,32]. This data can be split into two parts. One is for training and the other for testing. Let imagine χ as a training vector that will be input to the CNN model. The first layer is the convolution layer which follows according to Equation (6). Here $\otimes$ denotes convolution operator, filter is denoted Ғ and $\left(\mathrm{i},\mathrm{j}\right)$ denotes the corresponding spatial location.

$${\mathrm{CONVOLUTION}}_{\mathrm{i},\mathrm{j}}=\mathsf{\sigma }({\left(Ғ\otimes \mathsf{\chi }\right)}_{\mathrm{i},\mathrm{j}}+\mathrm{b})$$

(6)

Here σ (.) denotes activation function. For better convergence, the ReLU activation function is used in the present model as in Equation (7). This function gives output as same as input or zero.

$$\mathsf{\sigma }\left(\mathrm{x}\right)=\max \left(0,\mathrm{x}\right)$$

(7)

After a series of convolution layers, the feature vectors are converted into a single Flatten Vector (FV), which will be given as input to Fully Connected (FC) layers. In FC layers, two operations, pre-activation and activation, will be performed at every node. All the FC layers use the ReLU activation function. However, the last layer uses the softmax activation function, as shown in Equation (8).

$$\mathrm{Softmax}\left({\mathrm{x}}_{\mathrm{i}}\right)=\frac{{\mathrm{e}}^{{\mathrm{x}}_{\mathrm{i}}}}{{{\displaystyle \sum }}_{\mathrm{j}}{\mathrm{e}}^{{\mathrm{x}}_{\mathrm{j}}}}$$

(8)

This function gives output probabilities of each crop, and hence classification is possible.

The proposed method performance was tested with state-of-the-art methods (a brief description of the methods is given in Section 4). The experimental works were carried out using MATLAB R 2018b and Python with Google co-laboratory. The hardware utilized for the work was a personal computer with Intel(R) Core(TM) i5-6500 CPU with 3.20 GHz and 8 GB RAM.

3. Dataset Description

In the present work, experiments were conducted using two popular AVIRIS sensor-based datasets. These datasets are freely available and can be downloaded from [33]. The first hyperspectral data used is Indian Pines. These data were captured in the agricultural area of northwestern Indiana, USA, on 12 June 1992. The number of pixels is 145*145, collected in the form of 224 bands by covering the electromagnetic spectrum in the range of 400–2500 nm. Agriculture crops like corn and soya are covered in this data as 64% area. The vegetation types of grass and pastures are covered with 25% area [34,35,36].

The second hyperspectral data used is Salinas. These data were captured in the agricultural area of the Salinas Valley region, CA, USA, on 9 October 1998. The number of pixels is 512*217, collected in the form of 224 bands by covering the electromagnetic spectrum in the range of 400–2500 nm. Agriculture crops like vineyard fields, broccoli weeds, celery, fallow, and lettuce crops are covered with 100% area [37,38,39]. The class description and pixel samples information for the two datasets is shown in

Table 1. Number of samples per class and description of each class for two standard data sets of AVIRIS Sensor.

Indian Pines			Salinas
Name of the Class	Number of Pixel Samples per Band	Class Description	Name of the Class	Number of Pixel Samples per Band	Class Description
Alfalfa	46	Type of green grass	Brocoli_green_weeds_1	2009	Green colored vegetable
Corn-notill	1428	Corn crop cultivation without tillage	Brocoli_green_weeds_2	3726	Green colored vegetable
Corn-mintill	830	Corn crop cultivation with minimum tillage	Fallow	1976	Land Region
Corn	237	Corn crop	Fallow_rough_plow	1394	Land Region
Grass-pasture	483	Type of green grass	Fallow_smooth	2678	Land Region
Grass-trees	730	Type of green grass	Stubble	3959	Land Region
Grass-pasture-mowed	28	Type of green grass	Celery	3579	Green colored Plant region
Hay-windrowed	478	Row of cut small grain grass	Grapes_untrained	11,271	Type of Vineyard
Oats	20	Plant in Brown color	Soil_vinyard_develop	6203	Soil Region
Soybean-notill	972	Soya crop cultivation without tillage	Corn_senesced_green_weeds	3278	Green colored Plant
Soybean-mintill	2455	Soya crop cultivation with minimum tillage	Lettuce_romaine_4wk	1068	lettuce crop that grows in the fourth week
Soybean-clean	593	Soya Plant	Lettuce_romaine_5wk	1927	lettuce crop that grows in the fifth week
Wheat	205	Brown colored Wheat Plant	Lettuce_romaine_6wk	916	lettuce crop that grows in the sixth week
Woods	1265	Type of Tree	Lettuce_romaine_7wk	1070	lettuce crop that grows in the seventh week
Buildings-Grass-Trees-Drives	386	Building area	Vinyard_untrained	7268	Type of Vineyard
Stone-Steel-Towers	93	Tower area	Vinyard_vertical_trellis	1807	Type of Vineyard
Background	10,776			56,975

[40,41,42].

The major type of classes that exist in both Indian Pines and Salinas are shown in Figure 3. As per our interest for the present paper, different types of crops that exist in both Indian Pines and Salinas are also shown in Figure 4 [43,44,45,46].

The Indian Pines data belongs to the agricultural area of northwestern Indiana, USA. The area includes major portions of the Indian Creek and Pine Creek watersheds. Indiana is the tenth-largest farming state in the USA. More than 80% of the land in Indiana is dedicated to farms, forests, and woodland. The Salinas data belongs to one of the efficacious agricultural areas located in the central coast region of California, called the Salinas Valley, USA. This site is famous for producing most of the agricultural activities in the county due to its rich soil and plentiful underground water supplies [47].

The nomenclature for the individual classes is set according to the type of land, growing type, and spectral properties. For example, the classes “Brocoli_green_weeds_1” and “Brocoli_green_weeds_2” of Salinas’s data belong to the same type of land cover representing broccoli weeds, but they have different spectral properties due to their different conditions. The crop “Lettuce_romanine_4wk” refers to the lettuce crop that grows in the fourth week. The same terminology is used for the classes “Lettuce_romanine_5wk”, “Lettuce_romanine_6wk”, and “Lettuce_romanine_7wk”. From the Indian Pines data, “Corn-notill” represents cultivation without tillage and “Corn-mintill” represents cultivation with minimum tillage. The same terminology is used for the classes “Soybean-notill” and “Soybean-mintill”. These two classes, along with “Soybean clean”, represent different growing periods of the same soybean crop.

More information about the data and subclass regions in both datasets is presented in Figure 5.

On the whole, it can be concluded that the selected data is more suitable for crop classification applications. Indian Pines data consists of 64% pixel regions as different types of crops, and for Salinas data, crop regions are found to be 56%.

4. Experimental Results and Analysis

This section consists of parameter tuning, complete experimental results, and analysis. Crop classification results are evaluated against standard metrics.

4.1. Subsection Parameter Tuning in Band Selection

AVIRIS Sensor data is observed to be spread over three partitions. The first partition is the visible region (VIS) in the wavelength region of 400–700 nm and is equipped with 32 bands (d₁). The second partition is the near-infrared region (NIR) in the wavelength region of 700–1000 nm and is equipped with 40 bands (d₂). The third partition is the short wave infrared region (SWIR) in the wavelength region of 1000–2500 nm and is equipped with 148 bands (d₃).

The threshold value to select bands from the VIS region is set as 4.6 (δ1), and the number of bands selected is 15 (m₁) for Indian Pines data, and for Salinas data, these values are set as 4 and 12, respectively. The threshold value to select bands from the NIR region is set as 0.2 (δ2), and the number of bands selected is 15 (m₂) for Indian Pines data, and for Salinas data, these values are set as 0.11 and 16, respectively. The threshold value to select bands from the SWIR region is set as 0.62 (δ3), and the number of bands selected is 15 (m₃) for Indian Pines data, and for Salinas data, these values are set as 0.99 and 15, respectively.

The red band and green band information is essential to calculate NDVI and MNDWI. These are shown in Figure 6 for both datasets [48,49].

The δ1, δ2, and δ3 values were set as shown in Figure 7. Using the number of available bands (224) from the AVIRIS Sensor, we considered 1/5th of the number of the bands. Accordingly, we set m₁, m₂, and m₃ values.

From Indian Pines data, a total of 45 (m) bands were chosen with following band numbers:

18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 37 38 39 41 42 43 44 45 47 48 49 50 51 52 53 151 152 153 154 155 156 157 158 159 160 161 162 163 219 220.

From Salinas data, a total of 42 (m) bands were chosen with following band numbers:

21 22 23 24 25 26 27 28 29 30 31 32 40 41 42 44 45 46 47 48 50 51 52 53 54 55 56 108 109 110 154 155 156 157 158 159 160 161 162 163 164 165.

The graphical representation, which shows the entropy of visible region bands, NDVI of NIR region bands, and MNDWI of SWIR region bands, is in Figure 7 for both datasets.

4.2. Parameter Tuning in Classification Using CNN

After the selection of representative bands, it is necessary to perform classification. The detail of the split between training and testing samples of both datasets is presented in

Table 2. Details of training and testing samples per band for the data sets.

Dataset Name	Total Number of Pixel Samples	Number of Training Pixels	Number of Test Pixels	Number of Background Pixels
Indian Pines	10,249	7679	2569	10,776
Salinas	54,129	40,593	13,536	56,975

.

Parameter tuning for common parameters of the CNN model is presented in

Table 3. Selected Parameters for CNN Model for two data sets.

S.No	Parameter Type	Value
1	Loss Function	Categorical cross entropy
2	Model	Sequential
3	Epochs	100
4	Optimizer	Adam
5	Batch size	2**3
6	Convolution Layer: filters_1	2**6
7	Convolution Layer: filters_2	2**7
8	Dropout rate	0.35
9	Fully Connected Layer:Units_1	2**8
10	Fully Connected Layer:Units_2	2**7
11	Fully Connected Layer:Units_3	2**6
12	Fully Connected Layer:Units_4	2**5

.

CNN network model for Indian Pines data is presented in

Table 4. CNN Model used for Indian Pines data.

Layer (Type)	Output Shape	Param #
conv2d_10 (Conv2D)	(None, 5, 5, 64)	10,432
conv2d_11 (Conv2D)	(None, 5, 5, 128)	73,856
dropout_5 (Dropout)	(None, 5, 5, 128)	0
flatten_5 (Flatten)	(None, 3200)	0
dense_23 (Dense)	(None, 256)	819,456
dense_24 (Dense)	(None, 128)	32,896
dense_25 (Dense)	(None, 64)	8256
dense_26 (Dense)	(None, 32)	2080
dense_27 (Dense)	(None, 16)	528

, and the CNN network model for Salinas data is presented in

Table 5. CNN Model used for Salinas data.

Layer (Type)	Output Shape	Param #
conv2d_6 (Conv2D)	(None, 5, 5, 64)	11,584
conv2d_7 (Conv2D)	(None, 5, 5, 128)	73,856
dropout_3 (Dropout)	(None, 5, 5, 128)	0
flatten_3 (Flatten)	(None, 3200)	0
dense_16 (Dense)	(None, 128)	32,896
dense_17 (Dense)	(None, 64)	8256
dense_18 (Dense)	(None, 32)	2080
dense_19 (Dense)	(None, 16)	528
conv2d_6 (Conv2D)	(None, 5, 5, 64)	11,584

.

4.3. Results with Accuracy Measures

In this work, we used both quantitative and qualitative result analysis to show the performance of the proposed approach for crop classification. Three standard evaluation quantitative metrics were used [42] and are defined as:

• Overall Accuracy (OA): The percentage of correctly labeled pixels in the crop classification;
• Average Accuracy (AA): Average percentage of correctly labeled pixels for each crop;
• Class-wise Accuracies (CA): Percentage of correctly labeled pixels in each crop.

The OA, AA, and CA values of Indian Pines and Salinas, along with execution time, are shown in

Table 6. Details of class-wise accuracy of Indian Pines and Salinas data.

Indian Pines		Salinas
Class ID	Accuracy	Class ID	Accuracy
1	0.6667	1	1.0000
2	0.9664	2	0.9388
3	0.9712	3	0.9453
4	0.9667	4	1.0000
5	1.0000	5	0.9925
6	0.9891	6	0.9970
7	0.9000	7	0.9966
8	1.0000	8	0.9411
9	1.0000	9	0.9974
10	0.9794	10	0.9817
11	0.9935	11	0.9652
12	0.9597	12	0.9751
13	1.0000	13	1.0000
14	0.9937	14	0.9366
15	0.9485	15	0.9004
16	1.0000	16	0.9912
AA	95.84%	AA	97.24%
OA	97.62%	OA	96.08%
Execution Time (in seconds)	460.45	Execution Time (in seconds)	1024.35

.

The qualitative metric used in this work is the classification map. Figure 7 shows the classification map of the Indian Pines data, and Figure 8 shows the classification map of the Salinas data [49,50,51].

For Indian Pines data, the highest classification result was obtained for two crop classes: (Corn-mintill: class number “3”) with an accuracy of 97.12%, and (Soybean-mintill: class number “11”) with an accuracy of 99.35%. These two crop class regions are circled in Figure 8d of the classification map result.

For Salinas data, the highest classification result was obtained for two crop classes: (Fallow_rough_plow: class number “4”) with an accuracy of 100%, and (Lettuce_romaine_6wk: class number “13”) with an accuracy of 100%. These two crop class regions are pointed out in Figure 9d of the classification map result.

4.4. Discussion

This subsection discusses the classification accuracies of classes from the two datasets and also elaborates on the comparison of the proposed framework with the state-of-the-art methods.

From the Indian Pines data, it was observed that there are 6 types of crops, Corn-notill, Corn-mintill, Corn, Soybean-notill, Soybean-mintill, and Soybean-clean, with class numbers 2, 3, 4, 10, 11, and 12. The robust implementation of the classification method resulted in accuracies of 96.64%, 97.12%, 96.67%, 97.94%, 99.67%, and 95.97%, respectively. The accuracy range was found to be 95.97–99.35%.

From the Salinas data, it was observed that there are 6 types of crops, Fallow, Fallow_rough_plow, Fallow_smooth, Lettuce_romaine_4wk, Lettuce_romaine_5wk, and Lettuce_romaine_6wk, with class numbers 3, 4, 5, 11, 12, and 13. The robust implementation of the classification method resulted in accuracies of 94.53%, 100%, 99.25%, 96.52%, 97.51%, and 100%, respectively. The accuracy range was found to be 94.53–100%.

The proposed method for HSI classifications was compared with four state-of-the-art methods, including 3DGSVM [52], CNN-MFL [53], SS3FC [54], and WEDCT-MI [30], to prove the effectiveness in classifying the different crop regions.

The first method used for the comparison is the integration of 3-dimensional discrete wavelet transform and Markov random field for hyperspectral image classification called 3DGSVM [52]. In this work, more importance was given to spatial information. 3DDWT is used to extract spatial features. Probabilistic SVM coupled with MRF-based post-processing was used for HSI classification.

The second method used for the comparison is Hyperspectral Image Classification Using Convolutional Neural Networks and Multiple Feature Learning called CNN-MFL [53]. In this work, multiple features were extracted first, followed by several CNN blocks for each set of features. Here, geometric features were incorporated using attribute profiles. This is a novel technique that takes advantage of multiple feature learning and CNN to perform accurate HSI classification.

The third method used for the comparison is Spectral–Spatial Exploration for Hyperspectral Image Classification via the Fusion of Fully Convolutional Networks called SS3FC [54]. This method used spectral, spatial, and semantic information along with Fusion of Fully Convolutional Networks for HSI classification. A novel technique for the balanced splitting of the training/test dataset was introduced to solve the insufficient training samples problem.

The fourth method used for the comparison is unsupervised band selection based on weighted information entropy and 3D discrete cosine transform for hyperspectral image classification called WEDCT-MI [30]. In this work, original HSI data was first converted in discrete cosine transform-based coefficient matrices. The weighted entropy was calculated to quantify each band. Then, top-ranked bands were selected. Finally, SVM was used for classification.

Figure 10 shows the crop classification from Indian Pines and Salinas datasets. It can be seen from Figure 10 that the proposed band selection approach is effective in extracting the bands which contain much information about the crops. This is due to the inclusion of the physical properties of light in partitioning the bands and the biological properties of plants in band quantization.

4.5. Crop-Wise Analysis

The crop-wise analysis [55] on the two datasets is shown in this subsection.

4.5.1. Corn Crops

The first crop used for the comparison is “Corn-notill,” class number 2 from the Indian Pines data. For this crop, the proposed method outperformed the state-of-the-art methods 3DGSVM, SS3FC, and WEDCT-MI with higher accuracy of 96.64%. The CNN-MFL method had 94.82% accuracy, which is on par with the proposed method. The second crop used for the comparison is “Corn-mintill”, class number 3 from the Indian Pines data. For this crop, the proposed method outperformed the state-of-the-art methods 3DGSVM, SS3FC, and WEDCT-MI with higher accuracy of 97.12%. The CNN-MFL method had 96% accuracy, which is on par with the proposed method. The third crop used for the comparison is “Corn”, class number 4 from the Indian Pines data. For this crop, the proposed method outperformed the state-of-the-art methods SS3FC and WEDCT-MI with higher accuracy of 96.67%. The methods 3DGSVM and CNN-MFL had 96.64% and 96% accuracy, respectively, which are on par with the proposed method.

4.5.2. Soya Crops

The first crop used for the comparison is “Soybean-no till”, class number 10 from the Indian Pines data. For this crop, the proposed method outperformed the state-of-the-art methods 3DGSVM, CNN-MFL, SS3FC, and WEDCT-MI with higher accuracy of 97.94%. The second crop used for the comparison is “Soybean-min till”, class number 11 from Indian Pines data. For this crop, the proposed method outperformed the state-of-the-art methods 3DGSVM, SS3FC, and WEDCT-MI with higher accuracy of 99.35%. The third crop used for the comparison is “Soybean-clean”, class number 12 from the Indian Pines data. For this crop, the proposed method outperformed the state-of-the-art methods 3DGSVM, SS3FC, and WEDCT-MI with higher accuracy of 95.97%. The method CNN-MFL had 95% accuracy, which is on par with the proposed method.

4.5.3. Fallow Crops

The first crop used for the comparison is “Fallow”, class number 3 from the Salinas data. For this crop, the proposed method outperformed the state-of-the-art methods SS3FC and WEDCT-MI with higher accuracy of 94.53%. The methods 3DGSVM and CNN-MFL were slightly more accurate than the proposed method. The second crop used for the comparison “Fallow-rough-plough”, class number 4 from the Salinas data. For this crop, the proposed method outperformed the state-of-the-art methods 3DGSVM, CNN-MFL, SS3FC, and WEDCT-MI with higher accuracy of 100%. The third crop used for the comparison was “Fallow-smooth”, class number 5 from the Salinas data. For this crop, the proposed method outperformed the state-of-the-art methods 3DGSVM, CNN-MFL, and SS3FC with higher accuracy of 99.25%. The method WEDCT-MI had 99.03% accuracy, which is on par with the proposed method.

4.5.4. Lettuce Romaine Crops

The first crop used for the comparison is “Lettuce-romaine-4wk”, class number 11 from the Salinas data. For this crop, the proposed method outperformed the state-of-the-art methods 3DGSVM and SS3FC with higher accuracy of 96.52%. Methods CNN-MFL and WEDCT-MI were slightly more accurate than the proposed method. The second crop used for the comparison is “Lettuce-romaine-5wk”, class number 12 from the Salinas data. For this crop, the proposed method outperformed the state-of-the-art methods 3DGSVM, CNN-MFL, and WEDCT-MI with higher accuracy of 97.51%. Method SS3FC was slightly more accurate than the proposed method. The third crop used for the comparison is “Lettuce-romaine-6wk”, class number 13 from the Salinas data. For this crop, the proposed method and CNN-MFL had 100% accuracy and outperformed the remaining state-of-the-art methods.

4.6. Ablation Study

We conducted an ablation study to verify the effectiveness of the proposed band selection. There are two modules in the proposed methodology known as Partition and Ranking. We conducted experiments by varying the modules (replace Proposed partition based on VIS, NIR, and SWIR regions into Coarser partition into 3 regions and replace Proposed ranking based on Entropy, NDVI, and NDWI into random selection). The ablation analysis for Indian Pines and Salinas data is shown in

Table 7. Ablation experiments of proposed network.

Partition		Ranking			OA	AA
Coarser Partition in to 3 Regions	Proposed Partition Based on VIS, NIR, and SWIR Regions	Entropy Based Ranking	NDVI Based Ranking	NDWI Based Ranking
Indian Pines data
	✓	✓	✓	✓	97.62%	95.84%
✓		✓	✓	✓	77.76%	79.31%
	✓				82.51%	84.45%
✓					69.72%	62.81%
Salinas data
	✓	✓	✓	✓	96.08%	97.24%
✓		✓	✓	✓	80.5%	74.6%
	✓				83.9%	86.76%
✓					48.92%	51.33%

. The accuracy values show that the proposed band selection method has a positive impact on the hyperspectral classification.

4.7. Application of Proposed Methodology with Other Satellite Data

In order to test the adaptability and the effectiveness as in [56], we applied the proposed framework on the WHU-Hi-HongHu dataset [57,58,59]. This data consists of a complex agricultural area with a variety of crops. The data were acquired in Honghu City, China. The number of pixels is 940 × 475, with 270 bands acquired from 400 to 1000 nm wavelength. The data were acquired using an unmanned aerial vehicle (UAV)-borne hyperspectral system with high spatial resolution of 0.043 m. Out of 22 class regions, 15 classes were found with class numbers 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 20.

Table 8. Ablation experiments of the proposed network.

Class No	Class Name	Accuracy in %
1	Red roof	99.63
2	Road	94.76
3	Bare soil	99.43
4	Cotton	99.82
5	Cotton firewood	96.85
6	Rape	99.62
7	Chinese cabbage	97.81
8	Packchoi	97.93
9	Cabbage	99.59
10	Tuber mustard	98.45
11	Brassica parachinesis	97.75
12	Brassica chinesis	96.38
13	Small brassica chinesis	97.81
14	Latuca sativa	99.02
15	Celuce	97.21
16	Film covered lettuce	99.72
17	Romaine lettuce	98.41
18	Carrot	99.63
19	White radish	97.70
20	Garlic sprout	98.82
21	Broad bean	94.88
22	Tree	99.01

shows class-wise accuracies after the application of the proposed framework on the WHU-Hi-Hong Hu dataset. We used all the parameters similar to the Indian Pines and Salinas datasets. It can be concluded that the proposed framework was successful in classifying, with an average accuracy of 98.56% for 15 crop classes.

5. Conclusions

In this paper, an approach for crop classification for HSI images is proposed. Firstly, the bands are selected based on agricultural phenology, where biophysical properties of plants are also taken into consideration. Then, a two-dimensional CNN is trained with the extracted bands. The proposed method is tested and validated on two benchmark datasets. The average accuracy of the crops corn and soya from Indian Pines is 96.81% and 97.75%. For the Salinas dataset, the average accuracy of the crops fallow and lettuce romaine is 97.93% and 98.01%, respectively. The results clearly show the effectiveness of the proposed band selection in selecting the required features necessary for crop classification. In the future, we can incorporate spatial information along with spectral information and extend it to more crops from the datasets.