A Novel Hybrid Learning System Using Modified Breaking Ties Algorithm and Multinomial Logistic Regression for Classification and Segmentation of Hyperspectral Images

Abstract

A new methodology, the hybrid learning system (HLS), based upon semi-supervised learning is proposed. HLS categorizes hyperspectral images into segmented regions with discriminative features using reduced training size. The technique utilizes the modified breaking ties (MBT) algorithm for active learning and unsupervised learning-based regressors, viz. multinomial logistic regression, for hyperspectral image categorization. The probabilities estimated by multinomial logistic regression for each sample helps towards improved segregation. The high dimensionality leads to a curse of dimensionality, which ultimately deteriorates the performance of remote sensing data classification, and the problem aggravates further if labeled training samples are limited. Many studies have tried to address the problem and have employed different methodologies for remote sensing data classification, such as kernelized methods, because of insensitiveness towards the utilization of large dataset information and active learning (AL) approaches (breaking ties as a representative) to choose only prominent samples for training data. The HLS methodology proposed in the current study is a combination of supervised and unsupervised training with generalized composite kernels generating posterior class probabilities for classification. In order to retrieve the best segmentation labels, we employed Markov random fields, which make use of prior labels from the output of the multinomial logistic regression. The comparison of HLS was carried out with known methodologies, using benchmark hyperspectral imaging (HI) datasets, namely “Indian Pines” and “Pavia University”. Findings of this study show that the HLS yields the overall accuracy of {99.93% and 99.98%}_{Indian Pines} and {99.14% and 99.42%}_{Pavia University} for classification and segmentation, respectively.

1. Introduction

Hyperspectral imaging (HI), introduced at National Aeronautics and Space Administration (NASA)’s Jet Propulsion Laboratory [1], is a system that consists of spatial in conjunction with spectral coordinates forming a hyperspectral cube. This imaging system is being used in diverse fields viz. security control [2], pharmaceutical analysis [3], agriculture quality [4], biomedical sciences [5], and many others. Hyperspectral remote sensing is an active analysis in the field of the land area [6] and recent examples of remote sensing is an airborne visible/infrared imaging spectrometer (AVIRIS) sensor [7], operational at the NASA Propulsion Laboratory. Nowadays, AVIRIS sensors are capturing more than 200 spectral bands against each pixel, carrying huge information about an object, which is much better than previous primitive sensors [8]. One of the most popular and important areas of processing hyperspectral remote sensing is classification. Here the task is to classify each pixel of the hyperspectral image by assigning a label for generating a thematic land-cover map [9]. In this regard, several studies have been carried out in the last decade by applying a variety of machine learning and image processing methods to extract useful information from hyperspectral data [10].

Hyperspectral image analysis can classify the materials in an area under observation in a better way based on the huge spectral information obtained but at the cost of high spectral dimensionality. On the contrary, high dimensionality brings along with it the curse of the dimensionality problem that destroys classification performance (Feng et al., 2017). Furthermore, the Hughes effect arises due to high dimensionality in the hyperspectral data that pose critical problems for supervised learning methods to perform effectively [10,11], and sometimes one pixel contains a lot of spectral information with spatial information lost. In a similar manner, sometimes, a single channel contains spatial information, but spectral information is lost [12,13]. Attempts have been made to address the problem, such as that by Fauvel et al. [14]. They described all the details about spectral-spatial advancements in the scenario of classification.

Two kinds of features are mainly extracted from remote sensing data for classification [15].

• First, the main feature is that spectral information lies in the third dimension against each pixel entry in a data cube, and it can be extracted for the exploitation of linear separability of the classes. Numerous methods have been developed for extraction, such as independent component analysis [16], linear spectral unmixing [17,18], and maximum noise fraction [19].
• The second information is the spatial information for which numerous methods have been used, such as manifold regularization [20,21], kernel methods [22,23], and morphological analysis [13,24].

During the last decade, different conventional machine learning methods have used a large number of training samples for training over data [25,26]. In remote sensing datasets, normally, a limited number of reliable samples are available. Wang et al. [27] addressed this issue so that all labeled samples are not used in the training process. Wang et al. suggested that prominent samples should be selected only for training, based on active learning (AL), so that the maximum number of samples would be available during the testing phase [28,29], leading to an agreeable generalization accuracy. Li et al. [30,31,32] have shown that there are different ways to use AL for remote sensing, namely random selection (RS), maximum entropy (ME), mutual information (MI) [33,34], breaking ties (BT) [35], and modified breaking ties (MBT) [36].

Bruzzone et al. showed a significant improvement in classification with extracted spectral and spatial features from the dataset. Better classification is achieved when reliable samples are being used in training [37]. Samples usually are very expensive with regard to time and cost, and it is not so common to utilize this type of large ground truth information. Schölkopf and Smola [22] addressed the issue that kernelized methods that are insensitive towards the utilization of large dataset information due to the curse of dimensionality. Good generalization capability can be achieved using different kernelized methods with even a limited number of reliable training samples [23,38]. There exist numerous methods for the use of large ground truth information such as SVM [39], semi-supervised SVM [40], multinomial logistic regression (MLR) [41], partial least squares regression [42], and artificial neural networks [43]. In both spectral and spatial information, researchers have started to employ neural networks such as densely connected multiscale attention networks [44], end-to-end fully convolutional segmentation networks [45], and convolutional neural networks (CNN) with random forest [46] and Ghostnet [47].

In this work, a hybrid learning system (HLS) is introduced and tested that categorizes hyperspectral images into segmented regions with discriminative features using reduced training size. It is based on a semi-supervised learning approach, using an MBT algorithm for AL and multinomial logistic regression for unlabeled HI data categorization. For the training phase, a smaller number of samples are selected using the AL method (modified breaking ties). HLS is a combination of supervised and unsupervised training with generalized composite kernels generating posterior class probabilities for classification. Finally, Markov random fields are used to get the best segmentation label by using class prior labels from the output of the multinomial logistic regression. The proposed methodology is tested by comparing it with known methodologies for hyperspectral image classification and segmentation. Two benchmark HI datasets, namely “Indian Pines” and “Pavia University” are used for experimentation. The article is organized as follows: Section 2 describes the Methodology, including dataset description with processes and stages involved in HI; Results and Discussion are reported in Section 3 with a comprehensive comparison with existing researchers’ work, and Section 4 provides the concluding remarks.

2. Materials and Methods

The hyperspectral image classification and segmentation tasks are related problems. The goal of image classification is to assign a class label to each pixel in the hyperspectral image whilst segmentation refers to the partition of the set of image pixels into the collection of sets. These sets are declared regions where the pixels in each set are close to each other in some sense. Notably, the term classification is used when there is no spatial information taken into account. On the other hand, the term segmentation is used when spatial prior information is being considered. Let P is a set of n pixels of HI as given by $\mathrm{P}\equiv \left\{1, 2, \dots , \mathrm{n}\right\}$. Similarly, $\mathrm{M}\equiv \left\{1, 2, \dots , \mathrm{N}\right\}$ where M is a set of integers consisting of dataset labels. Further, let X is a set of d-dimensional vectors of image pixels represented as $\mathrm{X}=\left({\mathrm{x}}_1,{ \mathrm{x}}_2, \dots ,{ \mathrm{x}}_{\mathrm{n}}\right)\in {\Re }^{\mathrm{d}\times \mathrm{n}}$. Y is shown below as the set of labels for a dataset $\mathrm{Y}=\left({\mathrm{y}}_1,{ \mathrm{y}}_2, \dots ,{ \mathrm{y}}_{\mathrm{n}}\right)$. The purpose behind this problem formulation is to classify each pixel ${\mathrm{x}}_{\mathrm{i}}\in \mathrm{P}$ and assign them their best label ${\mathrm{y}}_{\mathrm{i}}\in \mathrm{M}$ and also segment them into different regions ${\mathrm{R}}_{\mathrm{i}}\subset \mathrm{P}$ based on labels ${\mathrm{y}}_{\mathrm{i}}$ for pixel ${\mathrm{X}}_{\mathrm{i}}$ [28]. The flow chart for the proposed methodology, namely the hybrid learning system (HLS), is presented in Figure 1. After training on AL-based samples, posterior probabilities were generated by a generalized composite kernels algorithm. On these probabilities, the maxim-a-posteriori algorithm was employed to find the best class probability/label against each training sample. On these classification labels, an α-expansion algorithm is run to generate the segmentation map. Generally in classification, spectral–spatial kernels are computed over a sample with its immediate and farther neighbors to assign it the probabilities, but in segmentation, the best label is assigned based on immediate neighbors. We have used two datasets to conduct our experimentation. Initially, each dataset is passed to extended attribute profiles (EAPs) for the components’ selection. After selecting the components, the MBT algorithm was applied for AL and multinomial logistic regression is applied for unlabeled HI data. Then, the output from multinomial logistic regression is used for segmentation. The method is iterated over ten Monte Carlo runs to get optimal results.

2.1. Datasets

In this study, two sets of landscape images were used for HI-based classification: the “Indian Pines” and “Pavia University” datasets. The AVIRIS sensors were used to acquire the Indian Pines dataset collected over the Northwestern Indiana region. It is 3D-data where the first two dimensions, 145 × 145, represent spatial information while the third one corresponds to 202 spectral channels wavelengths ranges from 400–2500 nm. Each pixel has a spectral resolution of 10 nm and represents an area of 20 × 20 m². There are 16 classes in this dataset with 10,366 instances. Although the dataset is imbalanced, the proposed methodology uses fewer instances for training sessions, thereby relieving the inherent effect of the unbiased nature of the dataset. For regions visualization, an image of spectrum for band (150) in grayscale is shown in Figure 2a.

The Pavia University dataset has been acquired using a ROSIS sensor by capturing the Pavia University, Italy region. It consists of 3D-data where the first two dimensions, 610 × 340, express the spatial resolution while the third dimension represents 103 spectral wavelength channels in the range 430–860 nm. Each pixel has a geometric resolution of 1.3 m. This dataset has 9 classes and 207,400 instances. For regions visualization, the spectral band, namely number 90, in the grayscale channel has been shown in Figure 2b.

2.2. Multinomial Logistic Regression (MLR)

The set $\mathrm{c}\equiv {\left[{\mathrm{c}}^{{\left(1\right)}^{\mathrm{T}}},{ \mathrm{c}}^{{\left(2\right)}^{\mathrm{T}}}, \dots ,{ \mathrm{c}}^{{\left(\mathrm{N}-1\right)}^{\mathrm{T}}}\right]}^{\mathrm{T}}$ represents regressors (logistic) and ${\mathrm{c}}^{\left(\mathrm{r}\right)}$ is the rth class set of regressors [28]. The posterior-class probabilities, as well as regressors, are calculated by the MLR model as given by

$$\mathrm{p}\left({\mathrm{y}}_{\mathrm{i}}={\mathrm{r}|\mathrm{x}}_{\mathrm{i}}, \mathrm{c}\right)=\frac{\exp \left({\mathrm{c}}^{\left(\mathrm{r}\right)}\mathrm{f}\left({\mathrm{x}}_{\mathrm{i}}\right)\right)}{{\sum }_{\mathrm{r}=1}^{\mathrm{N}}\exp \left({\mathrm{c}}^{\left(\mathrm{r}\right)} \mathrm{f}\left({\mathrm{x}}_{\mathrm{i}}\right)\right)}$$

(1)

The important point here is that the value of ${\mathrm{c}}^{\left(\mathrm{r}\right)}=0$ because the density is not dependent on translation on the regressor ${ \mathrm{c}}^{\left(\mathrm{r}\right)}$. The function $\mathrm{f}\left(\mathrm{x}\right)\equiv {\left[{\mathrm{f}}_1\left(\mathrm{x}\right),{ \mathrm{f}}_2\left(\mathrm{x}\right), \dots ,{ \mathrm{f}}_{\mathrm{k}}\left(\mathrm{x}\right)\right]}^{\mathrm{T}}$ where k represents the total number of feature vectors. The f (linear/nonlinear) is given by $\mathrm{f}\left({\mathrm{x}}_{\mathrm{i}}\right)\equiv {\left[1,{ \mathrm{x}}_{\mathrm{i},1}, \dots ,{ \mathrm{x}}_{\mathrm{i},\mathrm{j}}\right]}^{\mathrm{T}}$, where ${\mathrm{x}}_{\mathrm{i},\mathrm{j}}$ represents the $\mathrm{j}$th element of ${ \mathrm{x}}_{\mathrm{i}}$. The kernels are given by $\mathrm{f}\left({\mathrm{x}}_{\mathrm{i}}\right)\equiv {\left[1,{ \mathrm{F}}_{\mathrm{x},{\mathrm{x}}_1}, \dots ,{ \mathrm{F}}_{\mathrm{x},{\mathrm{x}}_{\mathrm{l}}}\right]}^{\mathrm{T}}$, here, ${\mathrm{F}}_{{\mathrm{x}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{j}}}:=\mathrm{F}\left({\mathrm{x}}_{\mathrm{i}},{ \mathrm{x}}_{\mathrm{j}}\right):=\left(\mathsf{\zeta }\left({\mathrm{x}}_{\mathrm{i}}\right), \mathsf{\zeta }\left({\mathrm{x}}_{\mathrm{j}}\right)\right),$ and some kind of symmetry also exists in the function F. Parameters are mapped in a nonlinear manner. For this, radial basis function has been employed represented as:

$$\mathrm{F}\left(\mathrm{x}, \mathrm{m}\right)\equiv \frac{\exp (-||\mathrm{x}-\mathrm{m} {||}^2)}{\left(2{\mathsf{\rho }}^2\right)}$$

(2)

It is important to learn the class densities for regressors’ estimation. The class densities are estimated using labeled and unlabeled samples, where ${\mathrm{S}}_{\mathrm{L}}\equiv \{\left({\mathrm{y}}_1,{ \mathrm{x}}_1\right), \dots , \left({\mathrm{y}}_{\mathrm{L}},{ \mathrm{x}}_{\mathrm{L}}\right)$ represents labeled samples (small set) and ${\mathrm{S}}_{\mathrm{U}}\equiv \left\{{\mathrm{x}}_{\mathrm{L}+1}, \dots ,{ \mathrm{x}}_{\mathrm{L}+\mathrm{U}}\right\}$ represents unlabeled samples (large set). Posterior density using labeled as well as unlabeled data is given by:

$$\begin{gathered} p\left({\mathrm{c}|\mathrm{Y}}_{\mathrm{L}},{ \mathrm{D}}_{\mathrm{L}},{ \mathrm{D}}_{\mathrm{U}}\right)\propto p\left({\mathrm{Y}}_{\mathrm{L}}{|\mathrm{D}}_{\mathrm{L}},{\mathrm{D}}_{\mathrm{U}}, \mathrm{c}\right)p(\mathrm{z}|{\mathrm{D}}_{\mathrm{L}},{ \mathrm{D}}_{\mathrm{U}}) \\ \equiv p\left({\mathrm{Y}}_{\mathrm{L}}{|\mathrm{D}}_{\mathrm{L}},\mathrm{c}\right)p\left({\mathrm{c}|\mathrm{D}}_{\mathrm{L}+\mathrm{U}}\right), \end{gathered}$$

(3)

where ${\mathrm{Y}}_{\mathrm{L}}\equiv \left\{y_1, y_2, \dots , y_{\mathrm{L}}\right\}$ represents set of labels in ${ \mathrm{S}}_{\mathrm{L}}$, ${\mathrm{D}}_{\mathrm{L}}\equiv \{x_1, \dots , x_{\mathrm{L}})$ represents feature vectors set in ${ \mathrm{S}}_{\mathrm{L}}$, and ${\mathrm{D}}_{\mathrm{L}+\mathrm{U}}$ represents labeled as well as unlabeled sets of samples. The maximum-a-posteriori (MAP) estimation for regressors is computed as given by $\hat{\mathrm{c}}=\arg \underset{\mathrm{c}}{\max }\left\{\mathrm{l}\left(\mathrm{c}\right)+\log p\left({\mathrm{c}|\mathrm{X}}_{\mathrm{L}+\mathrm{U}}\right)\right\}$, where

$$\begin{gathered} \mathrm{l}\left(\mathrm{c}\right)\equiv \log p\left({\mathrm{Y}}_{\mathrm{L}}{|\mathrm{D}}_{\mathrm{L}}, \mathrm{c}\right)\equiv \log {\prod }_{\mathrm{i}=1}^{\mathrm{L}}p(y_{\mathrm{i}}|x_{\mathrm{i}},\mathrm{c}) \\ \equiv {\sum }_{\mathrm{i}=1}^{\mathrm{L}}(x_{\mathrm{i}}^{\mathrm{T}}{ \mathrm{c}}^{\left(y_{\mathrm{i}}\right)}-\log {\sum }_{\mathrm{j}=1}^{\mathrm{K}}\exp (x_{\mathrm{i}}^{\mathrm{T}}{\mathrm{c}}^{\left(\mathrm{j}\right)})) . \end{gathered}$$

(4)

Here, $\mathrm{l}\left(\mathrm{c}\right)$ is the log-likelihood function over c provided the samples have labels ${\mathrm{S}}_{\mathrm{L}}$ and $p(\mathrm{c}|{\mathrm{D}}_{\mathrm{L}+\mathrm{U}})$ where the latter works as prior information over c. Further, the Gaussian prior is given by $p\left(\mathrm{c}|\mathsf{\Gamma }\right)\propto \exp \left\{-\frac{1}{2}{\mathrm{c}}^{\mathrm{T}}\mathsf{\Gamma }\mathrm{c}\right\}$, where $\mathsf{\Gamma }$ is a precision matrix and $\mathsf{\Gamma }=\mathsf{\Gamma }\left({\mathrm{D}}_{\mathrm{L}+\mathrm{U}}\right)$ is built in such a way that the regressors c is promoted by the density $\mathrm{p}\left(\mathrm{c}|\mathsf{\Gamma }\right)$. This leaves samples in the class having similar features f(x) (either labeled or unlabeled). It is important here to compute the distance between features.

2.3. Regression Estimator Using MAP

EM-algorithm is used iteratively for computation of MAP for c. The general forms of expectation and maximization steps are given by $\mathrm{J}\left({\mathrm{c}|\mathrm{c}}_{\mathrm{t}}\right)\equiv \mathrm{E}\left[\log \mathrm{p}\left(\mathrm{c}, \mathrm{e}|\mathrm{A}\right) |{ \mathrm{c}}_{\mathrm{t}}\right]$ and ${\mathrm{c}}_{\mathrm{t}+1}\in \arg \underset{\mathrm{c}}{\max }\mathrm{J}\left({\mathrm{c}|\mathrm{c}}_{\mathrm{t}}\right)$, where vector ${ \mathrm{e}}_{\mathrm{i}}$, is a scaling factor for i = 1, …, N-1, representing missing variables and $\mathrm{A}\equiv \left\{{\mathrm{D}}_{\mathrm{L}},{ \mathrm{X}}_{\mathrm{U}}\right\}$ is the set having instances of the labeled and unlabeled dataset. Now we can compute the Gaussian prior by using the sequence of $p\left({\mathrm{c}}_{\mathrm{t}}|\mathrm{A}\right)$ where t = 1, 2, … is an increasing function.

2.4. EMAPs with Spatial Information

The morphological profiles (MPs) and attribute profiles (APs) were determined to merge the spatial and spectral information to assist classification. The attribute filters (AFs) for computing APs, using gray level images were defined in a mathematical morphological framework that operated by combining components at different threshold levels. The filtering operation applied on grayscale images with given threshold values checks whether the current component is going to merge lower gray levels or otherwise. Here, merging of components to lower gray level means thinning, and higher gray level means thickening. In this technique, the entire feature (spectral band) extracted from the original pixel was not considered, and several principal components are taken for this technique and the remaining components are ignored. This yields extended attribute profiles (EAPs). In this research work, the first three principal components are considered that hold most of the associated variance. The stacked EAPs, using different types of attributes, are called extended multi-attribute profiles (EMAP). These are used to enhance extractability related to spatial features from a landscape.

2.5. Approach of Generalized Composite Kernels

Generalized composite kernels (GCKs) are used for spectral-spatial classification. In the proposed methodology, stacked cross-kernels are used as: $\mathrm{K}\left({\mathrm{x}}_{\mathrm{i}},{ \mathrm{x}}_{\mathrm{j}}\right)\equiv {[K^{\mathsf{\omega }} (x_{\mathrm{i}}^{\mathsf{\omega }}, x_{\mathrm{j}}^{\mathsf{\omega }}), K^{\mathrm{s}} (x_{\mathrm{i}}^{\mathrm{s}}, x_{\mathrm{j}}^{\mathrm{s}}), K^{\mathsf{\omega }\mathrm{s}} (x_{\mathrm{i}}^{\mathsf{\omega }}, x_{\mathrm{j}}^{\mathrm{s}}), K^{\mathrm{s}\mathsf{\omega }} (x_{\mathrm{i}}^{\mathrm{s}}, x_{\mathrm{j}}^{\mathsf{\omega }})]}^{\mathrm{T}}$ and the input function $h(.)$ is given by:

$$\begin{array}{c}h\left(x_i\right)\equiv {\left[1, K^T\left(x_i, x_1\right), K^T\left(x_i, x_2\right), \dots ,K^T\left(x_i, x_L\right)\right]}^T\\ \hspace{1em} \equiv [1, K^{\omega }\left(x_i^{\omega }, x_1^{\omega }\right), K^{\omega }\left(x_i^{\omega },x_2^{\omega }\right),\dots ,K^{\omega }\left(x_i^{\omega }, x_L^{\omega }\right),\\ K^s\left(x_i^s, x_1^s\right), K^s\left(x_i^s,x_2^s\right),\dots ,K^s\left(x_i^s,x_L^s\right),\\ K^{\omega s}\left(x_i^{\omega },x_1^s\right), K^{\omega s}\left(x_i^{\omega },x_2^s\right),\dots ,K^{\omega s}\left(x_i^{\omega },x_L^s\right),\\ K^{s\omega }\left(x_i^s,x_1^{\omega }\right),K^{s\omega }\left(x_i^s,x_2^{\omega }\right),\dots ,{\left(K^{s\omega }\left(x_i^s,x_L^{\omega }\right)\right]}^T\end{array}$$

(5)

Therefore, MAP estimation for $\hat{\mathrm{c}}$ may be written as:

$$\begin{gathered} \widehat{c}=\arg \underset{c}{\max }\{l\left(c\right)+\log p\left(c|D_{L+U}\right)+ \\ {\sum }_{i=1}^L{\sum }_{j=1}^L\left(\begin{array}{c}{c}_{j+1}^{\left(y_i\right)}{K}^{\omega }\left(x_i^{\omega },x_j^{\omega }\right)+c_{j+L+1}^{\left(y_i\right)}{K}^s\left(x_i^s,x_j^s\right)\\ +c_{j+2L+1}^{\left(y_i\right)}{K}^{\omega s}\left(x_i^{\omega },x_j^s\right)\\ +c_{j+3L+1}^{\left(y_i\right)}{K}^{s\omega }\left(x_i^s,x_j^{\omega }\right)\\ -\log {\sum }_{k=1}^K\left(\begin{array}{c}\exp c_1^k+c_{j+1}^{\left(k\right)}{K}^{\omega }\left(x_i^{\omega }, x_j^{\omega }\right)\\ c_{j+L+1}^{\left(k\right)}{K}^s\left(x_i^s,x_j^s\right)+c_{j+2L+1}^{\left(k\right)}{K}^{\omega s}\left(x_i^{\omega },x_j^s\right)\\ +c_{j+3L+1}^{\left(k\right)}{K}^{s\omega }\left(x_i^s,x_j^{\omega }\right)\end{array}\right)\end{array})\right\} \end{gathered}$$

(6)

Here, we have used variable splitting and augmented Lagrangian optimization technique to deal with large kernel sizes for improved computational performance [30].

2.6. Isotropic Multi-Level Logistic (MLL) Prior

A logical approach for image segmentation is that the contiguous pixels mostly lie in the same class. This approach has been exploited here with background information where image modeling, isotropic multi-level logistic (MLL) prior has been used. Piecewise-smooth segmentations are encouraged by this prior by using Markov random fields (MRF) where the Gibbs energy is computed. Therefore, the prior model for segmentation is given by:

$$p\left(\mathrm{y}\right)=\frac{1}{\mathrm{z}}\exp \left(-{\displaystyle \sum }_{\mathrm{k}\in \mathrm{K}}{\mathrm{P}}_{\mathrm{k}}\left(y\right)\right)1.$$

(7)

Here, z is a normalizing factor and ${\mathrm{P}}_{\mathrm{k}}$ is called prior potential and k is a set of cliques over the image. The potential over cliques is computed by:

$$-{\mathrm{P}}_{\mathrm{k}}\left(y\right)=\{\begin{array}{c}{\mathrm{v}}_{{\mathrm{y}}_{\mathrm{i}},} if \left|\mathrm{k}\right|=1 \left(\mathrm{for} \mathrm{single} \mathrm{cligue}\right)\\ {\mathsf{\mu }}_{\mathrm{k}}, if \left|\mathrm{k}\right|>1 and {\forall }_{\mathrm{i},\mathrm{j}\in \mathrm{c}} y_{\mathrm{i}}=y_{\mathrm{j}} \\ -{\mathsf{\mu }}_{\mathrm{k}}, if \left|\mathrm{k}\right|>1 and {\exists }_{\mathrm{i},\mathrm{j}\in \mathrm{c}} y_{\mathrm{i}}\ne y_{\mathrm{j}}\end{array}’$$

(8)

It is clear from Equation (8) that the neighboring pixels have the same labels. The Equation (7) becomes flexible by introducing $\exp ({\mathrm{v}}_{{\mathrm{y}}_{\mathrm{i}}})\propto \mathrm{p}\left({\mathrm{y}}_{\mathrm{i}}\right)$ and ${\mathsf{\mu }}_{\mathrm{k}}=\left(\frac{1}{2}\right)\mathsf{\mu }$ > 0 and it is given by:

$$\mathrm{p}\left(y\right)=\frac{1}{\mathrm{z}}\exp ({\sum }_{\mathrm{i}\in \mathrm{P}}{\mathrm{v}}_{y_{\mathrm{i}}}+\mathsf{\mu } {\sum }_{\left(\mathrm{i},\mathrm{j}\right)\in \mathrm{K}}\mathsf{\delta }\left(y_{\mathrm{i}}-y_{\mathrm{j}}\right)),$$

(9)

where $\mathsf{\mu }$ is controlling the level of smoothness and $\mathsf{\delta }\left(y_{\mathrm{i}}-y_{\mathrm{j}}\right)$ determines the higher probability towards label assignment.

2.7. Performance Measures

The performance measures for classification as well as segmentation for HLS are overall- and average- accuracies, including Kappa statistics. We have calculated the Overall Accuracy (OA), Average Accuracy (AA) [28], and Kappa ($\mathsf{\kappa }$) statistics [48] using the following relationships as given by $\mathrm{OA}=\frac{{\sum }_{\mathrm{i}=1}^{\mathrm{N}}{y}_{\mathrm{i}}}{\mathrm{N}}$, where ${\mathrm{y}}_{\mathrm{i}}$ represents true predicted instances and N is the total number of actual instances in the experiment. Similarly, $\mathrm{A}=\frac{{\sum }_{\mathrm{i}=1}^{\mathrm{L}}\frac{y_{\mathrm{i}}}{{\mathrm{L}}_{\mathrm{i}}}}{\mathrm{L}}$, where $y_{\mathrm{i}}$ represents true predicted instances of the ith class, ${\mathrm{L}}_{\mathrm{i}}$ is the number of actual instances of ith class while L indicates the total number of classes in the experiment, and $\mathsf{\kappa }=\frac{\mathrm{P}\left(x\right)-\mathrm{P}\left(y\right)}{1-\mathrm{P}\left(y\right)}$, where $\mathrm{P}\left(x\right)$ represents the actual observed agreement and $\mathrm{P}\left(y\right)$ represents the chance agreement. This measure is used to test inter-rater reliability. It shows the reliability of the data collected in the experimentation having the right representation of measured variables [49].

3. Results and Discussion

This section focuses on the results of experiments performed for the classification and segmentation of the HI scans. Firstly, a brief description of the experimental setup is provided followed by the experimental results with a discussion of the proposed methodology. Finally, a comprehensive comparison with the existing techniques is made at the end.

3.1. Experimental Setup

The experimentation was carried out using a Dell Alien machine, Intel (R) Core (TM) i7—7700 HQ CPU @ 2.80 GHz, having 32 GB RAM. The operating system used was Microsoft Windows with open source software and library programs.

Numerous parameters for computing EMAPs have been found, concerning mean value 2.5–10%, standard deviation value 2.5%, and threshold range is (200–500). The experiment is performed by running independent Monte Carlo iterations 10 times and has empirically found the optimum number of samples to generate outstanding results. Before and after this sampling threshold, accuracy suffered degradation.

After classification, the MAP estimation was passed to the segmentation algorithm, where the MLL algorithm was used for segregating the classes. Here, different accuracy measures were tabulated for each class corresponding to AL methods, namely RS, ME, MI, MB, and MBT. The classification results in terms of overall for all these AL techniques, in the case of the Indian Pines dataset, varied from 98.13 to 99.93%. The Indian Pines ground truth is pictorially shown in Figure 3a, while the classification and segmentation pictogram are shown in Figure 3b,c, respectively. These results may be compared subjectively based on the perceptual information within the original images shown in Figure 3a. The quantitative performance of classification and segmentation is illustrated in

Table 1. Classification accuracy of Indian Pines dataset (class-wise) on 10 independent repetitions using active learning techniques.

Class No.	Class Name	#Samples		Classification Accuracy over Different AL Methods
		Train	Test	Random Selection	Maximum Entropy	Mutual Information	Breaking Ties	Modified Breaking Ties
1	Alfalfa	5	50	100.00	100.00	100.00	100.00	100.00
2	Corn-no till	5	1429	99.90	100.00	100.00	100.00	100.00
3	Corn-min till	5	829	100.00	100.00	100.00	100.00	100.00
4	Corn	5	229	100.00	100.00	100.00	100.00	100.00
5	Grass-pasture	5	492	93.54	90.38	82.06	90.37	97.87
6	Grass-tree	5	742	100.00	100.00	100.00	100.00	100.00
7	Grass-pasture-mowed	5	21	100.00	100.00	100.00	100.00	100.00
8	Hay-windrowed	5	484	100.00	100.00	100.00	100.00	100.00
9	Oats	5	15	100.00	100.00	100.00	100.00	100.00
10	Soyabeans-no till	5	961	100.00	100.00	100.00	100.00	100.00
11	Soyabeans-min till	5	2463	100.00	100.00	100.00	100.00	100.00
12	Soyabeans-clean till	5	609	100.00	100.00	100.00	100.00	100.00
13	Wheat	5	207	100.00	100.00	100.00	100.00	100.00
14	Woods	5	1289	100.00	100.00	100.00	100.00	100.00
15	Building-grass-trees-drives	5	375	100.00	100.00	100.00	100.00	100.00
16	Stone-steel towers	5	90	100.00	100.00	100.00	100.00	100.00
Overall Accuracy				99.92	99.39	98.13	99.27	99.93
Average Accuracy				99.59	99.39	98.87	99.39	99.86
Kappa Statistics				98.99	99.77	97.38	98.96	99.84

and

Table 2. Segmentation accuracy of the Indian Pines dataset (class-wise) using HLS.

Class No.	Class Name	Segmentation Accuracy over Different AL Methods
		Random Selection	Maximum Entropy	Mutual Information	Breaking Ties	Modified Breaking Ties
1	Alfalfa	100.00	100.00	100.00	100.00	100.00
2	Corn-no till	100.00	100.00	100.00	100.00	100.00
3	Corn-min till	100.00	100.00	100.00	100.00	100.00
4	Corn	99.84	100.00	99.56	100.00	100.00
5	Grass-pasture	94.16	92.50	84.50	93.46	99.63
6	Grass-tree	100.00	100.00	100.00	100.00	100.00
7	Grass-pasture-mowed	100.00	100.00	100.00	100.00	100.00
8	Hay-windrowed	100.00	100.00	100.00	100.00	100.00
9	Oats	100.00	100.00	100.00	100.00	100.00
10	Soyabeans-no till	100.00	100.00	100.00	100.00	100.00
11	Soyabeans-min till	100.00	100.00	100.00	100.00	100.00
12	Soyabeans-clean till	100.00	100.00	100.00	100.00	100.00
13	Wheat	100.00	100.00	100.00	100.00	100.00
14	Woods	100.00	100.00	100.00	100.00	100.00
15	Building-grass-trees-drives	100.00	100.00	100.00	100.00	100.00
16	Stone-steel towers	100.00	100.00	100.00	100.00	100.00
Overall Accuracy		99.44	99.23	98.50	99.59	99.98
Average Accuracy		99.62	99.51	99.00	99.59	99.98
Kappa Statistics		99.21	98.93	97.19	99.43	99.98

for Indian Pines. The results were outstanding for classification for the Indian Pines dataset as shown in Table 1, having OA, AA, and kappa statistics as 99.93, 99.86, and 99.84%, respectively. Whereas for segmentation, each of the performance measures (OA, AA, and kappa) have been found as 99.98%, as shown in Table 2. For classification, most of the past methods were based on the exploitation of spectral information [50]. Researchers having similar thoughts relied on the fact that single spectral information is enough for better classification [14,51,52]. These extracted spectral features, representing independent characteristics, were fed to the classifier without using the spatial information in contiguous pixel dependency. Tarabalka et al. [53] addressed the solution of this issue, that exploitation of the spectral information only is not enough for classification. Spatial information should incorporate with spectral information for better results [15,30,32,54,55,56,57]. Li et al. [28] have also shown the importance of spatial information with spectral information. Jain et al. [58] have elaborated hyperspectral image classification with a combination of SVM using self-organizing maps (SOMs). This technique is divided into two phases; the first one is based on the training of the SVM using SOM, while the second is based on the calculation of probabilities by finding the interior and exterior pixels, so the best label should be decided based on an optimal threshold. The comparison of existing techniques using the Indian Pines datasets is shown in

Table 3. Comparison of classification and segmentation accuracies of the Indian Pines dataset (class-wise) on 10 repetitions using 5 training samples of each class.

Sr. No.	References	Classification Accuracy			Segmentation Accuracy
		OA	AA	Kappa	OA	AA	Kappa
1	Li et al. [28]	19.08	11.64	6.66	14.37	17.73	12.34
2	Li et al. [30]	38.32	48.30	40.04	42.05	62.41	39.04
3	Li et al. [31]	55.55	64.32	50.04	62.49	70.92	57.75
4	Li et al. [32]	48.32	64.38	44.19	82.84	88.32	78.19
5	Proposed (HLS)	99.93	99.86	99.84	99.98	99.98	99.98

(Section 3.2).

In the Pavia University experiment, the pictorial representation of ground truth, classification and segmentation landscapes of our proposed methodology are shown in Figure 4a–c. The representation of nine classes, i.e., asphalt, bare soil, bitumen, bricks, gravel, meadows, metal sheets, shadows, and trees, have been used to produce the ground truth. Every class with separate color can be seen after the application of the algorithm. Similar to the previous experimentation, empirically found number of samples using different active learning algorithms are computed by ten independent Monte Carlo runs. The classification results of each class are shown in

Table 4. Classification accuracy of the Pavia University dataset (class-wise) on 10 independent repetitions using HLS.

Class No.	Class Name	# Samples		Classification Accuracy over Different AL Methods
		Train	Test	Random Selection	Maximum Entropy	Mutual Information	Breaking Ties	Modified Breaking Ties
1	Asphalt	5	7174	98.28	97.81	96.52	99.13	99.01
2	Bare Soil	5	19,184	99.88	99.99	100	99.98	99.98
3	Bitumen	5	2486	96.68	95.63	93.04	98.27	98.05
4	Bricks	5	3583	99.96	99.99	100.00	100.00	100.00
5	Gravel	5	1605	99.99	100.00	100.00	100.00	99.99
6	Meadows	5	5556	83.60	88.87	96.05	89.92	90.03
7	Metal Sheets	5	1700	100.00	100.00	100.00	100.00	100.00
8	Shadows	5	4191	100.00	100.00	100.00	100.00	100.00
9	Trees	5	1173	99.98	99.98	99.65	99.48	99.99
Overall Accuracy				99.09	99.33	99.25	99.37	99.14
Average Accuracy				97.59	98.03	98.36	98.53	98.56
Kappa Statistics				98.23	98.72	98.93	99.09	99.01

as OA, AA, and k-statistics as 99.14, 98.56, and 99.01%, respectively. The segmentation results in tabular form are illustrated in

Table 5. Segmentation accuracy of the Pavia University dataset (class-wise) using HLS.

Class No.	Class Name	Segmentation Accuracy over Different AL Methods
		Random Selection	Maximum Entropy	Mutual Information	Breaking Ties	Modified Breaking Ties
1	Asphalt	98.97	98.95	98.48	99.75	99.64
2	Bare Soil	99.75	99.92	99.88	99.87	99.85
3	Bitumen	98.19	97.98	97.09	99.62	99.44
4	Bricks	99.99	100.00	100.00	100.00	100.00
5	Gravel	100.00	100.00	100.00	100.00	100.00
6	Meadows	86.74	92.95	97.26	94.72	94.90
7	Metal sheets	100.00	99.99	99.99	99.99	99.84
8	Shadows	100.00	100.00	100.00	100.00	100.00
9	Trees	100.00	100.00	100.00	100.00	99.99
Overall Accuracy		99.34	99.59	99.55	99.69	99.42
Average Accuracy		98.18	98.87	99.19	99.33	99.29
Kappa Statistics		98.45	99.21	99.32	99.48	99.28

for different active learning strategies. The segmentation results of the proposed methodology for the Pavia University dataset are: (OA, AA, and k-statistics as 99.42, 99.29, and 99.28%, respectively).

The existing models [28,30,31,32] were tested on freely available datasets, and further, the results were compared with the proposed framework (Table 3 and

Table 6. Comparison of classification and segmentation accuracies of the Pavia University dataset (class-wise) on 10 repetitions using 5 training samples of each class.

Sr. No.	References	Classification Accuracy			Segmentation Accuracy
		OA	AA	Kappa	OA	AA	Kappa
1	Li et al. [28]	2.15	3.08	0.80	3.15	5.89	1.21
2	Li et al. [30]	12.84	67.87	9.54	14.30	77.50	10.42
3	Li et al. [31]	42.91	62.01	18.24	48.83	66.13	22.99
4	Li et al. [32]	43.02	31.73	8.18	40.14	38.81	10.78
5	Proposed HLS	99.14	98.56	99.01	99.42	99.29	99.28

) using five samples from each class (Table 1 and Table 4).

3.2. Comparison with Existing Techniques

At the end of the experimentation phase, a comprehensive comparison of classification and segmentation was conducted on existing methodologies. The comparison was performed on five training instances for the Indian Pines and Pavia University datasets, respectively, as shown in Table 3 and Table 6. It can be noted that RS, ME, MI, and BT showed relatively poor performance as compared to MBT. This was attributed to the fact that former techniques usually select samples from complex areas, having the least confidence, which causes difficulty in learning procedure [36]. In comparison, the MBT outperformed as compared to the other active learning techniques, as it selected only those samples that were lying in the vicinity of high-density samples in each class.

Mura et al. [59] introduced morphological attribute profiles (MAPs). Morphological profile filters are used to extract structural information. These calculated profiles are more flexible and lead to better investigation of the contextual information by using different morphological attribute transformations. Makantasis et al. [60] employed a convolutional neural network to encode spectral and spatial information. The output is fed to a multi-layer perceptron for classification. The overall accuracy of 98% was achieved when the author used 30 samples of the Indian Pines dataset with deep neural networks. The relatively low performance by using even higher number of samples is the inherent capability of deep learning strategies as given in [61]. The remarkable segmentation results of the proposed methodology were owed to the use of well-classified data before the segmentation phase. The algorithm consumed relatively more resources for computation as compared to conventional solutions. Similarly, a semi-supervised approach for segmentation was introduced by Li et al. [42] in which the posterior class probabilities were calculated by MLR algorithm. Maximum-a-posteriori (MAP) was computed on this output to calculate the best classification label. The multi-level logistic (MLL) algorithm was then employed with Markov random fields (MRF) to compute the segmentation [44,45]. Moreover, MRF works on the model of Gibbs energy to calculate the best segmentation label, based on current pixel potential for the neighboring pixels, to decide whether this pixel belongs to the current region or not [46,47].

4. Conclusions

There always exists information corresponding to each channel (wavelength) during the acquisition process in hyperspectral imaging (HI). Since diffuse scattering and reflection are specific to material dependence, this dimension has enormous applications in biomedical, geography, defense, and chemistry. In this paper, a novel methodology is introduced that is called hybrid learning system (HLS), which categorizes hyperspectral images into segmented regions with discriminative features using reduced training size. HLS is a semi-supervised learning approach using active learning, a supervised learning technique, and unsupervised learning-based MLR for classification and segmentation of HI. The proposed method outperformed with a fewer number of training instances. For merged spatial and spectral information, EMAPS are computed first for each dataset to give improved classification results. Generalized composite kernels have been used in the MLR algorithm for classification to produce the posterior class probabilities against each instance for better segmentation of landscapes. The comparison of work has been carried out using benchmark datasets, i.e., “Indian Pines” and “Pavia University” with known methodologies. The HLS based HI has been found to yield an overall accuracy of {99.93% and 99.98%} _{Indian Pines} and {99.14% and 99.42%}_{Pavia University} for classification and segmentation, respectively.