SHO-CNN: A Metaheuristic Optimization of a Convolutional Neural Network for Multi-Label News Classification

Abstract

News media always pursue informing the public at large. It is impossible to overestimate the significance of understanding the semantics of news coverage. Traditionally, a news text is assigned to a single category; however, a piece of news may contain information from more than one domain. A multi-label text classification model for news is proposed in this paper. The proposed model is an automated expert system designed to optimize CNN’s classification of multi-label news items. The performance of a CNN is highly dependent on its hyperparameters, and manually tweaking their values is a cumbersome and inefficient task. A high-level metaheuristic optimization algorithm, spotted hyena optimizer (SHO), has higher advanced exploration and exploitation capabilities. SHO generates a collection of solutions as a group of hyperparameters to be optimized, and the process is repeated until the desired optimal solution is achieved. SHO is integrated to automate the tuning of the hyperparameters of a CNN, including learning rate, momentum, number of epochs, batch size, dropout, number of nodes, and activation function. Four publicly available news datasets are used to evaluate the proposed model. The tuned hyperparameters and higher convergence rate of the proposed model result in higher performance for multi-label news classification compared to a baseline CNN and other optimizations of CNNs. The resulting accuracies are 93.6%, 90.8%, 68.7%, and 95.4% for RCV1-v2, Reuters-21578, Slashdot, and NELA-GT-2019, respectively.

1. Introduction

News can be presented in the form of newspapers, periodicals, television, radio, blogs, news articles, etc. News media have been essential, critical, and informative from the beginning of time. News can be plain text and extremely complicated at the same time. This research accentuates news articles and their classification. News classification poses challenges, as news contains multiple semantics [1]. For instance, a single news story may contain information from multiple categories, such as computers, exports, and the economy. Legacy machine learning (ML) techniques classify news as a single-label classification task. Text classification is a primary task in natural language processing (NLP) [2]. It is especially useful for the classification of rich semantic text such as news articles. Multi-label text classification (MLTC) is the process of associating a text with numerous classification labels.

Over the years, researchers utilizing deep learning (DL) techniques have demonstrated its effectiveness in numerous domains [3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18]. Among various DL techniques, convolutional neural networks (CNNs) do not use matrix multiplication, rather they use a special technique called ’convolution’. The combination of convolution and pooling layers is used for the classification task [19,20]. Deep learning creates a learning model and forms learning patterns from a learning dataset and then efficiently predicts labels based on that learning [21]. CNNs have attained top performance in text categorization because of their outstanding capability of capturing local relations of temporal or hierarchical structures [22,23,24].

Kalchbrenner et al. [22] modeled words with stacking of numerous layers of convolution. Collobert et al. [23] and Yu et al. [24] both simulated text using a single convolutional layer. According to Kim et al. [25], the use of many convolutional layers is beneficial in the process of extracting high-level abstract characteristics. According to Yin et al. [26], the network can better handle sentences of varying lengths due to the use of a pooling operation. Furthermore, they investigated multichannel convolution architecture and variably sized feature detectors. Starting out, Conneau et al. [27] used a deep convolutional architecture consisting of 29 convolutional layers. To achieve superior results in detecting correlations among data, they increased the layer count to 49. However, in this context, they were unable to reach state-of-the-art results.

In order to significantly outperform long short-term memory (LSTM)-based approaches, Gu et al. [28] coupled a language CNN with recurrent highway networks. One study [29] employed LSTM to manage the flow of data throughout the network. The input window size in [28,29] was restricted due to the usage of a recurrent framework. There is also difficulty in representing hierarchies and identifying long-term dependencies [30]. Unlike other methods, CNN’s computational time scales linearly with the length of the sequences. CNNs have an advantage over recurrent neural networks since they not only obtain long-range information but also obtain a hierarchical representation of the input words [30].

The effectiveness of CNN is highly dependent on setting the external parameters, which vary based on data type, size, and origin, and therefore must be manually adjusted for each unique data set. These parameters are known as hyperparameters. The primary objective of this study is to obtain the ideal set of hyperparameters in the shortest possible time without any manual intervention, hence producing a classification model with improved performance and better time efficiency.

Recently, various metaheuristic algorithms have been proposed as effective methods for optimization [31,32,33,34]. Meta-heuristic algorithms used in such techniques have proven to be effective at reducing the search space needed to find an optimal solution. Unfortunately, once these algorithms are applied to huge search areas, they tend to use up a lot of time as well as computational resources and fall back on local optima. They also suffer from premature convergence and do not provide a middle ground between local and global searches. Thus, it is necessary to successfully fill the void left by metaheuristic optimization techniques when working with a wide search space to implement task-specific computational applications.

In order to get better outcomes, there is need for the development of more-comprehensive multi-text classification methods. In the disciplines of engineering and research, the utilization of the original heuristic algorithm is an extremely common practice [35]. Because of its excellent search performance, the spotted hyena optimizer, also known as the SHO algorithm, is utilized in the fields of engineering and science.

SHO is a metaheuristic optimization technique inspired by the principles of social and hunting behavior of spotted hyenas [36,37,38,39,40,41]. During the training process of the proposed model, SHO selects the optimal values of hyperparameters to tune the CNN. The best value-selection procedure involves turning the hyperparameters into vectors and then using advanced mathematical modeling to search for and assign values within the defined parameter range [38,39]. As this algorithm is inspired by spotted hyenas, it imitates the hunting behavior of hyenas, i.e., searching, surrounding, and attacking the target. The exploration and exploitation processes of SHO are highly precise and effective. The exploration and exploitation tasks are group-based, so it never descends into local optima. The fast convergence rate along with the robustness make SHO highly adaptable. These characteristics of SHO make it highly suitable for exploring new domains [40,41]. SHO has already achieved global success in optimization and is regarded as one of the top exploration techniques [42,43,44,45]. Therefore, the purpose of this study is to utilize the highest strengths of metaheuristic techniques to overcome CNN’s limitations in the field of news MLTC for improved classification accuracy and efficiency.

This research focuses on the use of CNN along with SHO to provide an expert system for automatically optimizing hyperparameters, including learning rate, momentum, number of epochs, batch size, dropout, number of nodes in hidden layers, and activation functions, to ensure the best possible accuracy and efficiency of an MLTC of news. Four publicly accessible news datasets, namely RCV1-v2 [46], Reuters-21578 [47], Slashdot [48], and NELA-GT-2019 [49], are used to evaluate and validate the proposed model. The results are extremely encouraging and reflect the model’s high convergence rate, which result in improved CNN performance in terms of accuracy and efficiency. The salient contributions of this research are:

• A metaheuristic-optimized SHO-CNN model for multi-label news text classification;
• Automation of hyperparameter optimization of a CNN using SHO;
• Evaluation and comparison of SHO-CNN with other metaheuristic optimization algorithms

The contents of the paper are further arranged as follows: Section 2 represents related multi-label text classification work along with hyperparameter optimization of a CNN and related techniques are also reviewed. Section 3 illustrates the proposed methodology with the help of an algorithm and related steps. Section 4 represents the evaluations, results, and discussion. Finally, Section 5 concludes the research and provides recommendations for related future work.

2. Related Work

The literature review includes information along with a background study of the related models and techniques. The summary of the literature review and the limitations in the field of MLTC are also discussed in this section.

2.1. Multi-Label Text Classification

MLTC is a technique used to allocate multiple labels to a singular document in the field of NLP [50]. The primary objective of NLP is to utilize mathematical models and algorithms to enable computers to better comprehend and rationalize associations inside and amongst documents [51]. MLTC is primarily used for document classification in several applications [52]. Additionally, MLTC is commonly utilized for the classification of web content [53,54] and recommendation systems to better comprehend context [55,56]. MLTC is able to assign various labels to a single object. For better understanding, assume the list of labels as $l=\left\{L_1,L_2,L_3,\dots L_z\right\}$, where l must assign n labels for L subsets to a related instance x [57]. In contrast to conventional text classification methods, MLTC allows several linked labels to be applied to each instance [58]. The primary goal of MLTC is to determine the maximum number of labels for an instance.

A hierarchical structure of classes has been proposed, where classes are inherited from other classes; if one object is assigned to a class, it may automatically relate to other classes as well. This technique is known as Hierarchical Multi-Label Text Classification (HMLTC) [59]. Another corresponding framework has been proposed, known as the Hierarchical Cognitive Structural Learning Model (HCSM). Furthermore, the study [59] has employed an Attentional Ordered Recurrent Neural Network (AORNN) for generating vector-based calculations to gather knowledge from data, along with a Hierarchical Bidirectional Capsule (HBiCaps) to utilize the said vectors in each iteration. One more related model is the Hierarchical–Attentional Graph Convolutional Recurrent Neural Network (H–AGCRNN), which modifies the vector formation algorithm for semantic extraction. A hierarchy-based update for knowledge extraction has been performed for the Hierarchical Attentional Graph Taxonomy Capsule–Graph Convolutional Network (HAGTP–GCN). Experiments were performed on four datasets (RCV1, EUR-Lex, WOS-46985, and Patent); the results suggest that HCSM was more useful than H-AGCRNN and HAGTP-GCN.

Ibrahim et al. presented a Generic Hybridized Shallow Neural Network (GHS-NET) to employ CNN along with Bidirectional LSTM (bi-LSTM) for gathering data-related information [60]. The model was evaluated on three datasets related to the medical literature, i.e., Chemical Exposure Analysis, Cancer Hallmarks, and Medical Information Mart for Intensive Care (MIMIC-III). The experimentation process contained a variety of environment settings related to filter sizes, pooling, and activation functions.

Wang et al. proposed a model called Multi-Label Reasoner (ML-Reasoner) [50]; it utilizes a binary classifier to generate classes along with an argumentation process for the classification of data. The process consists of four steps; the first step is parsing all data for text embeddings. Converted vectors are used for the CNN Encoder. The label encoder assigns labels to documents. Later, a sigmoid function is used as an activation function to normalize the output. The said model was evaluated on two datasets (AAPD, RCV1V2). The outcomes demonstrated that the ML-Reasoner model outperformed CNN and LSTM.

Benites and Sapozhnikova modified a fuzzy Adaptive Resonance Associative Map with a hierarchy model known as the Hierarchical ARAM neural network (HARAM) [61]. This approach was designed to enhance the classification effectiveness of large datasets with intricate structures. Chen and Ren proposed Latent Wordwise Label information (MLC-LWL) for multi-label text classification [62]. This model captures information for each word to form a label and then classifies each label based on the matching structure between them. The said model was evaluated and had higher accuracy compared to primary machine learning techniques.

Wang, Tianshi, et al. merged a dynamic semantic representation model and a deep neural network as DSRM-DNN [16]. It made use of a word embedding model along with a clustering technique. DSRMDNN elements are measured and selected based on the word score. Considering backpropagation properties, a deep belief neural network is integrated for text classification. The classification process reconditions newly discovered words along with minimally occurring words throughout the cycle. The model was evaluated using three datasets, i.e., RCV1-v2, Reuters-21578, and EUR-Lex. The results suggest that the performance of the model under consideration is better than that of MultiLabel Decision Tree (ML-DT), Multi-Label k-Nearest Neighbor (ML-KNN), Binary Relevance (BR), Classifier Chains (CC), Multi-Label Neural Networks (MLNN), HARAM, Convolutional and Recurrent Neural Networks (CNNRNN), Hierarchical Label Set Expansion (HLSE), and Supervised Representation Learning (SERL) models.

2.2. Hyperparameter Optimization of CNN

Conventional AI-based neural networks use matrix multiplication, but CNN uses the convolution method for mathematical calculations and computing of the classification process [63,64,65]. There are also variable parameters that affect the performance of CNNs; they are known as hyperparameters. These parameters are external to the CNN and can highly influence the classification output. The determination of highly optimal hyperparameters guarantees a high outcome in terms of accuracy and efficiency, but it is a time-exhausting manual process, even for an expert with plenty of experience in the domain. In a complex CNN configuration, there are numerous hyperparameters along with distinct ranges. It is tedious to manually choose, evaluate, and tune each parameter and their combinations. This necessitates the development of an automated procedure to determine the appropriate hyperparameter values.

In order to tune hyperparameters of a simple CNN, Random Search (RS) and Grid Search (GS) models are widely used [66,67,68]. These methods are non-adaptive in nature since they rely on readily available results. GS is feasible for a CNN with a small number of hyperparameters, as it evaluates all the combinations of hyperparameters. Therefore, increasing the number of hyperparameters exponentially increases the number of possible combinations, resulting in poor performance and resource utilization [67]. In comparison to the tuning of hyperparameters by using GS [68], Bergstra and Bengio have used RS for similar experiments [66]. Their results demonstrated that RS is time-efficient and generates more-precise models. The prevailing performance of nonadaptive techniques can be further improved by manually modifying the hyperparameter space.

Adaptive methods such as Bayesian optimization and metaheuristics-based models employ knowledge from data in the early phases [69,70,71]. Bayesian optimization models are considered to be more straightforward to implement than metaheuristics models. Metaheuristics-based models are best suited for progressive techniques, as they provide efficient handling for a solution space. Neuroevolution is an application of evolutionary computation related to neural networks. The prominent research for this domain includes EPNet [72] and NEAT [73]. Over the years, the evolution of high-speed memory management, especially Graphics Processing Units (GPUs), has enabled highly complex networks to perform efficiently.

In another work, Sun, Xue, Zhang, and Yen proposed an evolutionary CNN known as EvoCNN [74]. It automatically generates a CNN architecture along with initial weights. Ma et al. also proposed models based on evolutionary algorithms [75]. Both proposed models were evaluated on complex networks and resulted in high efficiency. Lecun et al. incorporated Genetic Algorithms (GA) along with grammatical evolution for automatic generation and tuning of hyperparameters for CNNs [64].

The effectiveness of metaheuristics-based algorithms for optimizing CNN hyperparameters has been demonstrated over the years. One such method is known as Particle Swarm Optimization (PSO). PSO-based models have been evaluated using publicly available standard datasets. The results show that PSO-based models have decreased classification error [76,77].

Due to the success of metaheuristic algorithms, several other metaheuristics-based approaches have been proposed, including Differential Evolution (DE) [78], Harmonic Search (HS) [79], and Reinforcement Learning (RL) [80,81]. The proposed models aim to tune hyperparameters and search for the optimal CNN architecture [82,83]. Hakim et al. [36] used the imperialist competitive algorithm (ICA) and Greywolf optimizer (GWO) to enhance CNN performance. The results demonstrated that the CNN optimized with metaheuristic algorithms outperformed the original CNN.

Dhiman and Kumar proposed SHO [38], which is an adaptation of metaheuristic algorithms. The authors evaluated the model on complex networks and standard datasets. Due to its advanced search ability, the results indicated a rapid convergence rate.The SHO algorithm was applied to image matching in the study [42], which resulted in improved accuracy. The modified SHO was utilized for PID parameter optimization in VAR in [44] and contributed to an increase in the diversity of search options.

Kadry et al. [37] have used SHO for CNN-based model optimization. The authors optimized the deep and handcrafted features using SHO and later concatenated them to create a feature vector with rich representation. In other research, Luo, Li, and Zhou employed SHO in the field of image processing [42]. Panda and Majhi [43] used SHO for the prediction of gene selection features using improved multi-objective SHO. Table 1 presents a summary of metaheuristic techniques and their limitations.

Table 1. Summary of metaheuristic optimization techniques in the literature.

Name	Contributions	Limitations
Random Search (RS) [66]	It is a direct search technique that does not maximize the problem’s gradient. The program uses a random number generator to determine the values of the hyperparameters.	Methods such as this are not adaptive since they use predetermined outcomes. They are an inefficient means of going down, which only lowers the overall performance.
Grid Search (GS) [67,68]	A grid of values is formed by picking two or more parameter values and evaluating all possible permutations of those values.	Larger combinations tend to have a negative impact on efficiency and productivity.
Harmonic Search (HS) [79]	This method was developed specifically for the purpose of resolving difficulties involving optimization with a single target.	After increasing the parameters, the performance starts to suffer, which leads to less accurate results.
Particle Swarm Optimization (PSO) [76,77].	It optimizes a problem by repeatedly searching for better solutions until one is found that meets certain specified criteria.	Given the fixed length of the encoding string, it is required to manually adjust the dense layer parameters and their positions within the encoded text.
Imperialist Competitive Algorithm (ICA) [36]	It can distinguish between distinct sources from a mixed signal.	It does not adjust itself to the changing characteristics of a dynamic system.
Grey Wolf Optimizer (GWO) [36]	The algorithm models the social structure and hunting strategy of grey wolves in the wild. Since there are fewer control settings, it is simple to implement.	Its low accuracy is a result of its slow convergence rate.
Genetic Algorithm (GA) [64]	This technique, which is based on natural selection, is capable of solving both limited and unconstrained optimization problems.	Over time, it may settle on a solution that is less than ideal.
Differential Evolution (DE) [78]	In order to generate a new vector, this technique uses the dissimilarity between two or more existing vectors.	Finding the best way to start a population and set control settings can be difficult. Although the population may not have attained a local optimum, convergence can still take place.

The current literature review confirms the necessity for MLTC of news. A primary CNN is inefficient for MLTC tasks [52]. The performance of a CNN is extremely dependent on the tuning of its hyperparameters, and automation of hyperparameters is crucial work in the field of CNN optimization. The performance of the model can suffer from an increase in complexity at times [67]. Metaheuristic-based optimizations are effective for automating CNN hyperparameter tuning [69,70,75]. Metaheuristic-based optimizations are effective for automating CNN hyperparameter tuning [36,64,76,78,79], but there is no single best-performing solution for all optimization problems [70]. Furthermore, increasing the dimensions greatly diminishes the effectiveness of approaches such as PSO, DE, and GA [84].

The proposed model employs SHO [38] because it holds superior exploration capabilities, rapid convergence rate, and extremely valuable qualities to prevent falling for local optima. In addition, it has been evaluated on complicated networks [42,44]. The optimization of CNN hyperparameters, particularly for an MLTC model, reflects an ongoing challenge. Considering the criticality, importance, and effectiveness of news media, this challenge poses a superior, purposeful, and significant research problem.

3. Methodology

The proposed model employs a CNN for the task of MLTC. The characteristics of CNN for the task of text classification are exceptional. The performance of CNN depends heavily on its hyperparameters. These hyperparameters are responsible for the learning process and output of a model. The hyperparameters are always specified and updated manually to tune the model. To overcome this time-consuming and manual task of tuning a CNN, the proposed model employs SHO for the automated tuning of the CNN. SHO is based on a metaheuristic search algorithm; it fetches the set of hyperparameters and their values to establish the optimal values for each hyperparameter. In this research, the CNN is highly analyzed to optimize related hyperparameters, including batch size, learning rate, momentum, dropout, the number of epochs, the number of nodes, and the activation function. The proposed model for MLTC of news is shown in Figure 1.

Figure 1. Architecture of the proposed SHO-CNN for MLTC of news.

The operations of the proposed model can be divided into phases, beginning with preprocessing for cleaning datasets, word embeddings for vector representations, hyperparameter tuning by SHO, and classification output by CNN.

3.1. Preprocessing

Preprocessing is generally an initial step for every machine learning model. It involves cleaning and organizing raw data to make it suitable to develop and train a machine learning model. The data cleaning standard may vary as required by the model. Preprocessing text for the proposed model includes the following essential steps:

• Converst numbers to descriptive words;
• Remove all special characters;
• Remove spaces;
• Expand all abbreviations;
• Remove ineffective words;
• Detect and replace synonyms for words.

3.2. Word Embeddings

The function of word embedding is to convert textual words into their corresponding numerical vectors; this allows machine learning models to identify and learn hidden patterns among textual data for NLP. In the proposed model, the skip-gram model is used to generate vectors [85]. It involves semantics to create a vector for every vocabulary word, as represented in Figure 2. The related context properties are retrieved from vector documentation. Context properties are compared on the basis of similarity to the input. Input words are referred to as target words, and the output with expected context attributes is known as labels.

Figure 2. Skip-gram model.

The target vector and context labels are encoded using one of the primary methods, called ’one-hot encoding’, which is also known as “1’-of-N” encoding. One-hot encoding stores a vector in the form of 0 s and 1 s. It is most appropriate for determining the size and orientation of words [86].

3.3. SHO-CNN

Considering the social and hunting behavior of a metaheuristic SHO algorithm, SHO is integrated with CNN to optimize the hyperparameters. The course of the current study includes implementation of domain standards and publicly available datasets. Secondly, we build and train the proposed model. Afterward, we evaluate the performance of the proposed model. Finally, we compare the performance of the proposed model with other baseline models using advanced performance measures, including accuracy, precision, recall, and F-measure.

Input and output parameters can be set during the development of machine learning models; however, the ideal values for hyperparameters cannot be determined. No expert is definite regarding the nature and relationship between training datasets and data processing. To circumvent this difficulty, specialists manually supply an extensive range of values for each hyperparameter, which is a time- and resource-intensive process. Ideally, a model ought to be capable of automatically searching for and obtaining the optimal values for each hyperparameter. In general, these hyperparameters differ between models and datasets and cannot be learned directly from data. For the tuning and optimization of hyperparameter values, external optimization techniques are applied.

Dataset $DS$ used in this research is divided into training ${DS}_{Tr}$ and testing ${DS}_{Te}$ subsets ${{(DS}_{Te},DS}_{Tr}\in DS$). For every $DS$, there are various classification models C for a CNN. These models take hyperparameters $HP$ for the CNN into consideration, including the learning rate, momentum, number of epochs, batch size, number of nodes, activation function, and dropout. Let us consider a classification model with optimized hyperparameters as $C_{oHP}$, where $oHP$ is the desired optimized configuration of hyperparameters $(oHP\in HP)$. The optimized solution is calculated using SHO. The optimized solution minimizes the loss function at a rate of L. The hyperparameters of $C_{oHP}$ can be validated via Equations (1) and (2).

$$oHP\left(D{S}_{Tr}\right)=\underset{C_{oHP}\in C}{\mathrm{args} \min }L\left(C_{oHP},D{S}_{Tr}\right)+\rho \left(C_{oHP}\right)$$

(1)

$$\overline{oHP}=\underset{oHP\in HP}{\mathrm{args} \min }L\frac{\left(C_{oHP},D{S}_{Tr}\right),D{S}_{Te}}{\overline{C}\left(oHP\right)}$$

(2)

The population of spotted hyenas is initialized to begin the processing of the proposed model. Each hyena is defined as the combination of hyperparameters of CNN to be tuned. The fitness value of each hyena is calculated from the initialized population. The hyenas search for the best solution until the max number of iterations. The fitness of each hyena is checked after every iteration. The best value for a hyena is updated in the search space. A group of best solutions is maintained in the solution space. The procedure is terminated when the required condition is met or when the maximum number of repetitions has been reached. The algorithm for the proposed model is presented in Algorithm 1.

Algorithm 1 SHO-CNN

Require:  Size of population of Hyenas $\left(H\right),H(i=1,2,3,\dots ,n)$ Ensure:  Fittest $H_i$, Every member in set H is a combination of hyperparameters $\overrightarrow{hp}$ for the CNN, including learning rate, momentum, number of epochs, batch size, number of nodes, activation function, and dropout.   1: Initialize random H population of size n   2: Determine the fitness value of each search agent   3: While (epoch < max number of iterations)   4: For each H   5: Update the location of each search agent   6: End for   7: Update the control parameters $\overrightarrow{X}$, $\overrightarrow{Y}$, ${\overrightarrow{Distance}}_{hp}$, and $\overrightarrow{hp}$   8: Compute the fitness value of each H in a group   9: If current solution ${\overrightarrow{L}}_{hp}$ is better than the previous solution, update ${\overrightarrow{L}}_{hp}$ 10: Update the group ${\overrightarrow{C}}_{hp}$ with updated ${\overrightarrow{L}}_{hp}$ 11: Epoch = Epoch + 1 12: End while

3.4. Tuning Hyperparameters

MLTC for news includes the classification of text data; therefore a one-dimensional CNN is utilized for experiments. The primary structure of the CNN is followed in the experiments, which contains convolutional layers at the start along with pooling layers, activation layers, and a fully connected layer at the end. A stack of five convolution layers is used in the experiments. A Rectified Linear Unit (ReLU) is the prevalent choice for the activation function in the case of multi-label text classification problems. The proposed model also utilizes the ReLU activation function. Max-pooling is applied as a pooling function in pooling layers. Adam is used as an optimizer. CNN has several hyperparameters that can be tuned to optimize the classification output. Before initiating the training process, the upper and lower limit value ranges of the parameters are assigned. The hyperparameters of the CNN and the parameter of SHO along with the selected range of values are represented in Table 2 and Table 3, respectively.

Table 2. Hyperparameters of CNN.

Hyperparameter	Value
Learning rate (LR)	Min. value 0.001, Max. value 0.1
Momentum (MOM)	[0.9 to 0.99]
Dropout	[0.1 to 0.9]
Epochs	[30,60,120,150,200,300]
Batch size (BS)	[8,16,32,64]
Number of nodes	(1,256)
Activation function	(ReLU, Sigmoid)

Table 3. Parameters of SHO.

Hyperparameter	Value
Size of population	30
Control parameter ($\overrightarrow{hp}$)	(5,0)
Random vector $\overrightarrow{M}$	(0.5,1)
Number of iterations	200

4. Evaluation and Results

This section describes the datasets, baseline models, and evaluation matrices used in the evaluation process along with a comparison of the results. The best values of the optimized hyperparameters for the CNN by the proposed model are also described in the section below.

4.1. Experimental Setup

The development environment is based on Windows 10; system specifications are Intel(R) Core (TM) i5-1005G1 CPU @ 1.20 GHz–1.19 GHz with 16gb ram. A jupyter (https://jupyter.org/, accessed on 8 September 2022) notebook in VScode (https://code.visualstudio.com/, accessed on 8 September 2022) with Python (https://www.python.org/, accessed on 8 September 2022) programming is used to develop the proposed model. A Python package known as natural language toolkit NLTK (https://www.nltk.org/, accessed on 15 September 2022) is used for preprocessing data. The text is vectorized using the Keras (https://keras.io/, accessed on 15 September 2022) Pandas (https://pandas.pydata.org/, accessed on 15 September 2022) and NumPy (https://numpy.org/, accessed on 15 September 2022) are used for data manipulation and analysis.

A multi-label text classification model can be evaluated using a group of evaluation matrices to capture various related aspects. Accuracy, Precision, Recall, F-measure, MicroF1, and MacroF1 are used to evaluate and compare the proposed model [87].

4.2. Datasets

Four standard publicly available datasets for the text-based news domain are used in this research. These datasets are a benchmark and have been used to evaluate several state-of-the-art machine learning models. Further details of each dataset are described below.

4.2.1. RCV1-v2

RCV1-v2 is an updated version of Reuters corpus volume1. It has a collection of newswire articles produced by Reuters from 1996 to 1997. It contains 804,414 manually labeled news documents [46]. The news documents are categorized based on industries, topics, and regions.

4.2.2. Reuters-21578

Reuters-21578 is the collection of news from one of the globally reputed news agencies, “Reuters”. The collection of news is from the Reuters newswire spanning 1987 [47]. We separate the dataset into 10,789 usable text articles and further divide it into a subset of 8631 data points for training and 2158 for testing.

4.2.3. Slashdot

Slashdot is a dataset obtained from the Slashdot website, a technology-related news-sharing community platform [48]. The dataset contains a total of 291 label classes and 19,258 multi-labeled news articles for training and 4814 news articles for testing.

4.2.4. NELA-GT-2019

NELA-GT-2019 is one of the latest datasets from the Harvard Dataverse. It is a large, multi-labeled news collection of around 1.12 million news articles [49]. It was obtained from 260 mainstream and alternative news sources from January 2019 to December 2019.

4.3. Baseline Models

The following baseline models are used for comparison with the proposed model.

4.3.1. CNN

The results of the proposed model are compared with a CNN; experiments include a CNN network with five convolutional layers along with max pooling for the pooling layer functions.

4.3.2. GA-CNN

The proposed model is also compared with a Genetic-Algorithm-optimized CNN (GA-CNN) [88]. GA is inspired by the theory of evolution and uses nature operations such as mutation, crossover, and selection. It determines and selects the fittest individual for the propagation of the search.

4.3.3. PSO-CNN

PSO-CNN is a Particle swarm optimized CNN. PSO is a stochastic optimization technique; it employs the concept of social interaction to solve search problems. It iteratively uses several items to achieve the optimization goal [89].

4.3.4. DE-CNN

DE-CNN is a CNN optimized by DE. DE is a population-based metaheuristics algorithm; it uses a large population of search agents to interact with each other and predict the best search in the solution space [90].

4.4. Results

In this paper, the SHO-CNN model for MLTC of news is proposed. SHO is utilized to optimize the CNN’s hyperparameters. It is concluded that no single hyperparameter is responsible for CNN’s optimization. A change in the value of one hyperparameter can have an effect on the values of other hyperparameters. Moreover, these optimal values are consistently dependent on the optimized values of other hyperparameters. SHO-CNN is evaluated using four benchmark datasets in the domain of MLTC of news: RCV1-v2, Reuters21578, Slashdot, and NELA-GT-2019. As this research aims to expand CNN’s capabilities through the metaheuristic optimization of hyperparameters, the CNN is optimized using a variety of metaheuristic techniques to determine the optimal algorithm for optimization. The resultant best values for the hyperparameters of SHO-CNN are represented in Table 4.

Table 4. Best values of hyperparameters of SHO-CNN.

Dataset	Learning Rate	Momentum	Dropout	Epoch	Batch Size	Nodes	Activation Function
RCV1-v2	0.001	0.95	0.4	300	64	256	ReLU
Reuters21578	0.001	0.94	0.5	300	64	256	ReLU
Slashdot	0.001	0.95	0.4	300	128	256	ReLU
NELA-GT-2019	0.001	0.96	0.3	150	32	256	ReLU

The performance of the suggested SHO-CNN model is greater than that of the baseline approaches. It achieves accuracies of 93.65% and 90.81% for the RCV1-v2 and Reuters21578 datasets, respectively. The baseline model CNN achieves accuracies of 91.12%, 87.86%, 68.73%, and 95.45% for the RCV1-v2, Reuters21578, Slashdot, and NELA-GT-2019 datasets, respectively. MLTC of news using the GA-CNN achieved accuracies of 92.32% and 88.34% for the RCV1-v2 and Reuters21578 datasets, respectively. The experiments with the PSO-CNN resulted in accuracies of 64.37% and 94.92% for the Slashdot and NELA-GT-2019 datasets, respectively. These results also confirm the study [79], which states that SHO optimization is more accurate than a basic CNN-, PSO-, or GA-optimized CNN. The results, along with models and evaluation matrices, are presented in Table 5.

Table 5. Multi-label text news dataset specifications.

Dataset	Model	Accuracy	Precision	Recall	f-Measure
RCV1-v2	CNN	0.911	0.902	0.799	0.851
	GA-CNN	0.923	0.921	0.905	0.918
	DE-CNN	0.925	0.92	0.921	0.928
	PSO-CNN	0.928	0.921	0.923	0.918
	SHO-CNN	0.936	0.930	0.935	0.931
Reuters-21578	CNN	0.878	0.875	0.866	0.871
	GA-CNN	0.883	0.881	0.885	0.888
	DE-CNN	0.889	0.888	0.884	0.886
	PSO-CNN	0.896	0.895	0.889	0.888
	SHO-CNN	0.908	0.902	0.909	0.901
Slashdot	CNN	0.571	0.501	0.564	0.530
	GA-CNN	0.592	0.591	0.590	0.590
	DE-CNN	0.627	0.620	0.615	0.617
	PSO-CNN	0.643	0.643	0.645	0.644
	SHO-CNN	0.687	0.683	0.682	0.682
NELA-GT-2019	CNN	0.917	0.914	0.903	0.908
	GA-CNN	0.923	0.922	0.914	0.918
	DE-CNN	0.939	0.931	0.939	0.935
	PSO-CNN	0.949	0.940	0.928	0.934
	SHO-CNN	0.954	0.950	0.951	0.951

A primary CNN is assessed together with an improved GA-CNN with a uniform mutation probability of 0.01, a crossover probability of 1, and a scaling factor of 0.5. PSO-CNN is also used in the evaluation process, with social, cognitive, and inertia constants of 1, 1, and 0.3, respectively. In addition, SHO-CNN with a control value of 1 and a random vector of 0.5 is also deployed. All optimization techniques use a population size of 30 with a maximum iteration count of 200.

As seen in Table 5, it is evident that SHO-CNN outperforms the other metaheuristic optimizations of CNNs for MLTC. Furthermore, SHO-CNN also achieves higher performance when compared to the baseline and other state-of-the-art approaches. LSTM, an upgraded version of CNN, is optimized using SHO [45], resulting in a higher hamming loss than SHO-CNN but superior micro-f1 and macro-f1 performance. Table 6 shows a comparison of the proposed SHO-CNN model with various state-of-the-art approaches on the RCV1-v2, Reuters21578, Slashdot, and NELA-GT-2019 datasets.

Table 6. Comparison between proposed model and similar research.

Dataset	Model	Hamming Loss	Micro-f1	Macro-f1
RCV1-v2	BR [91]	0.864	0.852	0.662
	CC [92]	0.872	0.855	0.682
	LP [93]	0.874	0.858	0.691
	CNN [25]	0.891	0.849	0.727
	BERT [94]	0.733	0.877	0.667
	CNN–RNN [95]	0.872	0.849	0.752
	SGM [96]	0.814	0.869	0.712
	SGM-GE [97]	0.753	0.878	0.762
	Seq2Set [98]	0.736	0.879	0.751
	ML-R [50]	0.793	0.898	0.695
	Seq2Tree [99]	0.732	0.868	0.700
	TextRCNN [100]	0.765	0.815	0.592
	HiAGM [100]	0.768	0.839	0.633
	HTCInfoMax [101]	0.748	0.835	0.627
	HiMatch [102]	0.754	0.847	0.641
	SHO-LSTM [45]	0.737	0.913	0.781
	SHO-CNN (Proposed)	0.722	0.906	0.776
Reuters21578	BR [91]	0.434	0.835	0.402
	PCC [103]	0.442	0.834	0.398
	seq2seq-RNN [104]	0.384	0.858	0.437
	BERT [94]	0.261	0.905	0.640
	LBA [105]	0.254	0.908	0.675
	ML-KNN [106]	0.376	0.732	0.253
	ML-ARAM [61]	0.363	0.626	0.163
	ML-RNN [104]	0.332	0.858	0.457
	LaMP [107]	0.291	0.889	0.560
	MPVAE [108]	0.267	0.887	0.542
	MrMP [109]	0.254	0.893	0.591
	Class-wise cluster + FS [110]	-	0.800	0.725
	CB-NTR [111]	0.247	0.907	0.633
	SHO-LSTM [45]	0.240	0.910	0.690
	SHO-CNN (Proposed)	0.237	0.908	0.665
Slashdot	BR [91]	0.052	0.486	0.362
	BR-Support [112]	0.055	0.516	0.357
	PCC [103]	0.056	0.480	0.279
	seq2seq-RNN [104]	0.058	0.528	0.270
	set-RNN [112]	0.053	0.538	0.310
	BERT [94]	0.037	0.583	0.380
	LBA [105]	0.038	0.582	0.410
	CC [92]	0.057	0.480	0.373
	CNN [25]	0.049	0.512	0.412
	CNN-RNN [95]	0.046	0.530	0.469
	MAGNET [112]	0.039	0.568	0.475
	SHO-LSTM [45]	0.038	0.650	0.492
	SHO-CNN (Proposed)	0.037	0.632	0.453
NELA-GT-2019	BR [91]	0.033	0.870	0.792
	BR-Support [112]	0.040	0.846	0.783
	PCC [103]	0.009	0.922	0.913
	LBA [105]	0.008	0.953	0.9420
	CNN [25]	0.008	0.934	0.926
	CNN-RNN [95]	0.008	0.935	0.944
	SHO-LSTM [45]	0.007	0.972	0.970
	SHO-CNN (Proposed)	0.006	0.963	0.956

According to the mathematical model of the SHO algorithm, to find a suitable prey ${\overrightarrow{L}}_p$, the factor $\mid Y\mid >1$ facilitates the spotted hyenas to move away from the prey. This promotes exploration of the search space, which leads to finding diverse CNN structures during optimization. The vector M provides random values for exploration; this mechanism is very beneficial for avoiding local optima problems even when the SHO algorithm is in the exploitation phase [88].

The results show that the proposed SHO-CNN model can perform better in real-time applications. The distinguishing factor in the proposed work is the introduction of SHO in a CNN model. As per our knowledge, this is the first case where an SHO-optimized CNN architecture is implemented for multi-label news classification. Furthermore, SHO-CNN achieves the best classification accuracies among all the baseline and modern techniques under consideration. The result of the proposed model demonstrates the significance of CNN in the domain of MLTC. This proposed algorithm has a very fast convergence rate and can successfully reduce the problem of being trapped in local minima while training the CNN. It also illustrates the capabilities of SHO for optimizing the hyperparameters of CNN.

Transformer encoders are best if computation is not a concern. When classes are fairly distributed, a CNN is a successful model due to its diligent performance and assessment efficiency. A CNN can perform better even without pre-trained word embeddings. The CNN model performed faster; on average, the CNN was 17.6% quicker than RNN, 91.3% faster than the Transformer encoder, and 94.7% faster than BERT-Base. BERT-Base is the slowest model due to the huge number of top-layer parameters [113].

More recently, CNNs have garnered attention for NLP tasks due to their higher performance, particularly on lengthier texts [114,115,116,117]. This is mostly due to the fact that CNNs are easier to train than other NLP methods. In NLP applications, CNNs use a convolutional layer with only one-dimension, which collects information from neighboring words. The number of filters in a one-dimensional convolutional layer considers n number of words next to one other as one (n-gram) [118,119,120]. Furthermore, adding a higher n-gram method to CNN algorithms does not significantly increase the computing cost [113].

4.5. Ablation Study

We further classified CNN hyperparameters into two groups: group A includes critical hyperparameters such as LR, mom, number of nodes, and activation function. Group B comprises advanced hyperparameters such as dropout, epoch number, and batch size, which are thought to have a significant impact on performance. Initially, we evaluated the suggested SHO-CNN model with only Group-A hyperparameters, which resulted in a significant performance improvement compared to CNN’s baseline performance. Secondly, we analyzed SHO-CNN with Group-B hyperparameters; the results show that optimizing only the advanced hyperparameters is insufficient for achieving high-performance results. The comparisons of hyperparameters from groups A and B are shown in Figure 3. According to the ablation study, the vital or advanced hyperparameters alone are not adequate for achieving high performance; rather, a mix of these hyperparameters is the optimum approach for improved performance.

Figure 3. Comparisons based on ablation study.

4.6. Discussion

The suggested SHO-CNN model outperforms the baseline and state-of-the-art approaches, as determined by experimentation. The selection of hyperparameter values plays an important part in the performance of the deep learning model. Selecting hyperparameters using a thorough hyperparameter search is computationally infeasible. Furthermore, selecting a random initialization for hyperparameters could result in subpar performance. Moreover, the choice of hyperparameter values is dependent on the dataset. This section compares SHO-CNN to the current state-of-the-art in terms of convergence rate, epoch, batch size, and learning rate.

4.6.1. Convergence Rate

The convergence rate for the proposed model is compared with the other state-of-the-art optimization techniques using RCV1-v2, Reuters-21578, Slashdot, and NELA-GT-2019 datasets. In Figure 4, the x-axis represents the number of iterations, and the y-axis represents a measure of fitness. The proposed model appears highly convergent and outperforms other techniques in vogue. The results also indicate that DE has a higher convergence rate when compared to GA and PSO for both datasets. The performance of DE and PSO are relatively similar, and they both performed better than GA and a simple CNN.

Figure 4. Evaluations based on convergence rate.

4.6.2. Execution Time

Using the RCV1-v2, Reuters-21578, Slashdot, and NELA-GT-2019 datasets, the execution time of the proposed model is compared to that of existing cutting-edge optimization strategies. The x-axis of Figure 5 reflects the optimization strategies used to optimize CNN alongside a primary CNN, while the y-axis represents the percentage of time. The findings reveal that the proposed model’s efficiency is fairly similar to that of a standard CNN, but the proposed model has faster convergence and better performance than a standard CNN. The results also reveal that PSO is superior to GA and DE in terms of performance.

Figure 5. Evaluations based on execution time.

4.6.3. Epoch

The evaluation of the proposed SHO-CNN model based on epoch is represented in Figure 6. The results reveal that SHO-CNN initiated at an accuracy of 87.5% and acquired a higher accuracy of 93.65% after 300 epochs for RCV1-v2. This optimal number of epochs provided by SHO resulted in encountering the underfitting problem of CNNs. It is evident from the results that SHO-CNN achieved the highest accuracy. A plain CNN, without any optimization, begins at 84.9% and 82.4% accuracy for RCV1-v2 and Reuters-21578, respectively, which is the lowest amongst the group. In the case of Reuters-21578, the proposed model also surpassed all the other mentioned optimization techniques. The proposed SHO-CNN model initiated at 86.5% and achieved 90.81% accuracy with 300 epochs for Reuters-21578. It also achieved an accuracy of 95.45% with 150 epochs for the NELA-GT-2019 dataset.

Figure 6. Evaluations based on epochs.

4.6.4. Batch Size

Evaluation of the proposed SHO-CNN model based on the selection of batch size is represented in Figure 7. Optimal batch size decreases noise in the gradients, hence enhancing the classification accuracy. The evaluation process included batch sizes of 8, 16, 32, 64, and 128. The accuracies of the proposed model are 87.5%, 90.9%, 92.3%, and 93.65% for RCV1-v2 at batch sizes of 8, 16, 32, and 64, respectively. The proposed SHO-CNN model yields maximum accuracies for the RCV1-V2, Reuters-21578, Slashdot, and NELA-GT-2019 datasets at batch sizes of 64, 64, 128, and 32, respectively.

Figure 7. Evaluations based on batch size.

4.6.5. Learning Rate

The proposed SHO-CNN model, along with CNN, GA-CNN, DE-CNN, and PSO-CNN are also evaluated based on varying learning rates (Figure 8). Exploring the optimal learning rate is the critical factor for a model’s best performance. The learning rate is correlated with other hyperparameters of a CNN, including momentum and batch size. The evaluation range for the learning rate is set to 0.001, 0.01, 0.05, and 0.1. The accuracies of the proposed model were 93.65%, 92.3%, 90.9%, and 87.5% accuracy for RCV1-v2 at learning rates of 0.001, 0.01, 0.05, and 0.1, respectively. SHO-CNN achieved the highest accuracies for the RCV1-V2, Reuters-21578, Slashdot, and NELA-GT-2019 datasets at a learning rate of 0.001.

Figure 8. Evaluations based on learning rate.

In this research, a basic CNN is used in the proposed model to capture spatial correlations and hierarchical structures of text with exceptional accuracy without increasing the model’s complexity [114,115,120]. The classification of text employs a one-dimensional CNN, resulting in the fastest execution and testing times [113,116,117]. Adding more filters does not significantly extend the execution time [27,119,120]. Five convolutional layers are utilized by SHO-CNN to extract high-level abstract properties [22]. Furthermore, the CNN is optimized through numerous optimization strategies; the inclusion of SHO enhances the variety of search solutions for optimizing hyperparameters while preventing the occurrence of local minima [44,45].

Although the SHO-optimized CNN performed well for the task of multi-label news text classification, there are some limitations of this research that need to be considered. Firstly, this study was conducted solely using the RCV1-V2, Reuters21578, Slashdot, and NELA-GT-2019 datasets, so it may not be generalized for other text classification tasks. Secondly, it is observed that SHO is occasionally susceptible to local optima when using a lesser number of repetitions. More recently proposed optimization algorithms (such as Political Optimizer [121], Heap-based Optimizer [122], Harris Hawks Optimizer [84]) can be tried for the optimization of CNNs. The classification of SHO-CNN is solely based on news text; combining text and images can generate good results.

5. Conclusions and Future Research

This research proposes an expert automated method based on SHO of CNNs for the purpose of MLTC of news. SHO is a metaheuristic-based approach that provides higher exploration of the solution space while avoiding the problem of becoming trapped in local optima. SHO is used in the proposed model to tune the CNN hyperparameters for improved classification performance. Furthermore, the CNN algorithm helps to improve the performance by spontaneously extracting the discriminant features from the text. As a result, the proposed model, SHO-CNN, is highly convergent and best-suited for multi-label classification. The proposed model is evaluated and appraised on four standard benchmark news datasets. SHO-CNN addresses the complexity of deep learning models by utilizing a primary CNN to achieve optimal performance compared to various baseline models, state-of-the-art models, and other optimization techniques for CNN. The results indicate that the proposed model has accuracies of 93.65%, 90.81%, 68.73%, and 95.45% for the RCV1-V2, Reuters-21578, Slashdot, and NELA-GT-2019 datasets, respectively. The results also reveal that SHO-CNN enhances a basic CNN’s accuracies on the RCV1-V2, Reuters-21578, Slashdot, and NELAGT-2019 datasets by 2.5%, 2.95%, 11.6%, and 3.7%, respectively. Future studies should investigate optimization approaches for different DL models. A multimodal approach to the classification of news is also a viable future strategy, given the majority of news is now disseminated via the internet, particularly social media. In addition, a future study will incorporate a model with the capacity to classify multilingual news articles.