Investigation of Multiple Hybrid Deep Learning Models for Accurate and Optimized Network Slicing

Abstract

In 5G wireless communication, network slicing is considered one of the key network elements, which aims to provide services with high availability, low latency, maximizing data throughput, and ultra-reliability and save network resources. Due to the exponential expansion of cellular networking in the number of users along with the new applications, delivering the desired Quality of Service (QoS) requires an accurate and fast network slicing mechanism. In this paper, hybrid deep learning (DL) approaches are investigated using convolutional neural networks (CNNs), Long Short-Term Memory (LSTM), recurrent neural networks (RNNs), and Gated Recurrent Units (GRUs) to provide an accurate network slicing model. The proposed hybrid approaches are CNN-LSTM, CNN-RNN, and CNN-GRU, where a CNN is initially used for effective feature extraction and then LSTM, an RNN, and GRUs are utilized to achieve an accurate network slice classification. To optimize the model performance in terms of accuracy and model complexity, the hyperparameters of each algorithm are selected using the Bayesian optimization algorithm. The obtained results illustrate that the optimized hybrid CNN-GRU algorithm provides the best performance in terms of slicing accuracy (99.31%) and low model complexity.

1. Introduction

Fifth-generation (5G) and beyond networks are planned to enable a wide range of new applications like industry v4, automobiles, and smart cities, which have substantially higher performance and cost requirements than traditional mobile broadband services [1]. Therefore, these networks should have flexible and scalable structures for meeting these different constraints, including performance, security, availability, and cost minimization. To achieve these goals, network slicing has been proposed by researchers and industry as a key enabler to offering customized 5G network services using the same physical network infrastructure [2]. Thanks to Network Functions Virtualization (NFV) and Software Defined Networking (SDN) technologies, which paved the way to creating network slices that enable many logical networks to operate independently over shared physical infrastructure [3], each logical network can offer personalized services for a certain application scenario [4].

Network slicing creates multiple virtual networks in a single physical infrastructure, where each virtual network meets the requirements for a specific service.

In 5G networks, large amounts of data should be analyzed before decisions are made to select network slices to ensure that the network can adequately meet QoS requirements [4]. Therefore, machine and deep learning models can be used to analyze large amounts of data and make the most accurate predictions of network slices in 5G networks. Additionally, these models should be optimized in terms of complexity to provide fast decisions for network slicing [5].

The hybridization of different deep learning methods can leverage the complementary strengths of each model and improve generalization, therefore leading to better performance, robustness, or adaptability.

Convolutional neural networks have the advantage of extracting and reducing spatial features, whereas recurrent neural networks are superior at modeling sequential or temporal dependencies. Therefore, combining convolutional neural networks with recurrent neural network variations can lead to better accuracy when used for network slicing. Although combining different deep learning methods can result in relatively complex models, optimization techniques can be beneficial for reducing the model complexity by selecting the best hyperparameters while assuring the best model performance.

In this paper, an optimized hybrid deep learning-based wireless network slicing model for 5G networks is proposed to accurately select the appropriate network slices with less complexity.

The main contributions of this work are summarized as follows:

• Multiple hybridizations of DL models, including CNN-LSTM, CNN-RNN, and CNN-GRU, are investigated for building an accurate 5G network slicing model.
• A Bayesian-based algorithm is utilized for hyperparameter and model structure optimization to improve the accuracy and reduce the complexity of the proposed models.
• Two types of datasets from different repositories are utilized to analyze and evaluate the performance of the suggested model.
• The proposed approach is tested and evaluated using simulated network scenarios.

The rest of the paper is structured as follows. In Section 2, the most related works are discussed. The theoretical background of the main components of this work is described in Section 3. Section 4 illustrates the proposed methods. The evaluation of the proposed methods is described in Section 5. Finally, the conclusions are presented in Section 6.

2. Related Work

There are several research studies which focused on designing efficient decision making to select accurate slicing in a network environment. A concise review of the most relevant literature is illustrated below.

In [6], based on a device’s important attributes, a hybrid network slicing mechanism based on CNN-LSTM was presented for the best prediction of the most suitable network slice for all incoming network traffic. The CNN is responsible for resource allocation and slice selection, and the LSTM manages network slice load balancing. The necessity for efficient network slicing in 5G is critical in terms of lowering network operator costs and energy usage while maintaining service quality. The authors of [7] used application-specific radio spectrum scheduling to apply deep learning models to the radio access network (RAN) slicing architecture. In [8], the authors applied an auto-encoded neural network architecture which is known as the deep auto-encoded dense neural network algorithm for network slicing. The authors of [9,10,11] forecasted future traffic demand patterns using deep neural networks (DNNs) based on either the spatiotemporal linkages between stations or the real-time network traffic load. The research studies in [12,13,14] applied the recurrent neural network (RNN) for network slicing along with mobility prediction and management in wireless 5G networks. Additionally, in [15,16,17,18], the authors used a slicing strategy based on reinforcement learning (RL), where for network slicing, RL revises resource allocation choices. In [19], the authors proposed a framework for maximizing device application performance using optimized network slice resources. To solve network load balancing issues more effectively, a machine learning-based network sub-slicing framework in a sustainable 5G environment was designed, where each logical slice was separated into a resource-virtualized sub-slice. In [20,21], the authors utilized the Support Vector Machine (SVM) algorithm to automate network functions for the creation, construction, deployment, operation, control, and management of network slices. Likewise, the authors of [22] used various machine learning methods such as Extra Tree, AdaBoost, SVM, extreme gradient boosting (xGB), Light Gradient Boosting Machine (LGBM), k-nearest neighbors (k-NN), and multi-layer perceptrons (MLPs) for slice-type classification. In [23], different deep learning models such as the attention-based encoder decoder, MLP, LSTM, and GRU are used to predict network slice segmentation (streaming, messaging, search, and cloud).

Unlike the previous research studies, which focused on a standalone or single approach in network slicing classification, there is another research route that focuses on using hybrid deep learning methods in the context of network slicing. The work that used hybridization of deep learning methods for network slicing is summarized as follows.

The authors of [24] used a 5G network slicing dataset to build a hybrid learning technique that included three phases: data collection, Optimal Weighted Feature Extraction (OWFE), and slicing classification. To improve network slicing effectiveness and accuracy, the author used glowworm swarm optimization and deer hunting optimization algorithms with neural networks and deep belief network-based network slicing. For efficient network slicing in 5G networks, the authors of [25] suggested a three-phase process that consists of loading the dataset, optimizing using Harris Hawk Optimization (HHO), and a hybrid deep learning model for network slice classification. Initially, they imported the datasets and optimized the hyperparameters using HHO. After that, a hybrid deep learning model based on LSTM and a CNN is employed.

Although different deep learning techniques, including hybrid models, have been employed for 5G network slicing, more improvements are still required due to the continuous increase in the number of applications and user devices in 5G and beyond networks. This can be achieved through the investigation of different combinations of hybrid deep learning models and optimization techniques to achieve higher accuracy with consideration of complexity reduction. In addition, using different datasets for evaluating the model ensures the generalization and robustness of the final model.

3. Background

This section provides a theoretical background for the main component of this work, including the network slicing mechanism and the hybrid structure of a CNN with (RNN, LSTM, and GRU) deep learning models, along with the concept of Bayesian optimization for hyperparameter tuning.

3.1. Network Slicing

Network slicing is a key component of 5G technology for meeting a variety of user service requirements. In particular, operators create many logically distinct networks over a shared physical network infrastructure. This leads to logical resource separation between network slices and ensures that services on different network slices have no conflict with each other, as shown in Figure 1. Furthermore, network slicing technology can dynamically distribute resources to numerous slices on the same physical network in response to varying demands for network resources across slices [1,25]. In fact, there are three main categories identified for 5G mobile networks sliced by the International Telecommunication Union which provide services to different network users and applications, including enhanced mobile broadband (eMBB), ultra-reliable low-latency communications (URLCs), and massive Internet of things (MioT) [6].

The key challenge of network slicing is how to assign proper network slices to different incoming clients [2]. When an inaccurate model is used for slice selection, a significant number of users may be assigned to the incorrect slice, which results in slice overutilization [2,26]. This may lead to an increasing user drop rate, which ultimately degrades the network performance to meet the required QoS.

3.2. Hybrid Deep Learning Models

In the hybrid approach, the best merits of two or more algorithms are combined to increase the model’s accuracy. Recurrent and convolutional neural networks are used as a hybrid technique in this context for accurate slice selection. In this approach, in order to select a proper network slice, the present network connection type can be categorized using prior traffic connection records because the network traffic events follow time series patterns. In the hybrid paradigm, as illustrated in Figure 2, the CNN architecture can be utilized to extract the important features of input data through its convolution and max pooling process [27,28]. Then, the features produced by the CNN are fed to either an RNN, LSTM, or GRU to capture the time series patterns and achieve accurate network slice prediction.

The mathematical formulation of the hybrid deep learning models shown in Figure 2 is described as follows. The first stage produces a feature map ($Fmp$), which represents the output from the CNN stage as shown in Equation (1):

$$Fmp=\mathrm{C}\mathrm{N}\mathrm{N}\left(x_t\right)$$

(1)

Here, the CNN architecture is made up of convolution and max-pooling layers, and $x_t$ is the input feature vector labeled with a class. To capture the long-term temporal dependencies, the newly generated Fmp is fed into the second stage represented by the RNN, LSTM, and GRU.

The next subsections include both theoretical and mathematical explanations of CNN, RNN, LSTM, and GRU algorithms.

3.2.1. Convolutional Neural Network

A convolutional network, often known as a convolutional neural network (CNN), is a biologically inspired addition to traditional feedforward networks (FFNs) [25]. The CNN architecture includes convolution layers, pooling layers, and completely connected layers [28]. The CNN organizes its data in the shape of a two-dimensional mesh and a one-dimensional mesh for the time series data. The basic CNN architecture is shown in Figure 3.

A CNN is composed of a one-dimensional convolution layer, a one-dimensional max pooling layer, a fully linked layer, and a nonlinear activation function known as ReLU.

The input vector x represents a one-dimensional dataset of network traffic time series events. A one-dimensional convolution layer is used to constructs the feature map $fm$ by applying convolution operation on the input data features $f$ using a filter $W$. A newly generated feature map $fm$ is created from a set of features $f$ as shown in Equation (2):

$${hl}_i=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}(W_i{x}_{i+f+1}+b)$$

(2)

where $b$ is a biased term. The filter ${hl}_i$ is applied to each set of features $x$ which is learned by the filter $i$ to produce a feature map, followed by a max pooling operation where $hl$ = max $hl$. The max pooling process yields the most substantial attributes, with the highest value being chosen. Multiple features, on the other hand, acquire more than one feature, and those additional features are transmitted to the fully connected layer. The softmax function, which yields the probability distribution across each class, is included in a fully linked layer, which is formally defined as follows:

$$o_t=softmax(w_{ho}{h}_l+b_o)$$

(3)

where $w_{ho}$ is the weight matrix and $b_o$ is the output bias for each $t$ class.

3.2.2. Recurrent Neural Network

A kind of neural network known as a recurrent neural network (RNN) is dedicated to processing sequential data [29]. The data of the hidden layer ($H_t$) for the current input state ($X_t$) enters the state at the next moment. A basic RNN block is demonstrated in Figure 4.

Equations (4) and (5) below are used in the calculations of the model [29]:

$$h_t=\tanh \left(W_{xh}{x}_t+W_{hh}{h}_{t-1}+b_h\right)$$

(4)

$$y_t=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}(W_{hy}{h}_t+b_y)$$

(5)

where $x_t$ is the current input, $h_{t-1}$ is the previous moment’s hidden layer information, and h_t represents hidden layer information that will be employed in this node current and delivered to the further node. The three weight matrices are $W_{xh}$, $W_{hh}$, and $W_{hy}$. The bias vectors are $b_h$ and $b_y$. The hyperbolic tangent function is the activation function tanh. For the hybrid CNN-RNN, the input vector x will feed from Fmp, which denotes the new feature vectors that are obtained from the CNN.

3.2.3. Long Short-Term Memory

Long Short-Term Memory (LSTM) is a subtype of the RNN architecture. The general LSTM block is shown in Figure 5.

The formulas illustrated in Equations (6)–(10) are used to determine the LSTM architecture’s output and hidden layer parameters [30]:

$$i_t=\sigma (W_{xi}{x}_t+W_{hi}{h}_{t-1}+W_{ci}{c}_{t-1}+b_i)$$

(6)

$$f_t=\sigma (W_{xf}{x}_t+W_{hf}{h}_{t-1}+W_{cf}{c}_{t-1}+b_f)$$

(7)

$$c_t=f_t{c}_{t-1}+i_t\tanh (W_{xc}{x}_t+W_{hc}{h}_{t-1}+b_c)$$

(8)

$$o_t=\sigma (W_{xo}{x}_t+W_{ho}{h}_{t-1}+W_{co}{c}_t+b_o)$$

(9)

$$h_t=o_t\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left(c_t\right)$$

(10)

The structure equations consisting of the input gate (i), forgetting gate (f), exit gate (o), and cell state (c) are specified between Equations (6) and (8). The expression t indicates the current state, and the expression t − 1 indicates the previous state. In Equation (6), the weight matrix is represented by $W_{xi}$, $W_{hi}$, and $W_{ci}$, and the bias vector is $b_i$. In Equation (7), $W_{xf}$, $W_{hf}$, and $W_{cf}$ refer to the weight matrix, and $b_f$ denotes to the bias vector. In Equation (8), $W_{xc}$ and $W_{hc}$ denote the weight matrix, and $b_c$ represents the bias vector. $W_{xo}$, $W_{ho}$, and $W_{co}$ in Equation (9) represent the weight matrix, and $b_o$ is the bias vector. A sigmoid function $\sigma$ is also used in these equations. In Equation (10), tanh is the hyperbolic tangent function [29]. In the hybrid CNN-LSTM model, the input vector x is replaced by Fmp, which indicates the newly computed CNN feature vectors.

3.2.4. Gated Recurrent Units

Gated Recurrent Units (GRUs) are a kind of RNN that resembles an LSTM system but is more straightforward. Examining the GRU structure reveals that the approach differs from LSTM, in which a single gate unit manages the state unit’s update choice in addition to possessing the forgetting factor [31]. The structure of a simple GRU block is illustrated in Figure 6.

The update unit ($z_t$) and deletion unit ($r_t$) serve as the forget and state units that make up the GRU structure. Equations (11)–(14) comprise the GRU architecture’s equations:

$$r_t=\sigma (W_{ir}{x}_t+b_{ir}+W_{hr}{h}_{t-1}+b_{hr})$$

(11)

$$z_t=\sigma (W_{iz}{x}_t+b_{iz}+W_{hz}{h}_{t-1}+b_{hz})$$

(12)

$${\tilde{h}}_t=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}(W_{in}{x}_t+b_{in}+r_t\left(W_{hn}{h}_{t-1}+b_{hn}\right)$$

(13)

$$h_t=\left(1-z_t\right){\tilde{h}}_t+z_t{h}_{t-1}$$

(14)

As seen in the previous equations, the current time $t$ refers to the previous time $(1-t)$, and the hidden layer h_t represents the current input $x_t$. $W$ represents the weight matrix. $b$ is the bias vectors, $\sigma$ is the sigmoid function, and tanh is the hyperbolic tangent function. The feature vectors Fmp obtained from the CNN replace the input vector x in the hybrid CNN-GRU model.

3.3. Bayesian Optimization for Hyperparameter Tuning

Hyperparameters are commonly defined as the configuration of a neural network structure. These hyperparameters include model structure parameters and learning method parameters [32]. Choosing the right values of these hyperparameters is crucial to helping neural networks learn more quickly and perform better [33]. The manual selection of hyperparameter values via trial experiments is a time-consuming process, and the optimal value of a hyperparameter may not be found. Therefore, optimization algorithms can be applied to hyperparameter optimization (HPO) to find the best hyperparameter values and improve the performance of neural networks.

The Bayesian optimization (BO) approach for hyperparameters was introduced by Snoek et al. in 2012 [33]. In BO, as in random search, a subset of hyperparameter combinations is sampled. However, the BO and random search methods differ from each other in the combination selection stage [34].

The goal is to allow the model to work in a way that raises the final prediction score (e.g., accuracy value). Thus, it is possible to make fewer evaluations and save time. During the selection process, probability calculation is made with the Bayes theorem, and the selection process proceeds according to the results of this process [32], as shown in Equation (15):

$$\mathrm{P}\left(\mathrm{V}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{a}\mathrm{b}\mathrm{l}\mathrm{e}\right|\mathrm{H}\mathrm{y}\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{p}\mathrm{a}\mathrm{r}\mathrm{a}\mathrm{m}\mathrm{e}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{s})$$

(15)

After the probability model is obtained, the (variable-hyperparameter) combinations are selected from the observations where the performance probability is high. However, when the performance probability is low, the process will continue by considering another subset of the hyperparameter space [34].

As shown in Equation (16), BO solves problems by determining the set of hyperparameters $x$ that minimizes the objective function $f\left(x\right)$ in a finite domain, with lower and upper bounds on each variable:

$$x=\arg \min f\left(x\right)wherex\in X$$

(16)

The flow diagram of hyperparameters tuning using BO for the proposed hybrid deep learning model for network slicing is illustrated in Figure 7 [33].

In this work, the particular mathematical formulation of the model optimization can be described as follows. The main objective is to minimize the loss function $L\left(\mathrm{\varnothing }\right)$ of the deep learning model, where the $\mathrm{\varnothing }$ model parameters need to be optimized and L is the mean square error (MSE). These parameters include the learning rate $\eta$, batch size $\beta$, number of epochs $\epsilon$, dropout rate $\delta$, and number of units $\mu$. Therefore, the model parameters $\mathrm{\varnothing }$ can be expressed as shown in Equation (17):

$$\mathrm{\varnothing }=(\eta ,\beta ,\epsilon ,\delta ,\mu )$$

(17)

In order to control the model’s complexity and training efficiency, a constraint is imposed on these model parameters as shown in Equation (18):

$$\left[\begin{array}{c}{\eta }_{min}\le \eta \le {\eta }_{max}\\ {\beta }_{min}\le \beta \le {\beta }_{max}\\ {\epsilon }_{min}\le \epsilon \le {\epsilon }_{max}\\ {\delta }_{min}\le \delta \le {\delta }_{max}\\ {\mu }_{min}\le \mu \le {\mu }_{max}\end{array}\right]$$

(18)

where the min and max define the parameter space $\mathrm{H}$, which is shown in Equation (19):

$$\mathrm{H}=\left[{\eta }_{min},{\eta }_{max}\right]\times \left[{\beta }_{min},{\beta }_{min}\right]\times \left[{\epsilon }_{min},{\epsilon }_{max}\right]\times \left[{\epsilon }_{min},{\epsilon }_{max}\right]\times [{\mu }_{min},{\mu }_{max}]$$

(19)

The objective of BO is to minimize the loss function $MSE\left(\mathrm{\varnothing }\right)$ as illustrated in Equation (20) to find the best parameters that ensure the best model performance.

$$\underset{\mathrm{\varnothing }\in H}{\min }MSE\left(\mathrm{\varnothing }\right)$$

(20)

4. The Proposed Hybrid Models

The hybrid deep learning-based architecture for network slicing is presented in Figure 8. The proposed architecture includes a combination of CNN-LSTM, CNN-RNN, or CNN-GRU models to achieve an accurate slicing model. The proposed models would select the proper slice based on different (key performance indicator) parameters such as the user equipment category, packet delay budget, and maximum packet loss [2]. Additionally, to enhance the proposed model’s performance and complexity, hyperparameter optimization using the BO method is adopted. The details of the proposed models’ design will be explained in the next subsection.

Although each algorithm for the hybrid deep learning model was previously described separately, the detailed structure of the proposed hybrid deep learning structure for network slice prediction is shown in Figure 9. This structure employs CNN (two convolution and one max pooling) layers for the feature extraction process of the input data combined with either an LSTM, RNN, or GRU layer to support network slice prediction by the dense and the softmax layers.

4.1. Dataset Description

In order to evaluate the proposed methods, two of the most commonly used datasets were considered. DeepSlice was the first dataset, which was published by Thantharate et al. [2]. It was used in this study to estimate the appropriate network slice for all incoming queries. This dataset contains key performance indicators (KPIs) for networks and devices, along with different features including the user equipment (UE) category, QoS class identifier (QCI), packet delay budget, maximum packet loss, day and time information, whether conditions (normal or harsh) exist, and any additional information among the many types of devices that are connected to the internet.

The second dataset was 5G Network Slicing [24], which comprises 30,000 instances, 10 attributes, and three classes (eMBB, URLLC, and mMTC).

Table 1. Dataset descriptions [2,24].

Dataset Name	Instances	Features Number	Classes
DeepSlice	63,168	8	eMBB, URLLC, mMTC, and Master slice
5G Network Slicing	30,000	10	eMBB, mMTC, and URLLC

shows the primary properties of each dataset.

Each dataset was preprocessed through data cleaning, one-hot encoding, and min-max normalization steps.

4.2. Model Performance Evaluation Metrics

The accuracy, precision, recall, and F1 score performance matrices were used to evaluate the performance and effectiveness of the proposed hybrid deep learning models. The mathematical expression of each metric is described in

Table 2. Model evaluation metrics.

Criteria	Formulation	Description
Accuracy	$TP+TN/TP+TN+FP+FN$	This represents the proportion of correct predictions out of the total predictions of a classification model.
Recall	$TP/TP+FN$	This measures how many true positives a classification model correctly detects.
Precision	$TP/TP+FP$	This measures how many of the examples that a classification model predicts as positive are actually positive.
F1 Score	$2TP/2TP+FP+FN$	This provides a combined measure of the precision and recall performance of a classification model.

. The confusion matrix [35] was used to calculate these metrics. The confusion matrix consisted of four different numerical indices containing predictive and actual values. The confusion matrix is shown in Figure 10, where true positive (TP) and true negative (TN) represent correctly predicted values, while false positive (FP) and false negative (FN) represent incorrectly predicted values.

Another metric was used, namely the confidence interval. This was calculated using statistical distributions at a given confidence level. The margin of error helps measure the accuracy of predictions, while the confidence interval provides a degree of certainty in the results. The confidence interval CI of a model can be calculated as follows [36]:

$$CI=\rho \mp z\sqrt{{\displaystyle \frac{p(1-p)}{n}}}$$

(21)

where $\rho$ is the model accuracy, z is the level z score of 95%, and n is the number of samples.

5. Results and Discussion

The performance of the proposed hybrid models is examined in this section. The evaluation process is divided into two parts. The first part involves the proposed hybrid models’ evaluation, while the second part of the evaluation focuses on the network performance with a selected slicing model. The applicability of the proposed model was investigated individually using the DeepSlice dataset and the 5G Network Slicing dataset. Ten-fold cross-validation was used with a ratio of 70:30 for training and testing, respectively. The evaluation parameters described in Section 4.2 were considered. The proposed models were initially evaluated without using hyperparameter optimization, and then evaluation was achieved with the presence of the BO optimization algorithm. Initially, the hyperparameters illustrated in

Table 3. The manually selected hyperparameters.

Hyperparameter	Value
Activation Function	ReLU
Loss Function	Cross-entropy
Optimizer	Adam
Batch Size	64
Learning Rate	0.001
Number of Epochs	100
Dropout	0.2
LSTM, RNN, and GRU	2

were manually selected for each hybrid model using trial experiments.

The evaluation results for each model using the DeepSlice dataset and the 5G Network Slicing dataset are shown in

Table 4. The evaluation results of the proposed models using the DeepSlice dataset.

Model	Accuracy	Recall	Precision	F1 Score	Confidence Interval
CNN-GRU	97.08	97.09	97.48	97.28	96.92–97.24
CNN-LSTM	96.97	97.47	96.25	96.86	96.81–97.13
CNN-RNN	88.22	83.54	86.64	85.06	87.91–88.53

and

Table 5. The evaluation results of the proposed models with 5G Network Slicing dataset.

Model	Accuracy	Recall	Precision	F1 Score	Confidence Interval
CNN-GRU	93.77	93.02	92.96	92.99	93.46–94.08
CNN-LSTM	91.57	91.67	91.78	91.72	91.24–91.90
CNN-RNN	86.24	85.22	84.33	84.77	85.75–86.73

, respectively.

It can be noticed that for both datasets, the CNN-GRU model achieved the highest accuracy. The CNN-LSTM model achieved comparable performance to the CNN-GRU model. On the other hand, the CNN-RNN model achieved the worst performance compared with the CNN-LSTM and CNN-GRU models. This is because these two models (CNN-GRU and CNN-LSTM) are better at capturing the long-term dependencies from the data compared with the CNN-RNN model. It is worth mentioning that the models trained using a larger DeepSlice dataset obtained better accuracy, as shown in Table 4, compared with the models trained using the 5G Network Slicing dataset. Generally, a large dataset provides more information, which leads to more generalization in the model. Alternatively, the 5G Network Slicing dataset has additional features that could introduce noise or might be nonrelevant, therefore providing models with lower accuracy.

Regarding the complexity analysis of the proposed CNN-RNN, CNN-LSTM, and CNN-GRU methods, it is worth stating that the complexity difference mainly depends on the RNN, LSTM, and GRU parts of the method, since the CNN is common.

The complexity of the CNN algorithm depends on the input size, number of filters, filter size, and depth. The complexity of the RNN unit depends on the weights for the input and the hidden state only. The complexity of the GRU unit depends on the weights for the input and the hidden state, with additional update gate and reset gate weights. On the other hand, the complexity of LSTM units depends on four sets of weights for each gate or cell update. Therefore, the CNN-LSTM model is considered the most complex model, the CNN-GRU is the moderate model, and the CNN-RNN is the least complex model.

It is worth mentioning that the model complexity had an inverse proportion with the scalability of the model and a direct proportion with the training time when using larger data and model sizes.

Then, the BO optimization technique was used to tune the hyperparameters to improve the accuracy and reduce the complexity of the proposed models by reducing the number of model units.

Based on the constraints mentioned in Section 3.3, both the search range for the selected parameter and the optimal value were obtained using BO for the proposed models trained with the DeepSlice dataset and for the same models trained with the 5G Network Slicing dataset, as shown in

Table 6. Search values and the optimal hyperparameters using the DeepSlice dataset.

Parameter	Search Values	Optimized CNN-LSTM	Optimized CNN-GRU	Optimized CNN-RNN
Learning rate $(\eta$)	0.0001–1	0.001	0.001	0.0005
$\mathrm{Batch}\mathrm{size}(\beta$)	[16,32,64]	32	64	32
$\mathrm{Number}\mathrm{of}\mathrm{epochs}(\epsilon$)	50–250	77	62	95
$\mathrm{Drop}\mathrm{rate}(\delta$)	[0.1–0.5]	0.2	0.2	0.3
$\mathrm{Number}\mathrm{of}\mathrm{LSTM},\mathrm{RNN},\mathrm{and}\mathrm{GRU}\mathrm{units}(\mu$)	1–5	4	2	4

and

Table 7. Search values and the optimal hyperparameters using the 5G Network Slicing dataset.

Parameters	Search Values	Optimized CNN-LSTM	Optimized CNN-GRU	Optimized CNN-RNN
$\mathrm{Learning}\mathrm{rate}(\eta$)	0.0001–1	0.0003	0.0009	0.0003
$\mathrm{Batch}\mathrm{size}(\beta$)	[16,32,64]	32	16	32
$\mathrm{Number}\mathrm{of}\mathrm{epochs}(\epsilon$)	50–250	78	87	98
$\mathrm{Drop}\mathrm{rate}(\delta$)	[0.1–0.5]	0.2	0.3	0.3
$\mathrm{Number}\mathrm{of}\mathrm{LSTM},\mathrm{RNN},\mathrm{and}\mathrm{GRU}\mathrm{units}(\mu$)	1–5	5	2	4

, respectively.

It can be observed that after using the best optimized hyperparameters, the evaluation results for the optimized models using the DeepSlice dataset are shown in

Table 8. Performance analysis of proposed optimized models using the DeepSlice dataset.

Model	Accuracy	Recall	Precision	F1 Score	Confidence Interval
BO-CNN-GRU	99.33	99.41	99.22	99.31	99.10–99.56
BO-CNN-LSTM	98.98	98.9	99.35	99.12	98.69–99.27
BO-CNN-RNN	97.99	97.67	96.88	97.27	97.56–98.42

, and the results while using the 5G Network Slicing dataset are shown in

Table 9. Performance analysis of the proposed model with 5G Network Slicing dataset.

Model	Accuracy	Recall	Precision	F1 Score	Confidence Interval
BO-CNN-GRU	96.21	95.86	96.19	96.02	95.96–96.46
BO-CNN-LSTM	96.59	95.27	95.22	95.24	96.34–96.84
BO-CNN-RNN	93.21	92.34	94.47	93.39	92.88–93.54

.

It can be noticed that, based on the results for both datasets in Table 8 and Table 9, the BO-CNN-GRU model with optimized hyperparameters obtained the highest accuracy, followed by the BO-CNN-LSTM and BO-CNN-RNN models. The optimized BO-CNN-LSTM model achieved highly comparable results to the BO-CNN-GRU model. However, it is worth stating that the number of units (layers) in the BO-CNN-GRU model (two units) was smaller than the number of units in the BO-CNN-LSTM (four and five units) model. Therefore, the optimized BO-CNN-GRU model had less complexity.

To investigate the impact of hyperparameter optimization, a comparison between the performance of the proposed hybrid deep learning models with and without using hyperparameter optimization was achieved as shown in Figure 11 for both datasets.

The results in Figure 11 show that using hyperparameter optimization had a positive impact on increasing the model’s performance. The percentage of improvement in the proposed models in terms of accuracy with hyperparameter optimization is summarized in Figure 12.

Although the CNN-RNN did not achieve the highest results in terms of accuracy, the use of hyperparameter optimization resulted in significant improvement in the model’s performance.

Based on the evaluation results of the proposed hybrid deep learning models for 5G mobile network slicing, it was found that the hybrid BO-CNN-GRU model with optimized hyperparameters using the BO algorithm stratified the required aim of this research in terms of accuracy and complexity (accuracy of 99.31% and two GRU units). Therefore, it was selected as a slice prediction model.

Table 10. A comparison between the proposed method and the state of the art.

Method	Accuracy	Dataset
GS-DHOA-NN+DBN [24]	91	5G Network Slicing
DNN [2]	95	DeepSlice
CNN-LSTM [6]	93	DeepSlice
LSTM [23]	89	5G Network Slicing
HHO-CNN-LSTM [25]	94	5G Network Slicing
CNN [25]	88	5G Network Slicing
The proposed BO-CNN-GRU	96.02	5G Network Slicing
The proposed BO-CNN-GRU	99.31	DeepSlice

provides a comparison between the performance of the proposed best BO-CNN-GRU model in terms of accuracy and the state-of-the-art methods using similar datasets.

The comparison shown in Table 10 shows that the proposed optimized BO-CNN- GRU and BO-CNN-LSTM hybrid models outperformed the other similar, state-of-the-art methods in terms of accuracy. Based on the results presented in Table 10, the hybrid deep learning models [6,24,25] and the proposed BO-CNN-GRU model achieved higher accuracy when compared with the standalone deep learning models in [2,25,32]. This is due fact that the hybrid model can gain the best merit of the combined methods.

When comparing the proposed BO-CNN-GRU model to the GS-DHOA-NN+DBN [24] model, the proposed model can provide greater accuracy, since the CNN-GRU model is better in some aspects, such as learning spatial and temporal features, compared with neural networks and deep belief networks, which aim to learn unsupervised features only. Comparing the HHO-CNN-LSTM [25] model with the proposed BO-CNN-GRU model shows that our model provided greater accuracy because the GRU has a simpler structure, which reduces the risk of overfitting. Additionally, it was fine-tuned better by the proposed BO optimization.

The obtained results show that the Deepslice dataset provided the models with better accuracy compared with the 5G Network Slicing dataset, as the larger dataset presented numerous examples, enhancing the model’s potential to generalize and reduce overfitting.

The next evaluation stage will focus on comparing the network performance using the best obtained BO-CNN-GRU slicing model and the less accurate BO-CNN-RNN model.

In the first scenario, to verify the effectiveness of the optimized BO-CNN-GRU slicing model, a 12-h network simulation was carried out. Over 250,000 user connection requests were generated, which consisted of 50% eMBB, 20% mMTC, and 30% URLLC. Figure 13 shows the simulation results in terms of the slice utilization (which is the ratio between used resources and the available resources in a specific network slice) and user drop rate (which refers to the percentage of users within a specific network slice who were disconnected due to slice overutilization) for the network simulation.

Similarly, in the second scenario, a network simulation was carried out using the less accurate BO-CNN-RNN slicing model. The simulation results in terms of the slice utilization and user drop rate for the second scenario are shown in Figure 14.

To clarify the difference between the network performance using the accurate BO-CNN-GRU slicing model and the low-accuracy BO-CNN-RNN model, the average enhancement in slice utilization and the reduction in the user drop rate were calculated, as shown in

Table 11. A comparison between network performance using BO-CNN-GRU and BO-CNN-RNN slicing models.

Slice	Slice Utilization BO-CNN-RNN	Slice Utilization BO-CNN-GRU	Slice Utilization Enhancement (%)	Drop Rate BO-CNN-RNN	Drop Rate BO-CNN-GRU	Drop Rate Reduction (%)
eMBB	83.67	91.08	8.86%	0.38	0.14	63%
mMTC	89.67	93.17	3.90%	0.58	0.07	88%
URLLC	72.75	93.00	27.84%	0.28	0.22	22%

.

Based on the results in Table 11, the average improvement in the network’s slice utilization was 13.53%, and the reduction in the drop rate was 58% when the BO-CNN-GRU slicing model was used compared with the less accurate BO-CNN-RNN slicing model.

6. Conclusions

Network slicing is crucial for 5G wireless communication and beyond, as mobile operators can provide various service requirements. Inaccurate and slow slice allocation can negatively impact a network’s performance in providing the required QoS. This paper investigated different hybrid deep learning methods and hyperparameter optimization to obtain an accurate network slicing model with low complexity. The evaluation results show that the hybridization of CNN and GRU methods with hyperparameters optimized using the Bayesian method (BO-CNN-GRU) obtained the best accuracy of 99.31% for network slicing, which outperformed the state-of-the-art methods. The network simulation shows that the proposed accurate network slicing represented by the BO-CNN-GRU model provided better network performance in terms of slice utilization and user drop rate.

Although deep learning-based slicing mechanisms provide adequate performance, there are still some trade-offs to be considered, such as the computational costs associated with the resources used to run the deep learning-based slicing model. Additionally, there is the integration complexity, since the integration of a deep learning-based slicing model into the 5G infrastructure may require highly reliable and robust APIs, which can be complex to involve. Another factor to consider is security risks, as the deep learning model could be targeted by adversarial attacks that can affect the network performance. As a future direction, in order to address the aforementioned issues, we suggest investigating ultra-lightweight deep learning and secure deep learning methods for network slicing in 5G and 6G networks.