Network Traffic Prediction for Multiple Providers in Digital Twin-Assisted NFV-Enabled Network

Abstract

This manuscript investigates the network traffic prediction problem, with the aim of predicting network traffic on a network function virtualization (NFV)-enabled and digital twin (DT)-assisted physical network for network service providers and network resource providers. It faces several key challenges like data privacy and different variation patterns of network traffic for multiple service function chain (SFC) requests. In view of this, we address the network traffic prediction problem by jointly considering the above key challenges in this manuscript. Specifically, we formulate the virtual network function (VNF) migration and SFC placement problems as integer linear programming (ILP) that aim to maximize acceptance revenues, minimize network resource costs, minimize energy consumption, and minimize migration cost. Then, we define the Markov Decision Process (MDP) for the network traffic prediction problem, and propose a model and algorithm to solve the problem. The simulation results demonstrate that our algorithms outperform benchmark algorithms and achieve a better performance.

1. Introduction

Network function virtualization (NFV), as one of the core innovations in modern networking technologies, significantly enhances network flexibility and scalability by virtualizing functionalities of traditional hardware-based network devices and running them on general-purpose servers [1,2]. In practical network deployments, the unpredictability of user behaviors, the diversification of application requirements, and the dynamic fluctuations of network workloads pose substantial challenges to NFV networks [3]. In statically configured networks, resources are often over-provisioned to accommodate potential traffic spikes, resulting in resource wastage. To address these issues, NFV networks typically rely on dynamic resource management and scheduling strategies to flexibly adjust the deployment locations of virtual network functions (VNFs), the allocation of computational resources, and traffic routing paths, thereby optimizing the overall performance. However, in dynamic traffic environments, such real-time adjustments must balance multi-dimensional objectives, including migration costs, resource utilization efficiency, and operational expenditures.

On the other hand, digital twin technology has gradually emerged in various industries in recent years [4,5]. By creating a virtual replica of a physical entity or system, it enables the real-time monitoring, analysis, and optimization of the operation of physical systems [6]. Introducing digital twin technology into NFV networks allows for more flexible and efficient network management through virtualization, helping operators cope with complex network environments and increasing demands. In practical applications, digital twin technology typically requires large amounts of data for training and optimization. This data often contains sensitive information, including user privacy data, operators’ core business data, and network resource usage. Therefore, in the context of multi-party collaborative NFV networks, how to achieve efficient network management and optimization while ensuring data privacy and security has become an urgent issue to address.

Network service providers and network resource providers train the data locally and then upload the model updates (rather than the raw data) or operational results to the global model for consolidation. Multiple organizations can collaborate effectively while ensuring data privacy. In this way, there is no need to share or upload data, but it is necessary to transfer the parameters of the neural network, and the number of parameters in a typical neural network is quite large. Therefore, an LSTM structure based on low-rank adaptation (LoRA) [7] is used here. Only low-rank matrices are transferred, rather than all the parameters of the entire neural network.

In this article, we propose the utilization of LoRA and DDPG in a digital twin-assisted NFV network for network traffic prediction. Our specific contributions and innovations can be summarized as follows. (1) To tackle the issue of untimely VNF migration induced by dynamic changes in network traffic, we construct a system network model for multiple network service providers and network resource provider in a digital twin-assisted NFV network. This network system model allows all parties to train neural network models locally, protecting the privacy data of various network service providers and network resource providers. (2) We propose a mathematical model for multi-party merchants in a digital twin-assisted NFV network. In this mathematical model, we define the constraints and optimization objectives aimed at maximizing network revenue. (3) We propose a prediction algorithm which integrates the deep reinforcement learning algorithm and LoRA in LSTM. In addition, we give the Markov Decision Process (MDP) model for the network traffic prediction problem.

The remaining sections of this article are organized as follows. Section 2 provides an overview of related works. Section 3 defines the system network model, the mathematical definition, and the MDP definition of the problem. The solutions and algorithms are proposed in Section 4. The simulation results are presented in Section 5, and this manuscript is summarized in Section 6.

2. Related Works

In this section, we introduce some research on the application of digital twins in NFV networks.

A digital twin is a virtual model based on physical entities, which reflects the state and behavior of a physical device or system in real time by modeling and simulating its real-time data. Traditionally, digital twins are widely used in industrial manufacturing, IoT, and other fields, and with the rapid development of virtualization technology in communication networks, digital twins have gradually entered the network field, especially in the NFV architecture, which plays a crucial role.

Calle-Heredia et al. [8] propose a new network architecture that combines the concepts of digital twins and virtual networks to support the efficient operation of XR applications. The architecture creates a virtual network model through digital twins to reflect the behavior and demand of XR applications in the network in real time, so as to optimize the allocation and management of resources. Li et al. [9] present an innovative platform combining a digital twin and NFV, demonstrating how digital twin technology can be utilized to enhance the performance, reliability, and intelligence of virtual network architectures. The platform not only provides effective solutions for network failure prediction and recovery, resource optimization, etc., but also provides new operation and management ideas for future networks, such as 5G and future 6G networks. Tang et al. [10] proposed a digital twin-based VNF mapping and scheduling framework to optimize network resource scheduling in industrial IoT using SDN and NFV technologies. With real-time monitoring, prediction, and scheduling completed by using a digital twin, the system can effectively improve the resource utilization of VNFs and ensure the efficiency and reliability of industrial IoT services. The solution provides strong support for dynamic and intelligent network management in industrial IoT, and is particularly suitable for industrial application scenarios that require low latency and high reliability. Wang et al. [11] proposed a service function chain scheduling framework that combines digital twin and multi-intelligence reinforcement learning to provide innovative solutions to the resource optimization and service quality assurance problems in multi-domain computing power networks. Through real-time monitoring and prediction using digital twin technology, and intelligent cooperative optimization by multi-intelligence reinforcement learning, the system is able to provide efficient and reliable service function chain scheduling in dynamic and complex network environments. This research provides new ideas and technical support for service management and optimization in future multi-domain networks. Liu et al. [12] proposed an innovative approach combining deep reinforcement learning and digital twins to present a dynamic scheduling framework for network resource demand prediction and VNF migration optimization problems. The method can effectively improve the utilization of network resources and ensure the efficiency of VNF migration decisions and the stability of service quality. The framework is experimentally verified to perform well in multi-domain network environments, providing a new technical path for resource management and scheduling in next-generation networks. Li et al. [13] proposed an innovative framework combining digital twin and service function chaining (SFC) techniques for providing efficient service scheduling and resource optimization in MEC environments. With the real-time monitoring and prediction of digital twins, combined with the dynamic scheduling capability of SFC, the framework is able to provide high-quality services for various applications under variable network conditions and effectively optimize the resource utilization. The solution provides a new solution idea for service provisioning and optimization in MEC, which has a large application potential. Guo et al. [14] propose a new method, DTFL: a digital twin-assisted graph neural network approach for service function chain fault localization in a network function virtualization environment. By monitoring and simulating the network state in real time through digital twin technology, combined with a graph neural network’s processing of complex network topology relations, DTFL can efficiently and accurately localize the faults in service function chains. Experimental validation shows that DTFL exhibits significant advantages in both fault localization accuracy and computational efficiency, providing a new solution for fault diagnosis and optimization in NFV networks. Tang et al. [15] proposed a VNF migration method based on digital twin technology and resource prediction, which significantly improves resource utilization and migration efficiency in IoT networks. Through real-time monitoring, prediction, and intelligent scheduling, the system is able to realize seamless VNF migration with guaranteed quality of service. The experimental results show that the method can effectively reduce resource waste, improve the dynamic adaptability of the network, and provide new ideas and methods for VNF migration and resource optimization in future IoT networks. Huo et al. [16] demonstrates that joint sparse Bayesian learning enables the effective reconstruction of hybrid near- and far-field structures under low signal-to-noise ratio conditions. These results substantiate the potential of incorporating embedded architectural paradigms—including digital twins, service function chains, and provider partitioning—to improve learning efficiency and system robustness in complex communication environments.

The research findings mentioned above serve as the foundation for our current study. In the context of digital twin-assisted NFV networks, where data is centrally stored, the issue of privacy protection for all parties becomes even more pronounced. To address this problem, we have reconstructed the network system model, formulated a mathematical model of the problem, and developed implementation algorithms.

3. System Model and Problem Formulation

3.1. System Model

3.1.1. Digital Twin-Assisted Network Architecture

This manuscript focuses on a software-defined network (SDN) and NFV-based network, which contains access, edge, and core networks. Access and core networks may belong to one network resource provider’s domain or cross multiple network resource provider’s domains. Each domain has a centralized SDN controller, which can obtain complete knowledge of the domain’s network topology, resource capacity, resource demands, and so on [17]. The network architecture is described in Figure 1. It is assumed that the overall multi-domain is centralized by a global orchestrator SDN controller, and it can obtain a global view of the multi-domain network and knowledge of multi-domain resources status by collecting information from each domain’s centralized SDN controller [18,19]. The network edge has multiple types of end nodes, including hosts, data center servers, mobile phones, and other devices. The network service demands of these sites and devices are aggregated and submitted to various network service providers, who, in turn, make resource usage requests to network resource providers in the form of a service function chain (SFC). Network service providers use the resources of the network resource providers to deliver services to users. Service providers may offer the same services or different ones. Network traffic flows are transmitted to different users’ devices and data centers via the core network. In the whole physical network, physical nodes are assumed to be divided into five categories: (1) NFV-enabled nodes, (2) transport nodes, (3) digital twin nodes, (4) access nodes, (5) end nodes. NFV-enabled nodes process and transmit VNFs that are required for services, transport nodes can only transmit traffic flows, twin nodes collect and synchronize the data of VNFs, access nodes can transmit traffic flows in the access network, while end nodes request a service or provide a service [10]. Due to privacy concerns, network service providers are unwilling to share information about network traffic, and network resource providers are also reluctant to share information about the usage of underlying network resources. Therefore, network service providers may have their own digital twin nodes, which can collect dynamic network traffic information. At the same time, network resource providers also have their own digital twin nodes, which monitor VNFs in real time and prevent the service failure caused by VNF operation failure. In this way, the digital twin node of each network service provider has network traffic information of their own, and the digital twin node of the network resource provider has information about network resource usage. The network traffic of each SFC varies dynamically according to higher resource demands to minimize the need for SFC reallocation (VNF migration), which results in the overuse of network resources. At the same time, this cannot completely avoid resource reallocation. To alleviate this issue, recent approaches suggest dividing the day into multiple time slices, predicting the network traffic peak within each time slice before each time slice begins, and proactively reallocating resources. Considering the data privacy of each provider, a new framework can be applied to independently train each network traffic prediction model of each network service provider and share the local model training results with the network resource provider. The global model calculates the costs and benefits of the underlying physical network based on mappings and migrations, which are used to train the global model. During the prediction phase, network traffic is predicted based on the physical network resources of the network resource provider. We utilize the SDN controller to manage network data flows [20]. It is assumed that the SDN controller can gather information on the VNFs through the network resource provider’s digital twin nodes [10]. SDN controllers collect information from core network nodes (transport nodes, NFV-enabled nodes, and the digital twin nodes) and access network nodes (access nodes), such as locations, resource usage, and SFC mappings, in order to improve the quality of service (QoS) of SFCs. This framework can physically isolate privacy information while enabling cooperation.

3.1.2. Physical Network

The physical network is represented by an undirected graph $G=\left(V,E\right)$ , where V and E stand for the node set and the link set, respectively. V consists of the transport node set $V^{for}\sqrt{a^2+b^2}$ , the NFV-enabled node set $V^{ser}$ , the access node set $V^{acc}$, the end node set $V^{end}$, and the digital twin node set $V^{dt}$ , i.e., $V^{for}\bigcup V^{ser}\bigcup V^{acc}\bigcup V^{end}\bigcup V^{dt}=V$ and $V^{for}\bigcap V^{ser}\bigcap V^{acc}\bigcap V^{end}\bigcap V^{dt}=0$ . Transport nodes have forwarding devices and only forward traffic flow to other nodes, NFV-enabled nodes are data centers (DCs) that can not only forward flow but also carry VNF instances, access nodes can forward traffic flow between the core network and the edge network, end nodes are terminal nodes that belong to users or service providers, while digital twin nodes can construct DTs of a physical network and cannot deploy VNFs. Digital twin nodes that belong to network resource providers are represented by $V^{resdt}$ , and digital twin nodes that belong to network service providers are represented by $V^{serdt}$ . Each NFV-enabled node is endowed with certain computing, processing, and storage resources, and each digital twin node is endowed with certain computing and storage resources. We use $c_n^{comp}$ to denote the computing capacity of the NFV-enabled node $v_n\in V^{ser}$ , we use $c_k^{comp}$ to denote the computing capacity of the digital twin node $v_k\in V^{resdt}$ , and we use $c_h^{comp}$ to denote the computing capacity of the digital twin node $v_h^{edge}\in V^{serdt}$ . When $v_r\in V^{for}$ , the computing capacity $c_a^{comp}$ or $c_r^{comp}$ of access node $v_a$ or transport node $v_r$ is 0. Each NFV-enabled node and digital twin node is endowed with a certain storage resource. We use $c_n^{stor}$ to denote the storage capacity of the NFV-enabled node $v_n\in V^{ser}$ , we use $c_k^{stor}$ to denote the storage capacity of the digital twin node $v_k\in V^{resdt}$ , and we use $c_h^{stor}$ to denote the storage capacity of the digital twin node $v_h^{edge}\in V^{serdt}$. Each NFV-enabled node is endowed with a certain processing resource. We use $c_n^{pro}$ to denote the processing capacity of the NFV-enabled node $v_n\in V^{ser}$. We use $N_n$ to represent the neighbor node set of the node $v_n\in V^{ser}$ , we use $N_k$ to represent the neighbor node set of the node $v_k\in V^{resdt}$ , and we use $N_h$ to represent the neighbor node set of the node $v_h^{edge}\in V^{serdt}$ . Each link $e_{m,n}\in E$($e_{l,l^{\prime }}\in E$) has a bandwidth capacity $c_{m,n}^{bw}$($c_{l,l^{\prime }}^{\mathrm{bw}}$) and a link propagation latency $d_{m,n}^{paga}$($d_{l,l^{\prime }}^{\mathrm{prop}}$).

3.1.3. Scenario

Different physical nodes may have different constraints on the allowable VNF types and the allowable number of VNF instances. Each physical node can instantiate multiple VNFs, and each VNF can instantiate multiple instances on the physical node. We use $b_{n,v}$ to represent if the VNF $f_v$ can be instantiated on the physical node $v_n$, and $n_{n,v}^{max}$ to represent the maximum number of VNF $f_v$’s instances that can be instantiated on node $v_n$. Each instance on a physical node consumes CPU resources, and the basic computing resource requirement for instance $f_{v,i}$ is represented by $c_{v,i}^{comp}$. Each instance on a physical node consumes storage resources, and the basic storage resource requirement for instance $f_{v,i}$ is represented by $c_{v,i}^{stor}$. Each instance instantiated on a physical node has a processing capacity, and the total processing capacity of the instances on the physical node cannot exceed the total processing capacity of the physical node. The total processing capacity of a physical node is represented by $c_n^{pro}$, and the processing capacity of each instance on the physical node is represented by $c_{v,i}^{pro}$. Additionally, some VNFs can be shared among multiple SFCs on the physical node, while others cannot. Therefore, the set of VNFs V consists of the shareable set $V^{sha}$ and the non-shareable set $V^{unsha}$. The VNF instance can be shared among multiple different SFCs, and the VNF instance in $V^{sha}$ can’t be shared among different SFCs.

3.1.4. Service Function Chain Request

There is a set of service function chain requests (SFC requests) R in the physical network, and a set of network service providers Q. These SFCs belong to different network service providers. We assume that there are $\left|R\right|$ SFCs, and they belong to $\left|Q\right|$ network service providers. The SFC requests can be represented as $R=\left\{r_1,r_2,\dots ,r_{\left|R\right|}\right\}$, and the network service providers can be represented as $Q=\left\{q_1,q_2,\dots ,q_{\left|Q\right|}\right\}$. We use $b_{s,p}$ to represent if SFC request $r_s$ belongs to network service provider $q_p$. An individual SFC request $r_s$ can be represented by an ordered sequence of VNFs, the delay requirement, the packet size, and the resource requirements, which is represented as $r_s=\left\{f_s,d_s,l_s,c_s\right\}$. In the SFC request $r_s$, $f_s=\left(f_{s,1},f_{s,2},\dots ,f_{s,k},\dots ,f_{s,\left|F_s\right|}\right)$ represents the VNFs of the SFC request, and $f_{s,k}$ represents the kth VNF of the SFC, $\left|F_s\right|$ represents the number of VNFs in the SFC request $r_s$. $d_s$ represents the end-to-end latency requirement of the SFC request $r_s$ and $l_s$ represents the packet size of the SFC request $r_s$. $c_s=\left(c_s^{bw},c_{s,1}^{node},c_{s,2}^{node},\dots ,c_{s,v}^{node},\dots ,c_{s,\left|F_s\right|}^{node}\right)$ represents the resource requirements of the SFC request $r_s$, $c_s^{bw}$ represents the bandwidth requirement of the SFC request $r_s$, $c_{s,v}^{node}=\left(c_{s,v}^{comp},c_{s,v}^{stor},c_{s,v}^{pro}\right)$ represents the resource requirements of the VNF $f_v$ in the SFC request $r_s$, $c_{s,v}^{comp}$ represents the computing requirement of the VNF $f_{s,v}$, $c_{s,v}^{stor}$ represents the storage requirement of the VNF $f_{s,v}$, and $c_{s,v}^{pro}$ represents the processing requirement of VNF $f_{s,v}$.

3.1.5. Digital Twin

Digital twin nodes have two kinds of categories: one belongs to network service providers to collect user requests, understand service network traffic, and other information; the other belongs to network resource providers to collect the mapping and migration status of the underlying physical network and the resource usage of the underlying physical network. Digital twin nodes that belong to network service providers are connected to end nodes, while digital twin nodes that belong to network resource provider are connected to transport nodes, NFV-enabled nodes, and access nodes. We use $v_k$ to represent a digital twin node that belongs to a network resource provider. We use $b_{sv,nk}$ to represent if VNF $f_{s,v}$ is mapped to the node $v_n$ which is associated with digital twin node $v_k$, and $b_{sv,nk}^{\prime }$ to represent the same relationship in the previous time slice. The packet for collecting or updating data of VNF $f_{s,v}$ between the NFV-enabled node $v_n$ and the digital twin node $v_k$ passes a physical path. If physical link $e_{l,l^{\prime }}$ belongs to the physical path, we set $b_{sv,nk,l,l^{\prime }}=1$, otherwise, we set $b_{sv,nk,l,l^{\prime }}=0$. Constructing the DT of nodes (such as NFV-enabled nodes, access nodes, and transport nodes) consumes resources of CPU, storage, and bandwidth. $c_{sv,nk}^{comp}$ represents the CPU resources to construct the DT of VNF $f_{s,v}$ in the digital twin node $v_k$ for the NFV-enabled node $v_n$, $c_{r,k}^{comp}$ represents the CPU resources to construct the DT in the digital twin node $v_k$ for the transport node $v_r$, and $c_{a,k}^{comp}$ represents the CPU resources to construct the DT in the digital twin node $v_k$ for the access node $v_a$. $c_{sv,nk}^{stor}$ represents the storage resources to construct and maintain the DT of VNF $f_{s,v}$ in the digital twin node $v_k$ for the NFV-enabled node $v_n$, $c_{a,k}^{stor}$ represents the storage resources to construct and maintain the DT in the digital twin node $v_k$ for the access node $v_a$, and $c_{r,k}^{stor}$ represents the storage resources to construct and maintain the DT in the digital twin node $v_k$ for the transport node $v_r$. $c_{sv,nk}^{bw}$ represents the bandwidth resources to transmit the DT data of VNF $f_{s,v}$ between the physical node $v_n$ and the digital twin node $v_k$, $c_{a,k}^{bw}$ represents the bandwidth resources to transmit the DT data between the access node $v_a$ and digital the twin node $v_k$, and $c_{r,k}^{bw}$ represents the bandwidth resources to transmit the DT data between the transport node $v_r$ and the digital twin node $v_k$. $v_h^{edge}$ represents the digital twin node that belongs to the network service provider, and $b_{h,p}$ represents if the digital twin node $v_h^{edge}$ belongs to the network service provider $q_p$. Constructing the DT of end nodes (such as user nodes, service nodes, and IoT devices) also consumes resources of CPU, storage, and bandwidth. $c_{e,h}^{comp}$ represents the CPU resources to construct the DT in the digital twin node $v_h^{edge}$ for the end node $v_e$. $c_{e,h}^{stor}$ represents the storage resources to construct and maintain the DT in the digital twin node $v_h^{edge}$ for the end node $v_e$. $c_{e,h}^{bw}$ represents the bandwidth resources to transmit the DT data between the end node $v_e$ and the digital twin node $v_h^{edge}$.

3.1.6. Problem Objectives

Given the physical network, and SFC requests, the problem is to predict network traffic for migrating SFC requests while achieving the following goals.

(1) Maximize the Acceptance Ratio of SFC Requests: A higher acceptance ratio of SFC requests brings higher revenue from network service providers.

(2) Minimize the Cost of Migration: Under the premise of maximizing the acceptance ratio of SFC requests, minimize the migration cost, mainly the latency of migration.

(3) Minimize Energy Consumption: Under the premise of maximizing the acceptance ratio of SFC requests, minimize the energy consumption, mainly including the energy consumption of NFV-enabled nodes.

3.2. Problem Formulation

In this section, we mathematically formulate the problem.

3.2.1. Decision Variables

We use a binary decision variable $x_{s,v}$ to represent if VNF $x_{s,v}$ is migrated.

$$x_{s,v}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{VNF}{f}_{s,v}\in F_s\mathrm{is}\mathrm{migrated},\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(1)

We use a binary decision variable $y_{s,v,n}$ to represent if VNF $f_{s,v}$ is embedded in the physical node $v_n$.

$$y_{s,v,n}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{VNF}{f}_{s,v}\mathrm{is}\mathrm{embedded}\mathrm{on}\mathrm{node}{v}_n,\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(2)

We use a binary decision variable $z_{s,v,l,l^{\prime }}$ to represent if virtual link (VL) $l_{s,{\theta }_{s,v}}$ is mapped to the physical link $e_{l,l^{\prime }}$.

$$z_{s,v,l,l^{\prime }}=\left\{\begin{array}{cc}1,\hfill & \mathrm{VL}{l}_{s,{\theta }_{s,v}}\mathrm{is}\mathrm{mapped}\mathrm{to}\mathrm{physical}\mathrm{link}{e}_{l,l^{\prime }},\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(3)

3.2.2. Constrains

Equation (4) guarantees that each VNF of the accepted SFC request is processed by the NFV-enabled node only once, and it can only run on the NFV-enabled nodes that support the VNF.

$$\sum _{v_n\in V^{ser}}{y}_{s,v,n}·b_{n,v}=1,\forall r_s\in R,f_v\in F_s.$$

(4)

Equations (5) and (6) state that each accepted SFC request begins from the start node and ends at the destination node.

$$y_{s,{\theta }_{s,0}^{\prime },n_s^{\mathrm{src}}}=1,\forall r_s\in R.$$

(5)

$$y_{s,{\theta }_{s,|F_s|+1}^{\prime },n_s^{\mathrm{dst}}}=1,\forall r_s\in R.$$

(6)

Equation (7) guarantees that a digital twin node that belongs to a network service provider can only be associated with one network service provider.

$$\sum _{q_p\in Q}{b}_{h,p}=1,\forall v_h^{edge}\in V^{serdt}.$$

(7)

Equation (8) guarantees that the flow of each SFC request cannot split.

$$\sum _{v_m\in N_n}{z}_{s,v,nm}+\sum _{v_m\in N_n}{z}_{s,v,mn}\le 1,\forall r_s\in R,f_{s,v}\in F_s,v_n\in V^{ser},e_{m,n}\in E,m<n.$$

(8)

Equation (9) guarantees that the conservation of each accepted SFC request at each node.

$$\begin{array}{c}{\displaystyle \sum _{v_m\in N_n}}{z}_{s,v,nm}-{\displaystyle \sum _{v_m\in N_n}}{z}_{s,v,mn}=y_{s,v,n}-y_{s,{\theta }_{s,({\theta }_{s,v}-1)}^{\prime },n},\hfill \\ \forall r_s\in R, f_{s,v}\in F_s,v_n\in V.\hfill \end{array}$$

(9)

Where ${\theta }_{s,v}$ represents the location sequence of VNF $f_{s,v}$, and ${\theta }_{s,q}^{\prime }$ represents the qth VNF of SFC request $r_s$. Then, ${\theta }_{s,({\theta }_{s,v}-1)}^{\prime }$ represents the previous VNF of $f_{s,v}$.

Equation (10) guarantees that the processing capacity of each sharable VNF is not violated.

$$\sum _{r_s\in R}{b}_{s,v}·b_{n,v}·y_{sv,ni}·c_{s,v}^{pro}\le c_{v,i}^{pro},\forall v_n\in V^{ser},f_v\in F^{sha},f_{v,i}\in I_{n,v}.$$

(10)

Equation (11) guarantees that a non-shareable VNF can be instantiated on each physical node for only one SFC.

$$\sum _{r_s\in R}{b}_{s,v}·b_{n,v}=1,\forall f_v\in F^{unsha},v_n\in V^{ser}.$$

(11)

Equation (12) guarantees that the processing capacity of each non-sharable VNF is not violated.

$$b_{s,v}·b_{n,v}·y_{sv,ni}·c_{s,v}^{pro}\le c_{v,i}^{pro},\forall v_n\in V^{ser},f_v\in F^{unsha},r_s\in R,f_{v,i}\in I_{n,v}.$$

(12)

Equation (13) ensures that the delay constraint of each SFC flow is not violated.

$$\begin{array}{c}{\displaystyle \sum _{{\theta }_{s,v}=0}^{\left|F_s\right|}}{\displaystyle \sum _{e_{m,n}\in E}}\left({\displaystyle \frac{l_{s,v}}{c_s^{bw}}}+d_{m,n}^{paga}\right)·z_{s,v,mn}\hfill \\ +{\displaystyle \sum _{f_{s,v}\in F_s}}{\displaystyle \sum _{v_n\in V^{ser}}}\left({\displaystyle \frac{1}{c_{s,v}^{pro}}}+{\displaystyle \frac{1}{c_{s,v}^{pro}-{\lambda }_s}}\right)·y_{s,v,n}\le d_s^{max},\hfill \\ \forall r_s\in R.\hfill \end{array}$$

(13)

Equation (14) ensures that the bandwidth capacity of each link is not violated when not registering at the digital twin node.

$$\sum _{r_s\in R}\sum _{f_v\in F_s}\left(z_{s,v,l{l}^{\prime }}+z_{s,v,l^{\prime }l}\right)·c_s^{\mathrm{bw}}\le c_{l,l^{\prime }}^{\mathrm{bw}},\forall e_{l,l^{\prime }}\in E,l<l^{\prime }.$$

(14)

Equation (15) ensures that the bandwidth capacity of each link is not violated when registering at the digital twin node.

$$\begin{array}{c}{\displaystyle \sum _{r_s\in R}}{\displaystyle \sum _{f_v\in F_s}}\left(z_{s,v,l{l}^{\prime }}+z_{s,v,l^{\prime }l}\right)·c_s^{\mathrm{bw}}\hfill \\ +{\displaystyle \sum _{v_k\in V^{\mathrm{resdt}}}}{\displaystyle \sum _{v_n\in T_k}}{\displaystyle \sum _{r_s\in R}}{\displaystyle \sum _{f_v\in F_s}}\left(b_{sv,nk,l{l}^{\prime }}+b_{sv,nk,l^{\prime }l}\right)·c_{sv,nk}^{\mathrm{bw}}\le c_{l,l^{\prime }}^{\mathrm{bw}},\hfill \\ \forall e_{l,l^{\prime }}\in E,l<l^{\prime }.\hfill \end{array}$$

(15)

Among them, the registration of access nodes, end nodes, and transmission nodes is omitted.

Equation (16) ensures that the processing capacity of each NFV-enabled node is not violated.

$$\sum _{f_v\in F}{b}_{n,v}·n_{n,v}·c_{v,i}^{pro}\le c_n^{pro},\forall v_n\in V^{ser},n_{n,v}\le n_{n,v}^{max}.$$

(16)

Equation (17) states that each VNF can only be associated with one digital twin node.

$$\sum _{v_k\in V^{resdt}}{b}_{sv,nk}=\sum _{v_k\in V^{resdt}}{b}_{s,v}·b_{n,v}·y_{s,v,n}·b_{n,k}=1,\forall v_n\in V^{ser},r_s\in R,f_v\in F.$$

(17)

Equation (18) guarantees that the flow of the transmitting packet of the DT of VNF $f_{s,v}$ on the NFV-enabled node $v_n$ to the digital twin node $v_k$ that belongs to the network resource provider cannot split.

$$\begin{array}{c}{\displaystyle \sum _{v_{l^{\prime }}\in N_l}}{b}_{sv,nk,l{l}^{\prime }}+{\displaystyle \sum _{v_{l^{\prime }}\in N_l}}{b}_{sv,nk,l^{\prime }l}\le 1,\hfill \\ \forall r_s\in R,f_v\in F_s,v_n\in V^{\mathrm{ser}},v_k\in V^{\mathrm{resdt}},e_{l,l^{\prime }}\in E,l<l^{\prime }.\hfill \end{array}$$

(18)

Equation (19) guarantees that the flow of the transmitting packet of the DT of switch information on the transport node $v_r$ to the digital twin node $v_k$ that belongs to the network resource provider cannot split.

$$\begin{array}{c}{\displaystyle \sum _{v_{l^{\prime }}\in N_l}}{b}_{rk,l{l}^{\prime }}+{\displaystyle \sum _{v_{l^{\prime }}\in N_l}}{b}_{rk,l^{\prime }l}\le 1,\hfill \\ \forall v_r\in V^{\mathrm{acc}}\cup V^{\mathrm{for}},v_k\in V^{\mathrm{resdt}},e_{l,l^{\prime }}\in E,l<l^{\prime }.\hfill \end{array}$$

(19)

Equation (20) guarantees that the flow of the transmitting packet of the DT of service information on the end node $v_e$ to the digital twin node $v_h^{edge}$ that belongs to the network service provider cannot split.

$$\begin{array}{c}{\displaystyle \sum _{v_{l^{\prime }}\in N_l}}{b}_{eh,l{l}^{\prime }}+{\displaystyle \sum _{v_{l^{\prime }}\in N_l}}{b}_{eh,l^{\prime }l}\le 1,\hfill \\ \forall v_e\in V^{\mathrm{end}},v_h^{\mathrm{edge}}\in V^{\mathrm{serdt}},e_{l,l^{\prime }}\in E,l<l^{\prime }.\hfill \end{array}$$

(20)

Equation (21) guarantees that the computing capacity of each NFV-enabled node is not violated.

$$\begin{array}{c}{c}_n^{compused}={\displaystyle \sum _{r_s\in R}}{\displaystyle \sum _{f_v\in F_s}}{y}_{s,v,n}·c_{s,v}^{comp}+{\displaystyle \sum _{f_v\in F}}{b}_{n,v}·n_{n,v}·c_{v,i}^{comp}\le c_n^{comp},\hfill \\ \forall v_n\in V^{ser},n_{n,v}\le n_{n,v}^{max}.\hfill \end{array}$$

(21)

Equation (22) guarantees the computing capacity of each digital twin node that belongs to the network resource provider is not violated.

$$\sum _{v_n\in T_k}\sum _{r_s\in R}\sum _{f_v\in F_s}{b}_{sv,nk}·c_{sv,nk}^{comp}+\sum _{v_r\in T_k}{b}_{r,k}·c_{r,k}^{comp}+\sum _{v_a\in T_k}{b}_{a,k}·c_{a,k}^{comp}\le c_k^{comp},\forall v_k\in V^{resdt}.$$

(22)

Equation (23) guarantees the computing capacity of each digital twin node that belongs to the network service provider is not violated.

$$\sum _{v_e\in T_h}{b}_{e,h}·c_{e,h}^{comp}\le c_h^{comp},\forall v_h^{edge}\in V^{serdt}.$$

(23)

Equation (24) guarantees that the storage capacity of each NFV-enabled node is not violated.

$$\sum _{r_s\in R}\sum _{f_v\in F_s}{y}_{s,v,n}·c_{s,v}^{stor}+\sum _{f_v\in F}{b}_{n,v}·n_{n,v}·c_{v,i}^{stor}\le c_n^{stor},\forall v_n\in V^{ser},n_{n,v}\le n_{n,v}^{max}.$$

(24)

Equation (25) guarantees that the storage capacity of each digital twin node that belongs to a network resource provider is not violated.

$$\sum _{v_n\in T_k}\sum _{r_s\in R}\sum _{f_v\in F_s}{y}_{s,v,n}·b_{n,k}·c_{sv,nk}^{stor}+\sum _{v_r\in T_k}{b}_{r,k}·c_{r,k}^{stor}+\sum _{v_a\in T_k}{b}_{a,k}·c_{a,k}^{stor}\le c_k^{stor},\forall v_k\in V^{resdt}.$$

(25)

Equation (26) guarantees that the storage capacity of each digital twin node that belongs to a network service provider is not violated.

$$\sum _{v_e\in T_h}{b}_{e,h}·c_{e,h}^{stor}\le c_h^{stor},\forall v_h\in V^{serdt}.$$

(26)

The above constraints are summarized as follows in

Table 1. Constraints.

VNF		SFC Request	DT Node		Link	NFV Node
(4) One and only one NFV node mapped		(5)(6) One and only one beginning and end node	(7) Belongs to only one merchant		(14) Bandwidth capacity not violated	(16) Processing capacity not violated
(17) One and only one DT node associated		(8) Cannot split flow	(18) Transmitting flow for NFV node cannot split		(15) Bandwidth capacity not violated when registering	(21) Computing capacity not violated
Shareable	Non-shareable	(9) Conservation	(19) Transmitting flow for transport node cannot split			(24) Storage capacity not violated
(10) Processing capacity not violated	(11) Exclusivity in NFV node	(13) End to end delay constraint	(20) Transmitting flow for end node cannot split
	(12) Processing capacity not violated		DT of network service provider	DT of network resource provider
			(23) Computing capacity not violated	(22) Computing capacity not violated
			(26) Storage capacity not violated	(25) Storage capacity not violated

where bold italic font is used for titles and the rest are values.

3.2.3. Revenue and Cost Structure

(1)

Revenue of SFC Request Acceptance

We assume that the network resource providers charge fees according to the resource requirement of each SFC request [10]. The revenue is calculated using Equation (27).

$$r^{accept}={\eta }_1·\sum _{r_s\in R}\sum _{f_v\in F_s}{c}_{s,v}^{comp}+{\eta }_2·\sum _{r_s\in R}\sum _{f_v\in F_s}{c}_{s,v}^{stor}+{\eta }_3·c_s^{bw}·\left(\left|F_s\right|+1\right).$$

(27)

where ${\eta }_1$, ${\eta }_2$, and ${\eta }_3$ are the unit resource revenue.

(2)

Cost of Migration

The migration delay of the VNF includes propagation latency and transmission latency, and the transmission delay of the VNF information to the digital twin node also includes propagation latency and transmission latency [10]. The following section will analyze these four types of delays.

The propagation latency is the delay incurred when a packet propagates in a medium, and it mainly depends on the propagation distance and the propagation speed. Let $c^{paga}$ represent the propagation speed, and let $d_{l,l^{\prime }}$ represent the distance of physical link $e_{l,l^{\prime }}$. The propagation latency from NFV-enabled node $v_n$ to NFV-enabled node $v_{n^{\prime }}$ is defined as follows.

$$d_{sv,n{n}^{\prime }}^{paga}=\sum _{e_{l,l^{\prime }}\in E}{z}_{s,v,l{l}^{\prime }}·d_{l,l^{\prime }}^{paga}=\sum _{e_{l,l^{\prime }}\in E}{z}_{s,v,l{l}^{\prime }}·{\displaystyle \frac{d_{l,l^{\prime }}}{c^{paga}}}.$$

(28)

The propagation latency from the NFV-enabled node $v_{n^{\prime }}$ to the digital twin node $v_{k^{\prime }}$ is defined as follows.

$$d_{sv,n^{\prime }{k}^{\prime }}^{paga}=\sum _{e_{l,l^{\prime }}\in E}{b}_{sv,n^{\prime }{k}^{\prime },l{l}^{\prime }}·d_{l,l^{\prime }}^{paga}=\sum _{e_{l,l^{\prime }}\in E}{b}_{sv,n^{\prime }{k}^{\prime },l{l}^{\prime }}·\left({\displaystyle \frac{d_{l,l^{\prime }}}{c^{paga}}}\right).$$

(29)

Let $c^{migbw}$ represent the transmission bandwidth for migrating the VNF. The transmission latency $d_{sv,n{n}^{\prime }}^{tran}$ from $v_n$ to $v_{n^{\prime }}$ is defined as follows.

$$d_{sv,n{n}^{\prime }}^{tran}={\displaystyle \frac{c_{s,v}^{stor}}{c^{migbw}}}·\sum _{e_{l,l^{\prime }}\in E}{z}_{s,v,l{l}^{\prime }}.$$

(30)

Let $c_{sv,n^{\prime }{k}^{\prime }}^{bw}$ represent the transmission bandwidth for transmitting the VNF information between the NFV-enabled node $v_{n^{\prime }}$ and the digital twin node $v_{k^{\prime }}$. The transmission latency $d_{sv,n^{\prime }{k}^{\prime }}^{tran}$ from the NFV-enabled node $v_{n^{\prime }}$ to the digital twin node $v_{k^{\prime }}$ is defined as follows.

$$d_{sv,n^{\prime }{k}^{\prime }}^{tran}={\displaystyle \frac{c_{sv,n^{\prime }{k}^{\prime }}^{stor}}{c_{sv,n^{\prime }{k}^{\prime }}^{bw}}}·\sum _{e_{l,l^{\prime }}\in E}{b}_{sv,n^{\prime }{k}^{\prime },l{l}^{\prime }}.$$

(31)

The migration latency of each VNF can be expressed as follows.

$$d_{s,v}^{mig}=d_{sv,n{n}^{\prime }}^{tran}+d_{sv,n{n}^{\prime }}^{paga}+d_{sv,n^{\prime }{k}^{\prime }}^{tran}+d_{sv,n^{\prime }{k}^{\prime }}^{paga}.$$

(32)

Define the migration latency of each SFC as follows.

$$c_s^{mig}=\sum _{f_v\in F_s}{x}_{s,v}·d_{s,v}^{mig}.$$

(33)

Define the migration cost at the time slice alternation as follows.

$$c^{migrate}=\omega ·\sum _{r_s\in R}{c}_s^{mig}.$$

(34)

where $\omega$ is the unit cost of migration latency.

(3)

Cost of Placement

A greater reward is obtained when the SFC delay and the construction delay of SFC twin information are shorter. We calculate the placement cost according to the end-to-end latency of SFC and the conducting latency of SFC in digital twin nodes that belong to network resource providers.

The end-to-end latency of SFC is analyzed as follows:

$$d_s=\sum _{{\theta }_{s,v}=0}^{|F_s|}\sum _{e_{m,n}\in E}\left({\displaystyle \frac{l_{s,v}}{c_s^{bw}}}+d_{m,n}^{paga}\right)z_{s,v,mn}\sum _{f_{s,v}\in F_s}\sum _{v_n\in V^{ser}}\left({\displaystyle \frac{1}{c_{s,v}^{pro}}}+{\displaystyle \frac{1}{c_{s,v}^{pro}-{\lambda }_s}}\right)y_{s,v,n}.$$

(35)

The construction delay of VNF twin information includes the construction delays of NFV-enabled nodes, accessing nodes, and transport nodes in the digital twin nodes. Since the twin information of accessing nodes and transport nodes is relatively small, we will only analyze the construction delay of VNF twin information for NFV-enabled nodes here.

The propagation latency from the NFV-enabled node $v_n$ to the digital twin node $v_k$ is defined as follows.

$$d_{sv,nk}^{paga}=\sum _{e_{l,l^{\prime }}\in E}{b}_{sv,nk,l{l}^{\prime }}·d_{l,l^{\prime }}^{paga}=\sum _{e_{l,l^{\prime }}\in E}{b}_{sv,nk,l{l}^{\prime }}·{\displaystyle \frac{d_{l,l^{\prime }}}{c^{paga}}}.$$

(36)

The transmission latency from the NFV-enabled node $v_n$ to the digital twin node $v_k$ is defined as follows.

$$d_{sv,nk}^{tran}={\displaystyle \frac{c_{sv,nk}^{stor}}{c_{sv,nk}^{bw}}}·\sum _{e_{l,l^{\prime }}\in E}{b}_{sv,nk,l{l}^{\prime }}.$$

(37)

Define the placement cost of all SFC requests as follows.

$$c^{place}=\mu ·\left[\sum _{r_s\in R}{x}_s{d}_s+\sum _{r_s\in R}\sum _{f_v\in r_s}\left(d_{sv,nk}^{tran}+d_{sv,nk}^{paga}\right)\right].$$

(38)

where $\mu$ is the unit cost of placement latency.

(4)

Cost of Operation

We define the power consumption of physical nodes as follows [21,22]:

$$e_n={\epsilon }_0+({\epsilon }_1-{\epsilon }_0)·{\displaystyle \frac{c_n^{compused}}{c_n^{comp}}}.$$

(39)

where ${\epsilon }_0$ is the basic energy consumption per unit of the NFV-enabled node, and ${\epsilon }_1$ is the maximum energy consumption per unit of the NFV-enabled node.

Define the operation cost between two migrations as follows.

$$c^{operate}=\gamma ·l_t^{mig}·\sum _{v_n\in V^{ser}}{e}_n.$$

(40)

where $\gamma$ is the unit operation cost of energy consumption and $l_t^{mig}$ is the time length between two migrations.

(5)

Punishment For Denying SFC Requests

The punishment is computed according to the resource requirement of each denied SFC request. The punishment is calculated as follows.

$$r^{punish}={\mu }_1·\sum _{r_s\in R}\sum _{f_v\in F_s}{c}_{s,v}^{comp}{\mu }_2·\sum _{r_s\in R}\sum _{f_v\in F_s}{c}_{s,v}^{stor}{\mu }_3·c_s^{bw}·\left(\right|F_s|+1).$$

(41)

where ${\mu }_1$, ${\mu }_2$, and ${\mu }_3$ are the unit resource punishment.

3.2.4. Optimization Objectives

(1)

Service Function Chain Deployment

In our SFC deployment (SFCD) model, the optimization objective is to minimize the operation cost and maximize the acceptance ratio. Overall, the SFCD problem can be formulated following integer linear programming (ILP) problem.

$$\begin{array}{ccc}\hfill & \mathrm{maximize}:r^{deploy}=r^{accept}-c^{operate}-c^{place}-r^{punish},\hfill & \hfill \mathrm{subject}\mathrm{to}:\left(4\right)–\left(26\right).\end{array}$$

(42)

(2)

Service Function Chain Migration

We formulate the SFC migration (SFCM) problem using the following integer linear programming (ILP) method, which aims to maximize the request acceptance ratio and minimize the migration and operation cost. The problem is formulated as follows.

$$\begin{array}{ccc}\hfill & \mathrm{maximize}:r^{mig}=r^{accept}-c^{operate}-c^{migrate}-r^{punish},\hfill & \hfill \mathrm{subject}\mathrm{to}:\left(4\right)–\left(26\right).\end{array}$$

(43)

Due to the NP hardness of the problems, they cannot be solved using the exhaustive search algorithm [23]. We use greeedy algorithms to solve the problems above.

3.3. Markov Decision Process

The knowledge layer is the core of the framework, where the DRL module of network resource providers solves the whole network traffic prediction problem. In this section, the network traffic prediction problem is modeled as a Markov Decision Process.

(1) States: The states include the state of the physical network and the state of the network traffic. The state of the physical network is stored in a network status database of the store collector that belongs to the network resource provider, and the state of the complete network traffic is stored in a history database of the store collector that belongs to the network service provider. They are separately described below.

State of Physical Network: We define this state as a composite of nodes, instances, and links where SFC requests are embedded, and available resources of nodes, instances, and links. This state includes seven kinds of record matrices: the number record matrix of VNF instance $N={\left\{n_{n,v}\right\}}_{\left|F\right|\times \left|V^{ser}\right|}$, the placement record matrix of VNF instance $P^{vnf}={\left\{p_{n,v,i}^{vnf}\right\}}_{\left|V^{ser}\right|\times \left|F\right|\times n_{n,v}^{max}}$, the mapping path record matrix of virtual link $P^{vlink}={\left\{p_{s,{\theta }_{s,v}}^{vlink}\right\}}_{\left|R\right|\times \left(\left|V\right|+1\right)}$, the record matrix of residual processing capacity $C^{repro}={\left\{c_{n,v,i}^{repro}\right\}}_{\left|V^{ser}\right|\times \left|F\right|\times n_{n,v}^{max}}$, the record vector of residual CPU capacity $c^{recomp}={\left(c_1^{recomp},c_2^{recomp},\dots ,c_n^{recomp},\dots ,c_{\left|V^{ser}\right|}^{recomp}\right)}_{1\times \left|V^{ser}\right|}$, the record vector of residual storage capacity $c^{restor}={\left(c_1^{restor},c_2^{restor},\dots ,c_n^{restor},\dots ,c_{\left|V^{ser}\right|}^{restor}\right)}_{1\times \left|V^{ser}\right|}$, and the record symmetric matrix of residual bandwidth capacity $C^{rebw}={\left\{c_{m,n}^{rebw}\right\}}_{\left|V\right|\times \left|V\right|}$. In N, each element $n_{n,v}$ is a single value, where $n_{n,v}$ indicates the current number of VNF $f_v$’s ($f_v\in F$) instances in the NFV-enabled node $v_n\in V^{ser}$. In $P^{vnf}$, if $f_v\in F^{unsha}$, each element $p_{n,v,i}^{vnf}$ is a single value, where $p_{n,v,i}^{vnf}$ denotes the current SFC request of VNF $f_v$’s ($f_v\in F^{unsha}$) deployment in instance $f_{v,i}\in I_{n,v}$. If $f_v\in F^{sha}$, each element $p_{n,v,i}^{vnf}={\left(p_{n,v,i,1}^{vnf},p_{n,v,i,2}^{vnf},\dots ,p_{n,v,i,s}^{vnf},\dots ,p_{n,v,i,\left|R\right|}^{vnf}\right)}_{1\times \left|R\right|}$ is an $\left|R\right|$-dimensional vector, where $p_{n,v,i,s}^{vnf}=1$ indicates the VNF $f_v$ ($f_v\in F^{sha}$) of SFC $r_s$ ($r_s\in R$) is deployed in instance $f_{v,i}\in I_{n,v}$, otherwise, the value of $p_{n,v,i,s}^{vnf}$ is 0. In $P^{vlink}$, each element $p_{s,{\theta }_{s,v}}^{vlink}$ is a physical path. If VNF $f_{s,{\theta }_{s,\left({\theta }_{s,v}-1\right)}^{\prime }}$ and VNF $f_{s,{\theta }_{s,{\theta }_{s,v}}^{\prime }}$ embed to the same physical node, then, $p_{s,{\theta }_{s,v}}^{vlink}$ is null. In $C^{repro}$, each element $c_{n,v,i}^{repro}$ is a single value, where $c_{n,v,i}^{repro}$ denotes the current residual processing capacity of VNF $f_v$’s ($f_v\in F^{unsha}$) instance $f_{v,i}\in I_{n,v}$ in NFV-enabled node $v_n\in V^{ser}$. In $c^{recomp}$, each element $c_n^{recomp}$ is the current residual CPU computing capacity of the NFV-enabled node $v_n\in V^{ser}$. In $c^{restor}$, each element $c_n^{restor}$ is the current residual storage capacity of the NFV-enabled node $v_n\in V^{ser}$. In $C^{rebw}$, each element $c_{m,n}^{rebw}$ is the current residual bandwidth capacity of the physical link $e_{m,n}\in E$. If link $e_{m,n}$ does not exist, the element $c_{m,n}^{rebw}$ equals 0.

$$S_t^{phynet}=\left\{N,P^{vnf},P^{vlink},C^{repro},c^{recomp},c^{restor},C^{rebw}\right\}.$$

(44)

State of Network Traffic: Each network service provider has the complete network traffic information of their own. They are stored in their history database of the store collector. During the training process, records can be extracted from their history database, and each record is specified by a time slice. The record in the history database can be defined as follows. In $c^{restor}$, each element $c_n^{restor}$ is the current residual storage capacity of the NFV-enabled node $v_n\in V^{ser}$. In $C^{rebw}$, each element $c_{m,n}^{rebw}$ is the current residual bandwidth capacity of the physical link $e_{m,n}\in E$. If link $e_{m,n}$ does not exist, the element $c_{m,n}^{rebw}$ equals 0.

$$S_t^{histraffic}=\left\{x_t,y_t,y_{t-1}\right\}.$$

(45)

In Equation (45), $x_t$ indicates the latest history network traffic in a given time period for the specified time slice $s_t=\left(s_t^{date},s_t^{slice}\right)$, where $s_t^{date}$ indicates the date of the specified time slice, and $s_t^{slice}$ indicates the serial number of the time slice in date $s_t^{date}$. $y_t$ represents the actual network traffic for time slice $s_t$, and $y_{t-1}$ represents the actual network traffic for time slice $s_{t-1}$ which is the previous time slice of $s_t$.

(2) Action: The action is the predicted value of network traffic for the specified time slice. We combine low-rank adaptation and deep reinforcement learning to solve the network traffic prediction problem. Prediction operations are performed in the actor network of the global model and the neural network of the local model.

The input of the local model is the network traffic $S_{p,t}^{histraffic}$ owned by each network service provider, and the output is the prediction value of network traffic $a_{p,t}$. This process can be expressed as follows:

$$a_{p,t}=A_p^{local}\left(S_{p,t}^{histraffic}\right)$$

(46)

The network resource provider knows the combined traffic from SFCs of the same type; however, it does not know the individual parts. Therefore, the global model uses the total SFC traffic as its input. The local models use their own components as the input. In the actor network of the global model, the input is the actual total network traffic ${\mathrm{S}}_t^{histraffic\_total}=\left\{{\mathrm{x}}_{\mathrm{t}}^{\mathrm{total}},{\mathrm{y}}_{\mathrm{t}}^{\mathrm{total}},{\mathrm{y}}_{\mathrm{t}-1}^{\mathrm{total}}\right\}$ from each SFC of the same type, and the output is the new prediction value of the total network traffic ${\mathcal{A}}_t^{\prime }=a_{i,t}^{\prime }$ for SFC type i. This process can be expressed as follows:

$${\mathcal{A}}_t^{\prime }=\mathcal{A}\left({\mathrm{S}}_t^{histraffic\_total}\right)$$

(47)

In the critic network of the global model, the main task is to evaluate the prediction performance. The input of the critic network is the prediction values of network traffic ${\mathcal{A}}_t^{\prime }$ from the global model, the actual total network traffic ${\mathcal{A}}_t$, and the physical network status $S_t^{phynet}$. The output is the evaluation value. This process can be expressed by the following equation:

$$q_t=Q\left({\mathcal{S}}_t^{\mathrm{phynet}},{\mathrm{S}}_t^{histraffic\_total},{\mathcal{A}}_t^{\prime }\right)$$

(48)

(3) Reward: In our proposed framework, the reward represents the network performance, and it needs to be considered from four aspects: migration cost, energy consumption, request acceptance revenue, and request failure penalty.

When the network traffic prediction value is greater than or equal to the actual peak value, there will be no passive migration within the time slice. Migration will only be actively initiated at the beginning of the time slice. Then, the migration cost is defined as the sum of the migration delays during active migration; the energy consumption is the network power consumption after the active migration; the request acceptance revenue is the total amount of resources of the SFC requests accepted after the active migration; and the request failure penalty is the total amount of resources of the SFC requests that fail to map after the active migration. Thus, the network reward is

$$r^{mig,>}=r^{accept,a}-c^{operate,a}-c^{migrate,a}-c^{punish,a}.$$

(49)

When the network traffic prediction value is less than the actual peak value, passive migration will occur within the time slice. In this case, both active migration and passive migration occur. The migration cost is the sum of the migration delays during active migration and passive migration; the energy consumption is the network power consumption under the actual peak value of network traffic; the request acceptance revenue is the total amount of resources of the SFC requests accepted after passive migration; and the request failure penalty is the total amount of resources of the SFC requests that fail to map after passive migration. Thus, the network reward in this case is

$$r^{mig,<}=r^{accept,p}-c^{operate,p}-\left(c^{migrate,a}+c^{migrate,p}\right)-c^{punish,p}.$$

(50)

The actual reward value, however, must be calculated on a case-by-case basis. $r^{mig}$ and $r^{mig,>}$ are the same. The sole distinction lies in the computation of the passive migration latency between $r^{mig}$ and $r^{mig,<}$, which is performed in the same manner as described in Section 3.2.3 (2).

Specifically, we formulate the network traffic prediction as an optimization problem, balancing migration latency, operation cost, and acceptance ratio to find the optimal predicted value. To achieve this objective, we first define and solve the VNF migration problem for evaluating the effect of the network traffic prediction. Secondly, we apply the LoRA and DDPG model to protect the data privacy of all merchants. With the aforementioned MDP model, we characterize the system state, action, and reward. Finally, we need to design an efficient network traffic prediction algorithm which can predict the optimal value of network traffic. The details of the algorithms will be described in Section 4.

3.4. LSTM and LoRA Fusions

In this paper, we inject LoRA into the weight matrix of each gate unit in the LSTM (including the input gate, forget gate, output gate, and candidate state), i.e., introducing low-rank incremental terms into each linear mapping. The weights in the gate computation process are also injected with LoRA in the same way. In this way, the model achieves more efficient training and a controllable capacity in prediction modeling while retaining the original temporal modeling structure.

3.4.1. Basic LSTM Equation

The standard LSTM unit contains three gate mechanisms and one memory unit:

$$\begin{array}{cc}\hfill \mathrm{Input}\mathrm{gate}:& i_t=\sigma (W_{ii}{x}_t+b_{ii}+W_{hi}{h}_{t-1}+b_{hi}).\hfill \\ \hfill \mathrm{Forget}\mathrm{gate}:& f_t=\sigma (W_{if}{x}_t+b_{if}+W_{hf}{h}_{t-1}+b_{hf}).\hfill \\ \hfill \mathrm{Output}\mathrm{gate}:& o_t=\sigma (W_{io}{x}_t+b_{io}+W_{ho}{h}_{t-1}+b_{ho}).\hfill \\ \hfill \mathrm{Candidate}\mathrm{memory}:& {\tilde{c}}_t=tanh(W_{ic}{x}_t+b_{ic}+W_{hc}{h}_{t-1}+b_{hc}).\hfill \\ \hfill \mathrm{Memory}\mathrm{update}:& c_t=f_t\odot c_{t-1}+i_t\odot {\tilde{c}}_t.\hfill \\ \hfill \mathrm{Hidden}\mathrm{state}:& h_t=o_t\odot tanh\left(c_t\right).\hfill \\ \hfill \mathrm{where}& \\ \hfill & x_t:\mathrm{Input}\mathrm{at}\mathrm{time}\mathrm{step}t(\mathrm{flow}\mathrm{feature}\mathrm{vector}).\hfill \\ \hfill & h_t:\mathrm{Hidden}\mathrm{state}(\mathrm{time}\mathrm{series}\mathrm{representation}).\hfill \\ \hfill & \sigma :\mathrm{Sigmoid}\mathrm{activation}\mathrm{function}.\hfill \\ \hfill & \odot :\mathrm{Hadamard}\mathrm{product}(\mathrm{element}\text{-}\mathrm{wise}\mathrm{multiplication}).\hfill \end{array}$$

3.4.2. LoRA Mathematical Representation

The core of LoRA is to inject trainable parameters through low-rank decomposition.

$$\begin{array}{cc}\hfill \mathrm{For}\mathrm{the}\mathrm{weight}\mathrm{matrix}& W\in {\mathbb{R}}^{m\times n}:\hfill \\ \hfill W^{\prime }& =W+\Delta W=W+BA\hfill \\ \hfill \mathrm{where}& \\ \hfill & B\in {\mathbb{R}}^{m\times r}.\hfill \\ \hfill & A\in {\mathbb{R}}^{r\times n}.\hfill \\ \hfill & r\ll min(m,n)\mathrm{is}\mathrm{the}\text{low-rank}\mathrm{dimension}(\mathrm{typically}r=4–32).\hfill \end{array}$$

3.4.3. LoRA Mathematical Representation

The following steps comprise injecting LoRA into LSTM gates.

Input gate enhancement:

$$\begin{array}{cc}\hfill i_t& =\sigma \left(\underset{\mathrm{Original}\mathrm{item}}{\underbrace{W_{ii}{x}_t+W_{hi}{h}_{t-1}}}+\underset{\mathrm{LoRA}\mathrm{item}}{\underbrace{B_{ii}{A}_{ii}{x}_t+B_{hi}{A}_{hi}{h}_{t-1}}}\right)\hfill \\ \hfill \mathrm{where}& B_{ii}\in {\mathbb{R}}^{d\times r},A_{ii}\in {\mathbb{R}}^{r\times d},r\ll d.\hfill \end{array}$$

Similarly, apply to other gates, including $f_t$, $o_t$ and ${\tilde{c}}_t$.

For the local model, the LSTM model is initialized and trained, and the LSTM parameters are transmitted back to the global model. Then, during the subsequent DDPG training process, the local model trains the LoRA_LSTM hybrid structure, where only the LoRA structure is activated during training. As a result, at the end of each training round, the parameters of the LoRA matrix are transmitted back to the global model. The global model collects and aggregates the LSTM and LoRA structures from multiple SFCs of the same type to form the actor network model in the LoRA_DDPG structure. The training algorithm is shown in Section 4 below.

The structure is described in Figure 2.

4. Algorithms

The pseudocode for DDPG- and LoRA-based training for network traffic prediction is provided in Algorithm 1; the pseudocode for prediction is presented in Algorithm 2.

Algorithm 1 DDPG- and LoRA-Based Training

Input: Historical traffic data of each local network service provider; Historical total traffic data of each SFC; Physical network G; already arrived SFC requests ${}_{{\theta }_0}$; initial models $Q_{{\theta }_0}$, $Q_{{\theta }_0}^{-}$, ${\mathrm{\Omega }}_{{\theta }_0}$, ${\mathrm{\Omega }}_{{\theta }_0}^{-}$, ${\pi }_{{\theta }_1}$,…,${\pi }_{{\theta }_Q}$. Output: Improved models ${\mathrm{\Omega }}_{{\theta }_0}$, ${\pi }_{{\theta }_1}$,…,${\pi }_{{\theta }_Q}$. 1: Initialize each local model ${\pi }_{{\theta }_i}$ without LoRA which belongs to network service provider; 2: for epoch j do 3:       for each local model ${\pi }_{{\theta }_i}$ without LoRA do 4:           Calculate the MSE loss between the predicted and actual values for local model ${\pi }_{{\theta }_i}$ without LoRA; 5:            Update parameters of each local model ${\pi }_{{\theta }_i}$ without LoRA; 6:       end for 7: end for 8: The actor network of the global model gathers parameters from each local model ${\pi }_{{\theta }_i}$ without LoRA; 9: for epoch k do 10:      for epoch j do 11:            for each local model ${\pi }_{{\theta }_i}$ with LoRA do 12:                  Freeze all structures excluding LoRA; 13:               Calculate the MSE loss between the predicted and actual values for local model ${\pi }_{{\theta }_i}$ with LoRA; 14:                  Update LoRA parameters of each local model ${\pi }_{{\theta }_i}$ with LoRA; 15:            end for 16:       end for 17:      The actor network of the global model gathers LoRA parameters from each local model ${\pi }_{{\theta }_i}$ with LoRA; 18:      Aggregate local models into actor network ${\mathrm{\Omega }}_{{\theta }_0}$ of global model using parameters from step 10 and step 17 for the same SFC type; 19:      Predict $A_t^{\prime }$ using actor network ${\mathrm{\Omega }}_{{\theta }_0}$ of global model; 20:      Calculate reward $r_t$ according to Equations (49) and (50); 21:      Store experience (state, action, reward, next state, slice) into replay memory and select mini-batch samples; 22:      Update the online critic network $Q_{{\theta }_0}$ with SGD; 23:      Copy the parameters of the online networks ${\mathrm{\Omega }}_{{\theta }_0}$ and $Q_{{\theta }_0}$ to the target networks ${\mathrm{\Omega }}_{{\theta }_0}^{-}$ and $Q_{{\theta }_0}^{-}$. 24: end for

Algorithm 2 Predicting

Input: Physical network G, already arrived SFC requests R, actor network ${\mathrm{\Omega }}_{{\theta }_0}$ of global model. Output: Predicted network traffic. 1: $x_t^{total},y_t^{total},$$y_{t-1}^{total}$← Read the record of historical network traffic for the specified time slice $s_t$ from the memory database of the same SFC type; 2: $A_t^{\prime }\leftarrow$ Input $x_t$ into actor network ${\mathrm{\Omega }}_{{\theta }_0}$ of the global model for each SFC request to obtain predicted values of network traffic; 3: $S^{\prime }\leftarrow$ Migrate the VNFs and Virtual Links using a greedy algorithm; 4: $r_t\leftarrow$ Calculate the reward according to Equations (49) and (50).

Therefore, in the LoRA_DDPG algorithm, the data components of the local models from various network service providers are protected, as the algorithm eliminates the need to transmit traffic component data to the network resource provider. The global model of the network resource provider only requires aggregated local network traffic for training and prediction. Concurrently, the parameters of the low-rank matrices from the local models are transmitted to the global model. This approach not only preserves the global model’s ability to learn from local models but also significantly reduces the volume of parameters transmitted.

5. Simulation

5.1. Simulation Settings

In this section, we use Python 3.6 and Pytorch 1.12 to verify the proposed scheme. The simulation platform is built on an Intel Core computer with a 1.8 GHz central processor and 8 GB Random Access Memory.

In the simulation, we obtain the network topology with scale: [140,20,20,20,10,10,991], where the numbers represent the quantity of NFV-enabled nodes, DT nodes of network service providers, DT nodes of network resource provider, end nodes, access nodes, transport nodes, and physical links, respectively. The CPU capacity of nodes ranges from 100 to 200 cores, the storage capacity of nodes ranges from 100 to 200, the processing capacity of nodes ranges from 80 to 150, the link bandwidth capacities between two nodes ranges from 15 to 25 Mbps, and the link latency between two adjacent nodes ranges from 1 to 2 s. The experiment considers eight types of VNFs, and one of the VNFs is non-shareable. Each SFC randomly selects a few VNFs, and the length of each SFC is between two and six. The SFC latency limit ranges from 30 to 50 s. A total of 100 SFC requests enter the network sequentially, with around 10 SFC requests typically online simultaneously. The processing latency of VNF in the service node is from 2 to 4 s. The CPU requirement of each VNF is from one to two cores, the storage requirement of each VNF is from one to two, the processing resource requirement of each VNF is from one to two, and the bandwidth resource requirement of each virtual link is from 10 to 20 bps. The CPU requirement of each instance is from one to two cores, the storage requirement of each instance is from one to two, and the processing resource requirement of each instance is from 10 to 20. In addition, randomization is used to assign an SFC request to each type of network service provider, and the type number of network service providers is set to four.

For the LoRA_DDPG algorithm of network traffic prediction, the actor networks of the online and target network in the global model contain two hybrid layers, each fusing LSTM with LoRA, while the dimension of each LSTM part is 50 and the rank of each LoRA part is eight. The critic networks of the online and target networks in the global model contain 2 LSTM layers and the dimension of each LSTM layer is 50. Each local model of a network service provider also contains two hybrid layers, each fusing LSTM with LoRA, while the dimension of each LSTM part is 50 and the rank of each LoRA part is eight. The algorithm uses softplus as the activation function. The detailed simulation parameter settings are shown in

Table 2. Parameter settings.

Hyperparameters	Values
Learning rate	0.0001
Dropout factor	0.2
Number of network service providers	4
Number of time slices in one day	24
${\eta }_1$ in Equation (27)	3
${\eta }_2$ in Equation (27)	3
${\eta }_3$ in Equation (27)	3
$\omega$ in Equation (34)	20
${\epsilon }_0$ in Equation (39)	200
${\epsilon }_1$ in Equation (39)	300
$\gamma$ in Equation (40)	0.01
${\mu }_1$ in Equation (41)	1
${\mu }_2$ in Equation (41)	1
${\mu }_3$ in Equation (41)	1

.

5.2. Benchmark Schemes

We compare the performance of our proposed LoRA_DDPG approach against the following three baseline schemes.

(1) Baseline 1, LSTM : In the global model, we apply LSTM [24], and the predicted results are directly given to the network resource provider for resource reallocation. We label the results associated with this baseline as LSTM in the experiments.

(2) Baseline 2, DDPG_LSTM : In the local model, we apply LSTM as the base layer, and apply DDPG [25] with LSTM layers in the global model. The actor network in the global model and the local model uses LSTM as the foundational layer, which is the only difference from our proposed method. We label the results associated with this baseline as DDPG_LSTM in the experiments.

(3) Baseline 3, FL [26] : In the local model, we apply LSTM, and the parameters of the model are delivered to aggregate the global model. We label the results associated with this baseline as FL in the experiments.

We use the following performance metrics to evaluate the performance:

(1) Resource: The resource revenue of the accepted SFC requests.

$$r^{averae,accept}={\displaystyle \frac{{\displaystyle \sum _{i=1}^n}{r}^{accept}}{n}}.$$

(51)

where $r^{accept}$ is defined in Equation (27).

(2) Latency: This represents the migration latency of the physical network.

$$c^{averae,migrate}={\displaystyle \frac{{\displaystyle \sum _{i=1}^n}{c}^{migrate}}{n}}.$$

(52)

where $c^{migrate}$ is analyzed in Equation (34).

(3) Energy: The energy consumption of the network, which consists of the energy consumption of all servers.

$$c^{averae,operate}={\displaystyle \frac{{\displaystyle \sum _{i=1}^n}{c}^{operate}}{n}}.$$

(53)

where $c^{operate}$ is defined in Equation (40).

(4) Punish: The resource required by the failed SFC requests.

$$r^{averae,punish}={\displaystyle \frac{{\displaystyle \sum _{i=1}^n}{r}^{punish}}{n}}.$$

(54)

where $r^{punish}$ is defined in Equation (41).

(5) Reward: The linear combination of the above four metrics.

$$r^{averae,mig}={\displaystyle \frac{{\displaystyle \sum _{i=1}^n}{r}^{mig}}{n}}.$$

(55)

where $r^{mig}$ is calculated using Equation (49) or Equation (50).

5.3. Performance Evaluation

To examine the effectiveness of LoRA_DDPG in the global model, we train the LoRA_DDPG model with the parameters selected in the above analysis and then deploy it in the DRL module.

We record the reward values of the validation data during the training process in the global model. We conducted a total of 10 training rounds, with each round using a validation dataset of 144 data points. The reward outcomes for the 144 validation data points across the 10 rounds are presented in the figure. As can be seen from Figure 3, after three rounds of training, the reward outputs for the 144 validation data points in each of the subsequent seven rounds have become very similar. Therefore, the training process has essentially approached convergence after three rounds.

To examine the effectiveness of LoRA_LSTM in the local model, we train the LoRA_LSTM model with the parameters selected in the above analysis.

We record the MSE loss values of the validation data during the second training process (lines 10 to 16 in Algorithm 1) in the local model. As can be seen from Figure 4, the training algorithm of LoRA_LSTM has a rapid convergence trend.

Next, we compare the revenue for accepted SFC requests, the migration latency, the energy consumption, the punishment of refused SFC requests, and the total reward of different algorithms to validate the performance of the proposed algorithm.

In Figure 5, the resource revenue of the accepted SFC requests is plotted for the solutions obtained from LoRA_DDPG, DDPG_LSTM, FL, and LSTM. The results show that the resource revenue of the DDPG_LSTM algorithm is the highest. DDPG_LSTM increases by 4.21% compared with LoRA_DDPG; LSTM reduces by 15.51% compared with LoRA_DDPG; and FL reduces by 9.67% compared with LoRA_DDPG.

Figure 6 illustrates the migration latency values of the four algorithms. It can be observed from the figure that the VNF migration latency value of the LSTM algorithm is the highest. Conversely, the VNF migration latency value of the DDPG_LSTM algorithm is the lowest. DDPG_LSTM reduces by 23.45% compared with LoRA_DDPG; LSTM increases by 49.63% compared with LoRA_DDPG; and FL increases by 26.81% compared with LoRA_DDPG.

Figure 7 depicts the energy consumption of the four algorithms. LoRA_DDPG performs the best in saving energy, but the difference in value between them is very small. DDPG_LSTM increases by 0.17% compared with LoRA_DDPG; LSTM increases by 0.37% compared with LoRA_DDPG; and FL increases by 0.01% compared with LoRA_DDPG.

Figure 8 presents the total punishment for denying SFC requests. We can observe that DDPG_LSTM achieves the lowest punishment. DDPG_LSTM reduces by 11.59% compared with LoRA_DDPG; LSTM increases by 50.79% compared with LoRA_DDPG; and FL increases by 46.55% compared with LoRA_DDPG.

Figure 9 presents the total reward of these algorithms. DDPG_LSTM increases by 51.59% compared with LoRA_DDPG; LSTM reduces by 157.44% compared with LoRA_DDPG; and FL reduces by 94.61% compared with LoRA_DDPG.

Through the above comparison, we find that DDPG_LSTM and LoRA_DDPG result in a better performance, DDPG_LSTM results in the best performance, and LoRA_LSTM ranks second in prediction performance. However, LoRA_DDPG transmits a smaller number of parameters, and the analysis is as follows: Each LSTM layer has a parameter count of 4 * (input dimension * hidden dimension + hidden dimension * hidden dimension + hidden dimension). Thus, when the hidden dimension per layer is 50, the output dimension is 1, and the input dimension is 144, the first layer contains 39,000 parameters, while the second layer contains 20,200 parameters. Thus, for the DDPG_LSTM architecture, each local model needs to transmit 59,200 parameters per iteration. The parameter count of a low-rank matrix is (input dimension * rank + rank * output dimension). Thus, for the two-layer fusion of LSTM and LoRA, when the rank value is eight, the first layer has 6208 parameters, and the second layer has 3200 parameters. Thus, for the LoRA_DDPG architecture, each local model needs to transmit 9408 parameters per iteration. The amount of parameters transmitted from the local model to the global model under the LoRA_DDPG structure is reduced by 84.11% compared with DDPG_LSTM.

6. Conclusions

In this manuscript, we study the network traffic prediction problem for end-to-end network service deployment. We focus on three key challenges of the prediction problem in the DT-assisted and SDN/NFV-enabled network. For the first challenge of the definition of the mathematical model considering the DT and multiple merchants, we define the VNF migration problem for multiple merchants in the DT network. For the second challenge of better resource utilization, we carefully take the network service provider and network resource provider into account, and define the Markov Decision Process for the network traffic prediction problem. For the last challenge of protecting the privacy of each merchant and reducing model parameters for communication, we embed LoRA into LSTM, and propose the algorithm LoRA_DDPG. We evaluate the performance of LoRA_DDPG through experimental simulations, and the simulation results show that our algorithm outperforms benchmark algorithms and achieves a better performance.

In summary, this research not only presents a novel prediction methodology but also paves the way for building more intelligent, autonomous, and efficient future networks. The proposed framework finds concrete and impactful applications in the core challenges of 5G/6G evolution, namely agile network slicing and responsive edge computing. Future work will involve implementing and validating this framework in a real-world testbed emulating these specific scenarios.