4.1. PPDM Overview and Limitations
PPDM [
11] adopts the hierarchical SFC deployment architecture (
Figure 2) to preserve domain privacy. The architecture consists of two layers: a centralized Multi-Domain Controller (MDC) and multiple Local Controllers (LCs). The MDC, which is operated by the VNO, coordinates the end-to-end deployment of the SFC across multiple domains. Each LC, which is operated by the respective InP, is responsible for deploying VNFs and virtual links in the requested SFC within the domain. In order to preserve domain privacy, detailed substrate-level information is not shared among MDC and LC; instead, abstracted resource information called the Service Intention Response Matrix (SIRM) is shared among them. SIRM is a binary matrix where the (
i,
j) element
takes the value 1 if VNF
can be deployed on the substrate node
, and 0 otherwise.
Figure 3 illustrates the PPDM deployment procedure. When an SFC request arrives, the MDC distributes the request to all LCs. Each LC evaluates the feasibility of mapping VNFs onto its local substrate nodes based on its private resource information. Specifically, each LC constructs the corresponding elements of the SIRM, where
indicates whether a substrate node
can host VNF
. These SIRM elements are then returned to the MDC and aggregated into a global SIRM. The MDC subsequently feeds this SIRM to the DQN agent, which selects a substrate node for each VNF in the SFC. The corresponding virtual links are then mapped onto the shortest paths in the substrate network.
A key characteristic of PPDM is that the feasibility of all VNFs in the SFC must be evaluated before the final decision on the deployment of the SFC is made. During this SIRM construction stage, whenever a substrate node is deemed capable of hosting a VNF, the required resources are temporarily reserved, even though the final deployment has not yet been determined. This temporary reservation is referred to as virtual occupation.
To reduce unnecessary evaluation and limit the number of candidate nodes, PPDM applies a top- node selection strategy, where only a subset of candidate nodes with higher capacity is considered. Although this approach helps reduce the number of evaluated nodes, it does not eliminate the effect of virtual occupation.
Because all VNFs are evaluated in advance, a single substrate node can reserve resources for multiple VNFs simultaneously. As a result, the effective capacity observed during the evaluation stage can be significantly lower than the actual physical capacity available after deployment. The available resources of the substrate node
when evaluating the feasibility of VNF
are expressed as
where
is the set of VNFs whose feasibility has already been evaluated within the current SFCs and
is the set of VNFs belonging to the SFCs that are currently operating.
As provisional reservations in the second term accumulate, the remaining capacity observed during evaluation may become significantly smaller than the actual capacity available after the final deployment. Consequently, later VNFs may be incorrectly classified as infeasible, even though sufficient resources would exist if temporary reservations were released. This effect is reflected in the SIRM, where the feasibility indicators tend to become zero for all candidate substrate nodes. This situation is referred to as an ambiguous state, where no candidate node is marked as feasible (i.e., for all ). As a result, the DQN agent receives a degraded state representation with no feasible options, making it difficult to identify valid deployment decisions. This increases the likelihood of deployment failure and ultimately reduces the overall acceptance ratio achieved by PPDM.
4.2. -Batch Sequential Deployment Mechanism
As discussed earlier, PPDM evaluates the feasibility of an entire SFC in a single orchestration round. Consequently, each LC must maintain virtual reservations for all feasible VNFs throughout the orchestration cycle. This design results in cumulative virtual occupation across domains, which can significantly distort perceived resource availability.
To address this limitation, we propose the -Batch mechanism which evaluates the feasibility of consecutive VNFs rather than the entire SFC. For each domain k, the LC computes and returns a compact batch-based SIRM. As a result, only VNFs within the current batch contribute to temporary resource reservations, thereby reducing both the magnitude and the duration of virtual occupation within the domain.
Figure 4 illustrates the general deployment procedure of the proposed the
-Batch mechanism. When the MDC receives an SFC request, the complete SFC information is sent once to all LCs during the initialization phase. This information includes the SFC request identifier, VNF resource requirements, virtual link structure, and service ordering information.
After receiving the SFC request, each LC evaluates the feasibility of the VNFs according to the current batch identifier. For the current batch, the LC performs a local feasibility evaluation using its internal topology and resource information. Temporary resource reservations, referred to as virtual occupation, are applied only to VNFs within the current batch. Based on this evaluation, each LC constructs and returns a batch-level SIRM response to the MDC. The SIRM only contains binary feasibility indicators , where indicates that the substrate node can feasibly host VNF , and otherwise.
Based on the SIRM responses received, the MDC estimates the predicted resource condition of the substrate nodes and ranks the feasible nodes. The MDC then selects the top- candidate nodes from the feasible SIRM entries and uses the resulting candidate set as the input to the DQN-based node selection process. This candidate selection strategy follows the general SIRM-based orchestration principle, where the MDC makes deployment decisions using feasibility indicators rather than detailed domain-internal resource information. In this study, top- selection is performed at the MDC to support global candidate comparison across multiple domains.
After the DQN determines the deployment decision for the VNFs in the current batch, the MDC sends the selected deployment result to the corresponding LC. The deployment notification includes control information such as the request ID (req_id), batch ID (sub_sfc_id), acknowledgment flag (ack), selected substrate-node mapping, and the next batch identifier. These identifiers are used to track the progress of deployment, synchronize batch-by-batch execution, and support rollback if a later batch fails.
Upon receiving the deployment notification, the corresponding LC performs the actual VNF deployment on the selected substrate nodes and releases unused virtual occupations from the current batch. If the deployment succeeds, the LC sends a success notification to the MDC together with the identifier of the next batch to be processed. This procedure continues until all batches of the SFC are successfully deployed. If a later batch fails, the MDC triggers a rollback for previously deployed batches and marks the SFC request as rejected.
Compared to PPDM, which performs a single SIRM exchange for the entire SFC, the proposed method introduces multiple SIRM exchanges in batches. This increases the signaling frequency, as multiple rounds of interaction between the MDC and the LC are required. However, each exchange involves fewer VNFs, resulting in a smaller feasibility evaluation at each step.
The total number of orchestration rounds depends on the batch size B. Smaller batch sizes result in more rounds, leading to a longer deployment setup time but more accurate feasibility evaluation. In contrast, larger batch sizes reduce the number of orchestration rounds, which can shorten the overall deployment setup time at the cost of reduced feasibility accuracy.
Under the proposed the
-Batch mechanism, the virtual occupation is confined to the VNFs within the current batch, resulting in a smaller and shorter virtual occupation. Consequently, the effective available capacity of the substrate node
during the feasibility evaluation is given by:
where
denotes the set of VNFs within the current batch whose feasibility has been evaluated,
denotes the set of VNFs in the current SFC that have already been deployed before the current batch, and
denotes the set of VNFs belonging to SFC that are currently in operation. Note that the second term on the RHS represents the virtual occupation within the current batch, which is reduced compared to the virtual occupation over the entire SFC. The third and fourth terms represent the actual resource consumption of VNFs that have already been deployed in the current SFC and those belonging to active SFCs, respectively.
The proposed mechanism preserves privacy by limiting the information exchanged between the MDC and the LC. Each LC evaluates batch feasibility using its own local information, such as available resources, internal topology, and routing conditions. However, these detailed data are not shared with the MDC. The LC only returns binary SIRM feasibility indicators, which show whether a candidate substrate node can support a specific VNF in the current batch. Therefore, the MDC can make deployment decisions without knowing exact residual resources, internal topology, link states, or routing paths. However, because SIRM responses are exchanged repeatedly, the MDC may still observe coarse feasibility trends over time. For example, if a node is frequently feasible or becomes infeasible across several requests, the MDC may infer general resource availability or congestion. Thus, the proposed mechanism prevents direct disclosure of sensitive domain information, while repeated binary responses may still provide limited indirect information.
4.4. DRL Agent for Batch Deployment
Figure 5 illustrates the DQN-based learning framework adopted for the proposed the
-Batch mechanism. The general DRL formulation follows the conventional approach introduced in [
11], while our contribution focuses on the proposed the
-Batch mechanism. In this architecture, the multi-domain substrate network together with the proposed
-Batch mechanism forms the environment, while the MDC acts as learning agents. At each decision step, the agent observes the state
derived from the batch-level SIRM and selects an action
corresponding to the deployment of a single VNF onto a substrate node. This sequential decision process continues until all VNFs in the current batch are deployed.
The state corresponds to the feasibility information at the batch level represented by the SIRM, which preserves domain privacy by providing the agent only with feasibility indicators rather than fine-grained substrate resource details; once the SIRM for all VNFs in the batch is constructed, the DQN agent sequentially uses each row corresponding to VNF , containing feasibility information across all substrate nodes, as the state input to determine the deployment action.
The action represents the selection of a substrate node for the current VNF within the batch. The action space is discrete, and its size corresponds to the total number of substrate nodes in all domains. This design ensures that the agent operates in a finite action space. After all VNFs in the batch are deployed, the corresponding virtual links between the selected substrate nodes are activated. are routed using the shortest-path method.
The Q-network is implemented as a fully connected feedforward neural network that approximates the action-value function
. As shown in
Figure 5, the size of the input layer is equal to the state dimension, denoted by
state_size, where
state_size corresponds to the total number of substrate nodes in all domains. The network then processes the input through several hidden layers. The output layer contains
action_size neurons, where
action_size corresponds to the total number of substrate nodes in all domains. Each output neuron represents the Q-value of selecting one substrate node for the current VNF.
The reward function adopted in this environment follows the formulation proposed in [
11] and is defined as:
where each term captures a distinct system objective: (i) the feasibility reward
, which reflects the SIRM-based feasibility result for the current VNF; (ii) the global substrate utilization term
, which encourages efficient resource usage based on node utilization; (iii) the delay-related term
, which emphasizes the end-to-end delay performance; and (iv) the repetition penalty
, which discourages repeated node selections.
It should be noted that the proposed DQN model uses a fixed-dimensional SIRM-based state representation and a discrete action space corresponding to the candidate substrate nodes. Therefore, the model is trained with a specific network configuration. If the number and ordering of substrate nodes remain unchanged, the input and output dimensions of the DQN model are preserved, and the trained model can still be applied to updated resource availability or traffic conditions represented through the SIRM. However, when the number of domains or the number of substrate nodes changes significantly, the state and action spaces can also change, which can affect the performance of the trained model. In such cases, partial or complete retraining may be required.
The Algorithm 1 summarizes the general training and deployment procedure of the proposed -batch mechanism. For each SFC request arriving, the VNF chain is partitioned into fixed-size batches, and the VNFs within each batch are processed sequentially. At each decision step, the environment constructs the SIRM and provides the corresponding state to the DQN agent, which selects an action corresponding to the deployment of the current VNF. The environment then executes the action, updates the availability of resources, and computes the reward . If any VNF within a batch cannot be successfully deployed, the entire SFC request is rejected; otherwise, the process continues until all batches are successfully deployed. When each SFC deployed reaches its expiration time, the resources allocated to the expired SFC are released.
The agent is trained using the standard DQN framework. After executing the action
in state
, the transition tuple
is stored in the replay buffer. When sufficient samples are available, a mini-batch is drawn to update the Q-network. For each sampled transition, the target value is computed as
where
denotes the target network and
is the discount factor. The Q-network parameters
are updated by minimizing the temporal-difference loss
using stochastic gradient descent. This optimization step reduces the discrepancy between the predicted and target Q-values, thus improving future action selection.
| Algorithm 1 -Batch Sequential Deployment Mechanism for Multi-Domain SFCs |
- Input:
Set of SFC requests - Output:
Deployment decisions and trained DQN policy
- 1:
Initialize Q-network , target network , and replay buffer . - 2:
for each episode do - 3:
Reset environment and substrate resource state. - 4:
for each arriving SFC request do - 5:
Partition the VNF chain of into fixed batches . - 6:
- 7:
for each batch of SFC do - 8:
Construct batch-level SIRM for the current batch. - 9:
for each VNF do - 10:
Extract state from the SIRM row of . - 11:
Select action using an -greedy policy based on . - 12:
Execute deployment according to . - 13:
if deployment fails then - 14:
- 15:
break - 16:
end if - 17:
- 18:
Observe next state and terminal flag . - 19:
Store transition in replay buffer . - 20:
if is sufficient then - 21:
Sample a mini-batch from . - 22:
- 23:
Update Q-network by minimizing - 24:
end if - 25:
Periodically update target network . - 26:
end for - 27:
if not then - 28:
Trigger rejection or rollback procedure for . - 29:
break - 30:
end if - 31:
end for - 32:
if then - 33:
Accept SFC request . - 34:
else - 35:
Reject SFC request . - 36:
end if - 37:
end for - 38:
end for
|