Dynamic Selection Slicing-Based Offloading Algorithm for In-Vehicle Tasks in Mobile Edge Computing

Abstract

With the surge in tasks for in-vehicle terminals, the resulting network congestion and time delay cannot meet the service needs of users. Offloading algorithms are introduced to handle vehicular tasks, which will greatly improve the above problems. In this paper, the dependencies of vehicular tasks are represented as directed acyclic graphs, and network slices are integrated within the edge server. The Dynamic Selection Slicing-based Offloading Algorithm for in-vehicle tasks in MEC (DSSO) is proposed. First, a computational offloading model for vehicular tasks is established based on available resources, wireless channel state, and vehicle loading level. Second, the solution of the model is transformed into a Markov decision process, and the combination of the DQN algorithm and Dueling Network from deep reinforcement learning is used to select the appropriate slices and dynamically update the optimal offloading strategy for in-vehicle tasks in the effective interval. Finally, an experimental environment is set up to compare the DSSO algorithm with LOCAL, MINCO, and DJROM, the results show that the system energy consumption of DSSO algorithm resources is reduced by 10.31%, the time latency is decreased by 22.75%, and the ratio of dropped tasks is decreased by 28.71%.

1. Introduction

Mobile edge computing (MEC) involves deploying server nodes near users to address excessive latency issues and data flow congestion [1]. By enabling the deployment of functions and applications at the wireless access network (RAN) edge layer closest to user devices, MEC provides computing resources and high-frequency bandwidth to avoid network congestion caused by massive data. At the same time, user devices offload processing tasks to the network edge nodes and real-time access data from MEC edge nodes to achieve information exchange purposes [2,3]. However, the distributed deployment of MEC edge nodes and task offloading between nodes consume additional energy. Therefore, reducing the energy consumption of task offloading in the MEC environment while meeting the requirements of latency and quality of service (QoS) has become a significant challenge hindering the development of MEC applications.

The Internet of Vehicles (IoV) is one of the main application scenarios for task offloading in MEC [4]. In actual IoV applications, task offloading between in-vehicle terminal devices and edge servers is typically influenced by multiple factors [5,6]. Among them, considering only offloading tasks to edge nodes when confronted with a significant volume of service requests and a vast amount of data, it is inevitable that an imbalance in the load of edge nodes and data loss may occur. Furthermore, extensive data exchange can result in high time latency and energy consumption during communication, which will lead to mission failure. Especially in vehicular networks with heterogeneous latency and computing requirements because vehicles usually choose the closest or best communication quality Base Station (BS) or Road Side Unit (RSU) to perform computing offloading of vehicular tasks, and the high mobility of vehicles and short effective communication range can lead to frequent switching of edge servers, which can easily cause untimely offloading and communication failures [7]. Thus, the offloading algorithm of vehicular tasks needs a dramatic reduction in delay and energy consumption.

Network slicing [8] is introduced into the IoV to provide an economical and efficient solution for vehicle. Network slicing is one of the key technologies of software-defined networking (SDN) [9], which is the logical division of the common physical infrastructure into multiple network slices through various network access technologies to provide guarantees for heterogeneous quality-of-service performance and maximize network operator optimization goals. Each slice can act as an independent end-to-end network to support the service requirements of various in-vehicle applications [10,11]. However, the resource-constraint result in infrastructure providers not being able to satisfy all service requests [12]. Therefore, to tackle the problem that resource-constrained vehicular terminal devices can hardly handle massive computational demands in real-time, the issue of data transmission failure due to frequent changes in the Vehicular Ad-hoc Networks (VANETs) environment [13]. It is an urgent choice to build a lower cost computational offloading model using a user-oriented network slicing approach to handle the computational offloading of tasks to achieve the optimization goal of low latency and low energy consumption.

Most of the current offloading algorithms focus on leveraging Network Function Virtualization (NFV) and Software Defined Network (SDN) technologies to minimize the delay and energy consumption associated with task offloading [14,15]. However, these algorithms seldom consider the dependencies between tasks dependencies. The high-speed movement of vehicles requires frequent exchange of resource state information, which triggers untimely information updates and leads to low offloading efficiency of in-vehicle tasks [16,17,18,19]. For this reason, this paper considers the dependency relationships between in-vehicle tasks and integrates network slices into the edge server (ES), uses Dueling Network for model training, and dynamically updates the selection slice results. The main contributions are as follows.

• The Dynamic selection slicing-based offloading algorithm for vehicular tasks: To accomplish the optimization objective of decreasing the aggregate expense related to task offloading, the dynamic selection slicing-based offloading algorithm for in-vehicle tasks in MEC is proposed in this paper. The algorithm uses the Dueling Network combined with Deep Q-Network (DQN) in deep reinforcement learning to dynamically select the optimal slicing results and then update the optimal offloading policy in the effective interval.
• Performance Evaluation: The results show that the DSSO algorithm proposed in this paper improves the efficiency of multi-dimensional resource utilization and task completion rate and reduces the total cost of the offloading task compared with the LOCAL, MINCO, and DJROM.

The rest of this paper is organized as follows. Section 2 presents related work. Section 3 designs the in-vehicle tasks offloading model based on the dynamic selection slices. The DSSO algorithm is proposed in Section 4. Experimental comparison analysis is performed in Section 5. Finally, Section 6 is the conclusion of the paper.

2. Related Work

The rise of MEC facilitates mobile devices to offload tasks to nearby edge nodes for processing to reduce task time delays and decrease the discard ratio of delay-sensitive tasks [20,21,22,23,24,25]. Bi et al. [20] considered wireless power supply weighting and computational rate maximization in the MEC scenario and proposed a joint optimal offloading algorithm based on the alternating direction multiplicative decomposition technique, but the algorithm ignored the inter-task waiting delay during task offloading. Tang et al. [21] explored a distributed offloading algorithm for the Mobile Edge Computing (MEC) environment that relies on model-free deep reinforcement learning. The aim of this algorithm is to minimize the expected long-term cost. Cheng et al. [22] focused on reducing the UE processing delay and energy consumption for offloading tasks and proposed a distributed offloading dynamic optimization algorithm for the MEC server multi-user, but the algorithm did not consider the optimization of the algorithm using radio resources and extend to other task-offloading scenarios with the complex structure of multi-user and multi-cell. Kim et al. [23] proposed a predictive model to decompose and offload tasks to multiple MEC servers to reduce the execution time at each MEC server. However, they did not consider the complexity of real WANs and the queueing delay of MEC servers when tasks are queued. Li et al. [24] designed an online energy minimization algorithm for multiple IoT devices with latency constraints in edge computing environments. Poularakis et al. [25] studied the multi-dimensional limitations of storage, computation, and communication to support MEC service placement and joint optimization problems for request routing; however, the algorithm does not consider the coordination problem of the existence of backhaul links between the base stations.

Network slicing is applied to computational offloading. The task improves the utilization of multi-dimensional resources in the network by selecting the best performance slices for computational offloading [26,27,28]. Chen et al. [26] proposed a DDQN-based computational offloading strategy for MEC environments that considers the time-varying channel quality of users and base stations, the energy units received by the wireless environment; thus, the maximized long-term utility of the dynamically exploited network slices resulting from the arrival of computational tasks. Huynh et al. [27] studied a network slicing-based management framework that considers the uncertainty and timeliness of network resource requests and captures the sliced requests through a semi-Markov decision process to obtain the optimal resource allocation policy. Al-Khatib et al. [28] proposed a slicing scheme that is based on priority and reservation. The scheme is designed for diverse vehicle applications and enhances the utilization of network resources while guaranteeing network utility. However, this approach does not account for the distinction between prediction models and immediate allocation.

The offloading algorithm based on deep reinforcement learning (DRL) provides a prospective solution for addressing tasks offloading in vehicular networks [29,30,31,32,33]. Peng et al. [29] proposed a distributed DRL-based algorithm for network resource management in highly dynamic vehicle scenarios. The algorithm trains deep neural networks with multiple agents, where the MEC server acts as the agent, to make online decisions on vehicle computation offloading and resource allocation. Mlika et al. [30] introduced a model-free approach based on DRL to solve the problem of the wireless channel, power allocation, slice selection, and vehicle grouping in MEC-enabled IoV networks using non-orthogonal multiple access (NOMA) to make better use of scarce channel resources. Nassar et al. [31] designed a DRL-based computational offloading strategy that allocates limited resources to vehicles with heterogeneous delay requirements. Li et al. [32] presented a task partitioning and scheduling algorithm to solve workload allocation, but relying on task partitioning and scheduling alone cannot find the optimal offloading strategy in an effective interval. Huang et al. [33] proposed an online offloading algorithm that maximizes the weighted value and computational efficiency based on a DRL-based algorithm, but the mobility of the device can make the algorithm harder to converge.

Although the above algorithms consider offloading tasks to edge nodes in MEC to improve task offloading efficiency as much as possible, the requirements of relational-dependent and delay-sensitive tasks can hardly be satisfied. The high-speed movement of vehicles requires frequent exchange of resource status information, which leads to untimely information updates and low offloading efficiency of in-vehicle tasks. Therefore, the offloading algorithm based on dynamic selection slicing for vehicle tasks is proposed where the custom slice is applied to the algorithm.

3. System Model and Optimization Goal

3.1. Communication Model

The process of in-vehicle task offloading based on the dynamic selection slices in MEC is shown in Figure 1. First, the in-vehicle terminal device initiates a compute offload request to the SDN controller. The SDN controller and the slice orchestrator interact with information by providing available status through the RSU, which generates data results that are the criteria for determining whether to offload the pending tasks to the edge server. Then, the offloading task is offloaded to a suitable slice within the edge server. Finally, the offloading results are transmitted to the slices through the offloading decision, and the optimal policy for task offloading is returned to the in-vehicle terminal device. The parameters involved in this paper are listed in

Table 1. Parameters and Description.

Parameters	Description
$D_i$	Vehicle $i$
${\omega }_i$	In-vehicle task
$d_{{\omega }_i}$	In-vehicle task input size (MB)
$S_l$	Slice $l$
$A{P}_j$	Access point $j$
$y_i^l$	An indicator variable to represent whether the vehicle $D_i$ select slice $S_l$ offload or not
$T_{i,{\omega }_i}^{ex}$	The execution delay in the slice for vehicle $D_i$ (S)
$T_{i,j,{\omega }_i}^{ex,l}$	The task execution time delay (S)
$T_{{\omega }_1,{\omega }_i}^{delay}$	The communication delay (S)
$T_i^l$	The total time latency (S)
$E_i^{ex}$	The execution energy in the slice for the vehicle $D_i$ (J)
$E_i^{service}$	The energy consumption for the tasks ${\omega }_i$ to the slice $S_l$ (J)
$E_i^{trans,l}$	The energy consumption for the vehicle $D_i$ to transfer the task to the slice $S_l$ (J)
$E_i^l$	the total energy consumption (J)
$f_i{}^{local}$	The CPU cycle frequency of the vehicle $D_i$ (MHz)
$P_i$	The vehicle $D_i$ transmission power (W)
$\tau \left(t\right)$	Task deadline (S)
$\sigma$	The noise in the channel transmission (dB)
$Co{m}_l$	The total amount of computing resources in the slice $S_l$
${\lambda }_i^l$	The number of computing resources allocated to the vehicle $D_i$ in the slice $S_l$
$R_{i,j}^l$	The transmission rate (bps)
$B_{i,j}$	The network bandwidth (bit/s)
$h_{i,j}$	The channel gain between the vehicle $D_i$ and the access point $A{P}_j$

.

3.2. Offloading Model

The Dynamic Selection Slicing-based Offloading Algorithm for in-vehicle tasks, $\omega =\left({\omega }_1,\dots {\omega }_{i-1},{\omega }_i\right)$ represents vehicular tasks, and $x_i\in \left[0,1\right]$ indicates the offload strategy of the in-vehicle task. The dependencies of the in-vehicle tasks are described using the directed acyclic graph 2, and the nodes in Figure 2 show the computational offload tasks generated by the in-vehicle terminal devices, and the dependencies are represented between tasks using the directed edges of nodes. The node ${\omega }_1$ without a parent is considered to be the start of a task, while the node ${\omega }_5$ without a child is considered to be the end of a task, and the node ${\omega }_4$ is represented as dependent on ${\omega }_2$ and ${\omega }_3$. The white nodes state that the edge server is capable of offloading tasks; the shaded nodes indicate that the task can only be executed locally in the vehicle.

• Local offload

$x_i=0$ indicates that the task can only be executed locally in the vehicle. $C_{{\omega }_i}$ expresses various resources required per bit of a task, and $M$ shows the set of vehicles. Therefore, the execution time of in-vehicle tasks $T_i^{ex}$ can be expressed as:

$$T_i^{ex}=d_{{\omega }_i}·C_{{\omega }_i}/f_i^{local}\hspace{1em}\hspace{1em}\forall i\in M$$

(1)

when local execution is selected for vehicular tasks, the transmission time latency can be ignored. Total time latency $T^{local}$ is the execution time, which can be shown as:

$$T^{local}=T_i^{ex}\hspace{1em}\hspace{1em}\forall i\in M$$

(2)

The energy consumption $E^{local}$ of the vehicle $D_i$ processing task is represented as:

$$E^{local}=E_i^{ex}=d_{{\omega }_i}·C_{{\omega }_i}·{\left(f_i^{local}\right)}^2 \forall i\in M$$

(3)

According to Equations (1)–(3), it can be seen that the task latency and energy consumption are mainly determined by the CPU cycle frequency.

2.

Offloading to the slice

$x_i=1$ represents that vehicle load pending tasks to the slices to access edge layer services. The vehicle terminal is connected to the slices $S_l$ through several access points $A{P}_j$. Owing to the dependencies between most of the tasks, it is commonly the situation that $A{P}_j$ is too busy to accept the next task. Thus, a binary variable $y_i^l\in \left(0,1\right)$ is introduced to determine whether the edge server interface is idle or not. When it is $y_i^l=0$, the access point $A{P}_j$ is idle. Otherwise, it needs to wait in queue.

In Equation (4), ${\rho }_{i,j}^l$ denotes the amount of wireless resources provided to the vehicle $D_i$ by slice $S_l$ docked to the access point $A{P}_j$, $B_{i,j}$ indicates the network bandwidth, $P_i$ illustrates the transmission power of $D_i$, and $h_{i,j}$ indicates the channel gain between the vehicle $D_i$ and the access point $A{P}_j$. “m” denotes a globally fixed constant, and similarly, $P_m$ denotes the transmission power of $D_m$, $h_{m,j}$ shows the channel gain between $D_m$ and $A{P}_j$, and $\sigma$ represents noise in the channel transmission. The transmission rate $R_{i,j}^l$ can be expressed as:

$$R_{i,j}^l={\rho }_{i,j}^l·B_{j,l}·\log \left(1+\frac{p_i·h_{i,j}}{\sigma +{\displaystyle \sum _{y_{i,j}^l=1}{p}_m+h_{m,j}}}\right)$$

(4)

The task transmission delay $T_{i,j,{\omega }_i}^{trans,l}$ is denoted as:

$$T_{i,j,{\omega }_i}^{trans,l}=T_{i,1,{\omega }_i}^{trans,j}=d_{{\omega }_i}/R_{i,1}^l\hspace{1em}\forall R_{i,1,{\omega }_i}^l>0$$

(5)

Furthermore, the task execution time delay $T_{i,j,{\omega }_i}^{ex,l}$ is expressed as:

$$T_{i,j,{\omega }_i}^{ex,l}=T_{i,1,{\omega }_i}^{ex,l}=\left(d_{{\omega }_i}·C_{{\omega }_i}\right)/\left({\lambda }_i^l·Co{m}_l\right)\hspace{1em}\forall {\lambda }_i^l>0$$

(6)

If the pending task is a dependent task when sliced for computational offloading inevitably results in a queueing situation. ${\omega }_{i-1}$ outputs the offloading result and returns to the slice, and then, ${\omega }_i$ is transmitted and offloaded to the slice, and ${\delta }_{{\omega }_i}$ represents the offload result of the task ${\omega }_i$. Therefore, it is necessary to consider the communication delays arising from the dependencies between tasks for latency-sensitive tasks. Through Formula (7), the communication delay $T_{{\omega }_1,{\omega }_i}^{delay}$ between tasks is expressed as:

$$T_{{\omega }_1,{\omega }_i}^{delay}=\left\{\begin{array}{cc}\frac{{\delta }_{\omega }{}_{{}_i}}{R_{i,j}^l}& x_i=1,y_i=0\\ 0& x_i=1,y_i=1\end{array}\right.$$

(7)

The total time latency $T_i^l$ for offloading the task ${\omega }_i$ to the slice $S_l$ is shown as:

$$T_i^l=T_{i,j,{\omega }_i}^{ex,l}+T_{i,j,{\omega }_i}^{trans,l}+T_{{\omega }_1,{\omega }_i}^{delay}$$

(8)

The offloaded energy consumption $E_i^{service}$ for the tasks ${\omega }_i$ to the slice $S_l$ is expressed as:

$$E_i^{service}=P_l·\left(T_{i,j,{\omega }_i}^{ex,l}+T_{{\omega }_1,{\omega }_i}^{delay}\right)$$

(9)

Similarly, the energy consumption $E_i^{trans,l}$ required to support the vehicle $D_i$ to transfer the task to the slice $S_l$ can be given as:

$$E_i^{trans,l}=P·T_{i,j,{\omega }_i}^{trans,l}$$

(10)

Therefore, the total energy consumption $E_i^l$ for offloading tasks ${\omega }_i$ to slice $S_l$ can be shown as:

$$E_i^l=E_i^{trans,l}+E_i^{service,l}$$

(11)

3.

Optimization objectives

Throughout the overall Telematics system, energy consumption is a key metric to measure whether the offloading strategy is optimal. In this paper, the total vehicle energy consumption $E_1$ is expressed as:

$$E_1={\displaystyle \sum _{i\in M}\left(1-x_i\right)·E_i^{ex}+{\displaystyle \sum _{i\in M}{\displaystyle \sum _{j\in O}{\displaystyle \sum _{l\in P}{x}_i·y_i}}}·E_i^{trans,l}}$$

(12)

In Equation (12), $i\in M=\left\{1,2\cdots NM\right\}$, $j\in O=\left\{1,2\cdots NO\right\}$, $l\in P=\left\{1,2\cdots NP\right\}$, and $N$ denotes the set of natural numbers.

Similarly, the energy consumption $E_2$ for offloading the in-vehicle tasks to the slice can be represented as:

$$E_2={\displaystyle \sum _{i\in M}{\displaystyle \sum _{l\in P}{x}_i·y_i}}·E_i^{ex,l}$$

(13)

Therefore, the total energy consumption $E$ of the in-vehicle task offloading is shown as:

$$\mathrm{P}1:\hspace{1em}E=E_1+E_2={\displaystyle \sum _{i\in M}\left(1-x_i\right)·E_i^{ex}+{\displaystyle \sum _{i\in M}{\displaystyle \sum _{j\in O}{\displaystyle \sum _{l\in P}{x}_i·y_i}}}·E_i^{trans,l}}+{\displaystyle \sum _{i\in M}{\displaystyle \sum _{l\in P}{x}_i·y_i}}·E_i^{ex,l}$$

(14)

$$s.t.\left\{\begin{array}{l}C1:0\le f_i\le f_{i,\max },0\le p_i\le p_{i,\max }\\ C2:0\le {\displaystyle \sum _{i\in M}{\rho }_{i,j}^l}\le 1, \forall j\in O,l\in P\\ C3:x_i\le {\displaystyle \sum _{l\in P}{y}_i^l},x_i,y_i^l\in \left[0,1\right], \forall i\in M,j\in O,l\in P\\ C4:0\le {\displaystyle \sum _{i\in M}{\lambda }_i^l}\le 1, \forall l\in P\\ C5:{\lambda }_i^l\le y_i^l\le {\omega }_i{\lambda }_i^l, \forall i\in M,l\in P\\ C6:\left(1-x_i\right)\cdot E_i^{ex}+{\displaystyle \sum _{i\in M}{\displaystyle \sum _{j\in O}{\displaystyle \sum _{l\in P}{x}_i\cdot y_i^l\cdot E_i^l}}}\le {\tau }_i, \forall i\in M\end{array}\right.$$

In Equation (14), constraints ($C1$)–($C3$) denote restrictions on the CPU cycle frequency, transmission power, and the amount of wireless resources. Constraint ($C4$) indicates the computational resources available to the slice. Constraint ($C5$) ensures that the task is processed before the deadline. Then, the NP-hard problem P1 is defined as Theorem 1.

Theorem 1.

P1 is an NP-hard problem.

Proof of Theorem 1.

Assuming that vehicles select a slice for offloading, the transmitted power is set to a maximum value. One access point $AP$, the deadline ${\tau }_i$ of the task is large enough, ${\gamma }_1,{\gamma }_2$ represent the average number of vehicles distributed equally, and allocate wireless resource ${\rho }_{i,j}^l$ and computing resources ${\lambda }_i^l$ to vehicles, ${\rho }_{i,j}^l=1/{\gamma }_1$, ${\lambda }_i^l=1/{\gamma }_2$ in Equation (15), and the energy consumption $E_i^{l,*}$ can be expressed as:

$$E_i^{l,*}=\frac{{\gamma }_1·d_{{\omega }_i}·p_{i,\max }}{B_{i,l}·\log \left(1+\frac{P_{i,\max }·h_{i,j}}{{\displaystyle \sum _{m\in M}{p}_{m,\max }·h_{m,j}}}\right)}$$

(15)

In Equation (14), $x_i=1$ represents the vehicle to offload the pending tasks to the slice.

At this point, the entire system energy consumption can be expressed as shown in P2.

$$\mathrm{P}2: {\displaystyle \sum _{i\in M}{\displaystyle \sum _{l\in P}{y}_i^l}}·E_i^{l,\ast }$$

(16)

$$s.t.\left\{\begin{array}{l}C1:{\displaystyle \sum _{l\in P}{y}_i^l}=1, \forall i\in M\\ C2:{\displaystyle \sum _{i\in M}{\displaystyle \sum _{l\in P}{y}_i^l}}·{\rho }_{i,1}^l\le 1\end{array}\right.$$

In Equation (16), P2 is mapped to a multi-dimensional multi-choice knapsack problem (MMKP) [34], i.e., P2 is also an NP-hard problem. However, $x_i=0$ indicates offloading of in-vehicle tasks to the vehicle terminal, i.e., the task tasks are processed locally. Thus, the system energy consumption can be expressed as $E^{local}=E_i^{ex}$. The above two cases can be reduced to P1: $E=E_1+E_2={\displaystyle \sum _{i\in M}\left(1-x_i\right)·E_i^{ex}+{\displaystyle \sum _{i\in M}{\displaystyle \sum _{j\in O}{\displaystyle \sum _{l\in P}{x}_i·y_i}}}·E_i^{trans,l}}+{\displaystyle \sum _{i\in M}{\displaystyle \sum _{l\in P}{x}_i·y_i}}·E_i^{ex,l}$, and it can be concluded that P1 is an NP-hard problem. □

Similarly, the time latency $T_1$ of local vehicle $D_i$ processing is expressed as:

$$T_1=\left(1-x_i\right){\displaystyle \sum _{i\in M}{T}_i^{ex}}$$

(17)

The time delay for offloading the task to the slice $T_2$ can be shown as:

$$T_2={\displaystyle \sum _{i\in M}{\displaystyle \sum _{j\in O}{\displaystyle \sum _{l\in P}{x}_i\cdot y_i^l\left[T_{i,j,{\omega }_i}^{trans,l}+T_{i,j,{\omega }_i}^{ex,l}+\left(1-Z_{i,j}^l\right)T^{delay}\right]}}}$$

(18)

In Equation (18), $Z_{i,j}^l\in \left[0,1\right],i\in M,j\in O,l\in P$, and $z_{i,j}^l$ indicates whether the vehicle $D_i$ selects slice $S_l$ and uses the access point $A{P}_j$ to provide the amount of wireless resources. $z_{i,j}^l=0$ indicates that the pending resources in the vehicle $D_i$ have been successfully offloaded to the slice $S_l$; $z_{i,j}^l=1$ represents that the access point is busy and needs to wait in the queue.

The total time delay of the system can be calculated using Equation (19).

$$\mathrm{P}3: T=T_1+T_2$$

(19)

$$s.t.\left\{\begin{array}{l}C1:{\rho }_{i,j}^l\le Z_{i,j}^l\le {\omega }_i{\rho }_{i,j}^l, \forall i\in M\\ C2:y_i^l\le {\displaystyle \sum _{j\in o}{Z}_{i,j,{\omega }_i}^l}, \forall i\in M,l\in P\\ C3:x_i,y_i^l,z_{i,j}^l\in \left[0,1\right], \forall i\in M,j\in O,l\in P\end{array}\right.$$

Since the transmission delay is 0 and the execution delay is relatively small when the task is selected for local offloading, the execution delay can be ignored in this model. Similarly, the P3 problem is reduced to P4.

$$\mathrm{P}4: T={{\displaystyle \sum _{i\in M}{\displaystyle \sum _{j\in O}{\displaystyle \sum _{l\in P}{y}_i^l·T_{i,j,{\omega }_i}^{ex,l}}+\left(1-z_{i,j,{\omega }_i}^l\right)·T}}}^{delay}$$

(20)

$$s.t.\left\{\begin{array}{l}C1:{\displaystyle \sum _{l\in P}{y}_i^l}=1, \forall i\in M\\ C2:0\le z_{i,j,{\omega }_i}^l\le 1, \forall l\in P\\ C3:{\displaystyle \sum _{i\in M}{\displaystyle \sum _{j\in O}{\displaystyle \sum _{l\in P}{T}_{i,j,{\omega }_i}^{ex,l}}}}+\left(1-z_{i,l,{\omega }_i}^l\right)T^{delay}\le +\infty , \forall i\in M\end{array}\right.$$

In Equation (20), constraint ($C1$) guarantees that the slicing access point selected by the vehicle is free to receive the task ${\omega }_i$; constraint ($C2$) ensures that the task ${\omega }_i$ successfully selects the access point $A{P}_j$ for offloading the task; and constraint ($C3$) is to be sure that the task is processed before the deadline.

Therefore, the construction of the objective function $F$, the total cost of the task to offload the edge layer slices, and $F_{\min }$ can be formulated as:

$$F_{\min }={\beta }_{t}E+{\beta }_{e}E$$

(21)

In Equation (21), ${\beta }_t$ and ${\beta }_e$, respectively, denote the weighting factors of delay and energy consumption.

In a realistic in-vehicle task offload environment, the objective function $F$ must be non-static. Therefore, the in-vehicle task offloading problem can be regarded as a mixed shaping nonlinear problem (MINLP). By the above proof, it can be seen that P2 and P4 are NP-hard problems whose feasible solutions are nonconvex, the time complexity grows exponentially with the growing number of tasks, and it is clear that the traditional methods cannot make adaptive decisions based on the dynamic characteristics of vehicles in the Telematics environment. Consequently, this paper proposes an in-vehicle task offloading algorithm based on the dynamic selection slices is proposed in this paper.

4. The Dynamic Selection Slice Offload Algorithm for In-Vehicle Task

4.1. Resource Slicing Based on Markovian Dynamic Adjustment

The offloading problem for the in-vehicle tasks oriented in MEC can be summarized as a sequential decision problem, and the Markov decision process (MDP) precisely provides an excellent mathematical model for sequential decision-making. In this paper, the optimization of in-vehicle task offloading is converted into finding the optimal solution in a Markov decision process and the design idea is as follows: First, the RSU in the connected vehicle environment provides available status $S\left(t\right)$, and the agent performs placement decision actions $A\left(t\right)$. Then, the agent obtains the reward value $r\left(t\right)$ based on the current state $S\left(t\right)$ and action $A\left(t\right)$, updates the neural network parameters by Dueling DQN, and takes the next action $A\left(t+1\right)$. Finally, the DRL algorithm maps the state space to the action space by iteratively selecting actions that maximize the expected future reward, using the Q-learning algorithm $f:S\to A$, generating the slicing results to maximize the reward value, i.e., minimizing the expected cost. Based on the Markovian dynamic adjustment of the state space, action space, and reward, the specific explanation is as follows:

• State
  The state space is defined as the collection of all possible states that the agent can be in $S\left(t\right)=\left\{H\left(t\right),d_{{\omega }_i}\left(t\right),{\varphi }_{{\omega }_i}\left(t\right),\tau \left(t\right)\right\}$, and in this study, it includes the current status of network resources, such as wireless channel $H\left(t\right)$, input size $d_{{\omega }_i}\left(t\right)$, various required resources ${\varphi }_{{\omega }_i}$, and deadline time $\tau \left(t\right)$, available CPU as well as the current status of each slice and the vehicular location.
• Action
  The action space is the set of actions taken in the available states of the MEC-oriented vehicle networking when the agent processes the vehicular task ${\omega }_i$ in a given time slot $t$ corresponding to a certain state $S\left(t\right)$. The offloading decision action $A\left(t\right)$ was taken by the vehicle. To offload the vehicular task to the appropriate slice for execution, the action space is mapped to the set of slices within the edge server and denoted as $A_{{\omega }_i}\left(t\right)=\left\{1,\cdots ,0,0\right\}$. Among them, $A_{{\omega }_i}\left(t\right)=\left\{1,\cdots ,0,0\right\}$ represents offloading the in-vehicle task to the first slice for execution. Therefore, the action space for the in-vehicle task can be defined as: $A_{{\omega }_i}\left(t\right)=\left\{A_{{\omega }_1}\left(t\right),\cdots ,A_{{\omega }_{i-1}}\left(t\right),A_{{\omega }_i}\left(t\right)\right\}$.
• Reward
  The reward provides feedback to the agent about the action taken in a given state, and its positive or negative value reflects whether there is an advantage in MEC when available state information $S\left(t\right)$ makes offloading decision actions $A\left(t\right)$ or not. If the reward value is negative, it indicates the need for dynamic adjustment, which needs to interact with the current in-vehicle network in time to obtain an enormous reward value. Meanwhile, the reward value can be used as an indicator to evaluate the merit of the offloading strategy. Additionally, the reward value $r\left(t\right)$ can serve as an indicator for evaluating the quality of offloading strategies.

4.2. Dueling DQN-Based Dynamic Selection Slice Offload Algorithm

In this section, based on the Markov model of resource slicing obtained in Section 4.1, the Dueling DQN in deep reinforcement learning is used to dynamically select the slicing results and then update the optimal offloading strategy. Among them, Dueling DQN is a combination of Dueling Network and DQN algorithm, and the architecture is shown in Figure 3. Firstly, the input of Dueling DQN is the feedback of state information by using deep convolutional neural networks to extract information such as shoulder obstacle image features and visual field features around the vehicle. Secondly, the DQN algorithm decomposes the action value function into the state value V and the advantage value A associated with each action in the state by introducing a dyadic network with two fully connected layer branches. Finally, the final Q-network output is obtained by combining these values linearly.

In Dueling DQN, after the agent executes the action $a$, the state is changed from $s$ to $s^{\prime }$. The reward value $r$ is returned and stored in the experience pool. The sampling quaternions $\left(s,a^{*},r,s^{\prime }\right)$ from the experience pool for data training are based on the priority sampling method. Due to the dominant function ${{\rm E}}_{a~\pi \left(s\right)}\left[A^{\pi }\left(s,a\right)\right]=0$, when the selected action $a$ has zero advantage, the stability of the DSSO algorithm is improved by using decentralized operations. Instead of using the maximum value as in the DQN algorithm, the average value is used in the proposed algorithm, and the forward mapping is implemented using the last module of the network to obtain the true value $Q\left(s,a,\theta ,\alpha ,\beta \right)$ denoted as:

$$Q\left(s,a;\theta ,\alpha ,\beta \right)=V\left(s;\theta ,\alpha \right)+A\left(s,a;\theta ,\beta \right)-\frac{1}{\left|A\right|}{\displaystyle \sum _{a*\in A}A}\left(s,a^{\ast };\theta ,\beta \right)$$

(22)

In Equation (22), $\theta$ denotes the network parameters, $\alpha ,\beta$ represents the relevant parameters of the fully connected layer network, $a^{\ast }$ is the next action of $a$, and the corresponding state value of $a^{\ast }$ is $s^{\prime }$.

The DSSO algorithm predicts the total cost for in-vehicle task offloading, which can be expressed as:

$$y^{DuDQN}=r+\gamma \underset{a*}{\max }Q\left(s^{\prime },a^{*};{\theta }_i^{-},{\alpha }^{-},{\beta }^{-}\right)$$

(23)

The loss function $L_i^{DuDQN}$ is defined as the discrepancy between the predicted value $y^{DuDQN}$, obtained from the output of the neural network, and the true value $Q\left(s,a,\theta ,\alpha ,\beta \right)$, which can be shown as:

$$L_i^{DuDQN}={\mathrm{{\rm E}}}_{\left\{s,a,s^{\prime },r\right\}}\left[{\left(y^{DuDQN}-Q\left(s,a;\theta ,\alpha ,\beta \right)\right)}^2\right]$$

(24)

In Equation (23), ${\theta }_i{}^{-}$ denotes the update of the network parameters, and ${\alpha }^{-},{\beta }^{-}$ states the update of the parameters related to the fully connected layer network. The parameter $\theta$ are continuously copied to the Target network through the Evaluation network under the optimization step $C$. The neural network is then subjected to backpropagation using the loss function $L_i^{DuDQN}$ to update the parameters by comparing the magnitude of the reward value $r\left(t\right)=\phi /\left({\beta }_{t}T+{\beta }_{e}E\right),\phi >0$, when offloading a vehicle during a task to evaluate the effectiveness of offloading strategies.

4.3. Implementation of Offload Algorithm Based on the Dynamic Selection Slices

4.3.1. The Offload Algorithm Description

The DSSO algorithm pseudo code is shown in Algorithm 1.

Algorithm 1: The Dynamic Selection Slice Offloading Algorithm for In-Vehicle Task

Input:$\mathrm{wireless} \mathrm{channel} \mathrm{status} H\left(t\right)$$, \mathrm{input} \mathrm{size} d_{{\omega }_i}\left(t\right)$$, \mathrm{task} \mathrm{deadline} \tau \left(t\right)$    $\mathrm{In}-\mathrm{Vehicle} \mathrm{task} {\omega }_i$ and the maximum number of iterations $\xi$ Output: Task offload to slice strategy $\mathrm{\Psi }$ 1. Begin

2. $\mathrm{Initialize} \mathrm{the} \mathrm{DNN} \mathrm{parameters} {\theta }_1$ and set the training interval $\delta$ 3.  $\mathrm{for} t=1,2,3,\cdots ,\xi$ do 4.   $\mathrm{if} \mathrm{the} \mathrm{slice} \mathrm{access} \mathrm{point} \mathrm{is} \mathrm{idle} y_i^l=0$ then 5.     $\mathrm{Generate} \mathrm{initial} \mathrm{selection} \mathrm{slice} \mathrm{result} \left\{P_1\left(t\right),f_1\left(t\right),{\rho }_1\left(t\right),{\eta }_1\left(t\right)\right\}$ 6.    $\mathrm{if} t mod \delta =0$ then 7.     $\mathrm{Randomly} \mathrm{select} \mathrm{training} \mathrm{sets} \left\{H\left(t\right),d\left(t\right),\tau \left(t\right)\right\}$; 8.      Use Adam Optimizer to get the best training parameters $\theta$; 9.     end if 10.     Get the computational resource solution       $\mathrm{in} \mathrm{the} \mathrm{valid} \mathrm{interval} f_k\left(t\right)$$\mathrm{and} {\eta }_k\left(t\right)$; 11.     $\mathrm{obtain} \mathrm{the} \mathrm{radio} \mathrm{resource} \mathrm{allocation} \mathrm{solution} {\rho }_k\left(t\right)$       $\mathrm{and} \mathrm{transmit} \mathrm{power} \mathrm{results} P_k\left(t\right)$ by dichotomous method; 12.     $\mathrm{Obtain} \mathrm{selection} \mathrm{slice} \mathrm{result} \left\{P^{*}\left(t\right),f^{*}\left(t\right),{\rho }^{*}\left(t\right),{\eta }^{*}\left(t\right)\right\}$ 13.    $\mathrm{else} \mathrm{if} \mathrm{the} \mathrm{slice} \mathrm{access} \mathrm{point} \mathrm{is} \mathrm{busy} y_i^l=1$ then 14.     Wait and calculate the return to step 4 15.   $\mathrm{Calculating} \mathrm{the} \mathrm{reward} \mathrm{value} r\left(t\right)$ 16.  Update Slice Offloading Policy $\mathrm{\Psi }$ 17. End

4.3.2. Algorithm Complexity Analysis

Algorithm 1 summarizes that lines 4–14 are based on offloading the pending tasks to the network slices of the edge server, i.e., $x_i=1$, and the access point is free $y_i^l=0$, and get selection slice results $\left\{P^{*}\left(t\right),f^{*}\left(t\right),{\rho }^{*}\left(t\right),{\eta }^{*}\left(t\right)\right\}$. Line 13 indicates that if the access point is busy, the waiting delay $T^{delay}$ is recorded, and return to step 4. Judge whether this uninstallation solution is the best uninstallation strategy $\mathrm{\Psi }$ based on the reward value corresponding to the selection slice result.

In the DSSO algorithm, the time complexity is derived from the following components: (1) Time complexity of getting the selected slice section, which is $O\left(NM\cdot NP\cdot NO\right)$; (2) Time complexity of resulting resource allocation, which is $O\left(NM\cdot NP\right)+O\left(NM\right)$; (3) Time complexity of the resulting wireless resource allocation and transmit power setting, which is $O\left(NM\cdot NP\cdot NO\right)$. In summary, the overall time complexity is $O\left(NM\cdot NP\cdot NO\right)$.

5. Experimental Results and Analysis

5.1. Experimental Environment and Configuration

The performance of the proposed algorithm is evaluated by building an edge environment comprising an SDN controller, a slice orchestrator, and eight edge servers (ES). The end devices and servers were configured as listed in

Table 2. Terminal device and server configurations.

Device Type	Memory	CPU
ES 1~2	4 GB	Intel(R) Core(TM) i5-10210U CPU @ 2.11 GHz
ES 3~4	4 GB	Intel(R) Core(TM) i7-2450M CPU @ 2.60 GHz
ES 5~6	4 GB	Intel(R) Core(TM) i5-4210M CPU @ 2.50 GHz
ES 7~8	4 GB	Intel(R) Core(TM) i5-4590U CPU @3.30 GHz
Dell Inspiron 5490	8 GB	Intel(R) Core(TM) i5-10210U CPU @ 1.60 GHz
Lenovo Xiaoxin pro 14	8 GB	Intel(R) Core(TM) i5-12500H CPU @ 1.80 GHz
Lenovo ThinkPad	8 GB	Intel(R) Core(TM) i7-5600U CPU @ 1.80 GHz
HP 840G3	16 GB	Intel(R) Core(TM) i5-8350U CPU @ 1.80 GHz
Xiaomi 11	8 GB	Snapdragon 778G
Honor 70 pro	12 GB	Dimensity 8000
Galaxy Z Flip3	16 GB	Snapdragon 888
Samsung w23	16 GB	Snapdragon 8 + Gen1

. The coverage area is 200 × 200 m², and equipment is evenly distributed throughout the coverage area, as depicted in Figure 4. Among them, the SDN controller installation is based on the Open Daylight platform [35], Kubernetes control the ES, the end-to-end orchestrator [36] is installed as a slicing orchestrator, and the test dataset for the experiments uses the KITTI dataset [37].

The basic setup of the experiment is as follows: the vehicle detection task is assigned to two network slices with 28 vCPU cores and 20 MHz bandwidth, and the experiment adopts the method of controlling variables, which means that only one variable is changed in each experiment while keeping other variables the same. Each experiment is repeated ten times under the same conditions, and the average value is used as the final result to reduce the random effects of the experiment.

5.2. Experimental Parameter Setting

5.2.1. Reward Factor

The cumulative rewards generated by three different learning rates, $\delta =0.001$, $\delta =0.015$, and $\delta =0.01$ [38], to verify the feasibility analysis of the DSSO algorithm during the learning process with the number of iterations step = 1000 [22] are shown in Figure 5.

The reward variation caused by different learning rate $\delta$ is illustrated in Figure 5. The DSSO algorithm can be converged in $\delta =0.001$, $\delta =0.015$, and $\delta =0.01$. When $\delta =0.001$, the reward value converges most slowly among the three, which leads to low optimization efficiency; $\delta =0.015$ the cumulative reward converges more slowly and shows a dangerous trend; $\delta =0.01$ the reward value curve stability is optimal and connects to the local optimum stage when the number of iterations exceeds 648; therefore, this experiment selects $\delta =0.01$ as the learning rate of the DSSO algorithm.

5.2.2. Weight Factors

The effects of different weight factors on task time latency and energy consumption are shown in Figure 6. When the vehicle $D_i$ has sufficient computing power, it will pay more attention to shortening the task average latency when processing the task; when the vehicle $D_i$ has insufficient computing power, it will pay more attention to reducing energy consumption.

The relationship of energy consumption and time latency on different weight factors is demonstrated in Figure 6. When ${\beta }_t=0.9,{\beta }_e=0.1$, the task time latency of the tasks to be offloaded is slight, and the energy consumption is high. While ${\beta }_t=0.1,{\beta }_e=0.9$, the time latency of the task to be unloaded is max, and the energy consumption is minor. When ${\beta }_t=0.55,{\beta }_e=0.45$ the reward value is the highest, indicating that the offloading cost of the vehicular tasks under edge computing is the lowest. Therefore, ${\beta }_t=0.55,{\beta }_e=0.45$ is selected as the weight factor for the DSSO algorithm in this experiment.

5.2.3. Evaluation Metrics

To evaluate the feasibility of the proposed algorithm DSSO, energy consumption, average delay, and packet loss rate of task offloading are selected as performance metrics.

• Energy consumption: denotes the sum of the energy consumption of tasks on the vehicle terminals and the slices that offload the tasks to the edge server.
• Time latency: expresses the sum of task execution delay, transmission delay, and waiting delay.
• The ratio of dropped tasks: represents the drop ratio of in-vehicle task offloading, and this metric evaluates the completion efficiency of task offloading.

5.3. Experimental Results and Analysis

To verify the performance of the DSSO proposed in this paper, DSSO is compared with LOCAL, MINCO [39], and DJROM [40] for experiments.

• Impact of input data size on algorithm performance

To evaluate the impact of task input data size on algorithm performance, this group of experiments set the task deadline to 1 s and the task input data size to 16 MB~48 MB. The impact of task input data size on the energy consumption generated by the four algorithms is shown in Figure 7.

As can be seen from Figure 7, as the amount of in-vehicle task input data size increases, the energy consumption grows for all four algorithms; among them, the energy consumption value of MINCO is large, the energy consumption value of LOCAL algorithm is in the middle, and the energy consumption values of DJROM algorithm and DSSO algorithm are relatively low. This is because the MINCO algorithm reduces the task offloading delay by evaluating the service priority and data flow characteristics but does not restrict the energy consumption too much; the LOCAL algorithm only considers the available resources of the vehicle itself to process the task, and as the task input data volume increases, the available resources are not enough to support the task offloading, and the energy consumption value of LOCAL algorithm increases steeply; DJROM algorithm maximizes the number of computational bits by reducing the uplink power, which minimizes the energy consumption of task offloading; DSSO algorithm dynamically selects the optimal offloading strategy for the in-vehicle tasks by transforming the offloading problem of the in-vehicle task into a Markov decision process to solve the optimal slicing result problem, using the Dueling DQN method in DRL. The DSSO algorithm highlights the advantage of reducing the energy consumption value compared with the DJROM algorithm when the task input data volume is large.

The impact of the time latency of four algorithms with different task input data sizes is given in Figure 8. It can be observed from the chart that as the task input data size increases, the time latency of all algorithms shows an upward trend. The DJROM algorithm achieves higher rates by reducing the power of the uplink, resulting in the highest time latency for tasks. The LOCAL algorithm experiences a sharp increase in the time latency of processing tasks when the task input data size reaches a particular value. The MINCO and DSSO algorithms have relatively lower time latency values. However, the DSSO algorithm has a more substantial advantage over the MINCO algorithm when facing complex network environments with enormous task offloading sizes.

The point-fold line chart of four offloading algorithms on the ratio of dropped tasks is outlined in Figure 9. The result has shown that the packet loss rate of LOCAL remains consistent when the task is offloaded, and the DSSO algorithm has the lowest packet loss rate when offloading tasks compared to MINCO and DJROM, i.e., the highest task offloading completion rate. That is because DSSO uses the loss function and the experience playback mechanism in the neural network to obtain the optimal result of slicing after mapping the true values through the decentralized operation of Dueling Network and then updates the uninstallation strategy by comparing the magnitude of the reward value, and this significantly enhances the stability of the transmission of uninstallation data.

2.

Impact of task deadline on algorithm performance

To evaluate the effect of task deadline on algorithm performance, the average amount of task input data set for this group of experiments is 32 MB, and the task completion deadline varies from 0.6 s to 1.4 s. The energy consumption of different offloading algorithms with different task deadlines is outlined in Figure 10.

As can be seen from Figure 10, the energy consumption of the four algorithms decreases to varying degrees as the task deadline increases. Among them, the DSSO algorithm significantly reduces energy consumption as the task deadline increases. When the task deadline is 1.4 s compared with the LOCAL, MINCO, and DJROM algorithms, the energy consumption is reduced by 37.76%, 27.83%, and 12.15%, respectively. This is because the DSSO algorithm uses the Dueling Network in the Dueling DQN algorithm to train parameters, which only considers the advantage function that affects the offloading action, thereby reducing the energy consumption of task offloading.

The effect of task deadlines on the time latency of the four algorithms is illustrated in Figure 11. From this figure, it can be seen that the time latency of all four algorithms increases as the task deadline increases. Among them, DSSO offloads the task to the network slice in the edge server through the collaborative interaction between the in-vehicle terminal and the edge server, which dramatically reduces the time delay of the task by 24.32%, 36.67%, and 6.67% compared with LOCAL, DJROM, and MINCO when the task deadline is 1.2 s.

As can be seen from Figure 12, without considering energy consumption and time latency, the packet loss rate of all four algorithms exhibits a significant decreasing trend with the increasing task deadline. However, the DSSO algorithm shows the most obvious advantage because it dynamically adjusts the slicing through continuous updates of neural network parameters, thereby obtaining an optimal offloading strategy within the effective range.

6. Conclusions

In this study, the offloading problem of relationship-intensive and delay-sensitive vehicular tasks in MEC is investigated. The distributed offloading algorithm was designed to enable devices to make offloading decisions dispersedly, effectively solving the load imbalance problem at edge nodes. The Dynamic Selection Slicing-based Offloading Algorithm for in-vehicle Task in MEC was proposed. The offloading model of in-vehicle tasks is constructed by considering vehicle conditions, task dependencies, and available resources. The DRL approach was used to dynamically select the optimal task offloading strategy. The experimental results showed that the DSSO algorithm achieved the optimization goal when facing massive service requests and large data scales and performed better than similar algorithms in energy consumption, time latency, and task completion rate. However, due to the link disconnection caused by the high-speed movement of vehicles and the possible security hazards during data computation offloading. In the future, the security issues of offloading to in-vehicle task collaboration computing under the edge computing environment will be investigated.