Learning-Based Cost-Minimization Task Offloading and Resource Allocation for Multi-Tier Vehicular Computing

Abstract

With the fast development of the 5G technology and IoV, a vehicle has become a smart device with communication, computing, and storage capabilities. However, the limited on-board storage and computing resources often cause large latency for task processing and result in degradation of system QoS as well as user QoE. In the meantime, to build the environmentally harmonious transportation system and green city, the energy consumption of data processing has become a new concern in vehicles. Moreover, due to the fast movement of IoV, traditional GSI-based methods face the dilemma of information uncertainty and are no longer applicable. To address these challenges, we propose a T2VC model. To deal with information uncertainty and dynamic offloading due to the mobility of vehicles, we propose a MAB-based QEVA-UCB solution to minimize the system cost expressed as the sum of weighted latency and power consumption. QEVA-UCB takes into account several related factors such as the task property, task arrival queue, offloading decision as well as the vehicle mobility, and selects the optimal location for offloading tasks to minimize the system cost with latency energy awareness and conflict awareness. Extensive simulations verify that, compared with other benchmark methods, our approach can learn and make the task offloading decision faster and more accurately for both latency-sensitive and energy-sensitive vehicle users. Moreover, it has superior performance in terms of system cost and learning regret.

1. Introduction

With the fast development of 5G technology [1,2] and IoV [3], the vehicle has become a smart device with communication, computing, and storage capabilities. However, the limited on-board storage and computing resources often cause large latency for local task processing and result in degradation of system QoS as well as user QoE, which can hardly satisfy the latency-sensitive and computation-intensive demands of VUs for real-time application like online entertainment. To this end, VUs may offload tasks for MCC or MEC that provide service and cloud computing capabilities at the edge of the mobile network to relieve their own computational burden [4]. However, the edge server such as the BS needs to serve a huge number of mobile users simultaneously and cannot always provide timely response to VUs. Using RSUs to replace BSs is a feasible solution, but the implementation of massive arrangement of RSUs is costly. For example, installing RSU in only one US city costs about USD 18.63 billion, and the annual maintenance cost is approximately USD 1.1 billion [5,6]. Papers [7,8] propose to use PCRSUs, making it an alternative to RSUs. Moreover, driving vehicles around the VU with a similar trajectory may share idle computing and storage resources, forming the VCEC [9,10]. As a result, MTVC via BS, PCRSU, CDV, and VU local computing is emerging as a new paradigm for latency-sensitive and computation-intensive task processing in IoV. Notably, CDVs can provide idle computing resources by sharing redundant capabilities of vehicles with similar trajectories, while ESs ensure reliable queue length broadcasting through periodic feedback and channel verification to support accurate offloading decisions.

In the meantime, to build the environmentally harmonious transportation system and green city, the energy consumption of vehicle has become a great concern. Since task offloading may both save energy for VU and satisfy latency-sensitive as well as computation-intensive demands, how to strategically select the task processing location is a key issue. Note that the ES candidates for task offloading in IoV change over time due to the movement of both VU and VCEC edge servers. Thus, the GSI such as CSI and available server resource is uncertain for VU, resulting in traditional GSI-based offloading methods that are no longer applicable. Additionally, there is a pressing need for research into the problem of task conflict resolution in MTVC systems. Fortunately, a learning-based approach can be leveraged to give the solution [11].

To tackle the aforementioned challenges, in this paper we propose a T2VC model for IoV to minimize the system cost denoted by the sum weighted latency and power consumption. Due to the information uncertainty and dynamic offloading issues, a MAB-based QEVA-UCB solution for jointly optimizing system latency and energy consumption is proposed. QEVA-UCB takes into account several related factors, such as the task property, task arrival queue, offloading decision as well as the vehicle mobility, and selects the optimal location for offloading tasks to minimize the system cost with latency energy awareness and conflict awareness.

The contributions of this paper are as summarized as follows:

• We first propose the T2VC model for IoV, which systematically integrates three-tier edge servers (BS, CDV, PCRSU) and VU local computing. This comprehensive integration is rarely fully considered in existing MAB-based offloading frameworks. Arriving tasks are cached in a FIFO queue, and we formulate the problem of minimizing weighted latency and power consumption. The model targets the high-dynamic and CSI information uncertainty in IoV, where traditional GSI-based methods fail.
• We propose the QEVA-UCB solution based on MAB reinforcement learning. Unlike existing MAB-based offloading approaches, it integrates queue state awareness and energy volatility perception. It also comprehensively incorporates vehicle mobility and task offloading conflicts. We use Lyapunov optimization [12,13,14] to decouple long-term energy constraints and short-term decision making—a key consideration ignored by most existing MAB methods. Theoretical analysis verifies that QEVA-UCB has an upper bound for system cost.
• Extensive simulations are conducted to compare the proposed strategy with the naive UCB method, MV-UCB method, the ALTO method, and the optimal method. Results demonstrate that our approach can learn and make the task offloading decision faster and more accurately for both latency-sensitive and energy-sensitive VUs. Moreover, it has superior performance in terms of system cost and learning regret.

The rest of this paper is organized in such a manner. Section 2 presents related works. Section 3 introduces the system model of T2VC. Section 4 presents the task offloading problem and the constraints. QEVA-UCB algorithm and simulation results are given in Section 5 and Section 6. Section 7 concludes the paper.

2. Related Works

The literature on vehicular networks is vast. In paper [15], Liu et al. chose the single-layer MEC to offload tasks to edge nodes. Tang et al. of paper [16] considered offloading tasks to RSUs deployed on roadside, which could reduce task completion latency to a large extent. With the development of on-board storage and computing capability, vehicles can possess spare resources to help surrounding neighbors in a V2V manner. In paper [9], Hou et al. stated that vehicles travelling in the same direction could provide service to each other. Furthermore, in paper [17], Jiang et al. proposed a two-layer IoV model, considering that VUs might offload tasks to the BS or to the surrounding CDV. In paper [18], Tang et al. applied both RSU and travelling vehicles to act as edge servers and proposed a greedy heuristic based scheduling strategy to perform the tasks among vehicular fog computing. Additionally, in paper [19], Braga Reis et al. proposed the use of parked vehicles to gradually replace costly RSUs for passing VUs. However, none of the aforementioned works systematically consider all possible edge server candidates, i.e., BS, CDV, and PCRSU, which are included in our proposed T2VC model.

Many works have focused on task offloading and resource allocation in wireless communication system. In paper [20], Liu et al. respectively designed three algorithms based on heuristic search, reconstruction linearization and semi-definite relaxation to solve the problem of edge nodes selection, offloading sorting and task allocation for the URLLC MEC system. It realized the balance between latency and complexity. In paper [21], X. Hou et al. proposed the concept of EC-SDIoV to better utilize heterogeneous edge computing resources in variable IoV environments and to meet system latency constraints while ensuring reliability. In paper [22], Bute et al. proposed a distributed task offloading scheme that selected nearby vehicles with free computing resource to process tasks in a parallel manner. The proposed scheme could achieve good performance in terms of latency. In paper [23], Cai et al. studied the task offloading in fog-enabled internet of things networks, where Lyapunov optimization was leveraged to get the optimal results. In paper [24] Eltoweissy et al. designed two greedy heuristic methods to optimize the average response time while satisfying the long-term energy constraint. In paper [25], Tout et al. studied a cost-effective MEC-based method for offloading decision making to minimize processing/memory resource as well as energy. However, the above works mainly utilize traditional optimization methods to solve the task offloading problem while ignoring the high-dynamic characteristics of IoV. Due to the continuous changes in network status and information uncertainty, it is difficult to obtain an optimal solution.

The UCB reinforcement learning method is able to obtain fast convergence for offloading decision making under information uncertainty [26,27,28], which is especially suitable for IoV systems. In paper [29], Zhu et al. adopted the MAB to minimize computational latency for fog-enabled networks. And in paper [30], Sun et al. proposed a MAB-based task offloading strategy to optimize the vehicular cloud computing system latency. In paper [31], Gao et al. designed a learning-assisted green offloading method using the MAB approach to ensure that system energy consumption was within a bound. However, the aforementioned works ignore the long-term queue delay constraint, resulting in unrealistic practical execution. In paper [32], Qin et al. utilized vehicle association to reduce the overall energy consumption of IoV with long term constraint awareness. However, the objective is only to minimize the power consumption. Moreover, the offloading architecture did not take the multi-tier vehicular computing into consideration and was not comprehensive.

3. System Model

We consider a T2VC model for IoV in an urban scenario. As shown in Figure 1, there are three types of edge server, i.e., one BS server $SB$, multiple CDV servers $\mathbf{SF}$ and PCRSU servers $\mathbf{SP}$, denoted by the set $\mathbf{S}=\{SB,\mathbf{SF},\mathbf{SP}\}$. F CDV server vehicles within the communication range of VU V are denoted by $\mathbf{SF}=\{S{F}_1,\dots ,S{F}_f,\dots ,S{F}_F\}$. P PCRSU server cars are represented by $\mathbf{SP}=\{S{P}_1,\dots ,S{P}_p,\dots ,S{P}_P\}$. Each task of VU can be either processed locally or offloaded to the surrounding ESs (In this paper, the VU sends task request to neighbor ESs when its resource is unable to complete the task processing, in which case it lacks resource to serve other vehicles and will reject service requests from others.). Arriving tasks in the system are cached in a FIFO queue.

Figure 1. System model.

We use a time-slot model, where T discrete slots make up the overall time period, represented by $\mathbf{T}=\{1,2,\dots ,T\}$. In each slot, the VU determines whether a task is offloaded to at most one ES. Thus, task splitting offloading to more than one ES is out of the scope of this paper. Assume there are totally $PE$ periods with starting time $t_0$ as well as ending time $t_e$, during which the available ES candidates for task offloading in the T2VC system remain unchanged. Then, the duration of each period can be expressed as $(t_e-t_0)$.

There are four types of locations for task processing: (1) local execution, (2) offloading to a CDV server $S{F}_f$, (3) offloading to a PCRSU $S{P}_p$, (4) offloading to the nearby BS $SB$, which can be represented by $k\in \{0,1,2,3\}$:

$$\begin{array}{c}\hfill k=\left\{\begin{array}{c}0,\mathrm{local}\mathrm{processing}\\ 1,\mathrm{offloading}\mathrm{to}\mathrm{a}\mathrm{nearby}\mathrm{CDV}\\ 2,\mathrm{offloading}\mathrm{to}\mathrm{a}\mathrm{roadside}\mathrm{PCRSU}\\ 3,\mathrm{offloading}\mathrm{to}\mathrm{a}\mathrm{nearby}\mathrm{BS}\end{array}\right.\end{array}.$$

(1)

Thus, task offloading decision of the VU can be expressed as follows:

$$\begin{array}{c}\hfill L\left(t\right)=[k,a(t\left)\right],\end{array}$$

(2)

where k indicates the offloading type, and $a\left(t\right)$ denotes the specific location under this type. For example, $L\left(t\right)=[1,8]$ means that V offloads the task to the 8th CDV server in slot t. $k=0$ represents local processing and corresponding setting $a\left(t\right)=0$.

3.1. Latency Model

We consider three types of latency in T2VC, namely, task transmission latency, queuing latency and processing latency. Transmission latency is the time interval of task transmission from VU to ES, queuing latency is the task waiting time in the VU/ES queue, and processing latency is the task processing time at the server side. For local processing tasks, system latency consists of queuing time and local processing time, while, for offloading tasks, it includes all the three.

In the tth slot, the allocated CPU frequency to the task of VU is denoted by ${\mathbf{C}}^t=\{C_0^t,{\mathbf{C}}_s^t\}$, where the computational capacity of the VU is $C_0^t$, while the computational capacity of the other three ESs are expressed as $C_B^t$, ${\mathbf{CF}}^t=\{C{F}_1^t,\dots ,C{F}_f^t,\dots ,C{F}_F^t\}$ and ${\mathbf{CP}}^t=\{C{P}_1^t,\dots ,C{P}_p^t,\dots ,C{P}_P^t\}$, respectively. Assume each task has different input data size, and the ratio of output data to input data is the same [29]. Thus, let $y^t/x^t=\gamma$ (We set $\gamma$ as a fixed value to simplify the analysis of offloading latency and energy consumption. Heterogeneous vehicular applications can be adapted by adjusting $\gamma$, which does not affect the algorithm’s core logic). Define the computational latency as follows:

$$\begin{array}{c}\hfill {\tau }^c\left(t\right)=(1-\delta ){\displaystyle \frac{x^t{w}^t}{C_0^t}}+\delta {\displaystyle \frac{x^t{w}^t}{{\mathbf{C}}_s^t}},\end{array}$$

(3)

where $\delta$ is a binary variable; i.e., $\delta =0$ if the task is processed locally; otherwise, $\delta =1$. $w^t$ is the calculation intensity in CPU cycles per bit, indicating the CPU cycles required to calculate one bit of input data. $C_0^t$ and ${\mathbf{C}}_s^t$ are the computational resource available in slot t at the VU and ES side, respectively.

The channel gain can be calculated by $G=A_0{l}^{-2}$, where $A_0$ is the attenuation factor, and l is the distance between ES and VU [33]. Thus, the uplink task transmission latency is expressed as follows:

$$\begin{array}{c}\hfill {\tau }_L^u\left(t\right)={\displaystyle \frac{x^t}{B_L^u\left(t\right)log(1+\frac{P_L^u\left(t\right)G_L^u\left(t\right)}{{\sigma }^2})}},\end{array}$$

(4)

where $B_L^u\left(t\right)$ is the bandwidth for uplink transmission from VU to ES $L\left(t\right)$ in time slot t, $P_L^u\left(t\right)$ denotes the transmission power, $G_L^u\left(t\right)$ is the channel gain for uplink transmission from VU to ES $L\left(t\right)$ in time slot t and ${\sigma }^2$ represents the noise power. Since value of k in $L\left(T\right)$ is different, the transmission latency ${\tau }_L^u\left(t\right)$ might be CDV latency ${\tau }_{1,i}^u$, PCRSU latency ${\tau }_{2,i}^u$, or BS latency ${\tau }_{3,i}^u$.

Similarly, assume that the size of the downlink output data is $y^t$. The downlink transmission latency is expressed as follows:

$$\begin{array}{c}\hfill {\tau }_L^d\left(t\right)={\displaystyle \frac{y^t}{B_L^d\left(t\right)log(1+\frac{P_L^d\left(t\right)G_L^d\left(t\right)}{{\sigma }^2})}},\end{array}$$

(5)

where $B_L^d\left(t\right)$ is the bandwidth for downlink transmission from ES $L\left(t\right)$ to VU in time slot t, $P_L^d\left(t\right)$ is the transmission power, and $G_L^d\left(t\right)$ is the channel gain. The downlink transmission latency might also be divided into three categories.

Let $Q_L^t$ and $Q_0^t$, respectively, denote the queue length of the ES and VU in time slot t, which is broadcasted by ES at the beginning of each time slot [30]. Then the corresponding queuing latency can be expressed as follows:

$$\begin{array}{c}\hfill {\tau }^w\left(t\right)=(1-\delta )Q_0^w\left(t\right)TI+\delta Q_L^w\left(t\right)TI,\end{array}$$

(6)

where $TI$ is the time required to wait for 1 bit data in the processing queue.

Therefore, the total latency of a task for VU in time slot t can be expressed as follows:

$$\begin{array}{c}\hfill \tau \left(t\right)={\tau }^c\left(t\right)+{\tau }_L^u\left(t\right)+{\tau }_L^d\left(t\right)+{\tau }^w\left(t\right).\end{array}$$

(7)

In practice, VU does not know the edge server’s real-time remaining computing resource information. Moreover, since vehicles are moving over time, system CSI is also unavailable and changes rapidly.

3.2. Energy Consumption Model

If processing locally, only the computing energy consumption of the VU is considered. If task is offloaded, the system energy consumption comes mainly from the transmission power of VU and the computing power at the ES side.

For local processing, the energy consumption in time slot t can be expressed as follows:

$$\begin{array}{c}\hfill E_{loc}^t=(1-\delta ){\alpha }_0^t{w}^t{x}^t,\end{array}$$

(8)

where ${\alpha }_0^t$ represents the power consumption per CPU cycle.

For task offloading, the energy consumption can be expressed as follows:

$$\begin{array}{c}\hfill E_{off}^t=\delta ({\alpha }_s^t{w}^t{x}^t+{\beta }_s^t{x}^t),\end{array}$$

(9)

where ${\alpha }_s^t$ denotes the power consumption per CPU cycle of ES s, and ${\beta }_s^t$ denotes the energy for VU to transmit 1-bit data to ES s.

Thus, the total energy consumption in slot t can be expressed as follows:

$$\begin{array}{c}\hfill E^t=(1-\delta )E_{loc}^t+\delta E_{off}^t.\end{array}$$

(10)

Therefore, the system cost can be expressed as follows:

$$\begin{array}{c}\hfill U{L}^t=\omega {\tau }^t+E^t,\end{array}$$

(11)

where $\omega$ is the transmission latency weight.

4. Problem Formulation and Decomposition

In this paper, the objective function is to minimize the sum weighted latency and power consumption of the T2VC system, constrained by the long-term average energy budget, which can be expressed as follows:

$$\begin{array}{}\mathrm{(12a)}& \hfill & \mathbf{P}\mathbf{1}:\underset{{\{L,\delta \}}_{\forall t}}{\min }\sum _{t=1}^T{U}^t,\hfill \mathrm{(12b)}& \hfill & \mathrm{s.t.}\underset{T\to \infty }{lim}sup{\displaystyle \frac{1}{T}}\sum _{t=1}^T{E}_{k,a_j\left(t\right)}^t\le e_{k,a_j\left(t\right)},k\in \left\{0,3\right\},\hfill \mathrm{(12c)}& \hfill & {{\tau }^t\le TV}^t,\hfill \mathrm{(12d)}& \hfill & R_L^t\ge R_{min}^t,\hfill \mathrm{(12e)}& \hfill & \sum _{k=0}^3\delta \le 1,\forall s\in S,\forall t\in T,\hfill \mathrm{(12f)}& \hfill & \delta \in \left\{0,1\right\},\forall s\in \mathit{S},\forall t\in T,\hfill \mathrm{(12g)}& \hfill & k\left(t\right)\in \left\{0,3\right\},\forall t\in T,\hfill \end{array}$$

where (12b) denotes that the upper bound of the longer-term average system energy consumption can not exceed the maximum constraint. (12c) and (12d) are VU’s QoS requirements. (12e), (12f), and (12g) indicate that tasks can be processed locally or offloaded to at most one ES.

5. QEVA-UCB Solution for T2VC Task Offloading and Resource Optimization

5.1. Energy Budget Based on Virtual Queue Technology

For the offloading decision process, the long-term time-averaged energy consumption constraint in Equation (12b) is coupled with short-term offloading decision making, making P1 challenging to solve. Thus, in this subsection, a virtual queue $q_L^t$ is introduced to transform (12b) into the stability constraint of the virtual queue, which is applied to address the vehicle energy constraint.

The virtual queue $q_L^t$ is updated as follows:

$$q_L^{t+1}={\left[q_L^t-e_L\right]}^{+}+E_L^t,$$

(13)

where ${[·]}^{+}\stackrel{\Delta }{=}max\left\{·,0\right\}$.

According to the study of [31], if the stability of the queue $q_L^t$ is guaranteed, the energy consumption constraint can be satisfied. Then, Lyapunov optimization can be leveraged and Lyapunov function is expressed as the half of the sum of squares of $q^t$:

$$x\left({\mathit{q}}^t\right)\stackrel{\Delta }{=}{\displaystyle \frac{1}{2}}\sum _L{\left({\mathit{q}}^t\right)}^2,$$

(14)

where ${\mathit{q}}^t$ is the vector of all virtual queue backlog in time slot t.

Define the corresponding Lyapunov drift $\Delta x\left({\mathit{q}}^t\right)$ as follows:

$$\Delta x\left({\mathit{q}}^t\right)\stackrel{\Delta }{=}\mathbb{E}[x\left({\mathit{q}}^{t+1}\right)-x\left({\mathit{q}}^t\right)\mid {\mathit{q}}^t],$$

(15)

the size of $\Delta x\left({\mathit{q}}^t\right)$ indicates the variation in the queue between two nearby time slots.

Redefine the system cost function as follows:

$$U^t=U{L}^t+\mathbb{V}[(1-\mathbb{C})q_0^t{E}_{loc}^t+\mathbb{C}{q}_L^t{E}_{off}^t],$$

(16)

where $\mathbb{V}$ is an adjustable positive parameter.

On the basis of the study of [13,14], if $E_L^t\ge e_L$, the queue $q_L^t$ will become larger, and the cost $U^t$ of selecting it as ES becomes higher. Therefore, it will no longer be selected for offloading afterwards. Furthermore, if $q_0^t$ is smaller than other virtual queues, then VU prefers to process the task locally to reduce the energy consumption.

5.2. QEVA-UCB Algorithm

To tackle problem (16), we propose the QEVA-UCB algorithm for jointly optimizing system latency and energy consumption.

It can be seen from Equation (9) that the energy consumption of the system is determined by $x^t$ and the offloading location. Once the task-related parameter, for example, the size of input data, is determined, the task latency for different offloading decision can be calculated, then the system energy consumption is also determined, accordingly. Therefore, in this paper, we mainly focus on the learning to obtain the minimal system latency. The principle of the QEVA-UCB algorithm is shown below.

Firstly, VU connects with the newly joining ES for task offloading, and let ${\tau }_L^t=\overline{{\tau }_L^{t-1}}$ [30]. Then, the transmission latency is updated based on its local information as follows:

$$\begin{array}{c}\hfill {\tau }_L^t=\overline{{\tau }_L^{t-1}}-\sqrt{\epsilon \left(1-{\displaystyle \frac{\widetilde{d^t}}{\xi }}\right)\left(1-\widetilde{x^t}{w}^t\right)ln\left(t-t_a\right)\sum _{k=0}^3{\displaystyle \frac{{\left({\tau }_L^{t-1}\right)}^2}{n_L^{t-1}}}},\end{array}$$

(17)

where $\overline{{\tau }_L^{t-1}}$ is the average latency in previous $t-1$ time slots, and the second term is the one-sided confidence interval. The latency estimation in (17) deviates from standard UCB to adapt to IoV’s dynamic nature. Its design strictly follows UCB’s exploration–exploitation framework. $\epsilon$ is the exploration preference, which is a constant. $\widetilde{d^t}$ is the normalized distance between ES and VU (IoV task offloading latency is inherently dependent on transmission distance. Integrating normalized distance into the exploration term scales the exploration intensity based on transmission overhead, which is consistent with UCB’s goal of balancing performance stability and exploration efficiency). $\xi$ is the distance weight factor; it quantifies how ES-VU distance influences the algorithm’s candidate selection. Adjusting $\xi$ helps the algorithm prioritize ESs with more suitable distances. A smaller $\xi$ will increase the algorithm’s emphasis on distance, therefore choosing the ES with shorter distance. $\widetilde{x^t}$ is the normalized input data size. $w^t$ is the normalized computational intensity ($\widetilde{x^t}$ and $w^t$ follow UCB’s reward-scaling principle. They unify the units of task-related variables, avoiding exploration bias caused by large differences in data size or computation intensity between heterogeneous tasks. This design is derived from the task-aware optimization logic of UCB variants, which is essential for adapting UCB to edge computing scenarios with diverse tasks). $t_a$ indicates the time slot when the ES first appears, which guarantees the volatility-awareness of our solution. $n_L^{t-1}$ is the number of times the location L is selected for task processing before the time slot t.

Updating $\overline{{\tau }_L^t}$ and $n_L^t$ according to (18) and (19).

$$\begin{array}{c}\hfill \overline{{\tau }_L^t}={\displaystyle \frac{\overline{{\tau }_L^{t-1}}\times n_L^{t-1}+{\tau }_L^t}{n_L^{t-1}+1}},\end{array}$$

(18)

$$\begin{array}{c}\hfill n_L^t=n_L^{t-1}+1.\end{array}$$

(19)

VU always offloads tasks to location L with the minimum system cost, which is calculated by the following:

$$\begin{array}{c}\hfill L=argmin{U}^t.\end{array}$$

(20)

The above process is summarized in Algorithm 1. It learns and selects the location with the lowest system cost for task offloading.

Algorithm 1 consists of two phases: the learning phase and the decision-making phase. In the learning phase, the computational intensity and exploration preference are first initialized (Line 5). If a new ES enters the VU’s communication range, VU will offload its task to the new ES at least once to obtain the corresponding system latency (Line 6–9). In the offloading decision-making phase stage, VU updates the system latency of each time slot according to the previous learning results (Line 11) and calculates the energy consumption for each offloading position (Line 12). Then, VU calculates $U^t$ according to Equation (16) (line 13) and selects the optimal location with the lowest cost for task offloading (Line 14). Lastly, VU updates $\overline{{\tau }_L^t}$, $n_L^t$ according to Equations (18) and (19) (line 15).

Algorithm 1 QEVA-UCB Algorithm

1: Input: Initial computational intensity $w^0,\epsilon$ 2: Output: L, $\delta$ 3: for$t=1:T$ do 4:    Learning Phase 5:         Initialization $w^t,\epsilon$ 6:         if there is a new vehicle $S_{\mathrm{new}}$ to become an edge server then 7:          Connect VU and $S_{\mathrm{new}}$. 8:          Observe ${\tau }_{\mathrm{new}}^t$. 9:         end if 10:    Offloading Decision Making Phase 11:       Calculate latency ${\tau }_L^t$ according to Equation (17). 12:       Calculate energy consumption according to Equation (9). 13:       Calculate $U^t$ according to Equation (16). 14:       Offload the task to location L according to Equation (20). 15:       Update $\overline{{\tau }_L^t}$, $n_L^t$ according to Equations (18) and (19). 16: end for

Theorem 1.

For each vehicle and edge server, the average energy should be less than its maximum energy limit included in vector ${\mathit{e}}_L$ indicating that the QEVA-UCB algorithm satisfies the time average energy constraint (12b). Then, the following results can be obtained:

$$\underset{T\to \infty }{lim}sup\sum _{t=0}^{T-1}\sum _L\mathbb{E}\left[{\mathit{q}}^t\right]\le {\displaystyle \frac{\mathcal{E}+\mathbb{V}{U}_{max}^t}{\vartheta }},$$

(21)

where ${\mathit{q}}^t$ is a vector of all virtual queueing backlogs in time slot t. $\mathcal{E}\stackrel{\Delta }{=}{\displaystyle \frac{1}{2}}{\displaystyle \sum _L}\left(e_L^2+{E_L^t}^2\right)\le {\displaystyle \frac{1}{2}}{\displaystyle \sum _L}{e}_L^2+{\displaystyle \frac{1}{2}}(1-\delta ){\alpha }_{max}^t{w}^t{x}_{max}^t+{\displaystyle \frac{1}{2}}{L}^2\delta ({\alpha }_{max}^t{w}^t{x}_{max}^t+{\beta }_{max}^t{x}_{max}^t)$. $\vartheta$ is a positive constant satisfying $\vartheta >0$ such that $e_L-\vartheta$ is also in the feasible domain.

The proofs are given in Appendix A. For the proposed QEVA-UCB algorithm, the learning regret is defined as the expected cost gap between the learned strategy and the optimal strategy, calculated by the following:

$$R\left(T\right)=\sum _{t=0}^{T-1}(\mathbb{E}\left[{\left(U^t\right)}^{*}\right]-\mathbb{E}\left[U^t\right]).$$

(22)

Theorem 2.

Algorithm 1 has the upper bounded learning regret of

$$R\left(T\right)\le {\displaystyle \frac{\mathcal{E}}{\mathbb{V}}}+{\displaystyle \frac{1}{\mathbb{V}}}\sum _{t=0}^{T-1}{\sum }_{L\ne L^{*}}\left[{\displaystyle \frac{4\epsilon lnT}{\Delta {\tau }_L^t}}+(1+{\displaystyle \frac{{\pi }^2}{3}})\Delta {\tau }_L^t\right]$$

(23)

Proof of Theorem 2. 

Details are given in Appendix B. □

5.3. Complexity and Convergence

In this paper, the complexity of our proposed QEVA-UCB algorithm can be calculated as follows. In Line 6 the computational complexity of calculating the latency function of all ES candidates is $O\left(N\right)$, where the number of ES candidate in time slot t is N. The complexity of updating the offloading number $n_L^t$ is $O\left(1\right)$. The computational complexity of calculating the energy consumption of all ES candidates in Line 12 is $O\left(N\right)$, and the task offloading decision made in Line 14 is a minimum-seeking problem, with a complexity of $O\left(N\right)$. The complexity of updating the empirical bit offloading latency $U^t$ is $O\left(1\right)$. Therefore, the total computational complexity of running QEVA-UCB to offload one task is $O\left(N\right)$. Assume that there are totally M tasks required to be offloaded in the T2VC system. Since VU offloads the tasks independently, the total complexity is $O\left(MN\right)$ for each slot. From Theorem 2, we can see that there is an upper bound to the learning regret of the QEVA-UCB algorithm, and it will eventually converge to the optimal value through learning.

6. Simulation Results

For the simulation part, there are T = 2000 time slots, divided equally into four periods, each of which consists of 500 slots. The system consists of one VU, three CDVs, two PCRSUs and one BS. The distance weight $\xi$ = 1 [34]. All results are averaged over 10 independent runs to account for randomness from task arrival, channel conditions, and ES mobility. We consider two types of VU, i.e., latency-sensitive VU and energy-efficient VU, respectively. The former prioritizes task completion timeliness, so we set $\omega =100$ to make latency the dominant factor in system cost. The latter focuses on minimizing energy consumption, so $\omega =0.1$ is adopted to emphasize energy efficiency. The adjustable parameter $\mathbb{V}=1$, which is appropriate here as it balances the trade-off between system cost and long-term energy constraints under the specific simulation parameters including energy budget, task properties and ES characteristics. The exploration preference $\epsilon =2$ is chosen to balance exploration of new ES candidates and exploitation of known optimal locations, adapting to the dynamic IoV environment. During every time slot t, ${\alpha }_0^t$ is the power consumption per CPU cycle, which is sampled from distribution Unif $({10}^{-10},5\times {10}^{-10})$ J/cycle. ${\alpha }_s^t$ is the power consumption per CPU cycle of ES, which is sampled from distribution Unif $(5\times {10}^{-9},15\times {10}^{-9})$ J/cycle. ${\beta }_s^t$ is the energy for VU to transmit 1-bit data to ES, which is sampled from distribution Unif (${10}^{-7},{10}^{-6})$ J/bit. The energy budget is set as ${\mathit{e}}_L$ = 200 J [31]. Other parameters are shown in Table 1. Note that we adopt simulations instead of physical experiments to verify the proposed method, as simulations allow precise control of dynamic variables and fair algorithm comparisons, which are more practical for validating theoretical models in multi-tier vehicular computing. Physical experiments would involve high costs for deploying real-world ESs and VUs, and it is challenging to stably replicate the dynamic IoV environment.

Table 1. Primary simulation parameters.

Parameter	Symbols	Value
Bandwidth	B	10 MHz
Channel noise	${\sigma }^2$	${10}^{-13}$ W
Calculated intensity	$w^t$	${10}^3$ cycles/bit
Time slot	T	2000
Exploration preference	$\epsilon$	2
Transmitting power	P	$0.1$ W
Start time of the period	$t_e$	$0,501,1001,1501$
Data Sizes	$x^t$	$[0.5,1]$ Mbits
Maximum energy consumption	${\mathit{e}}_L$	200 J
VU Computing capability	$C_0^t$	$0.8$ GHz
CDV Computing capability	${CF}_f^t$	$[2,4]$ GHz
PCRSU Computing capability	${CP}_p^t$	$[0.5,1.8]$ GHz
Distance between CDV and VU	${DF}_f^t$	$[10,300]$ m
Distance between PCRSU and VU	${DP}_p^t$	$[300,500]$ m
Distance weights	$\xi$	1

The QEVA-UCB approach is compared to the following six methods.

1.

UCB method [35]: Traditional/naive upper confidence bound method.

2.

R-UCB method: Range-awareness is added to the traditional UCB algorithm.

3.

MV-UCB method [34]: ES occurrence-awareness is added to the traditional UCB algorithm.

4.

ALTO method [30]: Task load-awareness and ES occurrence-awareness are enhanced to the traditional UCB algorithm.

5.

TR-UCB method: Task load-awareness and range-awareness are added to the traditional UCB algorithm.

6.

Optimal method: For each time slot, VU selects the ES location with the lowest system cost for task processing. The optimal method is an oracle baseline with perfect real-time CSI and full ES state information. It makes offloading decisions without CSI uncertainty, providing a theoretical upper bound to evaluate the performance gap of practical algorithms.

To ensure fair comparison, all benchmark methods (naive UCB, MV-UCB, ALTO, and optimal) are implemented under identical system assumptions. Specifically, they share the same T2VC model structure, task arrival rules, ES computational capabilities and deployment ranges, energy budget limits, and communication channel parameters. No additional advantages or constraints are imposed on any method, guaranteeing the validity of performance comparisons.

6.1. Volatility of ES Candidates

In this subsection, in order to observe the impact of system volatility caused by vehicle mobility, three variables are fixed, i.e., input data $x^{t}0.6$ Mbits, output data $y^t=0.2$ Mbits, $w^t={10}^3$ CPU cycles/bit. New ES arrives and old ES leaves the communication range during each time period. The availability of ES candidate is shown in Table 2, where ✓ means ES is available, and × means ES is not available. Since ES processes tasks one by one, all computing resources are used, and the rest tasks are cached in the FIFO queue at each node.

Table 2. Volatility of ES Candidates.

ES	VU	CDV1	CDV2	CDV3	PCRSU1	PCRSU2	BS
Computing resources (GHz)	0.8	2	1.5	1.8	0.9	3	∞
Time periods 1	✓	✓	×	×	✓	×	✓
Time periods 2	✓	×	✓	×	×	✓	✓
Time periods 3	✓	×	✓	✓	×	✓	✓
Time periods 4	✓	✓	✓	✓	✓	×	✓

Figure 2 shows the latency-sensitive VU performance versus ES candidate volatility. It can be seen from Figure 2a that, for our learning-based method, due to the beginning of exploration, the system cost starts from more than 9. However, with the increasing of time, it finally converges to near optimal result of 2. Note that, there are some fluctuations when $t=501,1001,1501$. This is because ES candidate varies in these three time slots, and VU needs to restart to explore the better offloading decision. The alternating rise and fall of fluctuations stem from the differences in adding or removing ES candidates, the cost rises when two ES candidates was removed in slot 501, and it decreases when ES candidates was added in slot 1001 and 1501. A similar situation also occurred in Figure 3a. It can also be seen that our proposed solution converges to the optimal value faster than other benchmark methods, due to the fact that it considers both the ES occurrence and the real-time distance between VU and each offloading candidate. Figure 2b shows that our proposed approach has smaller regret value compared with other methods due to the same reason. Figure 2c shows the ratio of the optimal ES selection, which is denoted by the ratio of the number of optimal decision to the total number of choice up to the tth time slot. It can be seen that, compared with other methods, our QEVA-UCB algorithm can make more accurate offloading decisions of a more than 95% optimal ES selection ratio within each ES fluctuation time.

Figure 2. Latency-sensitive VU performance versus ES candidate volatility: (a) System cost. (b) Learning regret. (c) Ratio of optimal selection.

Figure 3. Energy-efficient VU performance versus ES candidate volatility: (a) System cost. (b) Learning regret. (c) Ratio of optimal selection.

Figure 3 shows the energy-efficient VU performance versus ES candidate volatility. It can be seen from Figure 3a that, for our learning-based method, due to the beginning of exploration, the system cost starts from more than 20. However, with the increasing time, it finally converges to a near optimal result of 5. Note that there are some fluctuations when $t=501,1001,1501$. This is due to the fact that the ES candidate varies in these three time slots, and VU needs to restart to explore the better offloading decision. It also verifies that our approach converges faster than other benchmark methods, due to the fact that it considers the data size, ES fluctuation, and the real-time distance between VU and each offloading candidate. Compared to our solution, the UCB algorithm converges slowly in the first two periods and is unable to learn the optimal offloading strategy because it cannot identify the fluctuation characteristics of ES. Figure 3b shows that our proposed approach has smaller regret value compared with other methods. Figure 3c shows the ratio of the optimal ES selection, and it can be observed that the QEVA-UCB algorithm is able to make more accurate decisions than other learning algorithms, which verifies that QEVA-UCB can effectively reduce the system cost of energy-efficient VU in case of ES volatility.

6.2. Data Size Influence

In this subsection, we assume that there is no ES volatility in each time slot and evaluate the learning performance under different data size. Some parameters are as follows: input data $x^t\in [0.5,1]$ Mbits, output data $y^t\in [0.2,0.4]$ Mbits, computing intensity $w^t\in [{10}^3,2\times {10}^3]$ CPU cycles/bit. Due to random arriving data size, the data volume processed in each time slot changes dynamically.

Figure 4 shows the latency-sensitive VU performance with random data size. It can be observed from Figure 4a that the system cost of the QEVA-UCB algorithm finally converges to about 3, which is near-optimal. Figure 4b shows that the our approach has the smallest learning regret. With the increase in time, the learning regret of QEVA-UCB gradually stabilizes and obtains superior behavior. Figure 4c shows the ratio of optimal selection. It can be seen that QEVA-UCB always has the best performance due to the fact that it can adjust the offloading decision according to the real-time data size and queue backlog. For example, the optimal ES selection ratio of QEVA-UCB is nearly two times higher than UCB.

Figure 4. Latency-sensitive VU performance with random data size: (a) System cost. (b) Learning regret. (c) Ratio of optimal selection.

The energy-efficient VU performance with random data size is shown in Figure 5. It can be observed from Figure 5a that, compared with other methods, QEVA-UCB performance is near-optimal. From Figure 5b it can be seen that QEVA-UCB has the smallest learning regret. For example, it can reduce the regret value by 30% compared to the ALTO algorithm. Figure 5c shows that the learning effect for energy-efficient VU is not as good as that of the latency-sensitive VU. However, the selection accuracy of our approach is still higher than other algorithms. Compared with the UCB algorithm, the optimal ES selection ratio is increased by 50%.

Figure 5. Energy-efficient VU performance with random data size: (a) System cost. (b) Learning regret. (c) Ratio of optimal selection.

Figure 6 takes the latency-sensitive VU for instance and gives the system performance versus the change in data size. It shows that the system cost gradually grows as data size increases. However, since the QEVA-UCB algorithm is data size-aware, the system cost is the closest to the optimal method. In particular, when the data size is 1 Mbits, the system cost of our approach can be reduced by 20%, 125% and 300% compared to the ALTO, R-UCB and naive UCB algorithm, respectively.

Figure 6. System cost versus the change in data size (taking latency-sensitive VU for instance).

6.3. Overall Performance of QEVA-UCB

As can be seen from Figure 7, the QEVA-UCB algorithm is able to learn the optimal offloading policy faster than others when both ES candidate and task data size are volatile. This is because QEVA-UCB takes into account the task property, task arrival queue, offloading decision as well as vehicle mobility comprehensively. Therefore, in the T2VC system, it can still maintain the minimum system cost despite the fluctuation of task size and ES candidate. Specifically, in the second period, compared to the MV-UCB algorithm, the QEVA-UCB algorithm reduces the system cost by about 20%.

Figure 7. Overall performance of QEVA-UCB (taking latency-sensitive VU for instance).

Figure 8 shows the impact of $\mathbb{V}$. The system power consumption decreases, and the system latency increases as $\mathbb{V}$ grows up. The reason lies in that, as $\mathbb{V}$ increases, the algorithm focuses more on the energy consumption rather than the latency. In this paper, $\mathbb{V}=1$ is chosen for simulation in order to achieve a balance between system cost and energy constraint.

Figure 8. Impact of $\mathbb{V}$.

7. Conclusions

In this paper, the T2VC model is proposed for the IoV system, and the QEVA-UCB algorithm is designed to cope with the high-dynamic and CSI information uncertainty issue. To avoid task arrival conflicts, we further introduce the task queuing policy. Under the constraints of energy conditions, the system cost denoted by the sum weighted latency and power consumption is minimized. Theoretical analysis and simulations confirm that QEVA-UCB, by integrating queue state awareness, energy volatility perception, and vehicle mobility adaptation, can make faster and more accurate offloading decisions for both latency-sensitive and energy-efficient VUs. It outperforms existing MAB-based approaches in system cost and learning regret, providing a practical solution for multi-tier vehicular computing task offloading. A key limitation of this work is its focus on a single VU scenario without considering multi-VU resource competition. Additionally, extreme network conditions like sudden communication interruptions are not fully addressed. In future work, we will extend the model to multi-VU scenarios and integrate anti-interference mechanisms to enhance practicality.