AptEVS: Adaptive Edge-and-Vehicle Scheduling for Hierarchical Federated Learning over Vehicular Networks

Abstract

Hierarchical federated learning (HFL) has emerged as a promising paradigm for distributed machine learning over vehicular networks. Despite recent advances in vehicle selection and resource allocation, most still adopt a fixed Edge-and-Vehicle Scheduling (EVS) configuration that keeps the number of participating edge nodes and vehicles per node constant across training rounds. However, given the diverse training tasks and dynamic vehicular environments, our experiments confirm that such static configurations struggle to efficiently meet the task-specific requirements across model accuracy, time delay, and energy consumption. To address this, we first formulate a unified, long-term training cost metric that balances these conflicting objectives. We then propose AptEVS, an adaptive scheduling framework based on deep reinforcement learning (DRL), designed to minimize this cost. The core of AptEVS is its phase-aware design, which adapts the scheduling strategy by first identifying the current training phase and then switching to specialized strategies accordingly. Extensive simulations demonstrate that AptEVS learns an effective scheduling policy online from scratch, consistently outperforming baselines and and reducing the long-term training cost by up to 66.0%. Our findings demonstrate that phase-aware DRL is both feasible and highly effective for resource scheduling over complex vehicular networks.

1. Introduction

Vehicular networks are evolving into the core infrastructure for a data-driven ecosystem within intelligent transportation systems (ITS), built upon a unified communication plane connecting vehicles, distributed edge nodes (ENs), and a central cloud server (CS) [1,2]. This data-rich ecosystem lays the foundation for a new frontier of large-scale machine learning (ML) applications, such as real-time road hazard detection and traffic object recognition [3,4]. However, the training process for these powerful applications fundamentally conflicts with the traditional centralized paradigm due to insurmountable communication and privacy barriers. Federated learning (FL) has emerged as the key distributed paradigm to overcome these obstacles by enabling collaborative training directly on vehicles without exposing raw data [5].

Nonetheless, organizing FL in large-scale deployment scenarios, such as within an entire city, remains a daunting challenge that imposes significant network coordination and communication overhead. This inherent complexity drives academia and industry to pragmatically focus on optimizing services within well-defined and representative hotspots or critical road segments, as illustrated in Figure 1. It is within these dense and dynamic road segments that the Hierarchical FL (HFL) framework becomes particularly attractive, utilizing an intermediate edge layer to create a scalable and efficient training architecture tailored to vehicular networks [6].

Despite its architectural advantages, deploying HFL in such vehicular networks, referred to as VHFL, unveils a dual challenge stemming from task diversity and vehicle mobility. First, as vehicular networks evolve into multi-service platforms, they are expected to support a broad spectrum of ML training tasks, such as autonomous driving perception, cooperative hazard detection, and infotainment recommendation. These tasks vary not only in computational complexity and communication overhead [7], but more importantly, each entails a multi-objective optimization problem that involves inherent trade-offs among model accuracy, time delay, and energy consumption [8]. To address this, the system necessitates establishing a principled and configurable metric that consolidates these conflicting objectives into a quantifiable training cost, and accordingly design task-specific scheduling strategies to satisfy performance requirements.

Second, the challenge of adapting to task diversity is further compounded by the inherent environmental dynamics of vehicular networks. In real-world deployments, vehicles continuously arrive at and depart from road segments covered by densely deployed ENs [9], giving rise to a multi-layered dynamic execution environment for VHFL. These dynamics include the quasi-static diversity of network topology across ENs, the heterogeneity of locally available data on vehicles, and the stochastic nature of vehicle mobility. Such fluctuations not only undermine communication stability but also reshape the statistical properties of training data at each EN, ultimately affecting the model convergence behavior [10].

The intricate interplay between task diversity and tightly coupled environmental dynamics underscores the need for jointly optimizing the number of participating ENs and the number of scheduled vehicles per EN, namely, the configuration of Edge-and-Vehicle Scheduling (EVS). However, this edge–vehicle collaborative scheduling optimization remains largely underexplored. Most existing works focus on optimizing client selection or resource allocation under a fixed EVS configuration, without adapting optimize the EVS configuration. Although fixed scheduling policies, exemplified by the static 30% client selection, have been effective in less dynamic or quasi-static Internet of Things (IoT) [11], we contend that the effectiveness of such static approaches is significantly limited by the profound heterogeneity and high mobility of vehicular networks.

To address these challenges, this paper proposes AptEVS, a phase-aware adaptive scheduling framework based on deep reinforcement learning (DRL) that minimizes the long-term training cost in VHFL systems. The main contributions of this work are summarized as follows:

(1)

Comprehensive Cost Metric: We formulate a holistic training cost metric for VHFL systems, integrating accuracy deviation cost (ADC), time delay cost (TDC), and energy consumption cost (ECC) to quantitatively evaluate performance against task-specific requirements.

(2)

Dynamic Configuration Analysis: Through systematic experiments, we analyze optimal EVS configurations. Our findings reveal that the optimal configuration is a dynamic variable, exhibiting high sensitivity not only to intrinsic task attributes and environmental dynamics, but crucially, to the evolution of the training phase. This establishes the necessity for a phase-aware, learning-based approach.

(3)

Phase-aware DRL Framework (AptEVS): We propose AptEVS, a novel phase-aware DRL framework for online EVS adaptation. Its core employs a lightweight mechanism to detect the current training phase and dynamically switches between two specialized algorithms: Structured Exploration for the Initial Training Phase (ITP) and Priority-Enhanced DQN Training for the Medium Training Phase (MTP).

(4)

Experimental Validation: Extensive simulations demonstrate that AptEVS consistently outperforms baseline methods, achieving significant reductions in long-term training cost. These results prove the feasibility and effectiveness of online, phase-aware scheduling in complex vehicular environments.

The remainder of the paper is structured as follows. Section 2 describes the related work. Section 3 presents the system model of VHFL along with the problem formulation. Section 4 discusses the motivation. Section 5 describes the detail of AptEVS. Then, Section 6 provides the performance evaluation. Finally, conclusions are drawn in Section 7.

2. Related Work

Resource management is a critical factor for ensuring the efficiency of FL in vehicular networks. Prior research has explored this problem from various perspectives, with most studies focusing on client selection and resource allocation in conventional single-layer FL architectures.

Early works primarily aimed to optimize a single performance metric. For instance, a variety of approaches have been proposed to improve model accuracy through vehicle selection [12,13,14], or to minimize training time delay by adjusting aggregation intervals and deadlines [15,16]. Recognizing the limitations of single-objective optimization, recent studies have shifted toward multi-objective formulations, introducing composite metrics to jointly balance time delay, energy consumption, and other system factors [17,18,19,20,21]. While these methods provide valuable insights into managing performance trade-offs, they often lack sufficient configurability to address the diverse and task-specific requirements found in vehicular scenarios.

The emergence of HFL and the challenges posed by vehicular mobility have introduced a more complex optimization landscape. Many foundational HFL studies have been conducted under relatively stable network settings, such as smartphones or stationary IoT sensors [6,11,22,23,24,25,26,27,28]. Only a few pioneering works have begun to investigate the unique challenges introduced by high-speed and persistent mobility in vehicular networks. These efforts can be broadly categorized into two directions: (i) enhancing model performance by mitigating data distribution shift [29,30,31], and (ii) improving system efficiency by analyzing the impact of mobility on time delay and convergence [10,32].

Notably, previous work [11] demonstrated that scheduling 30% of clients can significantly reduce training overhead while maintaining accuracy. Meanwhile, the work [33] proposed that scheduling fewer clients in the initial training phase and increasing participation in the final training phase can effectively save resources. However, most existing works concentrate on client-level resource optimization, overlooking a fundamental challenge in hierarchical settings: the joint optimization of the number of active ENs and the number of scheduled vehicles per EN, referred to as the EVS configuration. While prior studies have acknowledged the importance of adapting resource allocation to the training process in FL [34,35], scheduling strategies that simultaneously consider task characteristics, environmental dynamics, and the temporal evolution of training phases remain largely unexplored, particularly under multi-objective EVS optimization in dynamic vehicular networks.

3. System Model

Similar to existing works [6,20,36], we consider a VHFL system implemented within a representative road segment, as shown in Figure 2. The system consists of a cloud server (CS), a set of ENs $\mathcal{M}=\{1,2,\dots ,M^{\max }\}$, and a set of vehicles $\mathcal{N}=\{1,2,\dots ,N^{\max }\}$. The CS orchestrates the three-layer HFL process until the cloud model converges and achieves target accuracy. Following hierarchical federated averaging [6], the CS aggregates edge models every ${\kappa }_1$ rounds, while each EN aggregates local models from its connected vehicles every ${\kappa }_2$ local epochs. Each vehicle $i\in \mathcal{N}$ trains its local model using a private dataset ${\mathcal{D}}_i={(x_{i,j}\in {\mathbb{R}}^s,y_{i,j}\in \mathbb{R})}_{j=1}^{D_i}$. The key notations in this paper are summarized in

Table 1. Summary of key notations.

Symbol	Definition	Symbol	Definition
$N_m^{\max }$	maximum of vehicles in EN m	$\mathcal{N}$	set of vehicles
$M^{\max }$	maximum of ENs	$\mathcal{M}$	set of ENs managed by the CS
m	index of an EN	i	index of a vehicle
$D_i$	local data size of vehicle i	${\mathcal{D}}_i$	local dataset of vehicle i
${\kappa }_1$	cloud interval	${\kappa }_2$	edge interval
$(r,\tau )$	time slice of edge round $\tau$ in cloud round r	R	total number of cloud rounds
$\mathcal{R}$	set of cloud rounds	${\mathcal{M}}_r$	set of ENs scheduled by the CS
$M_r$	edge scheduling number	$N_r$	vehicle scheduling number per EN
${\mathcal{N}}_m^c(r,\tau )$	vehicles connected to EN m at time slice $(r,\tau )$	${\mathcal{N}}_m^s(r,\tau )$	vehicles scheduled by EN m at time slice $(r,\tau )$
w	training model of vehicles	$w^r$	cloud model in the r-th cloud round
$w_m^{r,\tau }$	edge model at time slice $(r,\tau )$	$w_i^{r,\tau }\left(j\right)$	local model of vehicle i after j-th local iteration
L	total number of local iterations	b	batch size
$\eta$	learning rate	$d_{\mathrm{edge}}$	edge spacing
$\lambda$	vehicle arrival rate	$\rho \left(v_i\right)$	probability density function of $v_i$
$v_i$	speed of vehicle i	$\overline{v}$	mean speed of vehicles
${\sigma }_v$	standard deviation of vehicle speeds	$A_r$	model accuracy after the r-th cloud round
$A_{\mathrm{target}}$	target accuracy	$f_i(r,\tau )$	actual computing frequency of vehicle i
$C_i$	computational cycles per sample for vehicle i	$c_i$	effective capacitance factor of the computing chipset
$P_i(r,\tau )$	actual transmit power of vehicle i	Z	model size
$b_i(r,\tau )$	number of RBs allocated to vehicle i at $(r,\tau )$	B	bandwidth of one RB

.

3.1. Training Procedure of VHFL

To formalize the training procedure, we define a time slice $(r,\tau )$ as the $\tau$-th edge round within the r-th cloud round. In particular, the proposed VHFL framework incorporates dynamic state monitoring at both the edge and vehicle layers. To ensure privacy preservation, vehicles upload only necessary non-sensitive state information through lightweight encryption [37] or differential privacy techniques [38]. Notably, existing incentive mechanisms [39] can ensure that vehicles share state updates with low overhead and preserved privacy. This supports the feasibility of our approach and enables the CS to obtain key information for the HFL process. Prior to each time slice, the CS performs EVS based on the current system state. The VHFL design explored additionally considers state information collection at both the edge and vehicle layers, allowing the CS to better adapt its scheduling decisions throughout the training process. The training process of VHFL in the r-th cloud round is described as follows.

(1) Information Collection and Adaptive Scheduling: Before each cloud round, the CS first collects non-sensitive state information from the edge and vehicle layers, as detailed in Section 5. Based on this state, the CS dynamically determines the scheduling configuration $(M_r,N_r)$, where $M_r$ is the number of scheduled ENs and $N_r$ is the number of scheduled vehicles per EN. The specific scheduling method is described in Section 5. Let ${\mathcal{M}}_r=\{1,2,\dots ,M_r\}$ denote the set of scheduled ENs in the r-th cloud round. In the $\tau$-th edge round, each scheduled EN $m\in {\mathcal{M}}_r$ randomly selects a subset of vehicles ${\mathcal{N}}_m^s(r,\tau )$ from its connected set ${\mathcal{N}}_m^{\mathrm{c}}(r,\tau )$ for local training.

(2) Hierarchical Distribution: The CS distributes the cloud model parameters $w^r$, obtained from the previous cloud round, to all scheduled ENs. Let $w^0$ denote the initialized cloud model parameters. During the time slot $(r,\tau )$, the scheduled EN m broadcasts its edge model $w_m^{r,\tau }$ to the available vehicles within its coverage area, where $w_m^{r,0}=w^r$.

(3) Local Training and Model Uploading: During time slice $(r,\tau )$, only the scheduled vehicles participate in local training. Specifically, each scheduled vehicle $i\in {\mathcal{N}}_m^s(r,\tau )$ initializes its local model with the corresponding edge model $w_i^{r,\tau }\left(0\right)=w_m^{r,\tau }$, and then updates it at the l-th local iteration via mini-batch stochastic gradient descent (SGD):

$$\begin{array}{cc}\hfill & w_i^{r,\tau }(l+1)=w_i^{r,\tau }\left(l\right)-\eta \nabla f_i\left(w_i^{r,\tau }\left(l\right),{\mathcal{D}}_{i,l}\right)\hfill \end{array}$$

(1)

where $\eta$ is the learning rate, and $\nabla f_i(w_i^{r,\tau }\left(j\right),{\mathcal{D}}_{i,l})$ denotes the stochastic gradient of the loss function computed on mini-batch ${\mathcal{D}}_{i,l}$ sampled from the local dataset ${\mathcal{D}}_i$. The local loss is estimated during training as

$$\begin{array}{cc}\hfill & F_i^{\mathrm{local}}(r,\tau )={\displaystyle \frac{1}{bL}}\sum _{l=1}^L{f}_i\left(w_i^{r,\tau }\left(l\right),{\mathcal{D}}_{i,l}\right),\hfill \end{array}$$

(2)

where $L={\kappa }_2{\displaystyle \frac{D_i}{b}}$ is the number of local iterations, b is the batch size, and edge interval ${\kappa }_2$ is the number of local epochs. Once training is complete, vehicle i uploads its final local model $w_i^{r,\tau }=w_i^{r,\tau }\left(L\right)$ and corresponding local loss $F_i^{\mathrm{local}}\left(w_i^{r,\tau }\right)$ to its associated EN. After ${\kappa }_1$ edge rounds, the scheduled EN m uploads its edge model $w_m^r=w_m^{r,{\kappa }_1}$ and edge loss to the CS.

(4) Hierarchical Aggregation: The scheduled ENs exchange reception confirmations of local models in the form of non-sensitive data. Once receiving all available local models, each scheduled EN computes the edge loss and updates its edge model by performing edge aggregation as follows:

$$\begin{array}{cc}\hfill & F_m^{\mathrm{edge}}(r,\tau )={\displaystyle \frac{{\sum }_{i\in {\mathcal{N}}_m^s(r,\tau )}{D}_i{F}_i^{\mathrm{local}}(r,\tau )}{{\sum }_{i\in {\mathcal{N}}_m^s(r,\tau )}{D}_i}},\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill & w_m^{r,\tau +1}={\displaystyle \frac{{\sum }_{i\in {\mathcal{N}}_m^s(r,\tau )}{D}_i{w}_i^{r,\tau }}{{\sum }_{i\in {\mathcal{N}}_m^s(r,\tau )}{D}_i}}.\hfill \end{array}$$

(4)

Upon receiving all edge model parameters after ${\kappa }_1$ edge rounds, the CS computes the cloud loss and updates the cloud model by performing cloud aggregation as follows:

$$\begin{array}{cc}\hfill & F^{\mathrm{cloud}}\left(r\right)={\displaystyle \frac{{\sum }_{m=1}^{M_r}{\sum }_{\tau =1}^{{\kappa }_1}{\sum }_{i\in {\mathcal{N}}_m^s(r,\tau )}{D}_i{F}_m^{\mathrm{edge}}(r,\tau )}{{\sum }_{m=1}^{M_r}{\sum }_{\tau =1}^{{\kappa }_1}{\sum }_{i\in {\mathcal{N}}_m^s(r,\tau )}{D}_i}},\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill & w^{r+1}={\displaystyle \frac{{\sum }_{m=1}^{M_r}{\sum }_{\tau =1}^{{\kappa }_1}{\sum }_{i\in {\mathcal{N}}_m^s(r,\tau )}{D}_i{w}_m^r}{{\sum }_{m=1}^{M_r}{\sum }_{\tau =1}^{{\kappa }_1}{\sum }_{i\in {\mathcal{N}}_m^s(r,\tau )}{D}_i}}.\hfill \end{array}$$

(6)

3.2. Mobility Model

We assume that all ENs managed by the CS are uniformly distributed along a road segment [40], with the distance between adjacent ENs defined as the edge spacing L. The set of road segments covered by all ENs is referred to as the candidate training section. As shown in Figure 2, the road segments covered by scheduled ENs form the active training section, where only the vehicles passing through during the current time slice are eligible to participate in the HFL process. We further assume that vehicle arrivals at the entrance of the candidate training section follow a Poisson process [20,41,42] with a vehicle arrival rate of $\lambda$ (in vehicles per second, veh/s).

Considering that the contribution of vehicle data to the cloud model decreases over time, the candidate training section is generally short. Therefore, speed variations across this section are negligible, allowing each vehicle to maintain a constant speed during the training period. Specifically, each vehicle generates its speed $v_i$ (in meters per second, m/s) randomly and independently from a truncated Gaussian distribution [13,20,43]. Let $\overline{v}$ and $\sigma$ denote the mean and standard deviation of vehicle speeds within the coverage area of the CS at a given time. The pair $(\overline{v},{\sigma }_v)$ represents the speed characteristics of vehicles within the coverage area of the CS. The probability density function of $v_i$ is given in [20] as

$$\begin{array}{cc}\hfill & f\left(v_i\right)=\left\{\begin{array}{c}{\displaystyle \frac{2e^{\frac{-{(v_i-\overline{v})}^2}{2{\sigma }^2}}}{\sigma \sqrt{2\pi }\left(erf(\frac{v_{\max }-\overline{v}}{\sigma \sqrt{2}}\right)-erf\left(\frac{v_{\min }-\overline{v}}{\sigma \sqrt{2}}\right))}},\\ v_{\min }\le v_i\le v_{\max },\\ 0,\mathrm{otherwise},\end{array}\right.\hfill \end{array}$$

(7)

where $erf\left(x\right)={\displaystyle \frac{2}{\sqrt{\pi }}}{\int }_0^x{e}^{-t^2}dt$ is the Gaussian error function. $v_{\min }$ and $v_{\max }$ representing the minimum and maximum speeds, respectively, are calculated as $v_{\min }=\overline{v}-2\sigma$ and $v_{\max }=\overline{v}+2\sigma$.

3.3. Performance Metrics

In VHFL, scheduling optimization entails navigating trade-offs among multiple performance objectives, namely model accuracy, training delay, and energy consumption. To address this multi-objective optimization in a unified manner, we formulate a composite performance metric that quantifies the long-term training cost. This metric integrates the three core dimensions using a weighted formulation, enabling it to flexibly represent and enforce task-specific performance preferences. We next formally define the individual cost terms of this metric.

(1) Accuracy deviation cost (ADC): To enhance optimization sensitivity near the target threshold, we follow the exponential penalty design proposed in [44,45] and define the ADC in cloud round r as

$$\begin{array}{cc}\hfill & C^{\mathrm{A}}\left(r\right)=max\{0,1-{\mathsf{\Xi }}^{(A_r-A_{\mathrm{target}})}\},\hfill \end{array}$$

(8)

where $\mathsf{\Xi }>1$ is a scaling factor that controls the penalty sensitivity. This function has two desirable properties: (i) When $A_r\ge A_{\mathrm{target}}$, the ADC is zero due to the max operator, thereby avoiding unnecessary optimization efforts beyond the required accuracy; (ii) When $A_r<A_{\mathrm{target}}$, the ADC increases exponentially with the accuracy shortfall, reinforcing the system’s responsiveness to underperformance.

The ADC can be interpreted as a normalized penalty score within the range $[0,1]$, quantifying the degree to which the current accuracy violates the target. A value of 0 indicates the accuracy target is satisfied, while values closer to 1 reflect more severe deviation. By adjusting $\mathsf{\Xi }$, the system can flexibly control the tolerance to accuracy deviation. For example, setting $\mathsf{\Xi }=1000$ enforces stringent adherence to the accuracy requirement, which is well-suited for safety-critical tasks.

(2) Time delay cost (TDC): For each scheduled vehicle i at time slot $(r,\tau )$, let $f_i(r,\tau )$ denote its allocated computational frequency, and $C_i$ the required cycles per data sample. The local computation delay is expressed as [15,41,46]:

$$\begin{array}{cc}\hfill & T_i^{\mathrm{cmp}}(r,\tau )={\displaystyle \frac{{\kappa }_2{C}_i{D}_i}{f_i(r,\tau )}}.\hfill \end{array}$$

(9)

Each vehicle uploads its model via the Orthogonal Frequency Division Multiple Access (OFDMA) channels [46]. Let $b_i(r,\tau )$ be the number of allocated resource blocks (RBs), and B denote the bandwidth per RB. The uplink bandwidth is then $B_i(r,\tau )=b_i(r,\tau )B$. Given the allocated transmit power $P_i(r,\tau )$, average channel power gain $h_i(r,\tau )$, model size Z, and noise power spectral density $N_0$, the communication delay is expressed as [41,47]

$$\begin{array}{c}\hfill T_i^{\mathrm{comm}}(r,\tau )={\displaystyle \frac{Z}{B_i(r,\tau ){log}_2\left(1+\frac{h_i(r,\tau )P_i(r,\tau )}{N_0{B}_i(r,\tau )}\right)}}.\end{array}$$

(10)

Due to vehicle heterogeneity, the cloud round delay is determined by the slowest participant [41]:

$$\begin{array}{cc}\hfill & T_{\mathrm{vel}}\left(r\right)=\sum _{\tau =1}^{{\kappa }_1}{T}_{\mathrm{vel}}(r,\tau )=\sum _{\tau =1}^{{\kappa }_1}\underset{m\in \mathcal{M}}{max}\underset{i\in {\mathcal{N}}_m^s(r,\tau )}{max}(T_i^{\mathrm{cmp}}(r,\tau )+T_i^{\mathrm{comm}}(r,\tau )).\hfill \end{array}$$

(11)

After ${\kappa }_1$ rounds, each EN uploads its aggregated model to the CS at a transmission rate $R_{\mathrm{e}}$. Following the assumptions in prior works [48], we consider these wired links to be stable and high-capacity. Since the ENs upload edge models in parallel, the wired time delay is $T_{\mathrm{edge}}\left(r\right)={\displaystyle \frac{Z}{R_{\mathrm{e}}}}$. Thus, the TDC in cloud round r is

$$\begin{array}{cc}\hfill & C^T\left(r\right)=T_{\mathrm{vel}}\left(r\right)+T_{\mathrm{edge}}\left(r\right).\hfill \end{array}$$

(12)

It is noted that downlink time delay is negligible compared to uplink time delay [49].

(3) Energy consumption cost (ECC): For each scheduled vehicle i at time slot $(r,\tau )$, let $c_i$ denote its effective capacitance factor of the computing chipset. The local energy consumption consists of two parts (the computation energy, $E_i^{\mathrm{cmp}}$, and the communication energy, $E_i^{\mathrm{comm}}$), given by

$$\begin{array}{cc}\hfill & E_i^{\mathrm{cmp}}(r,\tau )=c_i{\kappa }_2{C}_i{D}_i{f}_i^2(r,\tau ),\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill & E_i^{\mathrm{comm}}(r,\tau )=P_i(r,\tau )T_i^{\mathrm{comm}}(r,\tau ).\hfill \end{array}$$

(14)

The total energy consumed by all scheduled vehicles in cloud round r is

$$\begin{array}{cc}\hfill & E_{\mathrm{vel}}\left(r\right)=\sum _{\tau =1}^{{\kappa }_1}\sum _{i\in {\mathcal{N}}_m^s(r,\tau )}(E_i^{\mathrm{cmp}}(r,\tau )+E_i^{\mathrm{comm}}(r,\tau )).\hfill \end{array}$$

(15)

Let $P_{\mathrm{e}}$ denote the EN’s transmit power. The wired transmission energy for one EN is $E_{\mathrm{edge}}\left(r\right)=P_{\mathrm{e}}Z/R_{\mathrm{e}}$. Thus, the ECC in cloud round r is

$$\begin{array}{cc}\hfill & C^E\left(r\right)=E_{\mathrm{vel}}\left(r\right)+M_r{E}_{\mathrm{edge}}\left(r\right).\hfill \end{array}$$

(16)

Training cost: As discussed at the beginning of this section, scheduling optimization in VHFL must navigate trade-offs among model accuracy, training delay, and energy consumption. To unify optimization objectives, we define a long-term training cost that integrates the three dimensions using a weighted formulation. Similar weighted-sum approaches have been widely adopted in FL to balance performance metrics [11,18,44]. Let ${\alpha }_1$, ${\alpha }_2$, and ${\alpha }_3$ denote the weight coefficients for ADC, TDC, and ECC, respectively, with ${\alpha }_1+{\alpha }_2+{\alpha }_3=1$. These weights capture the task-specific performance preferences: (i) safety-critical tasks emphasize high-reliability accuracy, thus assigning a larger ${\alpha }_1$; (ii) time-sensitive tasks prioritize fast responses, leading to a higher ${\alpha }_2$; (iii) non-critical background tasks focus on energy efficiency, hence favoring a larger ${\alpha }_3$. To ensure numerical comparability across the cost components, we also introduce scaling factors ${\beta }_1$, ${\beta }_2$, and ${\beta }_3$ to normalize the magnitudes of ADC, TDC, and ECC. Accordingly, the training cost in cloud round r is defined as

$$\begin{array}{cc}\hfill & C\left(r\right)={\alpha }_1{\beta }_1{C}^{\mathrm{A}}\left(r\right)+{\alpha }_2{\beta }_2{C}^T\left(r\right)+{\alpha }_3{\beta }_3{C}^E\left(r\right).\hfill \end{array}$$

(17)

In practice, for a given task, the weights ${\alpha }_i$ are fixed according to its characteristics, system policies, or service-level agreements. In contrast, the scaling factors ${\beta }_i$ are dynamically computed at the start of training to normalize the cost components based on their absolute magnitudes, as detailed in Section 5.

3.4. Problem Formulation

For a VHFL system, we expect to reduce the long-term training cost (LTTC) as much as possible while meeting the target accuracy of the ML task. Specifically, before the training process begins, the CS adaptively determines the Edge-and-Vehicle Scheduling number $(M_r,N_r)$ in each cloud round. The problem formulation is as follows:

$$\begin{array}{cc}\hfill \mathbf{P}\mathbf{1}:& min\sum _{r=1}^{R}C\left(r\right)\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill & {\alpha }_1+{\alpha }_2+{\alpha }_3=1,\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill & 0\le {\alpha }_1,{\alpha }_2,{\alpha }_3\le 1,\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill & {\beta }_1,{\beta }_2,{\beta }_3>0,\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill & M_r\le M^{\max },M_r\in \mathbb{Z},\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill & N_r\le N_m^{\max },m\in {\mathcal{M}}_r,N_r\in \mathbb{Z},\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill & \begin{array}{ccc}\hfill F^{\mathrm{cloud}}(r-1)-F^{\mathrm{cloud}}\left(r\right)& <\delta ,\hfill & \hfill r\in \{\mathrm{R}-4,\dots ,\mathrm{R}\},\end{array}\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill & A_R\ge A_{\mathrm{target}}.\hfill \end{array}$$

(25)

The objective of the ML task is to minimize the long-term training cost over R cloud rounds, as formulated in Equation (18), where R denotes the number of cloud rounds required for the cloud model to reach the target accuracy on the test dataset. The training cost $C\left(r\right)$ incurred in each cloud round r is defined in Equation (17). Constraints (19)–(21) specify the bounds on the weighting factors ${\alpha }_i$ and scaling factors ${\beta }_i$, which balance and normalize the performance metrics. Specifically, constraint (22) restricts the number of scheduled ENs $M_r$ in cloud round r to be no greater than the total number of available ENs, $M^{\max }$. Similarly, constraint (23) limits the number of vehicles scheduled per EN $N_r$ to be no more than $N_m^{\max }=b_m$, where $b_m$ is the number of available RBs allocated for the ML task at each scheduled EN $m\in {\mathcal{M}}_r$. Both $M_r$ and $N_r$ are integer variables, reflecting discrete scheduling decisions. Constraints (24) and (25) ensure that the cloud model converges and meets the specified target accuracy $A_{\mathrm{target}}$, where $\delta$ represents the convergence threshold, and $A_R$ denotes the model accuracy in the R-th cloud round.

4. Motivation

This section shows that the optimal EVS configuration is a dynamic variable, highly sensitive to multiple hierarchical factors. We begin by analyzing the intrinsic training dynamics of HFL, specifically how static task attributes and training phase evolution shape a baseline strategy. We then examine how this baseline is further influenced by external vehicular environments. This multi-level analysis provides key insights into system optimization and underpins the design of our environment-aware adaptive scheduling framework.

4.1. Experimental Setup

This subsection outlines the simulation setup, including the general VHFL system and the controlled experimental design for evaluating EVS performance under various tasks and environmental conditions.

4.1.1. VHFL System Simulation Setup

We simulate the VHFL system described in Section 3. To emulate realistic vehicular communication conditions, the simulation incorporates a composite channel model that dynamically determines the channel power gain $h_i(r,\tau )$. This model integrates three standard physical effects: (i) a 3GPP-compliant Non-Line-of-Sight (NLOS) path loss model to account for signal obstruction in urban and highway environments [50]; (ii) a standard log-normal shadow fading component for large-obstacle attenuation [50]; and (iii) a Rayleigh fading model to capture multipath fluctuations consistent with the NLOS assumption [51]. The general parameters are listed in

Table 2. Simulation parameters for HFL.

Parameter	Value
Computational frequency of vehicle for three HFL tasks, $f_i^{\max }(r,\tau )$	$0.5$/1/4 GHz
Transmit power of vehicle, $P_i^{\max }(r,\tau )$	$[20,30]$ dBm
Bandwidth of one RB for three HFL tasks, B	180/180/360 kHz
Number of RBs at each EN, $b_m$	20
Carrier frequency, $f_c$	$3.5$ GHz
Effective capacitance factor of the computing chipset on vehicle i, $c_i$	${10}^{-27}$
Noise power spectral density, $N_0$	$-174$ dBm/MHz
Edge Spacing, L	300 m
Vehicle arrival rate, $\lambda$	1 veh/s
Speed characteristics of vehicles in the training area, $(\overline{v},{\sigma }_v)$	$(20,3)$ m/s
Uplink transmission rate of the ENs, $R_{\mathrm{e}}$	50 Mbps
Cloud interval, ${\kappa }_1$	5
Edge interval, ${\kappa }_2$	5
Learning rate, $\eta$	$0.01$
Decay rate, ${\zeta }_{\eta }$	$0.995$
Batch size, b	20
Convergence threshold for three HFL tasks, $\delta$	$0.01/0.1/0.1$
Data partitioning parameter, $\varphi$	5
Data size per vehicle, $D_i$	200
Weighting factors for ADC/TDC/ECC, ${\alpha }_1/{\alpha }_2/{\alpha }_3$	0.1/0.5/0.4
scaling factor of ADC, $\mathsf{\Xi }$	100

.

4.1.2. Controlled Simulation Settings

We evaluate the system on three representative learning tasks of varying complexity: MNIST [52], FMNIST [53], and CIFAR-10 [54]. The specific CNN models, their computational complexities, communication load, and target accuracies are detailed in

Table 3. Parameter setup of different HFL tasks.

Task Class	Computational Complexity (cycles)	Communication Load (bits)	Target Accuracy (%)
MNIST	102,412	13,492,544	98
FMNIST	867,804	13,504,832	85
CIFAR-10	6,291,652	83,333,632	75

. To emulate data heterogeneity, we employ two standard non-IID data partitioning schemes [55] to control for quantity-based imbalance (parameterized by $\varphi$) and distribution-based imbalance (parameterized by $\psi$) across vehicles. Default data distribution parameters are listed in Table 2.

This observational study aims to systematically characterize the performance landscape. To this end, for each scenario, we simulate a range of fixed EVS configurations $(M,N)$ for each cloud round, varying the number of scheduled ENs M and vehicles per EN N within the set $\{2,4,6,8\}$. This allows us to identify the ground-truth optimal EVS for each condition by comparing the long-term training cost. To isolate the impact of EVS configuration and ensure both reproducibility and fair comparison, we adopt random vehicle selection and uniform bandwidth allocation as controlled baselines, and fix the random seed across all experiments to eliminate stochastic variations such as model initialization and data shuffling.

4.2. Impact of Intrinsic Task Attributes

(1) Impact of Static Task Characteristics: The static task characteristics, specifically the computational complexity quantified by the required cycles per data sample $C_i$ and the communication load represented by the model size Z, define the baseline optimization challenge. Our experiments first confirm that the optimal EVS is highly task-dependent. As shown in Figure 3, tasks with higher complexity (e.g., CIFAR-10) benefit more significantly from an increased EVS size, whereas simpler tasks (e.g., MNIST) reach performance saturation earlier. Furthermore, it is also worth noting that the results in Figure 4 reveal that the EVS configuration that minimizes the long-term training cost is a complex trade-off and does not necessarily align with the one that optimizes any single performance metric in isolation. These findings highlight that a scheduler must be sensitive to the static task characteristics.

(2) Impact of Training Phase Evolution: As shown in Figure 3, the training process exhibits a time-varying dynamic. We characterize this training phase evolution by analyzing the first- and second-order differences of the learning curve, namely the learning speed $\mathsf{\Delta }A=A_r-A_{r-1}$ and the learning acceleration ${\mathsf{\Delta }}^2{A}_r=\mathsf{\Delta }{A}_r-\mathsf{\Delta }{A}_{r-1}$. Here,

• Initial training phase (ITP): Accuracy increases rapidly as the model is far from optimal and gradients are large. This results in significant performance gains and a convex learning curve: $A_r\ll A_{\mathrm{target}},\mathsf{\Delta }{A}_r\gg 0,{\mathsf{\Delta }}^2{A}_r<0$.
• Medium training phase (MTP): Accuracy improvement slows and begins to fluctuate. The model enters a local convergence region, where training shifts from coarse adjustment to fine-tuning and becomes more sensitive to data-induced variations: $\mathsf{\Delta }{A}_r\to 0^{+},{\mathsf{\Delta }}^2{A}_r$ changes sign frequently.
• Final training phase (FTP): The model approaches convergence and accuracy saturates. Gradients diminish, performance gains become negligible, and variations are mainly due to noise: $A_r\to A_{\mathrm{target}},\mathsf{\Delta }{A}_r\approx 0,\mathrm{Var}\left(A_r\right)$ becomes stable or slightly increases.

This observed three-phase learning behavior aligns with established SGD theory on non-convex landscapes [34]. The effect is especially evident in FL, where non-IID data slows convergence and often triggers distinct performance phase transitions [35]. Crucially, the phase transition reflects a fundamental shift in the optimization objective: from aggressively minimizing the ADC in the ITP to prioritizing the conservation of TDC and ECC in the MTP and FTP. To monitor this evolution accurately, model accuracy evaluations are conducted on the cloud server using a fixed, pre-sampled test dataset. This standard approach ensures an unbiased assessment of the cloud model and avoids evaluation errors caused by heterogeneous local data.

4.3. Impact of Environmental Dynamics

Beyond the intrinsic task attributes and training phase evolution, the external environment introduces another layer of complexity. We decompose these environmental dynamics into three distinct but interconnected layers:

(1) Impact of Quasi-Static Network Topology (Edge Spacing). We first analyze the network topology, using edge spacing $d_{\mathrm{edge}}$ as a representative parameter. Our results in

Table 4. Impact of environmental dynamics on the optimal EVS configuration.

Variable	Value	Optimal EVS	Cloud Rounds	ADC	TDC (s)	ECC (J)	LTTC
$d_{\mathrm{edge}}$ (m)	200	(2, 6)	58	17.2	438.5	4732.7	14,575.2
	300	(4, 4)	53	16.0	421.8	7663.9	15,213.7
	400	(6,2)	52	18.1	368.1	8094.2	14,257.4
$\varphi$	5	(4, 4)	53	16.0	421.8	7663.9	15,213.7
	2	(8, 2)	65	32.6	457.6	13,474.3	20,089.3
$\psi$	$0.6$	(6, 2)	121	33.7	981.7	17,097.2	31,999.4
	$0.4$	(8, 2)	135	37.1	1161.1	25,348.6	40,517.0
$\lambda$ (veh/s)	0.6	(4, 2)	65	22.1	452.0	6888.8	16,264.4
	1.0	(4, 4)	53	16.0	421.8	7663.9	15,213.7
	1.4	(4, 2)	55	15.5	436.6	7937.2	15,636.3
$(\overline{v},{\sigma }_v)$ (m/s)	$(10,3)$	(2, 4)	56	19.1	436.3	4036.5	14,438.7
	$(20,1)$	(8, 2)	52	15.9	365.6	10,784.7	15,046.4
	$(20,3)$	(4, 4)	53	16.0	421.8	7663.9	15,213.7
	$(20,5)$	(2, 2)	68	28.3	467.5	3520.0	15,926.8
	$(30,3)$	(2, 2)	68	29.5	466.8	3562.4	16,046.3

reveal a non-monotonic structural shift in the optimal EVS $(M,N)$ configuration as spacing increases from 200 m to 400 m, flipping from a vehicle-centric $(2,6)$ to an edge-centric $(6,2)$. This reflects a change in the dominant performance bottleneck. At small spacing, handover-induced straggler risk becomes the primary challenge, inflating the TDC. The optimal strategy therefore favors a more centralized structure (fewer ENs) to minimize instability. Conversely, at large spacing, the bottleneck shifts to communication inefficiency, prompting a more distributed topology (more ENs) to improve average connectivity. This indicates that the optimal EVS is highly sensitive to the physical infrastructure layout of ENs.

(2) Impact of Baseline Data Heterogeneity. Operating on this physical topology, the statistical distribution of data across the vehicles introduces the fundamental learning challenge. Our experiments in Table 4 confirm that a higher degree of non-IID data makes it more difficult to reduce the ADC. To ensure stable convergence, the scheduler is incentivized to increase the EVS size. However, this directly conflicts with the goal of minimizing TDC and ECC. The optimal EVS is therefore a direct reflection of the trade-off between the lowering ADC, TDC, and ECC.

(3) Synergistic Impact of Mobility Patterns. Finally, vehicle mobility patterns act as the dynamic mechanism that reshapes the data landscape within the constraints of the network topology. As observed in Table 4, mobility patterns synergistically influence the optimal EVS. We dissect these effects by analyzing the macro-level vehicle arrival rate and the micro-level vehicle speed characteristics.

• Vehicle arrival rate: The arrival rate $\lambda$ reveals a non-monotonic impact on the optimal EVS. At low rates, the system is data-starved, and the optimal EVS is small due to a lack of diverse candidates. At very high rates, the system becomes data-congested, as high vehicle density creates severe contention for limited communication bandwidth, increasing the TDC. This again forces the scheduler to select a smaller EVS to manage congestion. A moderate arrival rate provides the optimal balance of a rich data supply without prohibitive resource contention.
• Vehicle speed characteristics: The speed characteristics $(\overline{v},{\sigma }_v)$ illustrate the duality of mobility. The mean speed $\overline{v}$ dictates the primary trade-off: at low EVS, the bottleneck is stagnant data diversity, hindering ADC reduction; at high EVS, the dominant issue is system instability, increasing TDC. The standard deviation ${\sigma }_v$ further acts as a data diversity amplifier; greater speed variation enhances vehicle mixing, allowing the system to achieve its learning goals with a potentially smaller EVS.

These findings collectively demonstrate that the optimal EVS must adapt not only to the edge spacing and the data on the vehicles but also to the dynamic evolution of the data landscape, which is shaped by complex mobility patterns.

4.4. Theoretical Analysis

The necessity for dynamic EVS adjustment stems from the fundamental conflict between convergence gain and system overhead. On the one hand, expanding the scale of participation generates significant convergence gains. According to the theoretical convergence bound for FedAvg on non-IID data derived in [56] and extended to hierarchical settings, the number of communication rounds R required to reach a target accuracy is inversely proportional to the number of participating nodes, satisfying $R\propto {\displaystyle \frac{1}{\sqrt{MN}}}$. This implies that increasing the number of EN and vehicles per EN accelerates global convergence. On the other hand, expanding the scale incurs substantial system overheads. The cost per training round is positively correlated with the participation scale, where the ADC and TDC typically exhibit weak linear growth with respect to M and N, while the ECC often exhibits super-linear growth due to cumulative aggregation and transmission burdens. Due to the task diversity and environment dynamics, the task attributes that determine convergence gain and the environmental state that determines resource overhead show significant differences in different scenarios, causing the optimal balance point after weighing convergence gains and system costs to drift. Therefore, the long-term training cost is closely related to the EVS configurations, and the optimal configuration has strong scenario dependence.

5. AptEVS: Adaptive Edge-and-Vehicle Scheduling

Section 4 reveals that the optimal EVS configuration is highly sensitive to task dynamics, environmental variability, and training phases, exhibiting strong non-stationarity and nonlinearity. Such complexity limits the effectiveness of static or heuristic strategies. Given the problem’s nature as sequential decision-making under uncertainty with no tractable model, we adopt DRL to exploit its strength in learning adaptive policies in high-dimensional dynamic environments.

5.1. Framework Design Overview

A key challenge in applying DRL to VHFL is determining whether an agent can learn an effective scheduling policy online, within a dynamic and previously unseen vehicular environment, without relying on pre-trained models. Before a generalizable solution can be developed, it is necessary to first validate the feasibility of online learning in such settings. To this end, AptEVS is designed as a proof-of-concept framework aimed at demonstrating this feasibility. We intentionally configure the agent to learn from scratch in each distinct environment. If the agent can consistently converge from random initialization to a high-performing policy, it indicates that our DRL formulation has captured the essential dynamics of the problem, suggesting that the scheduling task is fundamentally learnable.

To tackle this learnable problem, a powerful DRL approach is required. While classic reinforcement learning algorithms like tabular Q-learning are effective for problems with discrete and manageable state spaces [57], they are ill-suited for our VHFL setting due to the curse of dimensionality. The system state $s_t$ that our agent must process is high-dimensional and composed of numerous continuous variables. Discretizing such a space would lead to a combinatorially explosive number of states, rendering a Q-table computationally infeasible. Therefore, we leverage a DQN-based approach, which employs a Deep Neural Network (DNN) as a powerful function approximator to learn the mapping from the high-dimensional, continuous state space to the optimal action values.

Building on this motivation, we present AptEVS, a phase-aware DRL framework that dynamically adjusts the agent’s learning strategy according to the current phase of the HFL process, specifically, the ITP, MTP, or FTP. The framework comprises two key components: (i) a lightweight module for detecting training phase transitions, and (ii) a set of phase-adaptive algorithms that jointly enhance exploration and convergence. The remainder of this section details the DRL environment formulation and the implementation of the phase-aware scheduling strategy.

5.2. Design of DRL Environment

Figure 5 shows the environment design overview for AptEVS. The DRL environment is defined by three standard elements, state, action, and reward, where each cloud round in VHFL corresponds to a discrete time step t. Below we detail each component.

(1) State: The state $s_t$ is a structured vector composed of two sub-components, defined as $s_t=[s^{\mathrm{task}},s_t^{\mathrm{env}}]$, allowing the agent to perceive multi-level dynamics for context-aware scheduling. Here,

• $s^{\mathrm{task}}=\{C,Z\}$ is the set of the static task characteristics in intrinsic task attributes, where C is computational complexity and Z is communication load. While model accuracies $\{A_t,A_{t-1)}\}$ are available, they are not included in the state but used by a separate phase detection module.
• $s_t^{\mathrm{env}}=\{d_{\mathrm{edge}},s_t^{\mathrm{data}},s_t^{\mathrm{mobility}}\}$ is the dynamic of the external environment.

  –

  $d_{\mathrm{edge}}$ is the current distance between neighboring ENs.

  –

  $s^{\mathrm{data}}=\{D^{\mathrm{avg}},D^{\mathrm{std}},S^{\mathrm{avg}},S^{\mathrm{std}}\}$ is the real-time statistical characteristics of the distributed data, where $D^{\mathrm{avg}}$ and $D^{\mathrm{std}}$ denote the mean and standard deviation of the local data sizes across vehicles, and $S^{\mathrm{avg}}$ and $S^{\mathrm{std}}$ represent the corresponding statistics of local label skew. These statistics are also reported by vehicles in a privacy-preserving form. Specifically, the label skew for client i is defined as $S_i\triangleq \mu {\sum }_{k=1}^K{({\alpha }_i^k-{\displaystyle \frac{1}{K}})}^2$ [58].

  –

  $s^{\mathrm{mobility}}=\{\lambda ,\overline{v},{\sigma }_v\}$ is the vehicle mobility pattern in the current candidate training segment. $\lambda$ indicates the current vehicle arrival rate, and $\overline{v}$ and ${\sigma }_v$ are the mean and variance of vehicle speeds.

(2) Action: The agent selects an EVS configuration as the action $a_t=(M_t,N_t)$, where $M_t$ is the number of selected ENs and $N_t$ is the number of scheduled vehicles per node.

(3) Reward: To provide a stable and effective learning signal, we design a novel reward function based on a short-term target accuracy $A_{\mathrm{target}}^{ST}$. This serves as a dynamic reference point to measure the model’s relative performance improvement. We define $A_{\mathrm{target}}^{ST}$ as the peak accuracy achieved within the last $R^{\mathrm{cycle}}$ rounds:

$$\begin{array}{c}\hfill A_{\mathrm{target}}^{ST}=max\{A_{t-R^{\mathrm{cycle}}},\dots ,A_{t-1},A_t\}.\end{array}$$

(26)

Based on this dynamic target, we then define a short-term accuracy deviation cost (ST-ADC), which penalizes deviations from this short-term peak:

$$\begin{array}{c}\hfill C^{\mathrm{STA}}\left(t\right)=\left\{\begin{array}{cc}{\displaystyle \frac{1}{2}}{\mathsf{\Xi }}^{(A_{\mathrm{target}}^{ST}-A_t)},\hfill & A_t>A_{\mathrm{target}}^{ST},\hfill \\ 1-{\displaystyle \frac{1}{2}}{\mathsf{\Xi }}^{(A_t-A_{\mathrm{target}}^{ST})},\hfill & A_t\le A_{\mathrm{target}}^{ST}.\hfill \end{array}\right.\end{array}$$

(27)

The final reward $r_t$ at time step t is computed as a weighted sum of ST-ADC, TDC, and ECC:

$$\begin{array}{c}\hfill r_t=-{\alpha }_1{\beta }_1{C}^{\mathrm{STA}}\left(t\right)-{\alpha }_2{\beta }_{2}T\left(t\right)-{\alpha }_3{\beta }_{3}E\left(t\right),\end{array}$$

(28)

where this raw reward is subsequently normalized to a consistent scale to ensure stable learning throughout the training process.

This novel formulation is motivated by the limitations of more straightforward approaches. A simple method, such as directly using the negative training cost $-C\left(t\right)$ from our global objective (P1) (18), is problematic due to the sparse and delayed nature of the learning signal. Furthermore, conventional reward designs in FL that rely on instantaneous accuracy metrics, such as $A_t$ [59], ${\mathsf{\Xi }}^{A_t}$ [44], or $(A_t-A_{t-1})$ [57], are suboptimal in dynamic environments like VHFL. As our analysis revealed in Section 4.2, such signals can become monotonic, less informative, or highly susceptible to noise from non-IID data.

This dynamic-target-guided design offers three benefits: (i) it avoids reward saturation by leveraging a moving accuracy baseline; (ii) it enhances action discriminability via relative performance feedback; and (iii) it suppresses noise sensitivity by anchoring to a periodically updated peak accuracy.

5.3. Phase-Aware Scheduling Algorithms and Workflow

The adaptive capability of AptEVS is realized through a set of specialized algorithms, orchestrated by a main coordinating workflow detailed in Algorithm 1. This main algorithm functions as a high-level scheduler. At the beginning of each training episode, the agent’s components are initialized (Lines 1–2). Then, in each round r, namely, time step t, the agent’s core logic is as follows: First, it observes the comprehensive system state $s_t$ (Line 5). It then calls the Adaptive Phase Detection Mechanism in Algorithm 2 to identify the current training phase based on recent performance (Lines 7–8). Based on the detected phase, the agent invokes the appropriate action selection subroutine: structured exploration in Algorithm 3 for the ITP (Line 9–13), priority-enhanced DQN training in Algorithm 4 for the MTP (Line 14–21), or the pure exploitation policy for the FTP (Lines 22–26). After executing the action and receiving a reward, the experience is stored and used to train the DQN networks via the classic Mixed Experience Replay (MER) mechanism (Lines 13 and 18). This entire process repeats, allowing the agent to continuously learn and converge towards a stable, high-performance scheduling policy that is tailored to the specific dynamics of the current environment.

5.3.1. Adaptive Phase Detection Mechanism

A cornerstone of AptEVS is its ability to perceive the training phase evolution of HFL. Based on our observations in Section 4, we designed the lightweight, rule-based mechanism detailed in Algorithm 2 to detect transitions between phases based on accuracy stagnation and policy stability.

Algorithm 1 AptEVS: phase-aware DRL scheduling
Input:  Discount factor $\gamma$, learning rate $\alpha$, update frequency of target network J, cloud interval ${\kappa }_1$, initial epsilon ${ϵ}_0$, decay coefficient ${\zeta }_{ϵ}$, and minimum epsilon ${ϵ}_{\min }$, mini-batch $b_{\mathrm{DQN}}$. Output:  Trained online network parameters $\theta$.
1:	Initialize DQN online and target networks $\theta ,{\theta }^{\prime }$, replay buffer $\mathcal{V}$.
2:	Initialize counters $k_1\leftarrow 0$, $k_2\leftarrow 0$, best action $a_{\mathrm{best}}$, counter threshold $K_1$ and $K_2$.
3:	for each cloud round $r=1,2,\dots ,R$ do
4:	Set the current time step t be r.
5:	Observe current system state $s_t$.
6:	// Adaptive phase detection
7:	Apply Algorithm 2, obtain ${\mathcal{P}}_t,\{k_1,k_2\}$.
8:	// Phase-specific action selection
9:	if ${\mathcal{P}}_t$ is ITP then	▹ DQN Training
10:	Apply Algorithm 3, obtain and execute $a_t$.
11:	Calculate normalized reward $r_t$.
12:	Store experience $(s_t,a_t,r_t,s_{t+1})$ in $\mathcal{V}$.
13:	Train DQN networks $\theta ,{\theta }^{\prime }$ using MER from $\mathcal{V}$.
14:	else if ${\mathcal{P}}_t$ is MTP then	▹ DQN Training
15:	Apply Algorithm 4, obtain and execute $a_t$.
16:	Calculate normalized reward $r_t$.
17:	Store experience $(s_t,a_t,r_t,s_{t+1})$ in $\mathcal{V}$.
18:	Train DQN networks $\theta ,{\theta }^{\prime }$ using MER from $\mathcal{V}$.
19:	if $a_t$ is a new best action then
20:	$a_{\mathrm{best}}\leftarrow a_t$.
21:	end if
22:	else
23:	// Phase switches to the FTP
24:	$a_t\leftarrow {arg\; max}_{a\in {\mathcal{A}}_t}Q(s_t,a;\theta )$	▹ DQN Inference
25:	Execute $a_t$.
26:	end if
27:	end for
28:	return  $\theta$

ITP → MTP Transition: Theoretically, the magnitude of accuracy gain ($\mathsf{\Delta }A$) alone serves as a sufficient indicator to distinguish between different training phases. However, strictly applying this theoretical threshold to single-round data is prone to error due to the stochastic volatility inherent in federated learning. To bridge the gap between this macroscopic theoretical trend and microscopic observational noise, we introduced a cumulative counting mechanism. Instead of relying on instantaneous values, this mechanism triggers a phase shift only when the accuracy gain satisfies the condition for a cumulative number of rounds. This effectively acts as a temporal filter, smoothing out transient fluctuations and ensuring that the detected phase transition is a stable trend rather than a result of stochastic noise. The transition from the ITP to the MTP is triggered when the accuracy gain stagnates, which we detect when the accuracy gain, $\mathsf{\Delta }{A}_t$, remains below a predefined threshold $\delta$ for $K_1$ consecutive rounds (Lines 4–6). Upon detecting this transition, a critical adaptation is performed: the action space is pruned for the subsequent MTP. This step leverages the exploration from the ITP to establish an empirical lower bound for effective EVS configurations, significantly accelerating the subsequent policy learning. Let $a_{\min }^{\mathrm{ITP}}=(M_{\min }^{\mathrm{ITP}},N_{\min }^{\mathrm{ITP}})$ be the final EVS configuration from the ITP. The original action space, $\mathcal{A}$, is defined as $\mathcal{A}=\{(M,N)\mid M\in \{1,\dots ,M^{\max }\},N\in \{1,\dots ,N^{\max }\}\}$. The new, refined action space for the MTP is then formally constructed by retaining only the configurations that dominate this empirically found lower bound: ${\mathcal{A}}_t\leftarrow \{(M,N)\in \mathcal{A}\mid M\ge M_{\min }^{\mathrm{ITP}},N\ge N_{\min }^{\mathrm{ITP}}\}$.

Algorithm 2 Adaptive phase detection mechanism
Input:  Current phase ${\mathcal{P}}_t$, accuracy history $\{A_{t-1},A_{t-2}\}$, counters $\{k_1,k_2\}$, best action $a_{\mathrm{best}}$, current action $a_t$, thresholds $\{K_1,A_{\mathrm{ITP}},K_2\}$. Output:  Next phase ${\mathcal{P}}_{t+1}$, updated counters $\{k_1,k_2\}$.
1:	Initialize ${\mathcal{P}}_{t+1}\leftarrow {\mathcal{P}}_t$, $k_1^{\prime }\leftarrow k_1$, $k_2^{\prime }\leftarrow k_2$.
2:	if ${\mathcal{P}}_t$ is ITP then
3:	// Check for ITP to MTP transition
4:	if $(A_t-A_{t-1})<\delta$ then
5:	$k_1^{\prime }\leftarrow k_1+1$.
6:	end if
7:	if ($k_1^{\prime }\ge K_1$ or $A_t>A_{\mathrm{ITP}}$) and $t>1$ then
8:	${\mathcal{P}}_{t+1}\leftarrow \mathrm{MTP}$	▹ Transition to MTP.
9:	Construct the new action space ${\mathcal{A}}_t$.
10:	end if
11:	else if ${\mathcal{P}}_t$ is MTP then
12:	// Check for MTP to FTP transition
13:	if $a_t=a_{\mathrm{best}}$ then
14:	$k_2^{\prime }\leftarrow k_2+1$.
15:	else
16:	$k_2^{\prime }\leftarrow 0$	▹ Reset counter.
17:	end if
18:	if $k_2^{\prime }\ge K_2$ then
19:	${\mathcal{P}}_{t+1}\leftarrow \mathrm{FTP}$.	▹ Transition to Inference Phase
20:	end if
21:	end if
22:	return ${\mathcal{P}}_{t+1}$, $\{k_1^{\prime },k_2^{\prime }\}$.

Algorithm 3 Structured exploration for ITP
1:	Input: Accuracy history $\{A_{t-1},A_{t-2}\}$, last action $a_{t-1}=(M_{t-1},N_{t-1})$.
2:	Output: Action $a_t=(M_t,N_t)$.
3:	if  $t==1$ then
4:	$(M_t,N_t)\leftarrow (M^{\max },N^{\max })$.	▹ Ensure a robust initial performance
5:	else if  $t==2$ then
6:	$(M_t,N_t)\leftarrow (M^{\min },N^{\min })$.	▹ Probe the baseline
7:	else
8:	$(M_t,N_t)\leftarrow (M_{t-1},N_{t-1})$.	▹ Retain previous configuration
9:	if $\mathsf{\Delta }{A}_{t-1}\le 0$ then	▹ If accuracy stagnates
10:	if $M_t<M_{\max }$ then
11:	$M_t\leftarrow M_t+1$	▹ Preferentially increase ENs.
12:	else if $N_t<N_{\max }$ then
13:	$N_t\leftarrow N_t+1$.
14:	end if
15:	end if
16:	end if
17:	return $(M_t,N_t)$.

MTP → FTP Transition: The MTP is the agent’s primary learning phase, where it refines its scheduling policy. The transition to the final, inference-only FTP is determined by the convergence of this learned policy. We assess convergence based on policy stability. Specifically, the agent tracks the current best-known action $a_{\mathrm{best}}$ (Line 13). A counter is incremented only when the agent’s chosen action $a_t$ in a round matches $a_{\mathrm{best}}$ (Line 14). If the agent discovers a new, superior action, $a_{\mathrm{best}}$ is updated, and the counter is reset (Line 16). The policy is considered to have converged when this counter reaches a predefined threshold $K_2$, signifying that the optimal action has remained stable for a sufficient number of consecutive rounds (Line 18).

Algorithm 4 Priority-enhanced DQN training for MTP
1:	Input: State $s_t$, epsilon $ϵ$.
2:	Output: Action $a_t$.
3:	if $rand\left(\right)<ϵ$ then	▹ Exploration
4:	Calculate the prioritization probability $p\left(a\right)$ (29) for each action $a\in {\mathcal{A}}_t$.
5:	Obtain action $a_t$ randomly from ${\mathcal{A}}_t$ according to $p\left(a\right)$.
6:	else	▹ Exploitation
7:	$a_t\leftarrow {arg\; max}_{a\in {\mathcal{A}}_t}Q(s_t,a;\theta )$.
8:	end if
9:	Update $ϵ$ with decay coefficient ${\zeta }_{ϵ}$.
10:	return $a_t$.

5.3.2. Phase-Aware DRL Algorithms

AptEVS employs customized DQN algorithms for the ITP and MTP, aiming to maximize learning efficiency while minimizing exploration overhead.

Structured Exploration for ITP. In the ITP, the objective is to quickly identify an EVS configuration that is good enough. We design Algorithm 3, which starts from the minimal EVS setting and increases its scale only when accuracy improvement stagnates. This not only reduces the long-term training cost during ITP but, more importantly, identifies a constrained and high-potential action space for the subsequent MTP, significantly accelerating the main DRL training process. The joint state–action space of HFL is vast, where large-scale configurations often yield high penalties. In the ITP, random exploration in these regions causes extreme variance in target Q-values, destabilizing training. Our “Start Small” strategy constrains exploration to a low-cost feasible region, reducing reward variance and providing a robust warm start for the policy.

Priority-Enhanced DQN Training for MTP. In the MTP, the agent performs fine-grained policy optimization within the reduced action space identified during ITP. As illustrated in Algorithm 4, we propose a priority-enhanced epsilon-greedy strategy, where the probability of exploring a specific action is inversely proportional to its resource cost $(M_t$·$N_t)$. This design ensures that lower-cost actions are explored more frequently, further improving training efficiency. The probability of selecting the action a is

$$\begin{array}{c}\hfill p\left(a\right)={\displaystyle \frac{1/(M_t·N_t)}{{\sum }_{({\hat{M}}_t·{\hat{N}}_t)\in {\mathcal{A}}_t}1/({\hat{M}}_t·{\hat{N}}_t)}}.\end{array}$$

(29)

In the MTP, the priority-enhanced strategy acts as guided exploration. By weighting exploration probabilities inversely to resource cost, it effectively performs importance sampling. This concentrates computational resources on potentially optimal subspaces rather than invalid high-cost regions, thereby improving sample efficiency and accelerating convergence.

Policy Deployment for FTP. During the final deployment phase, the agent shifts from training to pure inference. It ceases exploration and exclusively exploits the learned Q-network. Unlike the ITP (heuristic acceleration) and MTP (DRL-based learning), the scheduling design for the FTP focuses on exploitation rather than exploration. Our empirical analysis indicates that during the MTP, the global model’s accuracy fluctuations are highly sensitive to the scheduling actions $(M,N)$. This sensitivity provides rich feedback (gradients) for the DQN agent, enabling it to efficiently explore the state space and converge to an optimal policy. Once the DQN agent converges, the learned policy is deemed robust for the remaining convergence process. Consequently, in the FTP, AptEVS terminates the training of the DQN and switches to Inference Mode. The ENs and vehicles are scheduled directly based on the inference results of the trained DQN model.

6. Performance Evaluation

This section presents a comprehensive evaluation of the proposed AptEVS framework. We first establish the overall superiority of AptEVS through a performance comparison against a static baseline. Following this, we conduct two rigorous ablation studies to isolate and validate the individual contributions of the phase-aware mechanism and novel reward function. The evaluation concludes with an analysis of the framework’s robustness and computational overhead.

6.1. Experimental Setup

Consider a VHFL system with $M^{\max }=8$ ENs. It is evaluated on the MNIST, FMNIST, and CIFAR-10 tasks under two types of non-IID data partitioning. The DRL agent is realized using a fully connected DQN comprising two hidden layers of 128 and 64 neurons, respectively. Each layer adopts the ReLU activation function, and the network parameters are optimized using the Adam algorithm. The simulation parameters for these HFL tasks align with those specified in Section 4.1. Additionally, the hyperparameters of DRL for AptEVS are detailed in

Table 5. Simulation parameters for AptEVS.

Parameter	Value
Learning rate of DQN, $\alpha$	$0.01$
Discounted factor of DQN, $\gamma$	$0.90$
Initial epsilon of the epsilon-greedy strategy, ${ϵ}_0$	$0.95$
Decay coefficient of the epsilon-greedy strategy, ${\zeta }_{ϵ}$	$0.98$
Minimum epsilon of the epsilon-greedy strategy, ${ϵ}_{\min }$	$0.01$
Experience replay buffer size, V	32
Mini-batch size of DQN, $b_{\mathrm{DQN}}$	16
Update frequency of target network, J	4
Minimum of time step, $t_{\min }$	40
ADC factor, $\mathsf{\Xi }$	100
Update cycle for short-term target accuracy, $R^{\mathrm{cycle}}$	4
Accuracy improvement threshold for ITP, $\delta$	5%
Counter threshold for ITP, $K_1$	3
Counter threshold for the DQN training phase, $K_2$	5
Termination accuracy for ITP, $A_{\mathrm{ITP}}$	60%

.

6.2. Overall Performance Comparison

To evaluate the overall performance of AptEVS, we compare it with two representative baselines:

• Fixed proportional Edge-and-Vehicle Scheduling (FpEVS): A static baseline used to reflect uniform and task-independent scheduling commonly adopted in earlier studies. It schedules all ENs and 30% of vehicles in every training round, following the empirical setup in [11].
• Phase-aware Edge-and-Vehicle Scheduling (PahEVS): A dynamic baseline that adjusts the vehicle participation ratio across different training phases, 20% in ITP, 30% in MTP, and 50% in FTP, while keeping all ENs scheduled. This design is based on the core insight from [33] that scheduling fewer clients in early stages and more in later stages can reduce training cost.

Although FpEVS and PahEVS do not rely on explicit optimization, they reflect two common scheduling paradigms in FL and provide useful references for evaluating the effectiveness of our adaptive approach. Given the absence of prior studies employing DRL for the joint optimization of edge-and-vehicle scheduling quantities in HFL, no direct state-of-the-art DRL baseline exists for this specific problem formulation. Therefore, we benchmark AptEVS against representative static (FpEVS) and dynamic heuristic (PahEVS) methods. To further validate that the performance gains are intrinsic to our proposed mechanisms rather than the generic RL backbone, we conduct rigorous ablation studies. These studies isolate the contributions of the novel reward function and the phase-aware strategy, verifying their role in ensuring robust performance without relying on complex model-based assumptions.

To ensure the reproducibility and reliability of the reported performance, all experiments were conducted over five independent trials with distinct random seeds. In the accuracy plots, the solid curves represent the average accuracy, while the shaded regions reflect the fluctuation range across trials. For the bar charts, the bar height represents the mean value, and the black error bars explicitly denote the standard deviation.

As shown in Figure 6(a1–c1), AptEVS does not always require the fewest communication rounds to reach the target accuracy across different tasks. However, it consistently achieves the lowest long-term training cost, demonstrating superior efficiency under diverse task settings. As illustrated in Figure 6(a2–c2), AptEVS reduces the long-term training cost by up to 44.4%, 37.5%, and 29.2% compared with the static FpEVS policy for the MNIST, FMNIST, and CIFAR-10 tasks, respectively.

Interestingly, the dynamic PahEVS policy, despite its phase awareness, incurs an even higher long-term training cost than the static FpEVS. This counter-intuitive result highlights a critical weakness of heuristic-based scheduling, where PahEVS rigidly enforces a high participation rate (50%) in FTP, causing a surge in TDC and ECC that far outweighs the marginal gains in model accuracy. This underscores that coarse-grained adaptivity can be inefficient.

In contrast, by intelligently managing this trade-off, AptEVS achieves further cost reductions of 62.4%, 56.3%, and 66.0% compared to PahEVS on the same tasks. This superior efficiency stems from its adaptive scheduling policy, which dynamically balances the cost and benefits among ADC, TDC, and ECC. These results validate our core hypothesis: static or coarse-grained heuristic strategies are insufficient to address the heterogeneous and dynamic nature of VHFL. Instead, AptEVS confirms that fine-grained, cost-aware adaptivity is essential for true optimization in VHFL.

6.3. Ablation Study: Efficacy of the Reward Function Design

The design of the reward function is paramount as it directly guides the DRL agent toward the desired performance objectives. To rigorously validate the effectiveness of our proposed reward structure in AptEVS, we conduct a comprehensive ablation study. We compare the performance of our complete AptEVS algorithm against four distinct variants. Each variant preserves the core architecture of AptEVS but substitutes our reward function with an alternative design.

• AptEVS-A: Employs a reward based on the direct model accuracy $A_t$, as in [59]. The reward is defined as $r_t={\alpha }_1{\beta }_1{A}_t-{\alpha }_2{\beta }_{2}T\left(t\right)-{\alpha }_3{\beta }_{3}E\left(t\right))$.
• AptEVS-EA: Uses an exponential form of the accuracy ${\mathsf{\Xi }}^{A_t}$ as the primary reward component, following [44]. The reward is defined as $r_t={\alpha }_1{\beta }_1{\mathsf{\Xi }}^{A_t}-{\alpha }_2{\beta }_{2}T\left(t\right)-{\alpha }_3{\beta }_{3}E\left(t\right))$.
• AptEVS-ADC: Utilizes the ADC $C^{\mathrm{A}}\left(t\right)$ as the reward signal, a method used in [45]. The reward is defined as $r_t=-{\alpha }_1{\beta }_1{C}^{\mathrm{A}}\left(t\right)-{\alpha }_2{\beta }_{2}T\left(t\right)-{\alpha }_3{\beta }_{3}E\left(t\right)$.
• AptEVS-ACAR: A variant based on [57], which uses a weighted sum of the current accuracy and the change in accuracy from the previous round. Let ${\mu }_1$ and ${\mu }_2$ be weights such that ${\mu }_1+{\mu }_2=1$, and set $U^{\mathrm{A}}\left(t\right)={\mu }_1{A}_t+{\mu }_2(A_t-A_{t-1})$. The reward function is given by $r_t=\left\{\begin{array}{cc}{A}_t+r_1,\hfill & A_t<A_{t-1},\hfill \\ {\alpha }_1{\beta }_1{U}^{\mathrm{A}}\left(t\right)-{\alpha }_2{\beta }_{2}T\left(t\right)-{\alpha }_3{\beta }_{3}E\left(t\right),\hfill & A_t\ge A_{t-1}.\hfill \end{array}\right.$

As shown in Figure 7, AptEVS consistently achieves the lowest long-term training cost. This confirms our earlier analysis in Section 5.2: traditional rewards suffer from saturation, noise sensitivity, or limited discriminability. Our feedback signal, based on a short-term target accuracy anchor, provides more stable and informative learning guidance, enabling more efficient policy learning.

To further validate the effectiveness of our reward design, we observed the reward progression of AptEVS across edge rounds. Empirical results in Figure 8 show that the DRL-based policy quickly converges within the early stages of HFL training, less than 10% of the total training rounds across all evaluated tasks. This highlights that the exploration cost induced by our reward function remains limited, and that the learned policy is capable of generalizing effectively in later stages.

6.4. Ablation Study: Efficacy of the Phase-Aware Mechanism

To verify the impact of our proposed phase-aware mechanism, we conduct an ablation study comparing the full AptEVS framework against a degenerated version, AptEVS-vanilla, which removes this mechanism. As illustrated in Figure 9, the full AptEVS agent exhibits similar or even accelerated convergence while consistently achieving the lowest long-term training cost. This significant performance advantage stems from the phase-aware design. The ITP encourages structured exploration, allowing the agent to build a robust understanding of the environment and effectively prune the vast action space. Subsequently, the main MTP capitalizes on this knowledge, enabling focused policy refinement within a more promising region of the action space. This leads to a more stable and efficient learning process, culminating in a superior final policy.

6.5. Performance Robustness Across Diverse Environments

Having verified the core mechanisms, we now investigate how well AptEVS adapts under diverse and challenging environments. Rather than exhaustive benchmarks, we select representative scenarios to examine the behavior of the learned policy and its ability to navigate dynamic bottlenecks.

Table 6. Robustness evaluation of the proposed algorithm under systematic variations in key environmental parameters against a default configuration: edge spacing $d_{\mathrm{edge}}=300$ m, data partitioning parameter $\varphi =5$, arrival rate $\lambda =1.0$ veh/s, and speed characteristics $(\overline{v},{\sigma }_v)=(20,3)$ m/s. The best results are highlighted in bold.

Variable	Value	Method	Edge Rounds	ADC	TDC (s)	ECC (J)	Long-Term Training Cost
$d_{\mathrm{edge}}$ (m)	200	AptEVS (Ours)	$428\pm 10$	$53.1\pm 6.1$	$521.9\pm 11.2$	$\mathbf{8677.0}\pm \mathbf{1992.0}$	$7895.8\pm 798.7$
		FpEVS	$403\pm 8$	$20.0\pm 0.2$	$\mathbf{486.6}\pm \mathbf{10.6}$	10,918.3 $\pm 219.5$	$\mathbf{7318.1}\pm \mathbf{148.3}$
		PahEVS	$\mathbf{352}\pm \mathbf{6}$	$\mathbf{16.9}\pm \mathbf{0.2}$	$512.2\pm 10.9$	17,423.3 $\pm 458.9$	10,686.0 $\pm 375.3$
	400	AptEVS (Ours)	$395\pm 19$	$68.7\pm 8.6$	$\mathbf{523.8}\pm \mathbf{21.8}$	$\mathbf{6663.3}\pm \mathbf{709.3}$	$\mathbf{7313.5}\pm \mathbf{113.8}$
		FpEVS	$387\pm 8$	$17.4\pm 0.2$	$624.6\pm 12.3$	20,026.9 $\pm 315.3$	13,481.3 $\pm 229.5$
		PahEVS	$\mathbf{345}\pm \mathbf{11}$	$\mathbf{15.5}\pm \mathbf{0.2}$	$536.6\pm 18.8$	17,663.0 $\pm 799.6$	10,739.0 $\pm 310.2$
$\lambda$ (veh/s)	$0.6$	AptEVS (Ours)	$483\pm 9$	$71.4\pm 5.4$	$597.3\pm 23.4$	$\mathbf{9222.9}\pm \mathbf{1735.2}$	$8635.2\pm 1113.4$
		FpEVS	$425\pm 22$	$21.9\pm 0.4$	$\mathbf{506.3}\pm \mathbf{23.8}$	10,281.2 $\pm 383.3$	$\mathbf{6350.7}\pm \mathbf{252.1}$
		PahEVS	$\mathbf{402}\pm \mathbf{9}$	$\mathbf{18.3}\pm \mathbf{0.1}$	$619.5\pm 16.0$	21,030.0 $\pm 695.3$	12,761.4 $\pm 381.3$
	$1.4$	AptEVS (Ours)	$357\pm 19$	$49.8\pm 4.3$	$\mathbf{439.8}\pm \mathbf{21.4}$	$\mathbf{5200.0}\pm \mathbf{977.5}$	$\mathbf{5832.4}\pm \mathbf{455.1}$
		FpEVS	$\mathbf{277}\pm \mathbf{6}$	$\mathbf{13.4}\pm \mathbf{0.0}$	$461.9\pm 10.0$	16,362.1 $\pm 391.7$	10,761.3 $\pm 280.8$
		PahEVS	$320\pm 0$	$14.9\pm 0.2$	$478.3\pm 0.7$	15,710.2 $\pm 22.7$	$9499.8\pm 104.8$
$(\overline{v},{\sigma }_v)$ (m/s)	$(10,3)$	AptEVS (Ours)	$427\pm 18$	$75.8\pm 15.1$	$543.0\pm 17.6$	$\mathbf{8718.7}\pm \mathbf{2412.4}$	$\mathbf{8382.5}\pm \mathbf{1153.9}$
		FpEVS	$383\pm 5$	$16.8\pm 0.2$	$742.3\pm 11.9$	33,054.9 $\pm 498.5$	22,237.3 $\pm 508.0$
		PahEVS	$\mathbf{335}\pm \mathbf{4}$	$\mathbf{15.6}\pm \mathbf{0.1}$	$\mathbf{503.0}\pm \mathbf{8.2}$	16,784.4 $\pm 287.6$	10,263.2 $\pm 168.4$
	$(20,1)$	AptEVS (Ours)	$373\pm 6$	$64.7\pm 8.5$	$465.9\pm 27.3$	$\mathbf{5774.5}\pm \mathbf{484.4}$	$\mathbf{6500.9}\pm \mathbf{449.4}$
		FpEVS	$\mathbf{283}\pm \mathbf{2}$	$\mathbf{14.8}\pm \mathbf{0.3}$	$\mathbf{393.0}\pm \mathbf{4.6}$	11,331.6 $\pm 147.2$	$7352.6\pm 85.9$
		PahEVS	$345\pm 4$	$15.4\pm 0.1$	$519.7\pm 5.7$	17,656.7 $\pm 378.6$	10,718.1 $\pm 189.2$
	$(20,3)$	AptEVS (Ours)	$391\pm 29$	$57.5\pm 0.8$	$475.0\pm 55.0$	$\mathbf{5100.0}\pm \mathbf{1100.6}$	$\mathbf{6163.7}\pm \mathbf{442.5}$
		FpEVS	$\mathbf{298}\pm \mathbf{5}$	$\mathbf{14.6}\pm \mathbf{0.1}$	$\mathbf{416.7}\pm \mathbf{6.0}$	12,131.6 $\pm 197.3$	$7825.1\pm 116.9$
		PahEVS	$335\pm 12$	$15.9\pm 0.2$	$502.6\pm 22.4$	16,843.9 $\pm 825.7$	10,289.0 $\pm 478.6$
	$(20,5)$	AptEVS (Ours)	$411\pm 46$	$70.0\pm 5.3$	$521.3\pm 52.8$	$\mathbf{8106.6}\pm \mathbf{1333.1}$	$\mathbf{7813.3}\pm \mathbf{800.0}$
		FpEVS	$367\pm 2$	$17.9\pm 0.1$	$511.2\pm 3.9$	14,975.8 $\pm 99.2$	$9632.3\pm 61.3$
		PahEVS	$\mathbf{336.7}\pm \mathbf{8}$	$\mathbf{15.9}\pm \mathbf{0.2}$	$\mathbf{507.0}\pm \mathbf{15.2}$	16,897.1 $\pm 719.5$	10,339.3 $\pm 383.6$
	$(30,3)$	AptEVS (Ours)	$369\pm 26$	$60.2\pm 5.5$	$458.6\pm 36.2$	$\mathbf{5247.8}\pm \mathbf{1169.6}$	$6155.5\pm 478.5$
		FpEVS	$\mathbf{310}\pm \mathbf{15}$	$16.9\pm 0.2$	$\mathbf{380.6}\pm \mathbf{17.8}$	$8248.9\pm 395.8$	$\mathbf{5626.4}\pm \mathbf{258.4}$
		PahEVS	$345\pm 7$	$\mathbf{16.8}\pm \mathbf{0.2}$	$518.7\pm 11.9$	17,108.7 $\pm 442.6$	10,513.5 $\pm 256.7$

summarizes the long-term training cost across different edge spacings and mobility patterns. A striking observation is that AptEVS learns efficient scheduling strategies from scratch in the vast majority of scenarios, including unfamiliar environments, underscoring its robustness. In specific circumstances, the fixed configuration of FpEVS happens to coincidentally align with the near-optimal operating point by chance. In contrast, AptEVS is an online learning agent. This inherent exploration overhead causes a slight performance cost. Therefore, the zero-overhead FpEVS yields a slightly lower cost. However, this is merely situational. AptEVS achieves superior performance in the vast majority of scenarios. Its ability to remain effective globally, despite being slightly suboptimal in rare specific cases, strongly validates the robustness of the proposed method.

More importantly, the results reveal how AptEVS learns to dynamically adapt to diverse performance bottlenecks. First, the agent learns to modulate its EVS configuration in response to edge spacings, strategically balancing the handover-induced straggler risks of small edge spacings against the vehicle edge communication inefficiency of large spacings. Second, it learns to manage the trade-off between data sparsity at low arrival rates and resource congestion at high arrival rates. Finally, the agent demonstrates a sophisticated understanding of the duality of vehicle speed. It learns to counter the high instability risk of fast-moving environments by adopting a conservative, small-EVS policy to control the TDC. Concurrently, it learns to recognize and exploit the speed standard deviation ${\sigma }_v$ as a data diversity amplifier. The results show that when ${\sigma }_v$ is high, the agent strategically reduces the EVS size. This indicates it has learned that the increased vehicle mixing naturally helps satisfy the ADC requirement, freeing the agent to select a smaller, more resource-efficient configuration that minimizes TDC and ECC.

Adaptability to Diverse Preference Weights: It is worth noting that the weighting factors in Equation (28) embody the diverse service requirements of vehicular tasks. A key advantage of the proposed AptEVS framework is its inherent preference-aware adaptability. Since the DRL agent updates its action value estimates to derive a policy that maximizes the cumulative reward derived directly from these weighted costs, changes in weighting factors automatically reshape the reward landscape. Consequently, the agent dynamically adjusts its learned policy to align with the new weights without requiring modification to the algorithm’s structure. This ensures that AptEVS functions as a robust, general-purpose solver capable of satisfying varying task objectives.

6.6. Overhead Analysis

The cloud servers typically possess high computational capabilities, with an aggregate processing frequency equivalent to 50 GHz. The computational complexity of the DQN model in AptEVS is approximately ${10}^4$ cycles/sample. During the DQN training phase, whenever an HFL cloud round is completed, experience samples of $b_{\mathrm{DQN}}=16$ are used to train the DQN model, resulting in a runtime of approximately 3 µs. Additionally, the time delay taken by the DQN model for decision-making action is approximately 1 µs, so the total additional time overhead for AptEVS is only 4 µs. Given that the execution time for each cloud round in HFL is approximately 1 s, the time overhead introduced by AptEVS is virtually negligible. Due to the minimal computation time required, the corresponding energy consumption overhead is also minimal, exerting an insignificant impact on the overall system performance.

The state information required by AptEVS consists of low-dimensional scalars. The total size of this metadata is negligible compared to the high-dimensional model parameters exchanged in HFL. Therefore, the bandwidth consumed by state monitoring is insignificant. Additionally, processing this state information primarily involves calculating means and standard deviations. These basic statistical operations incur minimal CPU load on the powerful cloud server compared to the complex gradient calculations in HFL.

7. Conclusions

This paper investigated the critical problem of adaptive EVS in VHFL systems, where static strategies fall short due to the interplay between diverse task-specific requirements and dynamic vehicular environments. To address this, we proposed a unified long-term training cost metric that jointly captures ADC, TDC, and ECC, providing a principled basis for evaluating performance under diverse tasks. Our experimental analysis demonstrated that the optimal EVS configuration is not fixed but evolves dynamically with task complexity, environmental conditions, and training phase progression. Based on this observation, we introduced AptEVS, a DRL-based framework that learns optimal EVS configurations online. AptEVS integrates a lightweight training phase detection mechanism with a phase-aware scheduling strategy, employing structured exploration in the ITP and priority-enhanced DQN in the MTP. Extensive simulations verified that AptEVS consistently derives high-performance scheduling policies from scratch, achieving significant reductions in long-term training cost compared with the baselines.

Future work may focus on improving sample efficiency for faster convergence, potentially through transfer learning by augmenting the DRL state space with model structural information. This approach aims to facilitate robust adaptation across varying tasks and dynamic environments. In conclusion, this study demonstrates the feasibility and benefits of phase-aware DRL-based scheduling, offering a promising direction for efficient and robust HFL in realistic vehicular networks.