Performance Analysis of Unmanned Aerial Vehicle-Assisted and Federated Learning-Based 6G Cellular Vehicle-to-Everything Communication Networks

Highlights

What are the main findings?

• The paper explores the variations in queuing and processing delays in a UAV-assisted C-V2X communications environment, with a focus on the international mobile telecommunication (IMT)-2030-based UAV-assisted 6G C-V2X communications.
• The queuing delay and processing delay are compared in a federated learning scenario and a gross data offloading scenario.

What is the implication of the main finding?

• For gross data offloading, we consider event-triggered cooperative perception messages (CPMs) and periodically transmitted basic safety messages (BSMs). The implication of the main finding is that the delay in FL scenario is less than that in the gross data offloading scenario.
• The queuing stability is ensured in the problem formulation.

Abstract

The paradigm of cellular vehicle-to-everything (C-V2X) communications assisted by unmanned aerial vehicles (UAVs) is poised to revolutionize the future of sixth-generation (6G) intelligent transportation systems, as outlined by the international mobile telecommunication (IMT)-2030 vision. This integration of UAV-assisted C-V2X communications is set to enhance mobility and connectivity, creating a smarter and reliable autonomous transportation landscape. The UAV-assisted C-V2X networks enable hyper-reliable and low-latency vehicular communications for 6G applications including augmented reality, immersive reality and virtual reality, real-time holographic mapping support, and futuristic infotainment services. This paper presents a Markov chain model to study a third-generation partnership project (3GPP)-specified C-V2X network communicating with a flying UAV for task offloading in a Federated Learning (FL) environment. We evaluate the impact of various factors such as model update frequency, queue backlog, and UAV energy consumption on different types of communication latency. Additionally, we examine the end-to-end latency in the FL environment against the latency in conventional data offloading. This is achieved by considering cooperative perception messages (CPMs) that are triggered by random events and basic safety messages (BSMs) that are periodically transmitted. Simulation results demonstrate that optimizing the transmission intervals results in a lower average delay. Also, for both scenarios, the optimal policy aims to optimize the available UAV energy consumption, minimize the cumulative queuing backlog, and maximize the UAV’s available battery power utilization. We also find that the queuing delay can be controlled by adjusting the optimal policy and the value function in the relative value iteration (RVI). Moreover, the communication latency in an FL environment is comparable to that in the gross data offloading environment based on Kullback–Leibler (KL) divergence.

1. Introduction

Low-latency in vehicular communications is a critical requirement to enable sixth-generation (6G) applications comprising holography-assisted real-time autonomous driving, platooning, and futuristic infotainment services [1]. In the paradigm of 6G cellular vehicle-to-everything (C-V2X) communications, the sojourn time, which is the sum of delays on each link traversed by a packet, must be in the range of microseconds to milliseconds [2]. Enhanced data rates and traffic volume place additional constraints on sojourn time and transmission window size [3]. In addition, the different types of data packets pertaining to augmented reality (AR), immersive reality, virtual reality (VR), and autonomous vehicle’s navigation based on real-time holographic maps add to the processing delay. These variations also contribute to delays at the link level, consequently leading to an increase in sojourn time [4]. The high heterogeneity of packets and non-independently and non-identically distributed (non-i.i.d.) contents demand resilient and adaptive network response to mitigate delay [5].

Unmanned aerial vehicles (UAVs) are revolutionizing vehicular communication by serving as flying platforms that enable low-latency connectivity through advanced embedded aerial computing equipment [6]. UAV-assisted C-V2X communications integrate vehicles’ data sensing and communication with computing capabilities at the UAV [7]. Moreover, according to recent specifications in the international mobile telecommunication (IMT)-2030, integration of UAVs in C-V2X communications is expected to enhance coverage, reliability, latency, and energy efficiency in next-generation intelligent transportation systems (ITSs) [8]. Recent works have demonstrated that UAVs can serve as mobile infrastructure in drone-assisted vehicular networks (DAVNs) where deploying physical roadside infrastructure is either not feasible or inferior in performance [9]. However, their limited battery power significantly impacts the utility of UAVs in DAVN [10]. Furthermore, it is challenging to maximize the UAV coverage range, minimize data loss, and ensure low latency [11].

Figure 1 illustrates the uplink scenario of the UAV-assisted C-V2X communication model being studied, focusing on queuing and processing delays in uplink and downlink. It examines the implications of these delays on overall latency, throughput, and power optimization. By analyzing the effects of these delays on both power consumption and throughput, we evaluate how communication and power usage can be optimized in dynamic UAV-assisted vehicular networks [12]. This is achieved while addressing the stringent requirements of 6G applications within a federated learning environment. Here, each vehicle can transmit only within the transmit time interval (TTI) to the UAV. The packets are temporarily buffered in the in-vehicle queue, where they await scheduling and transmission over the communication channel [12]. Note that in a federated learning (FL) scenario, these data consist of processed local model parameters [13]. In gross data offloading scenario, the collected sensor data are buffered and directly transmitted to the UAV [14]. At the UAV, the incoming packets are queued until the embedded parameter server (PS) is available. This contributes to a second queuing delay at the UAV [15]. Furthermore, the FL scenario introduces additional processing delays due to the computation of global model parameters, which require aggregation and synchronization of locally trained models from a number of vehicles that gather non-i.i.d. data [12].

Conversely, in the traditional gross data offloading scenario, the processing delay (($D_{pr}$)) arises from handling and analyzing the vehicle’s raw sensor data at the PS embedded in a UAV [16]. Both FL and gross data offloading scenarios impose distinct computational overheads that critically impact the overall latency [17]. During a transmission window, the vehicle selects resources by identifying all potential single-subframe resources (CSRs) available in the current subframe. The availability of these CSRs is limited to the transmission windows when the UAV maintains a line-of-sight (LoS) connection with the vehicle. This ensures optimal quality of service (QoS) and reliable communication. Additionally, system performance varies with the UAV’s mobility, trajectory, and positioning, which significantly affect uplink transmission efficiency and interference [18]. According to the 3GPP standard, the transmission window for C-V2X communications is upper-bounded by a maximum latency constraint and a CSR should be selected within that window [19]. An upper bound on the maximum tolerable latency guarantees that all data packets are transmitted within a pre-specified time frame. This is essential for ensuring the timely delivery of information, which is crucial for the safety and effectiveness of real-time vehicular applications. The total delay, denoted by $\mathcal{D}$, is defined as the sum of two components, the queuing delay (${\mathcal{D}}_{que}$) and the processing delay (${\mathcal{D}}_{pr}$), as in Equation (1). Note that $\mathcal{D}$ varies with a number of system-level and network-level factors. Specifically, $\mathcal{D}$ varies with the following:

• Different classes of packets such as control data, signaling data, or raw sensor information packets that are transmitted with different priorities, resulting in varying queuing and processing delays [12].
• Larger packets require more time for processing and transmission, which directly increases both ${\mathcal{D}}_{\mathrm{proc}}$ and the waiting time in queues [20].
• The scheduling policy, window size, and channel access mechanisms that impact the duration for which a packet must wait in a queue before being transmitted, thus impacting ${\mathcal{D}}_{\mathrm{queue}}$.
• The underlying edge processing infrastructure at the vehicles, the processing server embedded in a UAV, the available bandwidth, queuing policies, and congestion significantly influence both queuing delay and processing delay [21].

$$\mathcal{D}={\mathcal{D}}_{que}+{\mathcal{D}}_{pr},$$

(1)

where the queuing delay (${\mathcal{D}}_{que}$) and the processing delay (${\mathcal{D}}_{pr}$) considered in this paper, which together constitute the delay ($\mathcal{D}$), are described as follows:

• Queuing delay (${\mathcal{D}}_{que}$): This is defined as the total time the packets spend in a queue at the UAV waiting to be transmitted to the PS plus the time packets spend waiting at the UAV buffer to be processed by the PS [3].
• Processing delay (${\mathcal{D}}_{pr}$): This represents the time taken to optimally partition a packet and process all the packets at the PS [22]. It includes the time taken at the UAV to compute the global model parameters under the gross data offloading scenario.

1.1. Contributions

In this paper, to resemble a real-world situation, we assume that the transmission and propagation delay are an additional 10% of the total delay ($\mathcal{D}$) by virtue of fifth-generation (5G) communication infrastructure. In existing approaches, Federated Averaging (FedAvg) functions as a collaborative learning algorithm where each participating vehicle performs local model training using stochastic gradient descent (SGD) by averaging the local model parameters [23]. Here, each vehicle’s contribution is proportional to its dataset size. Measuring the contribution of each vehicle based on local sensor data is statistically challenging [24]. Therefore, to effectively capture and analyze the delays caused by local model processing at each vehicle and the subsequent communication of model updates to the UAV, the iterations of the FL training procedure are modeled using Markov kernels [25].

The sequence of packet transmissions is represented as state transitions, where each state corresponds to a specific iteration, along with the associated local model computation time, channel conditions, and transmission scheduling. The evolution of the vehicles’ local model parameters in Federated Learning (FL) is characterized by a Markov chain framework where each state of the local model represents its parameter configuration at a particular iteration. The transitions between successive states are governed by a transition probability matrix [26]. Therefore, our objective is to analyze the delay scenario in an FL environment using a Markov decision process. The present work builds upon and extends the findings reported in our previous study where we formulated FL as a Markov decision problem and optimized the model convergence rate.

Moreover, leveraging the collective perception messages (CPM) and basic safety messages (BSM) defined by the Society of Automotive Engineers (SAE), we analyze how a UAV’s average energy consumption, measured in Joules/second, varies with the vehicle velocity, road length ($R_L$), and the number of vehicles (V) [18]. In addition, we assume an ideal and fair decision-making process by the UAV when determining a subset of the vehicles for communication and resource allocation in each TTI. Lastly, trajectory optimization and UAV localization and mapping to serve the maximum number of vehicles are critical to ensure optimal resource allocation and UAV power consumption. Here, we assume that the UAV follows an elliptical trajectory, ensuring that it consistently remains within the coverage zone of the vehicles and avoids drifting outside the communication range. The principal contributions of this manuscript are as follows:

• We investigate the balance between latency and utility when a UAV selects vehicles during each FedAvg iteration. This latency is benchmarked against the delay encountered in bulk data offloading scenarios.
• We analyze and investigate the interplay between latency and UAV energy consumption, examining how model convergence rate, delay variability, data sample size, and packet size interrelate. Additionally, we explore the failure rate in adhering to service time constraints relative to packet delay and UAV energy usage.
• We assess the frequency of constraint breaches as vehicle numbers rise within a realistic UAV-supported C-V2X communication framework. Consequently, we determine the optimal vehicle count that a UAV can accommodate without surpassing these constraints.

1.2. Organization

The rest of the paper is organized as follows. Section 2 provides a discussion of existing approaches to minimize delay in UAV-assisted vehicular communication. Section 3 introduces our system model along with the problem formulation. The problem formulation takes into account the limited battery capacity of UAVs, the varying data types, and the need to handle non-i.i.d. traffic across multiple vehicles. This section focuses on the optimization techniques employed to address the mixed-integer nonlinear programming problem which arises from the binary scheduling variables and the nonlinear constraints associated with UAV-assisted C-V2X communications. Section 4 discusses the solution approach. Section 5 discusses the results and compares the observed latency for varying packet lengths for varying arrival rates ($\lambda$) and service rates ($\mu$). We compare the results for both FL and conventional gross data offloading scenarios. Finally, Section 6 concludes the paper and proposes some avenues for future research work.

2. Related Work

Unmanned aerial vehicles (UAVs) are used in wireless communications as flying base stations to offer wireless network coverage in areas where deploying a conventional ground infrastructure is infeasible [27]. UAVs are also being deployed to offload traffic from terrestrial networks to function as relay nodes between users and base stations [28]. The UAVs enhance communication range in remote or dense urban areas by creating line-of-sight (LoS) links for wireless backhaul [29]. In the IMT-2030 6G vision, networks of UAVs are emerging as a key technology for achieving low-latency and massive machine-type communications. UAVs facilitate edge computing and content delivery in smart cities, as well as in AR and VR applications [27].

In vehicular applications, there are critical latency and reliability requirements for the vehicular communication packets transmitted frequently to a UAV [30]. Due to varying velocities of vehicles and UAVs and to minimize transmission delays, sensing-based semi-persistent scheduling (SPS) enables vehicles to sense prior transmissions from neighboring vehicles, hence reducing the likelihood of packet collisions [9]. In this approach, a state machine and queuing model represent the SPS mechanism, and the model comprises four parallel discrete-time Markov chains (DTMCs) where the first DTMC represents an SPS algorithm for selecting radio resources. The other two DTMCs represent a packet generator, and the last DTMC models the packet queues [31]. In other approaches, hidden Markov models are utilized for traffic generation, and these models are resolved iteratively. Steady-state probabilities are employed to assess key performance metrics, such as the probability of successful transmission and task offloading. These studies examined a scenario in which vehicles needed to adjust their data transmission rates based on feedback from other vehicles. Additionally, random packet arrivals contribute to increased delays [22]. An adaptive window-based transmission control system can maintain a consistent data transmission rate by setting a maximum limit on outgoing packets [32].

Furthermore, machine learning techniques have been used to model adaptive variations in transmission window between multiple vehicles [12]. Packet arrival and queuing statistics have been aggregated intelligently to reduce sojourn time by combining packet arrival and queuing statistics [22]. Optimization of queue lengths and wait times has been used to minimize delay in C-V2X networks [19]. Convex optimization techniques, such as interior-point methods, bisection search, and golden-section search, have been extensively used to address the server utilization problem [33]. However, the problem of PS utilization in FL-based C-V2X communications is inherently convex when the packet queuing policy relies on a time-based scheduling algorithm. Centralized solutions are not suitable for these networks due to the high heterogeneity and variability in the data packets [12].

Decentralized machine learning approaches in 6G vehicular networks aim at processing the data at the vehicles where they are generated [34]. Among various decentralized machine learning approaches to mitigate delay in C-V2X communication, an FL model is trained using SGD and the local update is transmitted to a central PS [22]. Using FedAvg, the PS selects a random subset of vehicles to broadcast the shared global model. Each vehicle performs SGD on its local data, transmitting the update to the PS, which aggregates the local model updates to construct an updated global model [35]. Repeated aggregation of local models from the same vehicle disproportionately affects the global model, diminishing the effectiveness of previously learned information. Recent approaches have introduced self-organizing and FL frameworks to improve scalability and autonomy in large-scale, distributed UAV-assisted vehicular communication networks [22].

Although a UAV can transmit data to multiple vehicles simultaneously, the scheduling policies prevent multiple vehicles from transmitting data back to the UAV at the same time [36]. Scheduled time slots notify vehicles of when data will be transmitted or received to obtain a time–frequency schedule for allocating transmission resources [27]. Model parameter transmission and scheduling are handled by the server, which is responsible for synchronizing data before scheduling the next transmission window [32]. Some FL approaches assume that communication links are perfect and do not take into account packet loss. However, in reality, packets are re-transmitted when there are link failures. Since the underlying constraints depend on the communication channel and are difficult to estimate during training, it is essential to optimize both the UAV trajectory and power usage to reduce delays effectively [37].

Some works have emphasized the spatio-temporal correlation in vehicular data transmitted at different intervals [38]. There is a temporal correlation when a vehicle transmits data over several consecutive transmission windows. In terms of spatial correlation, data from different vehicles become correlated due to their geographical proximity, resulting in similar data patterns among vehicles within the same cluster [39]. To enhance the efficiency of collaborative training and minimize communication delays, a DTMC-based model considers the entire network topology. The proposed approach is implemented using a MAC protocol with a time-slotted channel-hopping scheduling policy [25]. In existing FL approaches, the number of samples and data sensed by each vehicle and the extent to which a local model deviates from the global model are crucial for ensuring convergence in distributed learning and are not taken into account. In such scenarios, some vehicles with fewer sensors receive smaller weights in the aggregation, and the vehicles with the fewest data may be excluded [40].

Table 1. Summary of existing works, identification of gaps in performance analysis of UAV-assisted and FL-based 6G C-V2X communication networks, methodologies used, and proposed solution approach to address these gaps.

Reference	Key Problem Addressed	Methodology Used	Existing Gaps	Our Proposed Solution
[27,28,29,30]	UAVs are deployed as aerial base stations and wireless network relays for vehicular networks to achieve low-latency and reliable communication.	Use of UAVs to establish LoS links, edge computing, and traffic offloading in 6G-enabled vehicular networks and automotive applications.	High mobility of vehicles and UAVs leads to Doppler spreads, packet collisions, scheduling constraints, latency, and reliability issues.	Integrate UAV trajectory optimization with adaptive scheduling policies for latency reduction and reliable communication with vehicles in FL environment.
[9,31,32]	Minimizing packet collisions and transmission delays in UAV–vehicle communication networks.	State machine and queuing models with multiple DTMCs, hidden Markov models, and adaptive window-based transmission control.	The iterative solutions increase processing delay, random arrivals add to latency, optimization of adaptive window size is a constraint.	Propose FL-enabled dynamic packet scheduling combining DTMC with adaptive feedback.
[12,19,33]	Queue-length and queue-backlog optimization; sojourn time minimization in C-V2X networks.	Machine learning-based adaptive transmission window-size adjustment, convex optimization techniques such as interior-point, bisection search and golden-section search.	Centralized convex optimization approaches are not scalable to larger networks due to heterogeneous and non-i.i.d packets; limited application of time-bound scheduling approaches.	Design decentralized FL-based queue optimization integrated with UAV-assisted C-V2X communication; the proposed approach is verified for scalability.
[34,35]	Delay reduction in FL-enabled vehicular communications.	Decentralized FL with SGD and FedAvg aggregation; self-organizing learning for scalability.	FedAvg leads to bias from repeated updates, data heterogeneity is not considered which leads to sub-optimal global model.	Introduce weighted and fairness-aware aggregation with UAV-assisted scheduling to balance non-i.i.d and heterogeneous data distributions.
[27,32,36,37]	Task scheduling and resource allocation in UAV-vehicular communications using FL.	Time–frequency scheduling policies, model synchronization at a centralized server, UAV trajectory and power optimization.	Assumption of perfect channels; minimal retransmissions; UAV scheduling conflicts for vehicles with simultaneous transmissions.	Joint UAV trajectory, power allocation, and distributed scheduling using stochastic optimization approaches.
[25,38,39,40]	Correlation and heterogeneity in vehicular sensing data for FL convergence.	Spatio-temporal correlation models, DTMC-based topology-aware scheduling, MAC protocol with time-slotted hopping.	Vehicles with fewer sensors are excluded from participating in FL, unequal weight assignment in aggregation, optimal convergence not guaranteed.	Propose correlation-aware FL aggregation with UAV-assisted clustering to ensure fairness and improve FL model convergence.

summarizes existing works and identifies gaps in the performance analysis of unmanned aerial vehicle-assisted and federated learning-based 6G cellular vehicle-to-everything communication networks, along with the methodologies used and our proposed solution approach to address these gaps. In Table 1, we identify specific limitations in existing works, including limited scalability in dynamic UAV networks, insufficient incorporation of heterogeneous data distributions, and latency models that do not consider queuing dynamics. Our proposed framework addresses these challenges in several ways. First, we utilize a queuing architecture that captures both communication and computation delays. Second, we adapt the FL process to account for UAV mobility and variable LoS connectivity with vehicles. Finally, we propose an optimization strategy to minimize the end-to-end latency while preserving the UAV’s battery power consumption. Table 1 highlights our contributions, clarifies the research gap we address, and underscores the novelty and practical relevance of our proposed framework. Additionally, we acknowledge that most data processed at the in-vehicle edge server or UAV are generated by the vehicle’s sensors. Since vehicles are equipped with different types of sensors, this leads to data that are not independently and identically distributed (non-i.i.d.). Our investigation into UAV-assisted C-V2X communications with non-i.i.d. data distribution represents a novel contribution of our work.

3. System Model and Problem Formulation

3.1. UAV–Vehicle Queuing Model

Figure 2 illustrates our proposed communication architecture where a cluster ($\mathcal{C}$) consists of $\mathbf{V}$ vehicles represented as $\{{\mathbf{v}}_1$, …, ${\mathbf{v}}_n\}$. The sensor data are processed at an in-vehicle edge server (ES), and a local model is generated, while the global model is aggregated and maintained at the UAV. Upon the UAV’s request, an ES transmits its processed local model to the UAV. Subsequently, the UAV identifies a subset of $\mathbf{V}$ vehicles from the set $\{{\mathbf{v}}_1$, …, ${\mathbf{v}}_n\}$ for further processing in each transmission window ($L_w$). Upon arrival at the UAV, packets are either aggregated instantaneously or buffered in a queue for subsequent processing. To prevent the UAV from moving beyond the communication range of the vehicles, we restrict the UAV’s path to an elliptical path, as illustrated in Figure 2 and motivated by the findings in [41,42,43]. The UAV trajectory spans a spatial distribution of points that follow a Poisson distribution in the elliptical region at a predefined height ($\mathcal{H}$). The UAV’s capacity to serve vehicles is also partly determined by the available UAV transmission power ($P_t$) and $\mathcal{H}$.

For the in-vehicle ES, we assume multi-core central processing units (CPUs) with limited computing and bandwidth capabilities. A roadside unit (RSU) enables communication between vehicles and UAV, where $\mathbf{V}$ vehicles can offload the sensor data packets to up to N RSUs and one macro base station (MBS). Each offloading operation comprises the transmission of the FL model or gross data in the form of BSMs and CPMs. In this paper, we assume an embedded parameter server (PS) in the UAV that processes vehicular information and provides real-time driving assistance. We then study the impact of UAV’s available battery power, buffer size, packet arrival and departure rates, and transmission frequency on end-to-end latency between the vehicle and the UAV [44]. Then, we compare this communication performance under two different scenarios:

(i).

Assuming gross data offloading from vehicles to the UAV.

(ii).

Considering FL where non-i.i.d. sensor data are processed at the in-vehicle ES, and subsequently, only the completed model parameter updates are communicated to the UAV.

To model, analyze, and minimize delay, we utilize the $MMk$ queuing architecture to characterize packet flow and the packet arrival process to the queue [26]. A packet is allocated a radio resource by utilizing the steady-state probability (${\pi }_0$) of the waiting time, defined under the assumption of equiprobable states.

Table 2. Main parameters and symbols definition.

Symbol	Definition
$\mathcal{D}$	Total delay
${\mathcal{D}}_{que}$	Queuing delay
${\mathcal{D}}_{pr}$	Processing delay
${\mathit{\theta }}_t$	FL model parameters; impact the communication delay and bandwidth usage
${\mathit{\varphi }}_t$	Weight parameters: matrices that determine the behavior of the FL model
$L_w$	Transmission window length: time duration allocated for data transmission
${\tau }_u$	Uplink time for vehicles
${\tau }_d$	Downlink time for vehicles to receive the updated global model from UAV
${\mathcal{P}}_u$	UAV transmission power; impacts communication range and energy consumption
$\beta$	Predefined queuing probability to analyze system behavior under different traffic loads
${\theta }_v^i$	Hyper-parameters from vehicle v: size and frequency of updates to the UAV
${\mathit{q}}_i$	Spatial and temporal data describing UAV trajectory coordinates: position and movement
${\varsigma }_{i,j}^{\left(k\right)}\left(t\right)$	Status indicator of a TTI
${\rho }_i$	UAV’s server utilization; implies load balancing for QoS management and task scheduling
${\lambda }_i$	Packet arrival rate; impacts queue stability and latency in UAV-assisted C-V2X links
$k_i$	Number of edge servers that perform real-time data processing at vehicles to generate local models
${\mu }_i$	Packet departure rate; ensures data flow, low-latency, buffer management, congestion control
$X^{\left(k\right)},Y^{\left(k\right)}$	Random variables for latency analysis; represent uncertainties and variations in delay
${\pi }_0$	Steady-state probability: long-term probability that an agent is in a particular state
$\delta , b_{\mathrm{s}}, B_{\mathrm{s}}, {\Delta }_{\mathrm{s}}$	UAV occupancy: state transition parameters
${\pi }^{\ast }\left(s\right)$	Optimal policy that defines the best set of actions for the vehicles and UAV
$V_{\pi }\left(s\right)$	RVI value function; implies the expected reward of being in a particular state
$\mathbb{E}({\delta }^{\left(t\right)}\mid s^{\left(0\right)})$	Expectation with respect to an action
${\pi }_m$	Steady state probability that m ESs are occupied
${\pi }_{S_{\tau }}$	Mean sojourn time of each packet in state $s_i$: average time packets spend in a state
$P_Q^i$	Queuing probability
${\mathbb{R}}_{0,1}$	A randomly generated real number from 0-1
$f\{\lambda ,\mu \}$	A function to analyze latency
${\left[\delta \right]}_{\Delta }$	${\mathcal{D}}_{pr}$ accumulated after ${\mathcal{D}}_{que}$
$P\left(s^{\left(t\right)}\mid s,a\right)$	Probability of UAV’s state transitions
$\psi$	A negligible delay incurred while the UAV remains idle
$\left\{X\right\}$	Markov chain describing stochastic model of initial states of ES
$\left\{Y\right\}$	Markov chain describing stochastic model of initial states of UAV
$\sigma$	Variance in data slice to train a client; implies data diversity spanning different scenarios
$\zeta$	Ratio of samples in adjacent data slices; implies data distribution across vehicles in proximity

provides a list of key symbols and parameters used in this manuscript.

3.2. Problem Formulation

In this paper, by formulating the problem of delay minimization as a Markov decision process (MDP) and applying relative value iteration (RVI), we determine the overall delay in UAV-assisted C-V2X communication [45]. In the FL scenario, the process of transmitting local model parameters is modeled as a series of independent and mutually exclusive trials. Each trial leads to one of the following outcomes: either the model hyper-parameters are successfully transmitted to the UAV or they remain in the queue. The outcome of one transmission window is statistically independent of the outcomes in subsequent transmission windows. In contrast, the gross data offloading scenario, which involves BSMs and CPMs, is influenced by previous outcomes and states, impacting the future delay across the state space [15].

We aim to minimize the overall delay in a transmission window ($L_w$) with respect to packet priority and link duration. We also attempt to find the optimal number of vehicles that can be realistically served by a UAV subject to the constraints of its available battery power. The above MDP formulated in ($\mathit{P}1$) is expressed as a bin packing problem (BPP) for optimization [46]. At the in-vehicle ES, we account for both uplink and downlink capacities. The problem ($\mathit{P}1$) is a multi-objective optimization problem [15]. This is shown in Equation (2) as a weighted sum of delay and power components:

$$\begin{array}{cc}\hfill \mathit{P}1:& \underset{{\mathit{\theta }}_t,{\mathit{\varphi }}_t,{\mathcal{D}}_t}{\min }\{w_1\underset{S{P}_1}{\underbrace{\left[T{\tau }_u+\sum _{t=1}^T\min \left(\underset{v\in \mathcal{V}}{\max }\left(d_t\right),{\tau }_d\right)\right]}}+\hfill \\ & w_2\underset{S{P}_2}{\underbrace{\left[\sum _{t=1}^T{\mathcal{D}}_t\min (d_t,{\tau }_d)\right]}}+w_3\underset{S{P}_3}{\underbrace{\left[\underset{\theta ,{\mathcal{P}}_u}{\max }\sum _{i=1}^V\sum _{t=1}^T\left(P_u\left(t\right)\right)\right]\}}},\hfill \\ & \mathrm{subject}\mathrm{to}\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill C1:& P_Q^v\le \beta ,\forall v\in \mathbf{V}\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill C2:& 0\le {\mathcal{D}}_t\le {\mathcal{D}}_{\max },\forall v\in \mathbf{V},\forall t\in \mathcal{T}\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill C3:& {\rho }_t<1, \forall v\in \mathbf{V}, \forall t\in \mathcal{T}\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill C4:& {\theta }_v^t\in \{0,1\},\forall v\in \mathbf{V},\forall t\in \mathcal{T}\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill C5:& \left|\right|{\mathit{q}}_{i(t+1)}-{\mathit{q}}_{i\left(t\right)}\left|\right|\le v_{max}\left(t\right)t_i\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill C6:& \left|\right|{\mathit{q}}_{i(t+1)}-{\mathit{q}}_{i\left(t\right)}\left|\right|\ge d_{min}\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill C7:& {\mathit{q}}_{(x,y)}={\mathit{q}}_{(x_0,y_0)}·e^{-\alpha \left({\displaystyle \frac{x^2}{a^2}}+{\displaystyle \frac{y^2}{b^2}}\right)}+\mathcal{H}\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill C8:& N_u^t\le c_{uav},\forall n\in {\mathcal{P}}_{\mathcal{UAV}},\mathcal{B}\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill C9:& {\varsigma }_{i,j}^{\left(k\right)}\left(t\right)\in \{0,1\}\mathrm{and}{s}_{i,j}\left(t\right)\in \{0,1\}\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill C10:& w_1+w_2+w_3=1\hfill \end{array}$$

(12)

where the term ${\theta }_t$ represents the FL model parameters that impact the communication delay and bandwidth usage. The term ${\mathit{\varphi }}_t$ indicates the weight parameters that determine the behavior of the FL model. The variable $\mathbf{V}$ is the set of vehicles, $\mathcal{T}$ is the total number of TTIs, ${\tau }_u$ is the uplink transmission time per packet, and ${\tau }_d$ is the maximum tolerable delay per packet. The term ${\theta }_v^t$ represents the scheduling indicator (1 if vehicle v is served in the $t^{th}$ TTI, 0 otherwise) and $L_w$ is the transmission window length. The term $P_Q^v$ is the queue overflow probability for vehicle v, and ${\rho }_t$ is the queue utilization factor. The term $q\left(t\right)$ is the UAV position at the $t^{th}$ TTI and $\mathcal{H}$ is the UAV altitude. The term $c_{\mathrm{uav}}$ is the UAV serving capacity for a number of vehicles per TTI. The variable $\beta$ is the threshold for the queue overflow probability.

The constraints $C1$ through $C10$ are designed to ensure queuing stability, communication reliability, UAV mobility feasibility, and computational tractability. Specifically, constraint $C1$ ensures that the probability of queue overflow for each vehicle does not exceed a predefined threshold $\beta$, i.e., $P_Q^v\le \beta$. This mitigates excessive packet loss due to UAV buffer overflows during uplink transmission. Constraint $C2$ enforces both buffer size and delay bounds. It ensures that the number of queued packets does not exceed the physical buffer capacity and that the queuing delay does not surpass a maximum threshold ${\mathcal{D}}_{\max }$. Packets exceeding this delay are dropped, while the lower bound of zero implies that a packet may be served immediately upon arrival if the UAV’s processor in the embedded PS is idle. Additionally, the TTI duration is constrained to avoid violating ${\mathcal{D}}_{\max }$. Constraint $C3$ maintains queue stability for the M/M/k queuing model by enforcing the utilization factor ${\rho }_t<1$, which guarantees that the UAV’s service rate exceeds the packet arrival rate. This prevents backlog accumulation and ensures stable system operation. Constraint $C4$ captures the successful transmission status of model hyper-parameters ${\theta }_v^i$ from vehicle v to the UAV, where a binary value indicates whether the transmission is successful or queued. This helps ensure fairness in local model transmission opportunities for all the vehicles. Constraint $C5$ restricts the UAV’s maximum displacement between time slots by enforcing a speed limit $v_{\max }$, which ensures the UAV’s trajectory remains physically feasible and energy efficient. To further improve energy efficiency, constraint $C6$ prevents redundant hovering at a single location without communication gain.

The constraint $C7$ governs the UAV’s elliptical flight path, parameterized by the major and minor axes $(a,b)$ and scaled by a tunable factor $\alpha \in [0,1]$. When $\alpha =0$, the UAV is stationary at altitude $\mathcal{H}$; when $\alpha =1$, the elliptical path reaches its maximum size. The elliptical trajectory, supported by prior work [41,42,43], offers advantages in obstacle avoidance and motion stability compared to circular or rectangular paths. Constraint $C8$ ensures that the UAV’s available computing resources, ${\mathcal{P}}_{\mathrm{UAV}}$, are sufficient to process $N_u^t$ incoming packets during uplink, accounting for the data buffer $\mathcal{B}$, defined as the product of data rate and sojourn time. The buffer size was set to 1 GB in our model. Constraint $C9$ introduces a binary variable $\varsigma$ to indicate whether the UAV communicates with a vehicle during the $i^{\mathrm{th}}$ TTI, and the variable $s_{i,j}\left(t\right)$ specifies the type of message (BSM or CPM). Finally, constraint $C10$ enforces that the weight coefficients $w_1$, $w_2$, and $w_3$ in the multi-objective function sum to 1, balancing trade-offs among latency, queuing delay, and power consumption. To solve the resulting optimization problem $\mathbf{P}1$, we employ BCD, which iteratively optimizes over subsets of variables while keeping the others fixed. While the original problem includes non-convex rank-one constraints, we apply convex relaxations via semi-definite programming (SDP) and lifting techniques to obtain tractable approximations. Furthermore, the weighting coefficients $w_1$, $w_2$, and $w_3$ in Equation (2) influence the conditioning and scaling of the subproblems. Specifically, the weight $w_1$ corresponds to the weight assigned to the communication delay component, capturing the time required for data transmission between vehicles and the UAV. The weight factor $w_2$ represents the weight related to the processing delay, indicating the time taken for local computation and model updates. The weight factor $w_3$ denotes the weight associated with the energy consumption of the UAV, accounting for the power required to maintain the trajectory, communication with vehicles and to process data. If these weights are excessively large or poorly scaled, the solution may take longer to converge. Note, ($\mathit{P}1$) is a mixed-integer nonlinear programming (MINLP) problem due to binary variables and nonlinear constraints, which is solved using block coordinate descent (BCD) and convex relaxations.

The problem ($\mathit{P}1$) is a BPP, which aims to pack n items of sizes i ($0\le i<n$) in the lowest number of bins without exceeding the capacity of each bin. In the proposed problem, the n items are the data packets comprising CPMs and BSMs, as well as data packets comprising local model updates ($\theta$). The bins are the randomly varying transmission windows, and the resulting queue lengths are minimized [46]. The delay minimization problem is solved by selecting an optimal number of ESs that meet the constraints on the queuing probability. The square-root staffing rule requires $M/M/k$ networks to allocate more resources than the UAV’s demand to maintain stability and delay constraints.

4. Proposed Solution

The optimization problem $\mathit{P}1$ is split into three subproblems. The uplink and downlink communication scheduling and delay minimization are subproblems $S{P}_1$ and $S{P}_2$, and the UAV computing power optimization is $S{P}_3$. These are solved using Lagrangian dual-decomposition and successive convex approximation. We relax the binary variables to convert $\mathit{P}1$ into a linear program, making each subproblem convex with real variables. All subproblems are solved in parallel. The non-convex constraints involving binary variables are relaxed using continuous approximations and solved using integer programming techniques for each subproblem. The nonlinear UAV trajectory constraints are approximated to improve tractability. Figure 3 depicts the proposed solution approach.

Here, utilization (${\rho }_i$) of the $i^{th}$ ES, for a given number of ESs ($k_i$), arrival rate (${\lambda }_i$) and departure rate (${\mu }_i$), is given by:

$${\rho }_i={\displaystyle \frac{{\lambda }_i}{k_i{\mu }_i}},$$

(13)

The steady-state probability that a packet is queued at the $i^{th}$ ES can be expressed as:

$$\begin{array}{cc}\hfill P_Q^i& =\sum _{m=k_i}^{\infty }{\pi }_m,\hfill \end{array}$$

(14)

$$\begin{array}{cc}& ={\pi }_0{\displaystyle \frac{k_i^{k_i}}{k_i!}}{\displaystyle \frac{{\rho }_i^{k_i}}{1-{\rho }_i}},\hfill \end{array}$$

(15)

where ${\pi }_m$ indicates the steady-state probability of m occupied edge servers. The availability probability of the $i^{th}$ ES can be expressed as:

$${\pi }_0={\left[\sum _{m=0}^{k_i-1}{\displaystyle \frac{{\left(k_i{\rho }_i\right)}^m}{m!}}+{\displaystyle \frac{k_i^{k_i}}{k_i!}}{\displaystyle \frac{{\rho }_i^{k_i}}{1-{\rho }_i}}\right]}^{-1}.$$

(16)

The queuing delay (${\mathcal{D}}_{que}$) incurred by the $i^{th}$ ES is:

$$E\left[{\tau }_Q^i\right]={\displaystyle \frac{1}{{\lambda }_i}}·{\displaystyle \frac{{\rho }_i}{1-{\rho }_i}}·P_Q^i.$$

(17)

4.1. Intermittent States and State Transitions at the Vehicles

In FL, vehicles compute local model updates and transmit them to the UAV. The iterations in SGD with parameter weights $\theta$ are considered as a Markov kernel. By sampling the $i^{th}$ entry between successive local updates, we denote the number of hyper-parameters communicated in a window w by a random vector ${\theta }^{\prime }\left(w\right)$. Figure 4 depicts the state transition model for an in-vehicle ES with utilization $\rho$ and comprising three states $S_i$, $S_t$, and $S_{t+1}$, and P is the transition probability matrix. Given an initial state $S_i$ and initial weight vector $\theta$, we have $\theta P^n\to \theta$ for n state transitions and as $n\to \infty$. A Markov chain {X = $X_0,X_1,\dots ,X_n\}$ describes the initial states of the ES in the transition matrix, and the Markov chain {Y = $Y_0,Y_1,\dots ,Y_n\}$ describes the UAV states that vary according to a state transition probability. Let X and Y be independent processes. The Markov chain $P^{\ast }$ is defined over their joint state space, where each state is represented as a pair $(s_i,s_j)$. Within $P^{\ast }$, transitions are allowed from a state $(s_i,s_j)$ to any other state $(s_k,s_l)$.

Moreover, $X^{\left(k\right)}$ is a random variable that has an expectation given by $E\left(X^{\left(k\right)}\right)=p_{ij}^{\left(k\right)}$. As per constraints C1 and C4, $E(X^{\left(0\right)}+X^{\left(1\right)}+\dots +X^{\left(n\right)})=p_{ij}^{\left(0\right)}+p_{ij}^{\left(1\right)}+\dots +p_{ij}^{\left(n\right)}$ for set of states $S=\{s_1,s_2,\dots ,s_m\}$. A packet moves to state $s_j$ from current state $s_i$ with transition probability $p_{ij}$, and it remains in the current state with probability $p_{ii}$. For a Markov chain with m states, $p_{ij}^{\left(2\right)}={\sum }_{k=1}^m{p}_{ik}{p}_{kj}$, and a packet remains in a queue or moves from the queue to the UAV with the associated transition probability. The transition probability from state $s_i$ to state $s_j$ after k steps is denoted by $p_{ij}^{\left(k\right)}$. For a Markov chain with m states, the second-order transition probability is given by $p_{ij}^{\left(2\right)}={\sum }_{k=1}^m{p}_{ik}{p}_{kj}$, which represents the probability of transitioning from state $s_i$ to state $s_j$ in two steps, through an intermediate state $s_k$. In matrix $P^n$, the ${(i,j)}^{th}$ element $p_{ij}^{\left(n\right)}$ denotes the probability that a Markov chain initialized in state $s_i$ reaches state $s_j$ after n transitions. The time-dependent steady-state probability distributions are given in Equations (18) and (19) as:

$${\pi }_{S_i}={\displaystyle \frac{{(1-p)}^i}{1-{(1-p)}^m}}·{\pi }_{S_t},$$

(18)

and

$${\pi }_{S_t}={\displaystyle \frac{{(1-p)}^{i+1}}{1-{(1-p)}^m}}·{\pi }_{S_{t+1}},$$

(19)

where p is the steady-state transmission probability of a packet from a vehicle to the UAV without queuing. Parameter m is the number of states in the system and the terms ${\pi }_{S_t}$ and ${\pi }_{S_{t+1}}$ are the steady-state probabilities of states $s_t$ and $s_{t+1}$, respectively. The expected time spent by a packet in state $s_i$ until it is transmitted to the next state is a geometrically distributed random variable. The mean sojourn time of each packet in state $s_i$ is obtained from the state probability given by Equation (20):

$${\pi }_{S_{\tau }}={\displaystyle \frac{{\pi }_{S_{\tau }}·{\tau }_t}{{\pi }_T·{\tau }_T+{\sum }_{i=0}^{m-1}{\pi }_{S_t}·{\tau }_T+{\sum }_{i=0}^{m-1}{\pi }_{S_{t+1}}·{\tau }_{S_{t+1}}}}.$$

(20)

The state probability ${\pi }_T$ is the probability of a packet being in each state. The state probability implies the steady-state probability of the UAV-assisted C-V2X communication system in terms of the packet’s transition probabilities and the number of states m. The overall probability of being in a state is given by Equation (21):

$${\pi }_T={\displaystyle \frac{1}{\frac{1}{1-{(1-p)}^m}{\sum }_{i=0}^{m-1}\frac{{(1-p)}^i}{{(1-p)}^m}}}.$$

(21)

where $p_{ij}$ is the transition probability from state $s_i$ to state $s_j$. The term ${\pi }_{S_i}$ indicates the steady-state probability of being in state $s_i$. The terms ${\pi }_{S_t}$ and ${\pi }_{S_{t+1}}$ imply the steady-state probabilities of adjacent states $s_t$ and $s_{t+1}$. The terms ${\tau }_t$, ${\tau }_{S_t}$ are the sojourn times associated with states $s_t$ and $s_{S_t}$. The term p is the steady-state transmission probability of a packet from a vehicle to the UAV without queuing, and m is the number of states.

Note the inter-arrival time of packet i at the ES is equal to the processing time of packet i at the ES, and the service time of packet i at the UAV is represented by its corresponding service time. If packet i arrives at the UAV after the previous packet’s processing time has ended, it will wait for the previous packet’s wait time. In this case, the inter-arrival time and service time are updated as a function of the previous packet’s wait and service times. If a packet i arrives at the processing station before the packet $i-1$ departs, the packet i waits for a random wait time. This wait time is deterministic and is a function of the service time of the previous packet at the processing station. The inter-arrival time between packets $i-1$ and i is the sum of the service time of packet $i-1$, the wait time of packet i, and the service time of packet i. Additionally, the state set representing the arrival of packet $i-1$ at the UAV, starting its service time, and the simultaneous start of packet i at the edge server are described by the corresponding events. If packet i arrives at the UAV after packet $i-1$ has been processed and departed, it means the UAV is idle, and packet i starts its service time. The inter-arrival time between packets $i-1$ and i at the UAV is the sum of the service time of packet $i-1$ at the UAV and the processing time of packet i at the edge server. The global aggregation step in FL updates the model weights based on the local data slices and the priorities of the local models. Each local model is assigned a function score based on the dominant properties of the vehicle data to determine its contribution to the global model.

4.2. UAV Occupancy and State Transitions

Let $s^{\left(t\right)}\in \mathcal{S}$ denote the state of the UAV where

• ${\delta }^{\left(t\right)}$ is the duration for which a packet remains in the queue before arriving at the UAV.
• $b_{\mathrm{s}}^{\left(t\right)}\in \{0,1\}$ is the UAV occupancy indicator, where $b_{\mathrm{s}}^{\left(t\right)}=1$ indicates that a packet is in the UAV’s queue, and $b_{\mathrm{s}}^{\left(t\right)}=0$ indicates otherwise.
• $B_{\mathrm{s}}^{\left(t\right)}\in \{0,1\}$ is the queue occupancy indicator, where $B_{\mathrm{s}}^{\left(t\right)}=1$ indicates that a packet is waiting in the queue, and $B_{\mathrm{s}}^{\left(t\right)}=0$ indicates an empty queue.
• ${\Delta }_{\mathrm{s}}^{\left(t\right)}$ represents the duration for which a packet remains at the UAV.

The action taken by a UAV is denoted by $a^{\left(t\right)}$, where the binary value $a^{\left(t\right)}=1$ indicates that the vehicle transmits a packet at the beginning of the window, and $a^{\left(t\right)}=0$ indicates no transmission. A policy $\pi$ is defined as a random mapping from the state space $\mathcal{S}$ to the action space $\mathcal{A}$, specifying the action $a^{\left(t\right)}$ in state $s^{\left(t\right)}$. An optimal policy ${\pi }^{\ast }$ minimizes the average delay:

$${\pi }^{\ast }=\arg \underset{\pi }{\min }{V}_{\pi }\left(s^{\left(0\right)}\right).$$

(22)

Figure 5 shows that the vehicle ES may take the following actions: local model completion (C), non-completion (NC), model transmission (T), or non-transmission (NT) to the UAV. The action set is described as $\mathcal{A}$ = $\{C,NC,T,NT\}$ for various local models (${\theta }_n$) in the transmission windows ($L_w$). In Figure 5, the branch highlighted in $red$ implies that an action $NC$ cannot be followed by an action T, i.e., a non-completed local model cannot be transmitted by the ES. The UAV can either select or not select a local update in each transmission window. Each action is a function of the pair $(\theta ,w)$, where

• $\theta \in \{0,\dots ,{\theta }_{\max }\}$ is a stochastic process representing the number of packets transmitted;
• $w\in \{0,\dots ,W\}$ is a stochastic process denoting the number of transmission windows elapsed since the start.

The action where the UAV selects an incoming packet depends on queue backlog, arrival rate, and the size of an incoming packet. These factors affect UAV energy consumption and limit the subset of vehicles a UAV can select in each transmission window. At any state, either the UAV is processing packet $i-1$ when packet i arrives, or a packet arrives at an idle UAV. The state transition probability is described by different cases depending on the UAV’s status, including when the system transitions to an idle state or when it switches between different processing conditions. In particular, we look at the transitions involving the UAV’s idle time, the processing time of packets at the edge server, and the success or failure of packet transmission. When a packet is at the UAV, the probability of an unsuccessful transmission over a given number of consecutive windows is non-zero. The optimal policy minimizes the total cost by determining the best action based on the current state and available actions. The actions are chosen to minimize the expected delay and maximize the efficiency of packet processing, considering the transmission success rates and the idle states of the UAV. The possible actions are based on combinations of transmission and non-transmission decisions, which are executed according to the system’s current state. An optimal policy ensures the system adapts to the dynamic environment of packet arrivals, processing, and transmission at the UAV.

4.3. Impact of UAV Occupancy on Packet Queue

When the UAV is busy, incoming packets are queued until the UAV’s processor becomes available. The MDP is $(\mathcal{S},\mathcal{A},P(s^{(t+1)}\mid s^{\left(t\right)},a^{\left(t\right)}),C(s^{\left(t\right)},a^{\left(t\right)}))$, where $C(s^{\left(t\right)},a^{\left(t\right)})$ is a cost associated with taking an action $a^{\left(t\right)}$ in current state $s^{\left(t\right)}$. The long-term optimal policy minimizing average delay is:

$${\pi }_s^{\ast }=\arg \underset{\pi }{\min }\left\{\underset{T\to \infty }{\lim }{\displaystyle \frac{1}{T}}\sum _{t=0}^{T-1}\mathbb{E}[C(s, a)\mid s^{\left(0\right)}]\right\}.$$

(23)

For a UAV state $s=(\delta ,b_{\mathrm{s}},B_{\mathrm{s}},{\Delta }_{\mathrm{s}})$ transitioning to $s^{(t+1)}=({\delta }^{\left(t\right)},b_{\mathrm{s}}^{\left(t\right)},B_{\mathrm{s}}^{\left(t\right)},{\Delta }_{\mathrm{s}}^{\left(t\right)})$, where the transition probability is denoted as $P(s^{(t+1)}\mid s,a)$ = ${\mathbb{R}}_{0,1}$, where ${\mathbb{R}}_{0,1}$ is a real number in $[0,1]$. The corresponding value function satisfies:

$$V_s^{\ast }=\underset{a\in \mathcal{A}}{\min }\left\{C(s, a)+\sum _{\mathcal{S}}P(s\mid s^t, a)V_s^{\ast }\right\}.$$

(24)

The optimal policy is computed using RVI as:

$$\begin{array}{cc}\hfill {\pi }_s^{\ast }& \leftarrow \arg \underset{a\in \mathcal{A}}{\min }\left\{C(s,a)+\sum _{\mathcal{S}}P(s\mid s^t,a)V_s^{\ast }\right\},\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill V_s^{\ast }& \leftarrow C(s,{\pi }_s^{\ast })+\sum _{\mathcal{S}}P(s\mid s^t,{\pi }_s^{\ast })V_s^{\ast }-V_{s_{\mathrm{opt}}}^{\ast },\hfill \end{array}$$

(26)

where $s_{\mathrm{opt}}$ is a reference state.

Figure 6 illustrates the state transition model of a UAV. The UAV state is monitored using the timestamp of the most recently received packet. Let $T_i=t_i-t_i^{\left(t\right)}$ denote the time from packet generation at the vehicle to reception at the UAV and $I_i=t_i^{\left(t\right)}-t_{i-1}^{\left(t\right)}$ denote a packet’s inter-arrival time. These definitions allow the use of optimal packet transmission policies with a finite state space, ensuring that when the UAV becomes overloaded, the time-critical C-V2X transmission window is not further counted. The optimal policy minimizing long-term average latency depends on the set of admissible policies under the constraints in $\mathit{P}1$. The modified UAV state is defined in terms of $({\delta }^{\left(t\right)},b_{\mathrm{b}}^{\left(t\right)},b_{\mathrm{s}}^{\left(t\right)},B_{\mathrm{b}}^{\left(t\right)},B_{\mathrm{s}}^{\left(t\right)},{\Delta }_{\mathrm{b}}^{\left(t\right)},{\Delta }_{\mathrm{s}}^{\left(t\right)})$, where

(i).

${\delta }^{\left(t\right)}$: packet queuing time.

(ii).

$b_{\mathrm{b}}^{\left(t\right)}\in \{0,1\}$: buffer queue’s occupancy indicator.

(iii).

$b_{\mathrm{s}}^{\left(t\right)}\in \{0,1\}$: UAV occupancy indicator.

(iv).

$B_{\mathrm{b}}^{\left(t\right)}\in \{0,1\}$: occupancy indicator of the transmission queue.

(v).

$B_{\mathrm{s}}^{\left(t\right)}\in \{0,1\}$: UAV occupancy indicator for the transmission queue.

(vi).

${\Delta }_{\mathrm{b}}^{\left(t\right)}$: status update at the UAV.

(vii).

${\Delta }_{\mathrm{s}}^{\left(t\right)}$: duration for which a packet remains at the UAV.

The occupancy indicators $(b_{\mathrm{b}}^{\left(t\right)},b_{\mathrm{s}}^{\left(t\right)},B_{\mathrm{b}}^{\left(t\right)},B_{\mathrm{s}}^{\left(t\right)})$ provide the current status of the UAV, queue, and buffer.

4.4. Federated Learning Based on Message Priority

For FL model training, we used the publicly available V2X-Sim dataset composed of n elements with input samples $x_i$ and corresponding class labels $y_i$. As per Equation (27), to update the model parameters, the UAV monitors a set of vehicles ($\mathcal{V}$) and selects a subset of vehicles in a communication round. The aggregation weight ($\theta$) associated with vehicle $\mathcal{V}$ is mapped to a score function f based on a set of measurable criteria over clients $v_i\in \mathcal{V}$ to minimize $(f(x_i;\theta ),y_i)$, where $f(·)$ is the function score. Each vehicle $v\in$$\mathcal{V}$ owns its local data with a varying number of data points.

$${\mathbb{E}}_{{\zeta }_i}{\Vert \nabla L_i(\theta ;{\zeta }_i)-\nabla L_i\left(\theta \right)\Vert }^2\le {\sigma }^2,\forall i,\forall w$$

(27)

where $\sigma$ is the variance in data slices used to train each client. A set of measurable criteria $\{c_1,\dots ,c_m\}$ that characterize the data features in each communication round is encoded in the global model aggregation step. According to Equation (28), the in-vehicle ES calculates these values and communicates them to the UAV to update the model. In order to evaluate model priority, the UAV accumulates and aggregates the local model hyper-parameters and updates from $\mathcal{V}$ vehicles denoted by the m-tuple $\{c_1,\dots ,c_m\}$ and encodes the priority information of vehicle data in the weights used for aggregating vehicle model contributions. The term $\zeta$ is the the ratio of samples in adjacent data slices, there are n samples in a data slice, and L implies a loss function accumulated in training the FL model. The aggregated models are affected by the statistical aspects of the data and the skewness used to train the vehicles. By considering the variance of the scores, the UAV prioritizes different classes in each slice of data and takes into account the measure of similarity in number and size.

$${\displaystyle \frac{1}{n}}\sum _{i=1}^n{\Vert \nabla L_i\left(w\right)-\nabla L_{i-1}\left(w\right)\Vert }^2\le {\zeta }^2,\forall i,\forall w.$$

(28)

5. Simulation Results and Discussion

In the simulation, we assumed a Manhattan mobility model with vehicle speeds ranging from 1 to 100 km/h. In simulations, we used BSM and CPM packets as specified by the SAE. Note, BSM packets were periodically broadcast to convey vehicle location, road position, velocity, and traffic status, with inter-arrival times in the range 100 ms–1 s. CPM packets were event-triggered messages related to emergencies or hazardous driving conditions modeled as a Poisson process with frequencies of 100 ms, 200 ms, and 300 ms. The number of packet retransmissions was assumed to be between one and five. Packet arrival rates were set to $\lambda$ = 1000–2000 packets/s.

Table 3. Simulation parameters.

Parameter	Value
Number of $\mathcal{PS}$ in UAV	1
Vehicle mobility	Manhattan mobility
Edge server location	In-vehicle
Number of vehicles (V)	1–100
UAV deployment altitude	100 m–2 km
Distance between vehicles	10–100 m
UAV transmission power	20 dBm (100 mW)
Communication frequency	5.9 GHz
Payload size for BSM, CPM	1 byte–3 Megabytes
Vehicle transmission power	25 dBm (316.2 mW)
Road length	1–5 km
Vehicle speed	0–100 km/h
Standard deviation in speed	10 km/h
Dataset used	V2X-Sim
Payload size of FL models	1 byte–10 Megabytes
$T_{CPM}$	100, 200, 300, 500 ms
UAV receiving threshold	−80 dBm
$T_{BSM}$	100–1000 ms
Buffer size ($\mathcal{B}$) at UAV	1 GB
$\lambda$	1000, 2000 packets/s
Mean speed of vehicles	50 km/h

lists the main parameters considered in the simulations.

For the FL scenario, we used the V2X-Sim dataset, with 60% training data and 40% testing data. We considered one PS, and V was varied from 1 to 20. The PS was embedded in a UAV that spanned a stochastic trajectory at an altitude of 100 m–2 km. We used the V2X-Sim dataset to train each agent (vehicles and UAV) with a random 10% subset, allowing each agent to learn a distinct environment for generating local models and the initial global model [47]. At each episode, the dataset was shuffled, batched, and sliced for testing while training each client. The V2X-Sim dataset was processed on an Amazon Elastic Compute-2 (EC-2) instance. The agents were trained on the V2X-Sim dataset using Python version 3.13.5 programming language, TensorFlow v2.16.1, and TensorFlow Federated frameworks. Further details of the FL model are provided in [22].

5.1. Variation in Transmission Window Length ($L_w$) and Queue Backlog with Varying Number of Vehicles (V)

The length of a transmission window $L_w$ impacts the maximum transmission latency. A CSR is selected by a vehicle within this duration, and the UAV must be in its coverage range to avoid re-transmissions or packet drops. The sensing window comprised 1000 previous subframes. We varied $L_w$ as 100 ms, 50 ms, and 20 ms and investigated the delay experienced by various packets in FL as well as gross data offloading scenarios. Figure 7 shows latency constraint violations for different numbers of vehicles (V). Figure 8 depicts the total delay variation over training time slots in the FL scenario for a varying V.

5.2. Variation in Average Delay with Inter-Arrival Time of BSM and CPM Packets with (V)

Figure 9 depicts the comparison of queue backlog with time slots for a varying V. The queue backlog increases rapidly for the first 0-60 time slots and exceeds the maximum queue capacity. The UAV’s queue is prevented from overloading and hence does not become unstable. Furthermore, with an increase in V for the FL scenario, synchronous uploading of local models improves latency. While the UAV’s queue stability is ensured, the delay in uploading local models results in a higher total delay. The RVI algorithm enforces the delay constraint by significantly reducing backlogs, ensuring greater stability over extended time slots compared to existing approaches in [48,49,50]. Figure 10 illustrates the variation in delay with the BSM inter-arrival time, which remains unaffected by the varying number of vehicles due to sufficient resources for packet transmission. For $L_w=20$ ms, the mean queuing delay ranges from 18 ms to 22 ms. For $L_w=100$ ms, the delay gradually increases from 50 ms to 65 ms as $T_{BSM}$ varies from 0 to 800 ms, then rises sharply when incoming packets accumulate, reflecting UAV resource limitations. The highest average delay occurs for $L_w=100$ ms, while the lowest is 22 ms for $L_w=20$ ms, and the delay increases with higher packet arrival frequency. Furthermore, the larger the $L_w$, the higher the vehicle count V, and longer inter-arrival times lead to increased delays, as more BSM packets are queued due to limited UAV processing capacity.

5.3. Variation in Average Energy Consumption (Joules/s) of UAV with Varying V

Figure 11 shows the variation in average delay for CPM packets with the number of vehicles (V). For a large transmission window ($L_w=100$ ms), increasing V and packet transmissions does not significantly increase delay, which is between 5 ms and 8 ms. Reducing $L_w$ introduces random wait times, with the average delay rising to 20 and 40 ms. For $L_w=50$ ms and V = 1 and 10, the delay decreases sporadically by approximately 45%. When $L_w$ is set to either 20 ms or 50 ms, if the service rate $\mu$ exceeds the CPM repetition frequency, the average delay increases. Figure 11 also illustrates the average delay versus CPM inter-arrival time $T_{CPM}$ for various packet arrival rates $\lambda$, which are event-driven. For $L_w=50$ ms and 100 ms, a slight delay increase for a small $T_{CPM}$ is caused by congestion from higher packet transmission rates. Overall, the average delay increases with a higher $\lambda$ but shows minimal sensitivity to the packet service rate $\mu$.

Figure 12 shows the variation in UAV average energy consumption with V for different vehicle speeds and road lengths. UAV energy consumption (J/s) depends on packet transmission frequency and bandwidth. For a given altitude between the source vehicle and the destination UAV, increasing V leads to collisions and higher delays. Energy consumption increases as ${\mathcal{R}}_L$ varies. The UAV energy is consumed in packet transmissions, while idle, and during flight. Increasing road length or vehicle speed (with a constant V) raises UAV energy consumption, as greater inter-vehicle distances require the UAV to cover a larger trajectory. In gross data offloading, more retransmissions further increase UAV energy use. Figure 13 shows the average FL delay that depends on the packet delivery ratio. Furthermore, it is observed that in the case of FL, the processing delay is reduced considerably, but there is little impact on energy consumption. However, limiting vehicle speed may not always be an option because of driving speed constraints. Thus, to improve the performance, an alternate approach is to adjust the transmission window ($L_w$) such that the queue backlog is minimized. Hence, we conclude that with the correct choice of trade-off between vehicle speed, road length, $L_w$, and ($\lambda$,$\mu$) pairs, it is possible to minimize delay and maximize UAV energy. As shown in

Table 4. Comparison of applications of some recent deep learning methods in UAV–vehicle communications.

Reference	Proposed Method	Objectives	Cost Function	Reported Results
[12,22]	Personalized FL Attention mechanism	Minimize hyper-parameters UAV trajectory optimization	Latency Convergence time	Less communication overhead and faster FL convergence Reduced FL energy consumption by 10–15%
[22,48]	FL network selection as sequential MDP Constrained optimization of state-transition probabilities	Reducing FL cost Network selection Computation-efficient value iteration algorithm to solve the MDP	FL training loss FL cost function to integrate terrestrial and aerial edge computing with distributed FL	Improvement in accuracy Improvement in energy consumption, delay, and FL performance compared to benchmarks FL latency was upper-bounded to 3 s. In our work, FL latency was up to 40 ms.
[51]	Lyapunov optimization for resource allocation Novel delay, queue, and energy models	Optimal resource allocation Feedback at the vehicle or RSU Minimize energy and delay	Trade-off between optimization algorithm and energy consumption Queuing delay	Reduced energy consumption compared to existing algorithms. In our work, we considered up to 100 vehicles on a 4 km road length.
[50]	Joint task assignment, power allocation, and node grouping Minimize the energy consumption of devices	Enhanced cooperation of idle computation resources Non-orthogonal multiple access in millimeter-wave heterogeneous network	Communication delay Sub-optimal solution for power allocation Offloading task ratio	Reduction in data exchanged, queuing length reduction, and reduced energy consumption compared to benchmark methods. Our work achieved a further reduction in latency of 40%.
[49]	Joint network selection, computation offloading Sequential decision-making problem MDP	Value iteration to find an optimal policy Reduced task computation time Minimize latency and energy	% of data offloaded Latency Backhaul link congestion	Improved network performance in terms of latency for 0.6–0.14 Mbits of offloaded data Restricted energy consumption between 0.002–0.004 Joules
Our Work	MDP and DTMC FL and gross data offloading	Minimize queuing delay Minimize processing delay	Communication delay Convergence time	Low-latency communication between vehicles (V) and UAV; FL latency was between 15–40 ms Lower queue backlog by 20% compared to work in [51]

, our proposed solution reduced the model convergence time by 30% compared to previous machine learning approaches. In the case of UAV resource constraints, the FL approach ensures that data from all vehicles are used to synchronize the global update without missing significant information. Figure 13 implies that in the case of FL, more vehicles on smaller road length require less communication overhead with the UAV to improve the local model convergence and global model performance. The system throughput and the probability of data transmission at the $L^{th}$ transmission window are given by

$${\displaystyle s_{\left(L_w\right)}=\sum _{n=1}^N{\tau }_n^{\left(L\right)}\prod _{m=1,m\ne n}^N\left(1-{\tau }_m^{\left(L\right)}\right),\mathrm{for}2\le L_w\le L}$$

(29)

where ${\tau }_m$ is the interval between the packet transmitted at time $t_{m-1}$ and $t_m$. For all data slices, a pair of neighboring sensor data may differ in one or more features. The $KL$-divergence between gross data offloading and FL approaches indicates that the optimal policy prescribes actions at state $s=(\delta ,\psi ,0,\psi )$ (empty UAV) for varying $\mu$ and $\gamma$. When the current delay $\delta <10$ ms, the UAV remains idle; if $\delta$ exceeds 10 ms, it transmits the global model.

Figure 14 illustrates the evolution of queue backlog versus number of training time slots (ms) for various queue lengths, packet sizes, and V’s. Figure 14 indicates that as the number of training time slots increases, the queue backlog increases but becomes stable after a few iterations. The queue backlog is positively correlated with the number of training time slots, and it is possible to optimize the task execution to reduce queue backlog. With more training iterations, the backlog decreases, and the computation rate improves. Based on the trends observed in Figure 13 and Figure 14, we observe that the FL delay can be optimized by keeping the road length to 2 km and vehicle speed between 60 and 80 km/h. Note that implementing the optimal policies in real-world C-V2X systems requires the integration of automotive software and hardware. These optimal policies depend on sophisticated algorithms that dynamically adjust UAV behavior based on current network conditions and task requirements. A real-time decision-making framework is essential in C-V2X to enable continuous communication between vehicles and UAVs for adjusting offloading decisions. Additionally, edge computing capabilities are necessary to manage the computational load and minimize latency in task offloading. To support FL models and the related offloading mechanisms, both UAVs and vehicles must be equipped with specific hardware components. UAVs require powerful onboard processors to perform local model training, high-capacity batteries for extended operational time, and reliable communication interfaces to ensure integration with the C-V2X system. Vehicles must be embedded with C-V2X communication modules that support both vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication, along with edge computing nodes to assist in data processing. Furthermore, it is crucial for both UAVs and vehicles to utilize energy-efficient hardware that balances the computational demands of FL with the limitations of battery life. By addressing these considerations, we aim to bridge the gap between theoretical findings and real-world implementation, ensuring that the optimal policies proposed in this paper can be realized in 6G ITS.

As shown in Table 4, recent advancements in machine learning methods for UAV–vehicle communications have focused on optimizing resource allocation, reducing latency, and improving energy efficiency. The work in [51] introduced a Lyapunov-based optimization framework that modeled delay, queue, and energy relationships to achieve optimal resource allocation. By incorporating feedback mechanisms at both vehicles and RSUs, this approach minimized energy consumption and delay, outperforming traditional optimization algorithms. Similarly, the work in [50] emphasized joint task assignment, power allocation, and node grouping within a millimeter-wave heterogeneous network using NOMA to enhance the cooperation among idle computation resources. This method demonstrated significant improvements in energy efficiency, reduced queuing length, and minimized communication overhead compared to cooperation-free benchmarks. The work in [49] modeled the communication and computation trade-off as an MDP, integrating joint network selection and computation offloading. Using value iteration for policy optimization, this method effectively reduced task computation time and minimized overall latency and energy consumption. Moreover, attention-based personalized FL models [22] have focused on UAV trajectory optimization and communication efficiency. By personalizing local models and minimizing hyper-parameter tuning, these approaches achieved faster convergence by approximately 20%, lower communication overhead by approximately 15%, and improved UAV’s energy efficiency by 40%. The authors in [48] applied FL to a sequential MDP framework, formulating network selection as a constrained optimization problem that balanced training loss and system cost. Through a computation-efficient value iteration algorithm, these methods successfully integrated terrestrial and aerial edge computing with distributed FL, leading to improved accuracy, lower delay, and better energy consumption. Compared to these existing approaches, our proposed framework combines MDP and DTMC models with FL to minimize both queuing and processing delays.

Existing works based on conventional RL, such as those employing deep Q-networks and actor–critic architectures, primarily focus on improving vehicle mobility and centralized decision-making. These methods demonstrate improved task execution times and reduced energy consumption but yield high computational complexity. Moreover, some FL approaches [52,53] have aimed at optimizing network convergence and resource allocation by minimizing queuing delays and power consumption. These methods achieve significant reductions in data exchange by up to 79% and improve FL convergence times by approximately 56%. However, these approaches require careful user selection and joint learning strategies to maintain performance consistency across distributed nodes. Another distinguishing aspect of our work is the enhancement of model convergence through fine-tuning strategies. While previous FL methods have benefited from joint training across multiple datasets, they often encounter performance degradation under non-i.i.d. data distributions, particularly in FedAvg and FedSGD implementations. Our approach mitigates this issue, providing a consistent 5–7% improvement across agents. The adaptation of Q-learning with optimized discount factors further enables faster model convergence and more robust policy learning. Collectively, these results demonstrate that the proposed FRL framework offers a communication-efficient and scalable solution for vehicular communication systems compared to existing DRL and FL approaches.

Furthermore, fewer communication rounds were needed to aggregate model updates at the UAV due to faster convergence. Initially, all vehicles contributed equally to global model computation, whereas in the following iterations, the UAV assigned higher weights to vehicles with slower convergence to minimize the loss function. A variety of hyper-parameter vectors were generated for FL training with various batch sizes. At the same time, the UAV took into account the convergence time and loss function of each vehicle in order to average the local models from each vehicle and keep track of the number of incomplete local models. Based on earlier works summarized in Table 4, our method achieved around 30% reduction in model convergence time compared to previous machine learning approaches. In order to optimally utilize the UAV’s available battery power and communication resources, the proposed FL approach ensures that data from all vehicles are used to synchronize the global model hyper-parameter updates at the UAV without missing significant information. A limitation of the proposed work is that the data are generated according to a Poisson process with exponentially distributed processing time, without taking into account more general distributions. The authors in [26] compared the two enabling vehicular communication technologies, namely, C-V2X and IEEE 802.11p. The authors illustrated the variation in average delay and channel busy ratio vs. number of vehicles in C-V2X communications (without UAV) in [26]. The authors considered different types of messages transmitted between vehicles and between vehicles and infrastructure. However, in this work, a static base station was considered. We analyzed a similar communication setup with a significant enhancement where a UAV was included as a mobile base station or a parameter server with embedded computing capability. A challenge in our work was to minimize queue backlog at the UAV in order to preserve the battery power available at the UAV.

The authors in [26] also illustrated the variation in average delay vs. packet inter-arrival rate for varying parameters in C-V2X communication (without UAV) in [26]. The authors considered the periodically transmitted cooperative awareness messages and event-driven decentralized environmental notification messages. Furthermore, the authors obtained a closed-form solution for the steady-state probabilities of the models to derive expressions for some performance metrics. As an extension of this approach, we modeled periodically transmitted BSMs and event-triggered CPMs to study the delay performance of UAV-assisted C-V2X communications in gross data offloading scenario. This delay was then compared to that of the FL scenario for a varying number of vehicles, UAV altitude, road length, and packet size transmitted from vehicles to UAV. The authors in [26] provided a performance comparison between IEEE 802.11p and C-V2X mode 4 in terms of average delay, collision probability, and channel utilization and concluded that IEEE 802.11p was superior in terms of average delay, whereas C-V2X mode 4 excelled in collision resolution. In our work, we extensively studied the performance of C-V2X mode 4 by varying the inter-arrival time of packets. The authors in [26] also illustrated the variation in collision probability and channel utilization vs. number of vehicles in C-V2X communications without UAV in [26]. The authors varied the inter-arrival time between subsequent packets from 20 ms to 100 ms to study the collision probability and channel utilization vs. number of vehicles in C-V2X communication without UAV. In our work, we varied the inter-arrival time in a similar range to study the queue backlog at the UAV and the average delay for BSM, CPM, and FL scenarios. Moreover, we used OTFS modulation to ensure there were no collisions between the packets transmitted from multiple vehicles. The authors in [49] illustrated the variation in average latency cost, percentage of application latency failures, and average energy consumption in vehicles vs. number of vehicles without UAV. The authors compared various MDP schemes with a varying number of vehicles. While the authors in [49] studied the average latency cost, percentage of application latency failures, and average energy consumption in vehicles for around 1800 vehicles, we restricted the number of vehicles in our setup to 100. However, we incorporated a UAV in the setup and studied the energy consumption pattern of the UAV when different packets were transmitted from the vehicles to the UAV. We also studied the state-transition probabilities of vehicles and UAV at various vehicle velocities and road lengths.

The authors in [10] illustrated the variation in average throughput vs. vehicle density with and without UAVs. The authors’ work in [10] marks a significant breakthrough in integrating UAVs in vehicular networks. Our work is a significant extension of that work, where we have studied the performance characteristics of UAV-assisted C-V2X communications in FL environments. The authors in [10] considered an integrated simulation platform to conduct a case study comprising a traffic generator, a network simulator, and a data processor. However, a limitation of that approach was that the authors simulated a basic scenario where vehicles traveled along a two-lane straight highway. The setup consisted of two UAVs and compared the throughput and delay performance between UAV-assisted vehicular network and the 802.11p-based vehicular network. The average delay in the authors’ work was reported to be in the range of 25–30 ms, whereas in our work, we were able to achieve a minimal average delay of 20 ms for the FL scenario and 22 ms for the gross data offloading scenario.

6. Conclusions

This paper presented a Markov chain model to analyze 3GPP C-V2X mode 4 communication with a UAV. In scenario one, we considered FL where the sensor data were processed locally and transmitted to the UAV. In scenario two, we considered gross data offloading in the forms of BSM and CPM packets defined by the SAE. We considered a DTMC with the UAV initially having at most one active packet batch pertaining to the global model. Then, we initiated local model updates at the ES, which were then transmitted to the UAV for aggregation. The performance of a vehicle in both scenarios was studied based on average delay, queue backlog, and UAV energy consumption. Furthermore, the results revealed that in the FL scenario, the processing delay was less at the UAV, whereas the queuing delay depended on the transmission window $L_w$. The queuing delay also decreased with the control parameter. In addition, we conducted a delay analysis by calculating the average delay under the policies which considered the UAV occupancy status. The experimental simulations and numerical results revealed that in both scenarios, the optimal policy aimed to minimize the UAV’s energy consumption while also minimizing the cumulative queue backlog. As part of future work, we intend to study the variations in latency when the UAV remains within the vehicle’s LoS path for the entire transmission window, while maximizing the trajectory covered with the available battery power at a given altitude. Additionally, the authors aim to investigate the co-existence of hybrid classical–quantum communication in UAV–vehicle networks.