Traj-Q-GPSR: A Trajectory-Informed and Q-Learning Enhanced GPSR Protocol for Mission-Oriented FANETs

Abstract

Routing in flying ad hoc networks (FANETs) is hindered by high mobility, trajectory-induced topology dynamics, and energy constraints. Conventional topology-based or position-based protocols often fail due to stale link information and limited neighbor awareness. This paper proposes a trajectory-informed routing protocol enhanced by Q-learning: Traj-Q-GPSR, tailored for mission-oriented UAV swarm networks. By leveraging mission-planned flight trajectories, the protocol builds time-aware two-hop neighbor tables, enabling routing decisions based on both current connectivity and predicted link availability. This spatiotemporal information is integrated into a reinforcement learning framework that dynamically optimizes next-hop selection based on link stability, queue length, and node mobility patterns. To further enhance adaptability, the learning parameters are adjusted in real time according to network dynamics. Additionally, a delay-aware queuing model is introduced to forecast optimal transmission timing, thereby reducing buffering overhead and mitigating redundant retransmissions. Extensive ns-3 simulations across diverse mobility, density, and CBR connections demonstrate that the proposed protocol consistently outperforms GPSR, achieving up to 23% lower packet loss, over 80% reduction in average end-to-end delay, and improvements of up to 37% and 52% in throughput and routing efficiency, respectively.

1. Introduction

Flying Ad Hoc Networks (FANETs) are gaining traction in disaster relief, environmental monitoring, and communication relay due to their rapid deployment, flexible mobility, and broad coverage [1,2,3,4]. With the rapid development of the low-altitude economy, unmanned aerial vehicles (UAVs) have emerged as critical carriers of edge intelligence [5]. However, the limited capabilities of individual UAVs highlight the need for UAV swarms, which can perform complex tasks more efficiently and cost-effectively. As a representative application of FANETs, UAV swarms present significant challenges to routing protocol design due to their high-speed movement in three-dimensional space, energy constraints, frequent link disruptions, and dynamic changes in network density [6,7,8].

Conventional routing protocols can be categorized into two main groups: topology-based and location-based [9]. Classic topology-based protocols, such as AODV [10] and OLSR [11], maintain routing tables either passively or proactively. However, in FANETs, where network topology changes frequently, these protocols tend to generate outdated routing information and incur substantial control overhead. In contrast, Greedy Perimeter Stateless Routing (GPSR) [12], a prominent position-based protocol, relies solely on node location information, offering strong scalability. GPSR employs two forwarding strategies to ensure data delivery. GPSR employs two forwarding strategies to guarantee data delivery. In greedy mode, each node selects the neighbor nearest to the destination. When no such neighbor is available, indicating a routing void, the protocol transitions to perimeter mode, forwarding packets along the network boundary using the right-hand rule until greedy forwarding can resume.

GPSR’s greedy forwarding, while flexible and scalable, struggles in highly dynamic UAV swarms due to its reliance on single-hop neighbor information, resulting in degraded routing performance from frequent link interruptions and topology changes [13,14]. In recent years, researchers have enhanced GPSR by incorporating multi-hop information [15,16], energy level [17], and reinforcement learning techniques such as Q-learning to optimize routing decisions [18,19,20]. However, these approaches often rely on historical data for prediction, resulting in high computational overhead and underutilizing the future trajectory information of UAVs. In mission-oriented UAV swarms, nodes are typically equipped with planners to coordinate the swarm’s global trajectory, and trajectory data is shared within the swarm to facilitate collaborative task execution. Inspired by [21], our study innovatively integrates trajectory knowledge into GPSR routing by constructing a two-hop neighbor table at any given moment. This provides comprehensive spatiotemporal environmental awareness and, combined with multidimensional features and adaptive learning mechanisms, significantly improves routing stability and transmission efficiency.

The main contributions of this paper are as follows:

• Trajectory-Aware Neighbor Expansion: We propose a novel routing strategy for UAV swarms by leveraging mission-planner-derived trajectory knowledge. We develop a time-aware two-hop neighbor table that enables nodes to anticipate network dynamics, enhancing both local and global topological awareness. This proactive approach significantly reduces routing voids and fosters stable communication in dynamic environments;
• Q-Learning-Driven Routing Optimization: We introduce an adaptive routing framework for UAV networks, utilizing Q-learning to navigate their high dynamism. The framework incorporates a comprehensive state space, including two-hop distances, residual energy, queue lengths, and trajectory dynamics, and employs a multi-objective reward function with dynamically adaptive parameters. This design ensures balanced and efficient next-hop selection, promoting robust routing decisions under rapidly evolving network conditions;
• Latency-Focused Queue Scheduling: To improve real-time performance and load balance, we devise a composite delay model that leverages trajectory insights to refine packet transmission timing. This mechanism mitigates congestion, reduces end-to-end latency, and provides an effective queue management solution for high-load, multi-hop scenarios in UAV swarms;
• Comprehensive Simulation Validation: Through extensive ns-3 simulations, we assess Traj-Q-GPSR across diverse scenarios, varying node densities, UAV speeds, and CBR loads. Comparative evaluations reveal substantial enhancements in packet delivery ratio, end-to-end delay, routing efficiency, and throughput, demonstrating the robustness and effectiveness of our approach in dynamic UAV swarm settings.

2. Related Work

2.1. GPSR and Its Adaptations in Dynamic Networks

GPSR (Greedy Perimeter Stateless Routing) is a well-known position-based routing protocol, originally designed for vehicular ad hoc networks (VANETs) due to its scalability and low communication overhead [13,22,23,24]. However, when applied to flying ad hoc networks (FANETs), GPSR faces significant challenges caused by three-dimensional high-speed mobility, frequent link breaks, and highly dynamic topologies. The greedy forwarding strategy of GPSR tends to fall into routing voids, thus compromising delivery reliability and network robustness. To address these issues, recent studies have focused on enhancing link stability and improving the resilience of GPSR-based routing.

In [25], Alsalami et al. proposed the OLSR+GPSR protocol, combining the topology-driven OLSR with the location-driven GPSR. A fuzzy control system was used to dynamically adjust the HELLO message interval based on node velocity and positional prediction error, thereby improving link awareness. Moreover, the protocol integrates multiple metrics, including neighborhood density, velocity vector, buffer occupancy, and residual energy, into the MPR (Multi-Point Relay) selection process. Although the scheme enhances OLSR’s applicability to FANETs, it still requires partial topology maintenance and lacks predictive capability regarding future topology changes.

To address multipath diversity and load balancing, the AM-GPSR protocol [26] introduces an adaptive HELLO mechanism and a multipath greedy forwarding scheme. By adjusting the HELLO interval according to neighbor stability and maintaining multiple candidate paths with priority ranking, the protocol provides better resilience to link failures and improves balancing. However, the additional control overhead and reliance on single-hop information limit its scalability in large-scale UAV swarms.

SZLS-GPSR [27] introduces the concept of a “communication safe zone,” restricting candidate next-hop nodes to a stable sub-region within the communication radius. By jointly considering direction projection length, cumulative link duration, and movement direction within the zone, it significantly reduces the probability of falling into routing voids and improves packet delivery success. Nevertheless, this approach focuses mainly on local stability and does not exploit trajectory information to improve global topological awareness.

In UF-GPSR [28], Sudesh Kumar et al. introduce a utility function in the greedy forwarding process, which integrates residual energy ratio, distance reduction, movement direction, link risk, and average velocity to evaluate next-hop suitability. This enables more accurate and robust forwarding decisions.

In high-density networks, CF-GPSR [29] utilizes a cylindrical filtering method to constrain candidate nodes within a 3D cylinder between the source and the destination. Combined with metrics such as velocity alignment, ideal distance, residual energy, and movement angle, the protocol builds a composite utility function that enables low-overhead and reliable next-hop selection.

Overall, current GPSR adaptations improve performance in dynamic UAV networks by enhancing greedy forwarding with energy awareness, link stability, and multi-factor evaluation. However, most methods still rely on real-time single-hop neighbor information, lacking the ability to anticipate future topological changes. Moreover, few approaches have integrated machine learning-based decision-making to construct truly adaptive routing protocols.

2.2. Trajectory-Aware Routing in FANETs Networks

In FANETs, UAV mission planners can often generate highly accurate future trajectory sequences. Nevertheless, most existing studies rely on historical data for trajectory prediction rather than leveraging planner-generated trajectories directly [30].

In [17], the authors proposed EORB-TP, which predicts near-future movement based on time-series modeling of historical positions to estimate link duration and optimize forwarder set selection in opportunistic routing. Similarly, Vinti Gupta et al. in [31] applies an optimized 3D interpolation model for energy-efficient trajectory prediction. In [30], a prediction module is incorporated into a reactive greedy protocol to estimate link stability and improve path durability. While these approaches improve stability, they generally suffer from limited prediction accuracy and high computational complexity, which undermines real-time applicability.

Only a few works have directly utilized trajectory data from task planners for routing. In [21], the authors propose a mission-driven protocol where UAVs fly in a predefined formation and determine packet deliverability based on their current phase. Though effective in reducing packet loss in idealized scenarios, this method lacks flexibility in next-hop selection and adaptability to dynamic environments. Muhammad Morshed Alam et al. in [32] introduce a multi-agent deep reinforcement learning framework to jointly optimize trajectory, spectrum allocation, and routing. Although performance improves holistically, the solution’s high computational overhead and reliance on deep models hinder its deployment in resource-constrained UAV nodes.

In summary, trajectory-aware routing in FANETs is still underexplored. Most existing work depends on prediction rather than directly using planner-generated trajectories. Furthermore, such knowledge has not been fully integrated into position-based protocols like GPSR. To fill this gap, this paper proposes a novel integration of future trajectory information and Q-learning into GPSR, constructing time-aware two-hop neighbor tables to support adaptive and predictive routing decisions.

2.3. Q-Learning-Based Routing Protocols

The emergence of deep reinforcement learning (DRL) has led to increasing interest in intelligent routing protocols for FANETs [33,34,35,36,37,38]. Q-learning, as a model-free RL method, can iteratively refine routing decisions through interactions with the environment, making it particularly suitable for dynamic and rapidly evolving networks.

In [39], the authors extended traditional Q-learning by incorporating node velocity and direction into the state space, allowing decisions to align with mobility patterns. However, Q-FANET lacks mechanisms for predicting link failures or achieving effective load balancing. QTAR [38] (Q-learning-based topology-aware routing) improves upon this by introducing traffic-aware metrics into the reward function, enabling better congestion handling under high load, though it also does not consider trajectory dynamics. QMR [40] (Q-learning based Multi-objective optimization Routing) integrates multi-objective optimization into Q-learning via Pareto-based reward functions to balance latency, energy, and reliability. While theoretically powerful, its high-dimensional state-action space results in longer training times, limiting real-time applicability.

More recently, PER-D3QN [41] (Prioritized Experience Replay Dueling Double DQN) applies a prioritized experience replay mechanism and dueling architecture to mitigate Q-value overestimation and improve convergence. The approach builds congestion- and failure-aware state and reward models, showing improved robustness in dynamic FANET conditions. QFRP [37] (Q-Learning and Fuzzy Logic based Routing Protocol) enhances Q-table update granularity by combining HELLO and ACK signals, and integrates fuzzy logic to assess neighbors based on Q-value, link quality, and access delay. IQMR [15] (Improved Q-learning-based Multihop Routing) utilizes a $Q\left(\lambda \right)$ learning framework with an elaborate state space encompassing UAV status, connectivity, coverage, and collision risk, along with adaptive learning rates and dynamic HELLO intervals to improve convergence under varying network conditions.

Overall, Q-learning-based routing protocols are evolving toward multi-dimensional sensing, adaptive parameter tuning, and intelligent decision-making. Nevertheless, existing work primarily responds to current states without integrating future topology evolution or planner-based trajectory information. Our proposed method combines trajectory knowledge with Q-learning to construct a predictive, robust, and high-efficiency routing protocol tailored for dynamic UAV swarm networks.

3. System Model

3.1. Network Model

Figure 1 presents a high-level overview of the proposed system, which includes both the operational scenario and the internal routing logic within each UAV. In the illustrated mission environment, UAVs perform collaborative tasks over urban and forest areas with diverse mobility patterns. Each UAV is equipped with a routing module enhanced by trajectory knowledge and a Q-learning agent. The example highlights a typical routing void encountered when the GPSR protocol fails due to insufficient neighbor coverage. By contrast, the proposed method anticipates future link availability and dynamically selects more reliable multi-hop paths.

This paper introduces a UAV swarm that operates autonomously without ground station control, using only intra-swarm communication to execute tasks. The swarm consists of m low-altitude drones, $\mathcal{U}=\{U_1,U_2,$ …$,U_m\}$, initially distributed randomly in a 3D space. Trajectory information is shared within the swarm to coordinate task planning and execution. To support predictive coordination and efficient routing, each UAV periodically disseminates its future trajectory through extended beacon messages embedded in HELLO packets. Neighboring nodes maintain a time-indexed local trajectory table, which is updated upon reception to support anticipatory coordination and predictive routing decisions.

Each UAV node $U_i$ determines its three-dimensional coordinates at time t via GPS, represented as $P_i\left(t\right)=(x_i\left(t\right),y_i\left(t\right),z_i\left(t\right))$. We suppose that the 3D misssion area is limited in $(x_{min}\le x\le x_{max},y_{min}\le y\le y_{max},z_{min}\le z\le z_{max}).$ The swarm follows a predefined task planning algorithm to determine its collective trajectory, where the future position of each UAV is governed by its current state and planned motion:

$${\mathbf{p}}_i(t+\Delta t)=f({\mathbf{p}}_i\left(t\right),{\mathbf{v}}_i\left(t\right),{\mathbf{a}}_i\left(t\right),\Delta t)$$

(1)

Here, $f(·)$ represents the motion model function, ${\mathbf{v}}_i\left(t\right)$ denotes the velocity vector, and ${\mathbf{a}}_i\left(t\right)$ represents the acceleration vector. The trajectory planning algorithm ensures collision avoidance as a fundamental constraint, maintaining a minimum safety distance r between any two UAVs at all times:

$$∥q_i\left(t\right)-q_j\left(t\right)∥\ge r\forall i\ne j,t\ge 0.$$

(2)

where $∥·∥$ represents the Euclidean norm.

We assume that UAVs have a fixed communication range, allowing wireless communication between UAVs that are within this radius. Consequently, the network topology can be modeled as a time-dependent graph $\mathcal{G}\left(t\right)=\left(\mathcal{V}\right(t),\mathcal{E}(t\left)\right)$, where the edge set $\mathcal{E}\left(t\right))$ represents the dynamic communication links between UAVs.

3.2. Energy Model

Considering the heterogeneity in UAV energy levels, we assume that each UAV $U_i$ starts with an initial energy $E_{initial}$, which follows a random distribution within a certain range. As the mission progresses, energy is continuously consumed, primarily due to flight and communication.

Flight energy consumption is the primary contributor to overall UAV energy usage and is influenced by aerodynamic properties, thrust requirements, and velocity. It consists of two main components: hovering power consumption and motion power consumption.The power required to maintain a hovering state in the absence of wind resistance is given by

$$P_{\mathrm{hover},i}={\displaystyle \frac{mg}{\eta }}$$

(3)

where m is the UAV’s mass and $\eta$ represents the efficiency of the propulsion system, which quantifies the conversion between electrical power and mechanical thrust. During motion, energy consumption is largely determined by the UAV’s velocity and aerodynamic drag, expressed as

$$P_{\mathrm{motion},i}={\displaystyle \frac{1}{2}}{C}_d\rho A{v}_i^3$$

(4)

where $C_d$ is the drag coefficient, $\rho$ is the air density, A is the UAV’s frontal area, and $v_i$ is its velocity.The total flight energy consumption over a time interval $\Delta t$ is given by

$$E_{\mathrm{flight},i}={\int }_t^{t+\Delta t}{P}_{\mathrm{flight},i}dt={\int }_t^{t+\Delta t}{P}_{\mathrm{hover},i}+P_{\mathrm{motion},i}dt$$

(5)

The energy consumed for communication depends on the power required for information exchange with neighboring UAVs. It consists of three main components: transmission power, reception power, and idle power when the wireless module remains active but is not transmitting or receiving data. Assuming the transmission and reception power remain constant over time, the total communication energy consumption over $\Delta t$ is given by

$$E_{\mathrm{comm},i}=P_{\mathrm{tx},i}{T}_{\mathrm{tx}}+P_{\mathrm{rx},i}{T}_{\mathrm{rx}}+P_{\mathrm{idle},i}{T}_{\mathrm{idle}}$$

(6)

where $P_{\mathrm{tx},i}$ and $P_{\mathrm{rx},i}$ are the power required for transmission and reception, respectively, and $T_{\mathrm{tx}}$,$T_{\mathrm{rx}}$,$T_{\mathrm{idle}}$ represent the time spent in each corresponding state.

At any given time t, the remaining energy of UAV $U_i$ is

$$E_{\mathrm{res},i}\left(t\right)=E_{\mathrm{initial},i}-(E_{\mathrm{flight},i}+E_{\mathrm{comm},i})$$

(7)

If this energy level drops below the predefined threshold $E_{\mathrm{threshold}}$, the UAV is considered to have insufficient power to continue its mission and must disengage from the network to return to base.

3.3. Transmission Delay

Packet transmission delay in UAV networks is influenced by queuing delay $T_q$, channel contention delay $T_c$, and transmission delay $T_t$. Each UAV node maintains a FIFO-based transmission queue. To accurately estimate packet transmission delay, we adopt the M/M/1 queue model, which characterizes a single-server system with Poisson arrivals and exponentially distributed service times, making it well-suited for wireless communication modeling.

According to M/M/1 queue theory, the average queuing delay for a packet is

$$T_q={\displaystyle \frac{\rho }{\mu (1-\rho )}}$$

(8)

where the queue utilization factor $\rho =\lambda /\mu$ must satisfy $0\le \rho <1$ for stability. The service rate $\mu$ is determined by the available channel bandwidth B and the packet size L, such that $\mu =B\left(bps\right)/L\left(bits\right)$.

UAV nodes typically use the CSMA/CA mechanism for channel access, leading to additional contention delays. We adopt the Bianchi backoff model [42] to characterize this contention. In this model, a node undergoes multiple backoff stages before successfully transmitting a packet, where the backoff time per stage is

$$T_{\mathrm{backoff}}={\displaystyle \frac{C{W}_{min}+1}{2}}·T_{\mathrm{slot}}$$

(9)

where $C{W}_{min}$ is the minimum contention window size, and $T_{\mathrm{slot}}$ is the time slot duration. The expected number of backoff stages, $E\left[b\right]=(1-p_c)/p_c$ (where $p_c$ is the collision probability dependent on network load and neighboring nodes), determines the channel contention delay.

$$T_c=E\left[b\right]·T_{\mathrm{backoff}}$$

(10)

The transmission delay, which represents the time required to physically send the packet over the channel, is given by

$$T_t={\displaystyle \frac{L\left(bits\right)}{B\left(bps\right)}}$$

(11)

Combining the queuing, contention, and transmission delays, the total estimated delay for a packet is

$$T_{\mathrm{delay}}={\displaystyle \frac{\rho L}{B(1-\rho )}}+E\left[b\right]·T_{\mathrm{backoff}}+{\displaystyle \frac{L}{B}}$$

(12)

Due to dynamic topology changes in UAV networks, packet transmission delay depends on both the current queue state and the future network topology. To enhance routing efficiency, we predict the transmission time and assess whether a suitable next-hop neighbor will be available. For a packet p arriving at time t, the expected transmission time is

$$t_{\mathrm{send},p}=t+T_{\mathrm{delay}}$$

(13)

Using trajectory information shared among the UAV swarm, the set of next-hop neighbors at $t_{\mathrm{send},p}$ can be determined as:

$${\mathcal{N}}_i\left(t_{\mathrm{send},p}\right)=\{U_j\mid d_{ij}\left(t_{\mathrm{send},p}\right)\le R\}$$

(14)

where R is the fixed communication radius.If no next-hop neighbors are available at $t_{\mathrm{send},p}$, the packet is dropped to prevent unnecessary queuing:

$$\mathrm{If}{\mathcal{N}}_i\left(t_{\mathrm{send},p}\right)=\varnothing \mathrm{then}\mathrm{drop}\mathrm{packet}.$$

(15)

By incorporating trajectory-based predictions, this approach reduces unnecessary retransmissions, optimizes queue management, and improves overall network efficiency.

Figure 2 visualizes how this paper combines the trajectory information with the sending delay model to optimize the queue management of the nodes.

In the diagram, dashed blue lines represent the UAVs’ planned trajectories, while solid blue arrows denote available communication links. Under a task-driven deployment, UAVs $U_A,U_B,U_C$, and $U_G$ converge on the left to execute Task 1, whereas $U_D,U_E$, and $U_F$ move to the right for Task 2.

At time $t_k$, shown in panel (a), both $U_A$ and $U_E$ possess packets in their transmission queues addressed to $U_G$, and their queue lengths are equal. Using the predicted transmission delay, both packets are slated for dispatch at time $t_{k+5}$, as depicted in panel (b). Although no viable route from $U_A$ to $U_G$ exists at $t_k$, by $t_{k+5}$ UAV $U_A$ can reach $U_G$ via the stable multi-hop path

$$\mathcal{P}:A\to B\to C\to D\to G.$$

Conversely, $U_E$ becomes isolated, its neighbor set empties, and no path to $U_G$ is available at $t_{k+5}$. Consequently, its packet is proactively discarded. This anticipatory drop prevents wasteful transmissions and thus materially improves queue efficiency.

4. The TRAJ-Q-GPSR Algorithm

4.1. Trajectory-Informed UAV Routing

To address the inherent limitation of traditional GPSR protocols, where routing decisions rely solely on the current positions of nodes for greedy forwarding, this study introduces the integration of trajectory knowledge and a two-hop neighbor table. These enhancements enable UAV nodes to obtain a more comprehensive understanding of both spatial and temporal network dynamics. As a result, the likelihood of encountering routing voids is reduced, and both routing stability and path selection are significantly improved. The proposed approach first describes the acquisition of trajectory knowledge, followed by its mathematical modeling. Based on this foundation, a method is developed to construct a two-hop neighbor table that leverages the acquired trajectory information.

We assume that the mission planner computes the trajectory for each UAV within a future time window $[t_0,t_f]$. The trajectory ${\mathcal{T}}_i$ of UAV $U_i$ at time t is defined as a time-parameterized geometric state representation:

$${\mathcal{T}}_i\left(t\right)=\left(p_i\left(t\right),v_i\left(t\right),a_i\left(t\right)\right),t\in [t_0,t_f]$$

(16)

where $p_i\left(t\right),v_i\left(t\right)$ and $a_i\left(t\right)$ denote the position, velocity, and acceleration of UAV $U_i$ at time t, respectively.

To facilitate computation and storage, the trajectory is discretized into a finite time series:

$${\mathcal{T}}_i=\{(p_i\left(t_k\right),v_i\left(t_k\right),a_i\left(t_k\right))\mid t_k=t_0+k·\Delta t,k=0,1,\dots ,K\}$$

(17)

where $\Delta t$ represents the sampling interval, and K is the number of planned steps, determined by the planning algorithm’s prediction range.

For the entire UAV swarm, the set of all predicted trajectories within the given time window is defined as

$$\mathcal{T}=\bigcup _{i=1}^N{\mathcal{T}}_i.$$

(18)

This dataset encapsulates the motion information of all UAV nodes within the prediction horizon, providing valuable spatiotemporal knowledge for routing decisions.

In GPSR protocol, routing nodes can only sense their immediate neighbors at the current moment. Given the rapid changes in network topology, this often results in routing failures or frequent transitions into the perimeter forwarding mode. However, by sharing trajectory information, UAVs can proactively anticipate their future neighbors. The neighbor set ${\mathcal{N}}_i\left(t_k\right)$ of UAV $U_i$ at a future time $t_k$ is defined as the set of all nodes within its communication range R:

$${\mathcal{N}}_i\left(t_k\right)=\{U_j\mid \parallel p_i\left(t_k\right)-p_j\left(t_k\right)\parallel \le R\}.$$

(19)

By leveraging trajectory sharing, each UAV can obtain information about its neighbors’ neighbors, thereby constructing a two-hop neighbor table:

$${\mathcal{N}}_i^{\left(2\right)}\left(t_k\right)=\bigcup _{U_j\in {\mathcal{N}}_i\left(t_k\right)}{\mathcal{N}}_j\left(t_k\right).$$

(20)

Here, ${\mathcal{N}}_i^{\left(2\right)}\left(t_k\right)$ represents the two-hop neighbor set of UAV $U_i$ at a future time $t_k$.

Constructing a two-hop neighbor table at future time instances enables UAVs to make more informed routing decisions. This approach not only helps avoid local optima and reduces the likelihood of routing voids, but also supports the optimization of queue management strategies. By incorporating estimated packet transmission times, the method ultimately contributes to more robust and efficient routing performance.

The Traj-Q-GPSR theoretical framework of this paper is shown in Figure 3, which integrates trajectory knowledge with reinforcement learning to optimize routing decisions. The left part constructs a two-hop neighbor table using trajectory information. It evaluates node availability at the estimated transmission time based on a delay model, thereby determining whether to enqueue or drop the packet. The right part incorporates reinforcement learning to dynamically adjust routing weights. These adjustments are based on state features such as distance, queue length, and residual energy. This mechanism supports more efficient forwarding decisions and enables continuous policy updates.

4.2. Q-Learning Framework and Theory

Q-learning, a model-free reinforcement learning method, enables agents to learn optimal policies through interaction with their environment, adapting to unknown conditions. Its low complexity and adaptive nature make it ideal for optimizing routing decisions on edge devices like UAVs, particularly given the dynamic and uncertain topology of UAV networks caused by node mobility.

Q-learning is based on value iteration, and its theoretical foundation lies in the Markov Decision Process (MDP). An MDP provides a rigorous mathematical framework for describing how an agent learns the optimal policy in a stochastic environment, and is defined by the five-tuple $(S,A,P,R,\gamma )$. Here, the state space S represents all possible states of the system; in our study, the state includes the UAV node’s current position, velocity, neighbor information, and queue status. The action space A encompasses all possible actions that the agent can take, corresponding to the selection of a next-hop forwarding node. The state transition probability P defines the probability that the system transitions to a new state $s^{\prime }$ after the agent takes an action a in state s; this transition is influenced by the UAV’s motion model and changes in the network topology. The reward function R measures the immediate gain obtained by executing action a in state s, thereby reflecting the quality of that action. The discount factor $\gamma \in (0,1]$ is used to balance the importance of immediate and long-term rewards, with larger $\gamma$ placing more emphasis on long-term returns and smaller $\gamma$ on immediate gains.

The fundamental principle of Q-learning relies on an iterative update mechanism between states, actions, and rewards. Its objective is to find the optimal action-selection strategy for any state s by continuously exploring, experimenting, and updating Q-values, thereby enhancing routing efficiency and network adaptability. Specifically, when an agent in state s chooses an action a, it receives an immediate reward $R(s,a)$ and transitions to a new state $s^{\prime }$. The agent maintains a Q-value table $Q(s,a)$ and updates it using the Bellman equation:

$$Q(s,a)\leftarrow Q(s,a)+\alpha \left[R(s,a)+\gamma \underset{a^{\prime }}{max}Q(s^{\prime },a^{\prime })-Q(s,a)\right]$$

(21)

Here, $\alpha$ is the learning rate, determining the influence of newly acquired information on the existing Q-values, and ${max}_{a^{\prime }}Q(s^{\prime },a^{\prime })$ represents the maximum Q-value achievable from the new state $s^{\prime }$, i.e., the maximum expected future reward. Through this update mechanism, the Q-values gradually converge to their optimal values, guiding the agent to make efficient decisions in a dynamic environment.

4.3. Traj-Q-GPSR Algorithm

Traj-Q-GPSR integrates trajectory information and two-hop neighbor awareness into routing, enabling forward-looking Q-learning for efficient UAV data delivery. This section details the implementation of the Traj-Q-GPSR algorithm from four aspects: the design of the state and action spaces, the construction of the reward function, the dynamic adjustment strategy for Q-learning parameters, and the pseudocode representation.

4.3.1. Definition of State and Action Spaces

In this algorithm, each UAV node functions as an agent whose state representation comprises both physical attributes and network context. Specifically, the state includes the node’s current position, velocity, and acceleration, along with two-hop neighbor information obtained from shared trajectory data. Additionally, the queue status is incorporated to reflect the current load of the node. The state space is formally defined as follows:

$$s=\left(p_i,v_i,a_i,{\mathcal{N}}_i^{\left(2\right)},Q_i\right)$$

(22)

where $Q_i$ indicates the current queue length reflecting the load of the UAV. The action space A defines the decision options available to the UAV in each state. Given the high dynamism of UAV networks, conventional GPSR, which relies solely on the current positions of one-hop neighbors for greedy forwarding, often suffers from local optima and routing voids. Therefore, this study extends the action space to include two-hop neighbor information, allowing UAVs to make next-hop selections with broader topological awareness and achieve improved global routing performance. The action space is formally defined as

$$A=\{a\mid a\in {\mathcal{N}}_i^{\left(2\right)}\}$$

(23)

4.3.2. Design of the Reward Function

The reward function forms the cornerstone of Q-learning, directly shaping the optimization direction of routing decisions. It is designed to balance multiple objectives—transmission efficiency, energy distribution, network load, and mobility prediction—while improving stability by mitigating routing voids. The composite reward function is expressed as

$$R(s,a)=R_{\mathrm{two}-\mathrm{hop}}(s,a)+w_v{R}_v(s,a)+w_{h}P(s,a)$$

(24)

where $R_{\mathrm{two}-\mathrm{hop}}(s,a)$ denotes the two-hop reward, $R_v(s,a)$ is the velocity projection reward, and $P(s,a)$ is the routing void penalty term. The coefficients $w_v$ and $w_h$ represent the weights of the velocity reward and the void penalty, respectively.

• Two-hop reward
  The two-hop reward is composed of single-hop rewards and their extension. The single-hop reward $R(s\to n)$ evaluates the immediate benefit of selecting a neighbor $U_n$ as the next hop when UAV node $U_i$ is in state s. It considers three key factors: distance, residual energy, and queue load, defined as

  $$R(s\to n)=w_d{R}_d\left(n\right)+w_e{R}_e\left(n\right)+w_q{R}_q\left(n\right)$$

  (25)
  • Distance Reward $R_d\left(n\right)$
    Promotes selection of nodes closer to the destination, calculated as the proportion of distance reduced:

    $$R_d\left(n\right)={\displaystyle \frac{d_i-d_n}{d_i}}$$

    (26)
    Here, $d_i=\parallel p_i\left(t\right)-p_D\parallel$ is the Euclidean distance from the current node to the destination $U_D$, and $d_n=\parallel p_n\left(t\right)-p_D\parallel$ is the distance from the neighbor to the destination. If $d_n<d_i$, then $R_d\left(n\right)>0$, yielding a positive reward; otherwise, it is negative.
  • Energy Reward $R_e\left(n\right)$
    This prioritizes nodes with higher remaining energy to extend network lifetime:

    $$R_e\left(n\right)={\displaystyle \frac{E_n}{E_{max}}}$$

    (27)

    where $E_n$ is the remaining energy of $U_n$, and $E_{max}$ is the maximum energy capacity.
  • Queue Length Reward $R_q\left(n\right)$
    This prefers nodes with lower loads to reduce transmission delay:

    $$R_q\left(n\right)=1-{\displaystyle \frac{Q_n}{Q_{max}}}$$

    (28)
  Expanding the single-hop reward to consider the influence of the next-hop neighbors, the two-hop reward is defined as

  $$R_{\mathrm{two}}-\mathrm{hop}(s,a)=R(s\to n)+\sum _{m\in \mathcal{N}\left(n\right)}{\displaystyle \frac{1}{D_m}}R(n\to m)$$

  (29)

  where $R(n\to m)$ is the single-hop reward from $U_n$ to its neighbor $U_m$, and $D_m$ is the degree of $U_m$ (number of neighbors). The normalization factor ${\displaystyle \frac{1}{D_m}}$ prevents reward inflation in highly connected regions.
• Velocity Projection Reward
  In UAV networks, node mobility affects link stability. The velocity projection reward $R_v(s,a)$ leverages trajectory information to evaluate whether the selected next-hop UAV is moving toward the destination, improving adaptation to network dynamics.
  Given the velocity vector $v_n\left(t\right)$ of UAV $U_n$ and the directional vector $d=p_d-p_n\left(t\right)$ from $U_n$ to the destination $U_d$, the reward is defined as

  $$R_v(s,a)=cos\left(\theta \right)={\displaystyle \frac{v_n\left(t\right)·d}{|v_n\left(t\right)\left|\right|d|}}$$

  (30)
  A positive $cos\left(\theta \right)$ indicates movement toward the destination, yielding a positive reward; a negative value indicates movement away, resulting in a penalty. This encourages selecting UAVs whose motion trends enhance link availability.
• Routing Void Penalty
  To prevent routing voids—where no neighbor is closer to the destination than the current node—the penalty term refines void severity assessment. Void severity $H\left(s\right)$ is defined as

  $$H\left(s\right)=1-{\displaystyle \frac{N_{\mathrm{closer}}\left(s\right)}{N_{\mathrm{total}}\left(s\right)}}$$

  (31)

  where $N_{\mathrm{closer}}\left(s\right)$ is the number of neighbors closer to the destination than $U_i$, and $N_{\mathrm{total}}\left(s\right)$ is the total number of neighbors. The penalty is

  $$P(s,a)=-H\left(s\right)$$

  (32)
  If $H\left(s\right)=1$, indicating a full routing void, the penalty is maximized.

4.3.3. Adaptive Q-Learning Parameters

The learning rate $\alpha$ and discount factor $\gamma$ are critical parameters in Q-learning, directly influencing the convergence speed and decision-making performance of the algorithm. Given the high dynamics and task diversity in UAV networks, these parameters require adaptive adjustment. In rapidly changing environments, they should facilitate quick learning, while in more stable conditions, they enable fine-tuned decision-making. This approach ensures a balanced trade-off between exploration and exploitation, thereby enhancing adaptability to topology variations.

The learning rate $\alpha$ determines the extent to which new information influences the existing Q-values. A fixed $\alpha$ approach can struggle in dynamic environments: it may converge too slowly when the environment changes drastically or over-adjust and deviate from the optimal solution when the environment is stable. To address this, an adaptive adjustment mechanism based on Q-value fluctuations is proposed. When Q-values fluctuate significantly, indicating unstable learning, $\alpha$ should increase to speed up updates. When Q-values stabilize, $\alpha$ should decrease to refine the strategy. The formula for dynamically adjusting $\alpha$ is

$$\alpha ={\alpha }_{min}+({\alpha }_{max}-{\alpha }_{min})·{\displaystyle \frac{1}{1+|Q(s,a)-Q_{\mathrm{prev}}(s,a)|}}$$

(33)

where ${\alpha }_{max}$ and ${\alpha }_{min}$ are the maximum and minimum allowable values for the learning rate, $Q(s,a)$ and $Q_{\mathrm{prev}}(s,a)$ represent the current and previous Q-values, respectively. This strategy enables UAVs to rapidly adjust their policies in dynamic environments while maintaining stability during convergence.

The discount factor $\gamma$ determines the balance between immediate rewards and long-term returns. In UAV networks, changes in the neighbor set reflect the network’s topological instability. When the neighbor set changes significantly, future uncertainty rises, and $\gamma$ should decrease to reduce reliance on future rewards. When changes are minimal, $\gamma$ can increase to prioritize long-term planning. The neighbor change rate is defined as

$$\Delta N=|N\left(t_k\right)-N\left(t_{k-1}\right)|$$

(34)

Based on this, the adaptive adjustment formula for $\gamma$ is

$$\gamma ={\gamma }_{min}+({\gamma }_{max}-{\gamma }_{min})·e^{-\beta \Delta N}$$

(35)

where ${\gamma }_{max}$ and ${\gamma }_{min}$ denote the upper and lower bounds of the discount factor, and $\beta$ is a tuning parameter. When $\Delta N$ is large (indicating a highly dynamic environment), $\gamma$ decreases, focusing on immediate decisions over uncertain future rewards. When $\Delta N$ is small (stable environment), $\gamma$ nears ${\gamma }_{max}$, emphasizing long-term gains. This approach enhances adaptability in dynamic conditions while maintaining optimization in stable ones.

By dynamically adjusting the learning rate $\alpha$ based on Q-value fluctuations and the discount factor $\gamma$ based on neighbor change rates, this strategy enables Q-learning to quickly adapt to environmental shifts in UAV networks while ensuring long-term convergence stability. This mechanism significantly boosts the efficiency and robustness of routing decisions, offering an effective solution for learning and optimization in UAV network applications.

4.4. Pseudo-Code

The pseudo-code representation of the Traj-Q-GPSR algorithm proposed in this paper is summarized in Algorithm 1.

Algorithm 1 Traj-Q-GPSR Algorithm

Require: Global trajectory information $\mathcal{T}$, current time $t_k$, node energy, communication radius R Ensure: Updated Q-states and next-hop routing decisions       1: /*Initialization*/       2: for each UAV node i do       3:     Initialize the current state s based on $\mathcal{T}$       4:     Construct the two-hop neighbor table ${\mathcal{N}}_i^{\left(2\right)}\left(t_k\right)$       5:     Initialize the Q-table $Q(s,a)\leftarrow 0$       6:     Initialize parameters $\alpha ={\alpha }_{max}$ and $\gamma ={\gamma }_{max}$       7: end for       8: /*Packet Processing Logic*/       9: if A packet is pending for transmission then       10:     if Queue is at maximum capacity. then       11:         drop the packet       12:     else       13:         compute the estimated sending time $t_{\mathrm{send},p}$       14:         if ${\mathcal{N}}_i\left(t_{\mathrm{send},p}\right)=\varnothing$ then       15:            drop the packet       16:         else       17:            enqueue the packet       18:         end if       19:     end if       20: end if       21: /*Q-Learning Routing Decision*/       22: while network is active do       23:     for each UAV node i do       24:         if $E_{\mathrm{res},i}\left(t_k\right)\ge E_{\mathrm{threshold}}$ then       25:            if the queue is non-empty then       26:                if there’s a closer neighbor then       27:                    route based on Q-learning action space       28:                    forward the packet       29:                    compute immediate reward via (24).       30:                    update the Q-table       31:                    update $\alpha$ and $\gamma$ via (33) and (35)       32:                else       33:                    encountering routing voids       34:                    perimeter forwarding by right-hand rule       35:                end if       36:            end if       37:         else       38:            node i departs the network and returns to base       39:         end if       40:     end for       41: end while

5. Simulation Results and Discussions

5.1. Simulation Settings

In our simulations, we employ ns-3 as the network simulator and MATLAB R2022b for plotting the results. Notably, the mobility scenarios are generated using BonnMotion [43], a tool developed by the University of Bonn, which produces node movement trace files for the UAV network based on various mobility models. The detailed simulation parameters [15,44] are summarized in

Table 1. Simulation parameters.

Parameter	Value
Simulator	ns-3.31, MATLAB
Packet Size	512 bytes
Simulation Time	150 s
Simulation Area	$2000\mathrm{m}\times 2000\mathrm{m}\times 300\mathrm{m}$
Speed Range	10–30 m/s
Communication Range	250 m
Bandwidth	10 MHz
Number of Nodes	30, 40, 50, 60, 70, 80, 90, 100
Traffic Type	CBR
CBR Rate	2 Mbps
Number of CBR Connections	10, 20, 30, 40, 50
HELLO interval	1 s
Frequency of trajectory updates	5 Hz
MAC Protocol	IEEE 802.11p
Transport Protocol	UDP
Initial Energy	900–1000 J
Energy Threshold	100 J
Propagation Model	Nakagami Model
Mobility Model	Gauss–Markov Mobility Model

.

We adopt four performance metrics [22,38,44] to quantify the improvements in the proposed routing protocol, and—by holding all other factors constant—evaluate its behavior under varying node densities, mobility speeds, and CBR connections.

• Packet Loss Ratio (PLR):The PLR is defined as the ratio of lost packets to the total packets sent by the source, reflecting packet losses due to routing failures, congestion, or channel errors:

  $$\mathrm{PLR}={\displaystyle \frac{N_{\mathrm{lost}}}{N_{\mathrm{sent}}}}.$$

  (36)
  A lower PLR indicates higher reliability and robustness in dynamic topologies and interference-prone environments.
• End-to-End Delay (E2ED): E2ED measures the total time a packet takes from transmission at the source to reception at the destination, capturing the timeliness of the protocol:

  $$\mathrm{E}2\mathrm{ED}={\displaystyle \frac{1}{N_{\mathrm{recv}}}}\sum _{p=1}^{N_{\mathrm{recv}}}\left(t_{\mathrm{recv},p}-t_{\mathrm{send},p}\right),$$

  (37)

  where $N_{\mathrm{recv}}$ is the number of successfully received packets, and $t_{\mathrm{send},p}$, $t_{\mathrm{recv},p}$ are the send and receive timestamps of packet p.
• Routing Efficiency: To assess resource utilization in multi-hop scenarios, we define routing efficiency as the average ratio of successfully delivered packets per forwarding hop:

  $$\mathrm{RoutingEfficiency}={\displaystyle \frac{1}{N}}\sum \left({\displaystyle \frac{\mathrm{totalRx}}{\mathrm{totalTx}\times \mathrm{HopCount}}}\right).$$

  (38)
  This metric minimizes forwarding overhead while ensuring successful delivery. A higher value indicates better path optimization and resource efficiency.
• Throughput: Throughput quantifies the network’s capacity to successfully deliver payload data per unit time, in bits per second (bps):

  $$\mathrm{Throughput}={\displaystyle \frac{\mathrm{totalRx}\times \mathrm{PacketSize}\times 8}{\mathrm{simtime}}}.$$

  (39)
  It directly reflects the protocol’s data delivery capability and bandwidth utilization under given network conditions.

In addition, this study examines node energy consumption and routing overhead to complement the performance evaluation. Energy consumption reflects the protocol’s impact on forwarding-related power usage, while routing overhead quantifies control message load required for maintaining connectivity and route discovery under dynamic conditions.

5.2. Impact of Node Density

First, we analyze the impact of UAV node density on network performance. As shown in Table 1, a fixed simulation area is configured, and the node mobility speed ranges from 15 m/s to 25 m/s in this section. For each parameter setting, 30 independent simulations are conducted, and a 95% confidence interval is adopted. Figure 4 shows the impact of node density on network performance.

At any node density, the proposed Traj-Q-GPSR algorithm consistently achieves a significantly lower packet loss rate compared to the traditional GPSR. As node density increases, the packet loss curve reflects a trade-off between improved connectivity and intensified channel contention. Specifically, when the number of nodes increases from 30 to around 70, the connectivity rapidly improves and path breaks are significantly reduced. Leveraging trajectory prediction and two-hop queue optimization, Traj-Q-GPSR reduces the packet loss rate from approximately 33.13% to 25.5%. In contrast, traditional GPSR, relying solely on instant location and single-hop decision-making, struggles with frequent neighbor changes and inefficient recovery, leading to an increased packet loss rate, peaking at 40.66%. As the number of nodes continues to increase from 70 to 100, the network enters a high-density phase. The redundancy among nodes increases, and alternative routes become more available, allowing GPSR to partially compensate for local routing failures and reducing the loss rate to 35.46%. Meanwhile, Traj-Q-GPSR, by proactively filtering invalid packets and applying queue-aware intelligent routing, effectively mitigates congestion and queuing losses under dense scenarios, further reducing the packet loss rate to 22.76%.

As can be seen from the end-to-end delay curves in Figure 4b, across all regimes, Traj-Q-GPSR achieves substantially lower delays than the conventional GPSR. Although GPSR’s latency decreases modestly as density grows, it remains around 1000 ms. In contrast, Traj-Q-GPSR maintains delays below 500 ms, rapidly dropping to 191.94 ms at moderate densities and stabilizing near 100 ms at high densities. The observed performance gap stems from the fundamental differences in their forwarding strategies. GPSR employs a one-hop greedy forwarding approach, which often triggers perimeter recovery when routing voids occur, resulting in frequent detours and retransmissions. In contrast, Traj-Q-GPSR utilizes trajectory information and two-hop neighbor awareness to enable more forward-looking and informed relay selection. It also integrates queue-aware packet management to improve forwarding efficiency and reduce unnecessary overhead. Together, these innovations dramatically curtail the detour and retransmission overheads that dominate GPSR’s delay profile.

As shown in Figure 4c, routing efficiency for both Traj-Q-GPSR and GPSR decreases as node count grows from 30 to 100, but Traj-Q-GPSR consistently outperforms GPSR. At low densities, adding nodes rapidly improves connectivity but increases average hop-count and forwarding overhead even faster, causing the ratio ${\displaystyle \frac{\sum \mathit{totalRx}}{\sum (\mathit{totalTx}\times \mathit{HopCount})}}$ to drop. Once the topology nears saturation, gains in delivered packets and in transmitted-hop product balance each other, stabilizing efficiency. In dense regimes, Traj-Q-GPSR’s trajectory prediction and two-hop neighbor awareness fully exploit available paths, yielding low delay and high reliability, so its efficiency remains at a high plateau.

Figure 4d plots throughput as node count increases from 30 to 100. Enhanced connectivity and spatial reuse drive overall growth: GPSR rises from approximately $1.03\times {10}^5$ bps to $1.40\times {10}^5$ bps, whereas Traj-Q-GPSR climbs from $1.32\times {10}^5$ bps to $1.92\times {10}^5$ bps. This trend mirrors reductions in packet loss and latency, reflecting more multi-hop paths, fewer link failures and retransmissions, and lower MAC-layer contention at high density. Thanks to its fusion of two-hop neighbor information, trajectory prediction, and reinforcement-learning-based selection, Traj-Q-GPSR delivers about 28% higher throughput than GPSR at 30 nodes, increasing to 37% at 100 nodes, thereby markedly improving data-delivery efficiency in dense UAV networks.

5.3. Impact of Node Mobility Speed

Node mobility speed is a key factor affecting the stability and performance of FANETs. Higher mobility speeds result in more frequent topology changes and increased link-break probability, imposing greater demands on the robustness and adaptability of routing protocols. In this subsection, we investigate the performance of each protocol across four core metrics under various mobility speeds to assess their adaptability in highly dynamic environments. The network is fixed at 70 UAV nodes with 20 CBR pairs. Each speed scenario is simulated 30 times, and all results are reported with a 95% confidence interval.

As illustrated in Figure 5a, the packet loss rate of Traj-Q-GPSR remains nearly constant as the UAV average speed increases, while GPSR’s loss rate rises from 21.8% up to 26.8%, with a pronounced jump around 20 m/s and a further increase nearing 27%. This divergence stems from their differing robustness to high-mobility links: GPSR relies solely on one-hop greedy forwarding, so when nodes move rapidly links break more frequently and retransmissions surge. In contrast, Traj-Q-GPSR uses two-hop neighbor tables and trajectory information to select more stable relays. It also applies queue-based rules to avoid sending packets over links that are likely to break, reducing both retransmission overhead and packet loss. At 30 m/s, Traj-Q-GPSR’s loss rate is 17% lower than GPSR’s, clearly demonstrating its superior link reliability in highly dynamic environments.

As shown in Figure 5b, when the UAV speed increases from 10 m/s to 30 m/s, the end-to-end delay of GPSR first rises from 878.7 ms to 991.4 ms, and then drops to 827.5 ms. This trend is mainly caused by prolonged fallback to recovery mode under low-to-moderate mobility, where broken links result in delayed forwarding. At higher speeds, however, faster topology updates enable quicker discovery of alternative relay paths and reduce average waiting time. In contrast, Traj-Q-GPSR maintains a stable delay range of 134–170 ms across all speeds, benefiting from trajectory prediction for preemptive two-hop neighbor construction and Q-learning-based robust relay selection. This allows Traj-Q-GPSR to sustain higher routing connectivity and forwarding efficiency under dynamic conditions, achieving an approximate 80% reduction in overall delay.

Figure 5c shows that Traj-Q-GPSR consistently outperforms GPSR in routing efficiency, and both exhibit a synchronized “rise-then-fall” trend as speed increases. This behavior closely mirrors the combined characteristics of packet loss rate and end-to-end delay discussed earlier. At moderate speeds, topology updates occur at an optimal rate, yielding good inter-node connectivity and enhanced path validity, which in turn boosts successful packet delivery and reduces average hop count, driving up routing efficiency. As speed increases, frequent link breakages reduce route stability. Although delay may slightly decrease, rising packet loss lowers throughput, leading to reduced efficiency. By contrast, Traj-Q-GPSR leverages trajectory awareness and Q-learning to mitigate high-mobility link fluctuations, maintaining superior efficiency across all speed regimes—achieving a peak improvement of over 52% compared to GPSR.

As shown in Figure 5d, Traj-Q-GPSR consistently maintains a high and stable throughput performance, with values ranging from $1.8\times {10}^5$ bps to $2.0\times {10}^5$ bps. In contrast, GPSR achieves significantly lower throughput, fluctuating around $1.0\times {10}^5$ to $1.2\times {10}^5$ bps. This trend closely aligns with the packet loss rate and routing efficiency observed earlier, particularly at the speed of 15 m/s, where the lowest packet loss and highest efficiency result in elevated throughput, indicating a synergistic optimization of overall transmission quality. Even under high-speed mobility, Traj-Q-GPSR leverages trajectory awareness and reinforcement learning strategies to adapt swiftly, thereby preserving throughput stability. Overall, Traj-Q-GPSR consistently demonstrates superior link maintenance and transmission reliability across the entire speed range, significantly enhancing system throughput in dynamic network conditions.

5.4. Impact of CBR Connections

In this subsection, we investigate the impact of different CBR traffic intensities on network performance. By varying the number of CBR connections, we analyze the differences between Traj-Q-GPSR and traditional GPSR in the four key performance metrics mentioned above. This evaluation provides a direct insight into each protocol’s network carrying capacity and congestion resilience, assessing their adaptability and robustness under varying traffic loads. The simulation scenario in this section involves 70 UAV nodes with node mobility ranging from 15 m/s to 25 m/s. Each simulation setting was run 30 times, and the results are reported with a 95% confidence interval.

As shown in Figure 6a, as the number of CBR connections increases from 10 to 50, the network load intensifies, resulting in more frequent link contention and collisions, which collectively lead to a general rise in packet loss rate. Under such conditions, the traditional GPSR protocol sees its loss rate increase from approximately 33.4% to 39.3%, highlighting the limitations of its single-hop greedy forwarding strategy. This approach often leads to routing voids and excessive retransmissions under high load. In contrast, Traj-Q-GPSR achieves a packet loss rate of 28.5% even at 10 CBR connections, which rises to 33.3% at 20 connections due to increasing contention. However, aided by two-hop neighbor information, trajectory prediction, and queue-aware optimization, the loss rate decreases to 29.8% and 30.3% at 30 and 50 pairs, respectively—consistently maintained around 30%. Notably, under the heavy-load condition of 50 CBR pairs, Traj-Q-GPSR achieves a 30.3% loss rate, which represents a reduction of approximately 23% compared to GPSR. This result clearly demonstrates Traj-Q-GPSR’s superiority in mitigating congestion and avoiding routing voids recovery failures under high-traffic conditions.

As illustrated in Figure 6b, end-to-end delay increases significantly with the growth of CBR traffic intensity. This is primarily due to aggravated link contention, increased MAC-layer retransmissions, and the accumulation of queueing delays. In the traditional GPSR protocol, the delay surges from approximately 305.9 ms to 3209.5 ms, indicating the considerable overhead caused by frequent route recoveries and retransmissions triggered by its single-hop greedy forwarding under high traffic loads. In contrast, the delay in Traj-Q-GPSR only rises from 82.2 ms to 859.0 ms, consistently staying below one second. This trend is also consistent with the packet loss patterns observed in Figure 4a. As corroborated by earlier experimental results, Traj-Q-GPSR demonstrates substantial improvement in end-to-end latency, further validating its capability to maintain low delay in high-load communication scenarios.

As shown in Figure 6c, the routing efficiency of GPSR declines significantly with the increase in the number of CBR pairs. When the CBR count reaches 50, its routing efficiency drops by 40.4% compared to the case with only 10 CBR pairs. Moreover, during moderate traffic conditions, GPSR exhibits marked fluctuations, reflecting its unstable forwarding performance due to frequent routing voids and retransmissions under heavy loads. In contrast, Traj-Q-GPSR maintains a consistently high level of routing efficiency, with variations contained within 5% across all tested loads. Under the maximum traffic load of 50 pairs, Traj-Q-GPSR improves routing efficiency by 77.8% compared to GPSR, demonstrating a significantly enhanced data delivery success rate per unit of forwarding cost.

As illustrated in Figure 6d, the overall throughput increases steadily with the number of CBR communication pairs. This growth is attributed to the elevated traffic activity and improved resource utilization across the network. Throughout the process, Traj-Q-GPSR consistently achieves higher link utilization efficiency by leveraging future trajectory awareness and Q-learning-based routing decisions, particularly under high-traffic conditions. In contrast, GPSR exhibits significantly lower throughput. When the number of communication pairs reaches 50, the throughput of Traj-Q-GPSR reaches approximately $4.3\times {10}^5$ bps, which is double that of GPSR at only $2.14\times {10}^5$ bps. The performance gap widens as the traffic load increases, confirming the superior adaptability and scheduling efficiency of Traj-Q-GPSR in complex communication scenarios.

5.5. Energy Consumption

To comprehensively evaluate the proposed Traj-Q-GPSR protocol, we further compare its energy consumption with that of GPSR under varying node densities, as illustrated in Figure 7 and summarized in

Table 2. Energy consumption (J) based on the number of nodes.

Number of Nodes	30	40	50	60	70	80	90	100
Traj-Q-GPSR	27.1	53.3	112.5	172.3	232.8	290.1	345.5	374.7
GPSR	48.9	83.7	153.2	199.8	266.6	332.1	420.3	453.7

. By incorporating neighbor energy information into the routing decision process, Traj-Q-GPSR aims to enhance energy efficiency during data forwarding.

Compared to GPSR, Traj-Q-GPSR consistently exhibits marginally lower energy consumption across all tested node densities. As the number of nodes increases, the energy consumption of both protocols rises accordingly. In high-density scenarios with 100 nodes, Traj-Q-GPSR achieves a 17.4% reduction in energy consumption relative to GPSR.The improvement in energy efficiency is relatively moderate. This is primarily due to the additional overhead caused by the exchange of trajectory information. Nevertheless, the results highlight that Traj-Q-GPSR effectively reduces redundant transmissions and lowers routing maintenance costs in densely deployed FANET environments.

5.6. Routing Overhead

In the evaluation of routing overhead, this study considers two key dimensions: communication overhead and storage overhead. For communication overhead, as outlined in the system model section, trajectory information is periodically disseminated through HELLO packets that encapsulate positional beacons. The structure of the HELLO packet, as detailed in

Table 3. A HELLO packet header format table.

Field Name	Size (Bytes)
Packet Type	1
Number of Neighbors	2
Node ID	2
Current Coordinates ($x,y,z$)	24
Current Queue Length	1
Current Energy Level	2

,

Table 4. A format table of packet header.

Field Name	Size (Bytes)
Packet Type	1
Perimeter Mode Flag	1
Node ID	2
Destination Coordinates $(x,y,z)$	24
Update Timestamp	4
Perimeter Entry Coordinates $(x,y,z)$	24
Previous Hop Coordinates $(x,y,z)$	24
Future Trajectory (5 × $(x,y,z)$)	120

, extends the standard GPSR format by embedding trajectory-related fields, without introducing additional control message types. To rigorously quantify the cost of control traffic, we adopt the Byte-Level Routing Overhead Ratio (BROR) as a measurement metric [45]. This indicator captures the proportion of routing-related bytes relative to the total transmission, offering a fine-grained view of protocol efficiency at the byte level. The BROR is defined as

$$\mathrm{BROR}={\displaystyle \frac{\mathrm{Control}\mathrm{Bytes}}{\mathrm{Control}\mathrm{Bytes}+\mathrm{Average}\mathrm{Data}\mathrm{Payload}\mathrm{Bytes}}}$$

This formulation enables an objective assessment of protocol-induced overhead throughout the simulation period, independent of packet counts or sizes.

Figure 8 and

Table 5. Byte-level routing overhead ratio (BROR, %) based on the number of nodes.

Number of Nodes	30	40	50	60	70	80	90	100
Traj-Q-GPSR	1.43%	1.33%	1.17%	1.06%	1.01%	0.98%	0.96%	0.93%
GPSR	0.86%	0.82%	0.78%	0.71%	0.67%	0.65%	0.63%	0.61%

present a comparative analysis of the byte-level routing overhead ratio (BROR) for Traj-Q-GPSR and the conventional GPSR protocol under varying node densities. As anticipated, Traj-Q-GPSR incurs a slightly higher overhead due to the incorporation of trajectory-related information in control packets. Nevertheless, as the network density increases, improved connectivity and higher throughput contribute to a steady decline in BROR for both protocols. Importantly, the overhead gap between Traj-Q-GPSR and GPSR diminishes from 0.57% at low density to 0.32% in high-density scenarios. These findings suggest that, while the proposed protocol introduces marginally greater control traffic, this cost is effectively amortized in denser topologies. More significantly, the gains in routing stability and overall network efficiency justify the additional overhead introduced by the trajectory-aware mechanism.

This work extends the conventional one-hop neighbor table to a time-aware two-hop structure and employs Q-tables within a Q-learning framework, thereby incurring greater memory overhead compared to GPSR. Since storage requirements grow with neighbor count, node density is the dominant factor affecting memory cost.

Table 6. Storage overhead (average number of neighbor entries) based on number of nodes.

Number of Nodes	30	40	50	60	70	80	90	100
Traj-Q-GPSR	4.08	6.65	9.81	13.57	17.93	22.87	28.42	34.56
GPSR	1.58	2.13	2.67	3.22	3.76	4.31	4.85	5.40

presents the average number of neighbor entries maintained by Traj-Q-GPSR and GPSR under different node densities. The results indicate that Traj-Q-GPSR experiences accelerated growth in storage load as density increases, yet remains within acceptable limits, reflecting a balance between enhanced topological awareness and resource efficiency.

6. Conclusions

This work addresses the core challenges of routing in UAV swarm networks, including link instability, routing voids, and lack of adaptability under dynamic and mission-driven conditions. To overcome these issues, we propose a trajectory-informed and reinforcement learning-based routing framework that integrates future mobility knowledge into a time-aware two-hop neighbor structure. Coupled with an adaptive Q-learning mechanism and delay-aware queue scheduling, the proposed approach enhances routing stability and decision intelligence. In future work, we aim to extend the framework to handle uncertain or asynchronous trajectory updates and improve learning convergence under constrained communication conditions.