MSTAGNN-MARL: A Multi-Level Intelligent Decision Framework for Integrated Spatial-Temporal Conflict Resolution in High-Density Airspace

Abstract

The spatial and temporal conflicts within terminal maneuvering areas, particularly in multi-airport systems, are growing increasingly complex. Traditional independent processing methods face inherent limitations when dealing with multi-source uncertainties, dynamic weather conditions, and high-density operations. This paper proposes MSTAGNN-MARL that systematically integrates the resolution of spatial conflicts and temporal scheduling issues. This framework is based on four crucial innovations: First, a strategic-tactical-execution hierarchical architecture is constructed that integrates multi-criteria decision optimization with graph neural network-based multi-agent reinforcement learning. Second, an uncertainty perception mechanism is designed that explicitly encodes conflict features as dynamic edge attributes in social graphs, incorporating a real-time dynamic weather model and a Gaussian noise-based perception uncertainty model. Third, develop a compliance automated system for behavior cloning that learns the decision preferences of controllers to achieve human–machine collaboration and provide transparent visualization. Fourth, a robustness assurance mechanism for abnormal scenarios is constructed, employing behavior tree-driven emergency strategies to handle unexpected situations. Experiments demonstrate that the proposed method achieves an 89.3% conflict resolution rate, reduces average delays by 6 min compared to existing methods, and exhibits robust performance under varying traffic densities and dynamic weather conditions. Ablation experiments validate the effectiveness of the four innovations. This framework provides a new research paradigm for scheduling and decision-making in Intelligent Transportation Systems (ITS).

1. Introduction

The complexity of Air Traffic Management (ATM) continues to rise alongside the sustained growth in global aviation demand. According to projections by the International Civil Aviation Organization (ICAO), air traffic volume is expected to double by 2030. This will place unprecedented pressure on existing Air Traffic Control (ATC) systems, particularly in Terminal Maneuvering Areas (TMAs) where aircraft density is highest [1,2,3]. In multi-airport terminal areas, overlapping airspace shared by multiple airports creates numerous conflict hotspots where intersecting flight paths converge, significantly increasing the complexity of ATC.

Current air traffic operations face two critical and interrelated challenges. First, from the perspective of spatial conflict, aircraft operating in close proximity must maintain prescribed safety separation standards—typically 5 nautical miles (nm) horizontally and 1000 feet vertically [4]. The complex intersection of approach flight paths within the terminal area of multi-airport systems creates multiple potential conflict points. Traditional Conflict Detection and Resolution (CD&R) methods, while effective in low-density scenarios, struggle to address the cascading effects triggered by maneuvering in high-density environments [5]. Second, from the perspective of temporal conflicts, allocating time slot resources for shared entry points and destination airports can lead to severe time conflicts when multiple aircraft require simultaneous access. Aircraft scheduling must comply with wake separation requirements while minimizing delays; however, existing pretactical scheduling methods lack the adaptability needed to address dynamic operational uncertainties.

The fundamental limitation of existing research lies in its isolated processing of the two dimensions of space and time. Focusing solely on conflict resolution methods may generate trajectories that severely compromise scheduling efficiency, while pure scheduling optimization often overlooks spatial conflicts during flight operations. This disconnect between tactical conflict resolution and strategic scheduling leads to suboptimal solutions, making it difficult to implement effective solutions in actual operations.

Achieving effective joint optimization of conflict resolution and aircraft scheduling in the terminal area of multi-airport systems presents four critical challenges:

• System Integration Gap: Existing methods typically address only isolated problems in conflict resolution or aircraft scheduling, failing to effectively capture the coupling between spatial maneuvering and temporal sequencing. There is an urgent requirement for a systematic framework capable of integrating three levels: strategic planning, tactical resolution, and execution control [6,7,8].
• Multi-source Uncertainty: Terminal area operations in multi-airport systems involve multiple sources of uncertainty, including the time-varying intensity and movement characteristics of dynamic weather conditions, observation errors in aircraft state measurements by detection equipment, and uncertainties in trajectory prediction. Existing methods are predominantly based on deterministic modeling assumptions, resulting in solutions that lack robustness and exhibit poor transferability in practical applications [9,10,11,12].
• Transparency and Human-Centered Deficiencies: Despite demonstrating formidable decision-making capabilities, Deep Reinforcement Learning (DRL) methods operate as “black box” resulting in limited interpretability [13]. Air traffic controllers require transparent decision processes and solutions aligned with their operational practices. The absence of compliance automation mechanisms and decision transparency hinders the deployment of intelligent methods in practical ATC.
• Insufficient Robustness in Abnormal Scenarios: Actual operations encounter various abnormal scenarios, including severe weather, high-density traffic, and emergency conditions such as aircraft loss of control [14]. Existing methods primarily focus on normal operating conditions, lacking systematic robustness assurance mechanisms and emergency response capabilities for abnormal scenarios.

To address the aforementioned challenges, this paper proposes a multi-level strategic-tactical architecture with a graph neural network and a multi-agent reinforcement learning framework. The crucial innovations of this framework include:

• Multi-level Decision-Making Architecture: To address the challenge of system integration gaps, an integrated strategic-tactical-execution framework has been built. The strategic layer performs priority optimization and time slot allocation based on multi-criteria decision-making methods. The tactical layer employs Graph Neural Network Multi-Agent Reinforcement Learning (GNN-MARL) to achieve collaborative conflict resolution. The execution layer ensures plan implementation through behavior trees and rolling time domain trajectory planning. This architecture achieves systematic joint optimization of spatial conflicts and temporal scheduling, addressing the Trajectory-Based Operations (TBO) outlined in the SESAR 2030 and Federal Aviation Administration (FAA) NEXTGEN initiatives [15].
• Uncertainty Perception Mechanism: To address multi-source uncertainty challenges, this mechanism innovatively encodes conflict features as edge attributes in dynamic social graphs. A multi-source uncertainty modeling approach was simultaneously developed: a real-time dynamic weather model captures variations in weather unit radius and movement, while a Gaussian noise-based aircraft perception uncertainty model handles observation errors. The graph attention mechanism addresses partial observability issues in Partially Observable Markov Decision Processes (POMDP), providing a novel paradigm for distributed intelligent decision-making in complex transportation networks.
• Compliance Automation System: Addressing transparency and human factors challenges, it employs behavioral cloning technology to learn decision preferences of controllers, achieving a 93.45% accuracy rate in action classification. It provides transparent decision visualization, displaying action value ranking, conflict feature analysis, and decision confidence distribution of each agent [16]. The FAA right-of-way rules are embedded in behavioral trees to enhance interpretability. The system supports personalized and group conformity models to adapt to different operational habits of controllers. Controller acceptance validation experiments demonstrate a recommendation adoption rate of 78%.
• Robustness Assurance for Abnormal Scenarios: Addressing the challenge of insufficient robustness in abnormal scenarios, the system systematically handles aircraft loss-of-control situations through behavior trees and rolling horizon planning [17,18]. System performance was validated under dynamic weather conditions of varying severity, with sensitivity analysis demonstrating robustness against up to 7.5% observational noise. A priority-based real-time emergency response mechanism is provided.

To clarify the terminology used in this paper:

• “Scheduling without optimization” refers to First-Come-First-Served (FCFS) sequencing, where aircraft are processed in arrival order without considering system-wide efficiency or delay minimization.
• “Scheduling without conflict resolution” refers to pre-tactical planning that assigns time slots and sequences assuming nominal trajectories, without accounting for real-time spatial conflicts during flight operations.
• “Conflict resolution without scheduling” refers to tactical interventions that resolve immediate separation violations without considering the downstream impact on arrival sequences and runway utilization.

Our “joint optimization” approach simultaneously considers: (a) Temporal objectives: minimizing total delay while satisfying wake separation constraints, (b) Spatial objectives: maintaining required separation throughout flight trajectories, and (c) Coupling constraints: ensuring that conflict resolution actions remain compatible with scheduling constraints and vice versa.

The rest of this paper is structured as follows: Section 2 reviews relevant research and identifies existing gaps. Section 3 models the Combined Conflict Resolution and Aircraft Scheduling (CRAS) problem as a Markov Decision Process (MDP). Section 4 elaborates on the multi-level intelligent decision-making framework, detailing four crucial innovations. Section 5 presents comprehensive experimental results, including ablation studies and comparative analysis with state-of-the-art methods. Finally, the paper concludes by summarizing its contributions and discussing future research directions.

2. Related Work

Research in conflict resolution and aircraft scheduling has evolved from traditional mathematical optimization to meta-heuristic methods and then to reinforcement learning approaches.

Table 1. Summary of research work related to conflict resolution methods.

Time Period	Papers	System Integration	Uncertainty Management	Strategic Planning	Transparency and Compliance	Exception Handling	Methods
Traditional Mathematical Optimization and Rule-Based Methods Phase (2000–2012)	TCAS [4]	×, Only paired conflicts	×, Deterministic Assumption	×, Purely Tactical Response	√, Rule Interpretability	√, but Only Vertical Maneuver	Geometric Rules
	ACAS X [19]	×, Local Optimization	×, Deterministic MDP	×, No strategic layer	√, Partially explainable	×, Enhanced responsiveness	Markov Decision Process
	MIP-Trajectory Planning [20]	×, Single Conflict Resolution	×, Deterministic Modeling	√, Constrained Planning	√, Mathematical Optimality	×, High computational complexity	Mixed-Integer Programming
	Velocity Obstacle [21]	×, Geometric Avoidance	×, Certainty	×, No scheduling	√, Geometric Intuition	×, Poor scalability	Geometric Obstacle Avoidance
Heuristic Optimization Methods stage (2013–2018)	MIP-ALP [22]	√, Scheduling Optimization	×. Certainty	√, Time Slot Allocation	√, Optimal Solution	×, Minor issues	Precision Optimization
	Dynamic Planning-CPM [23]	√, Consider Constraints	×. Certainty	√, Dynamic Scheduling	√, Traceable	×, Exponential complexity	Dynamic Planning
	NSGA-II [24]	√, Multi-Airport Coordination	×. Certainty	√, Pre-tactics	√, Pareto frontier	×, High Recalculation Cost	Genetic Algorithm
	RHC + SA [25]	√, Rolling Optimization	√, Partial processing	√, Dynamic Characteristics	×, Black-box optimization	√, Local adjustment	Simulated Annealing
	ABC-Uncertainty [26]	√, Single objective	√, Partial processing	√, Robust Scheduling	×, Heuristic	√, Uncertainty Handling	Artificial Bee Colony
Deep Reinforcement Learning Method Stage (2019–2023)	Hierarchical DRL [27]	×, Dual Aircraft Scenario	×. Certainty	√, Path + Conflict	×, Black box	×, Limited Scenario	Hierarchical Reinforcement Learning
	MARL-Velocity [28]	√, Multi-intelligent	×. Certainty	×, Pure Tactics	×, Black box	×, Single-Dimensional Maneuverability	Multi-Agent Reinforcement Learning
	Q-learning Schedule [29]	√, Single runway	×. Certainty	√, Online Learning	×, Black box	×, Simple Scenario	Q-Learning
	Reward Shaping [30]	√, Guide Learning	×. Certainty	√, Incremental Reward	×, Black box	×, Training Dependency	Reward Shaping
	MADDPG-Hybrid [31]	√, Hybrid Mode	√, Multi-source Uncertainty	√, Partial Strategy	×, Black box	√, Uncertainty	Multi-Agent DDPG
Joint Optimization and Physical Constraint Stage (2021-present)	RHC + SA Combination [25]	√, Conflict + Schedule	√, Rolling Time Domain	√, Pre-tactics	×, Coupled Calculation	√, Limited scalability	Hybrid Optimization
	Decomposition MARL-Departure [22]	√, Coordinated Departure	×. Certainty	√, Multi-Airport	×, Black box	√, Only Departure	Decomposition Reinforcement Learning
	Improved MARL [17]	×	×	√, Partial Strategy	×, Black box	√, Weather Processing	Improved MARL
This Paper	MSTAGNN-MARL	√	√	√	√	√	Multi-level Intelligent Decision-Making

Note: Checked (√) = Yes; Unchecked (×) = No; System Integration = Whether aviation system is integrated into the mode; Uncertainty Management = Whether it has the ability to manage uncertainties within the airspace; Strategic Planning = Whether it has strategic planning capability; Transparency and Compliance = Whether the model is transparent and compliant; Exception Handling = Whether it has exception handling capability. Bold text indicates the superiority of the model proposed in this paper.

summarizes the developmental timeline of conflict resolution and aircraft scheduling research, presenting representative methods and their characteristics across different stages.

2.1. Conflict Resolution Methods Research Progress

Early conflict resolution methods primarily relied on geometric algorithms and predefined rules. The Traffic Collision Avoidance System (TCAS) provides vertical resolution recommendations based on relative aircraft positions, while the newer ACAS X system employs MDP to enhance collision avoidance performance. However, these approaches focus on handling pairwise conflicts and lack global optimization capabilities in dense traffic scenarios [4,19].

Optimization-based methods propose various optimization modeling strategies for conflict resolution problems, including Mixed-Integer Programming (MIP) for trajectory planning and Velocity Obstacle (VO) methods for conflict avoidance. Although these approaches theoretically obtain optimal solutions, their computational complexity increases sharply with the number of aircraft, resulting in poor scalability. This limitation restricts their application in real-time operational scenarios involving more than 10–15 aircraft [20,21].

Recent research advances have applied Deep Reinforcement Learning (DRL) to conflict resolution problems. Brittain et al. introduced a hierarchical deep reinforcement learning framework, assigning separate agents for path selection and conflict resolution, but their research was confined to two-aircraft conflict scenarios [27]. Ghosh et al. proposed a Multi-Agent Reinforcement Learning (MARL) approach, configuring an independent agent for each aircraft. However, their research primarily focused on velocity adjustment strategies, failing to encompass multi-dimensional maneuvers such as heading and altitude [28].

In summary, existing conflict resolution methods exhibit the following limitations: they typically focus on tactical-level solutions while lacking integration with strategic planning; they are based on deterministic environment assumptions, neglecting the impact of dynamic weather and sensor uncertainties; they lack mechanisms for decision transparency and compliance with controller operational preferences; and they demonstrate insufficient capability to handle abnormal scenarios.

2.2. Aircraft Scheduling Methods Research Progress

The Aircraft Landing Problem (ALP) has been extensively studied, with early research primarily employing exact optimization methods. Wang et al. modeled the problem as an MIP problem with runway capacity and time window constraints [22]. Brittain developed a dynamic programming algorithm for runway scheduling considering Constrained Position Movement (CPM) [23]. While these methods can provide optimal solutions for small-scale problems, their computational time increases exponentially with problem scale, making them impractical for real-time operations in the terminal areas of multi-airport systems.

To address computational complexity issues, researchers have turned to metaheuristic algorithms, including Genetic Algorithm (GA), Simulated Annealing (SA), and Ant Colony Optimization (ACO). Gui et al. employed the Non-Dominated Sorting Genetic Algorithm II (NSGA-II) to achieve cooperative scheduling in the terminal area of multi-airport systems [24]. Liang et al. combined Rolling Horizon Control (RHC) with SA to address the dynamic characteristics of the system [25]. Wang et al. employed an efficient Artificial Bee Colony (ABC) algorithm with a minimax regret criterion to handle uncertainty [26]. However, these methods focus on scheduling optimization during the pre-tactical phase, neglect spatial conflicts during flight operations, and require recalculation of the entire schedule when new aircraft enter the system, lacking autonomous adaptability.

Recent studies have begun exploring the application of reinforcement learning to aircraft scheduling problems. Guleria et al. applied the Q-learning method to single-runway scheduling [29]. Xu et al. introduced a reward shaping mechanism, including incremental step rewards to provide more effective learning guidance [30]. Wang et al. employed the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm to address hybrid runway operations considering multi-source uncertainties [31]. IJtsma and Chen developed reinforcement learning-based dynamic scheduling methods for drone swarms [32,33].

Overall, the current scheduling approach has the following shortcomings: it optimizes only temporal conflicts at runway thresholds while neglecting spatial conflicts in aircraft routes; it employs a deterministic modeling framework that cannot effectively handle multiple sources of uncertainty during operations; and it lacks seamless integration with real-time conflict resolution mechanisms.

2.3. Conflict Resolution-Aircraft Scheduling Joint Optimization

Research on the joint optimization of conflict resolution and aircraft scheduling remains relatively limited. Liang et al. integrated conflict resolution and scheduling optimization within a rolling time-domain framework using an SA algorithm, but its tight computational coupling hindered the method’s scalability [25]. Cai et al. investigated coordinated departure scheduling in the terminal area of the Beijing-Tianjin-Hebei multi-airport system using a decomposition-based MARL method. However, their research focused solely on the departure phase and did not address arrival scheduling [22].

Huang et al. represent a significant advancement in this field, proposing a joint optimization framework incorporating three improvements to MARL: dynamic weather modeling, priority policy design, and reward shaping mechanisms [17].

However, this work leaves several research gaps: the absence of an explicit strategic planning layer for long-term optimization; conflict features are not explicitly modeled as graph edge attributes, limiting the expressive power of graph neural networks; the lack of systematic controller preference learning mechanism and decision transparency assurance; and limited capability to handle emergency scenarios such as aircraft loss of control and severe weather conditions.

3. Problem Formulation

3.1. Problem Definition and Scope

3.1.1. Initial Assumption

This study examines the terminal area environment of a multi-airport system primarily utilizing free airway airspace, where autonomous aircraft operate based on advanced onboard automation equipment. The terminal area of a multi-airport system comprises terminal airspace shared by multiple airports. Approach fix points connect Standard Terminal Arrival Routes (STARs), with arriving aircraft entering from these approach fix points and proceeding to their destination airports. Dynamic hazardous weather units exist in the environment, with their location and intensity changing over time.

The autonomous joint optimization of aircraft occurs within a continuous time interval spanning from the entry time $t_{\mathrm{Entry}}$ to the destination arrival time $t_{\mathrm{Dest}}$. This interval is discretized into time steps $t\in t_{\mathrm{Entry}},t_{\mathrm{Entry}}+\Delta t,\dots ,t_{\mathrm{Dest}}$, where $\Delta t=30$ seconds, aligning with the recommended interval for TCAS [34].

The joint optimization problem is decomposed into two phases, which are as follows:

• Phase 1 (Spatial Conflict Resolution): This phase aims to generate conflict-free trajectories for all arriving aircraft within the terminal area of a multi-airport system. This phase must satisfy the following requirements: aircraft separation constraints to prevent separation loss, hazardous weather avoidance and real-time weather unit dynamics, and maneuvering constraints based on aircraft performance limitations.
• Phase 2 (Time Scheduling): This phase aims to build conflict-free arrival plans for each destination airport. This phase must satisfy the following requirements: maintaining wake separation intervals between consecutive arriving aircraft, adhering to CPM requirements to preserve arrival fairness, and minimizing delays relative to estimated arrival times.

This study is based on the following assumptions: (1) a two-dimensional uniform airspace representation (horizontal plane) is adopted; (2) only arriving aircraft are considered (departing aircraft are excluded); (3) the initial route from the approach fix to the destination is a straight-line path; (4) airport capacity is sufficient to accommodate all arriving aircraft; (5) velocity adjustments are permitted within operational constraints [34].

3.1.2. Aircraft Characteristics Modeling

This study considers heterogeneous aircraft operating within the terminal maneuvering area. Aircraft are categorized into four wake turbulence categories following ICAO standards: Super Heavy (A380), Heavy (B777, B747), Medium (A320, B737), and Light (general aviation). The aircraft characteristics influencing scheduling include:

The joint optimization problem is decomposed into two phases, which are as follows:

• Performance Envelope: Each aircraft category $i$ has specific performance constraints: (1) Velocity range: $[v_{\min }^t,v_{\max }^t]$, where Heavy aircraft: $[220,280]$ knots, Medium: $[200,260]$ knots, Light: $[150,200]$ knots; (2) Turn rate limit: ${\omega }_{\max }^i\in [{1.5}^{\circ }/\mathrm{s},3^{\circ }/\mathrm{s}]$ depending on aircraft category; (3) Climb/descent rate: $RO{C}^i\in [1000,2500]$ ft/min.
• Wake Separation Requirements: The minimum time separation $T\{Sep(m,n)\}$ between consecutive aircraft depends on the leader-follower weight category combination, as specified in

  Table 2. Wake Separation Matrix (seconds).

  	Following: Super	Heavy	Medium	Light
  Leader:
  Super	120	140	160	180
  Heavy	80	90	120	140
  Medium	60	60	60	80
  Light	60	60	60	60

  . For example, a Medium aircraft following a Heavy aircraft requires 120 s separation, while two Medium aircraft require 60 s.
• Approach Speed Profiles: Different aircraft types follow category-specific approach speed profiles, affecting their time-to-destination calculations and sequence optimization.

These heterogeneous characteristics are incorporated into both the strategic layer (for wake separation-aware time slot allocation) and the tactical layer (for performance-constrained maneuver selection).

3.2. Markov Decision Process Modeling and Safety-First Optimization Philosophy

3.2.1. Markov Decision Process Modeling

The joint optimization problem is modeled as an MDP defined by a six-tuple $\langle \mathcal{S},\mathcal{A},\mathcal{R},\mathcal{P},\gamma ,\mu \rangle$, where $\mathcal{S}$ represents the state space, $\mathcal{A}$ represents the action space, $\mathcal{R}:\mathcal{S}\times \mathcal{A}\times \mathcal{S}\to \mathbb{R}$ is the reward function, $\mathcal{P}:\mathcal{S}\times \mathcal{A}\to \Delta (\mathcal{S})$ is the state transition probability function, $\gamma \in (0,1]$ is the discount factor, and $\mu$ represents the initial state distribution.

State Space $\mathcal{S}$. At time step $t$, the observed state $s_t^i$ of aircraft $i$ contains the following information, defined as follows:

$$s_t^i=(p_t^i,p_{\mathrm{Entry}}^i,p_{\mathrm{Dest}}^i,h{d}_t^i,v_t^i,{\mathrm{RTA}}_t^{i,j},{\mathrm{seq}}_t^{i,j})$$

(1)

where $p_t^i=(x_t^i,y_t^i,h_t^i)$ represents the three-dimensional position, $h{d}_t^i$ is the heading, $v_t^i$ indicates the airspeed, ${\mathrm{RTA}}_t^{i,j}$ signifies the reference arrival time at the destination $j$, and ${\mathrm{seq}}_t^{i,j}$ denotes the arrival sequence.

Action Space $\mathcal{A}$. Each aircraft selects a discrete maneuver action at each time step, defined as follows:

$$a_t^i=(a_t^{\mathrm{Speed}},a_t^{\mathrm{Heading}})\in {\mathcal{A}}^{\mathrm{Speed}}\times {\mathcal{A}}^{\mathrm{Heading}}$$

(2)

where $a_t^{\mathrm{Speed}}\in \{1:Deceleration, 2:Normal, 3:Acceleration\}$ represents the speed maneuvers, $a_t^{Heading}\in \{1:Turnleft,2:Aim,3:Turnright\}$ represents heading maneuvers. These maneuvers are constrained by the maximum overload limit $[-1,2.5]g$ specified in CCAR-25.

Reward Function $\mathcal{R}$. The reward function is designed to balance multiple objectives, including conflict resolution, scheduling efficiency, and trajectory smoothness. At each time step, the reward for the aircraft $i$ is computed as:

$$\mathcal{R}(s_t^i,a_t^i,s_{t+1}^i)={\mathcal{R}}_{conflict}+{\mathcal{R}}_{schedule}+{\mathcal{R}}_{smooth}+{\mathcal{R}}_{destination}$$

(3)

where ${\mathcal{R}}_{conflict}=-\alpha \cdot n_{conflict}$ penalizes separation violations, with $n_{conflict}$ being the number of conflicts and $\alpha =50$; ${\mathcal{R}}_{schedule}=-\beta \cdot \left|T{D}^{\{i,j\}}\right|$ penalizes delays relative to reference arrival times, with $\beta =0.1$; ${\mathcal{R}}_{smooth}=-\delta \cdot {‖\Delta v_t^i‖}^2$ penalizes abrupt maneuvers, with $\delta =0.01$; ${\mathcal{R}}_{destination}=+{\gamma }_{dest}$ when aircraft successfully reaches its destination without conflicts, with ${\gamma }_{dest}=100$.

Transfer Function $\mathcal{P}$. The evolution of the system state follows the aircraft kinematic equations, as shown below:

$$p_{t+1}^i=p_t^i+v_t^i\Delta t,\hspace{1em}{v}_t^i\in [v_{{\min }^i},v_{{\max }^i}]$$

(4)

where the heading is updated based on the selected maneuver.

Strategy Function $\pi$. Policy $\pi$ maps states to action probability distributions $\pi (\cdot |s_t^i)$. The optimization objective is to find the optimal policy ${\pi }^{*}$ that maximizes the expected discounted cumulative reward:

$$J({\pi }^i)={\mathbb{E}}_{\{{\tau }^i~{\pi }^i\}}\left[{\displaystyle \sum _{k=0}^{K^i}{\gamma }^{k}R(s_{t_{Entry}+k}^i,a_{t_{Entry}+k}^i,s_{t_{Entry}+k}^i)}\right]$$

(5)

where ${\tau }^i=(s_{t_{\mathrm{Entry}}}^i,a_{t_{\mathrm{Entry}}}^i,\dots ,s_{t_{\mathrm{Dest}}}^i,a_{t_{\mathrm{Dest}}}^i)$ represents the trajectory of aircraft $i$ from entry to destination, $K^i=(t_{Dest}^i-t_{Entry}^i)/\Delta t$ represents the total number of time steps for aircraft $i$, and $\gamma \in (0,1]$ is the discount factor that balances immediate and future rewards. The expectation ${\mathbb{E}}_{\{{\tau }^i~{\pi }^i\}}$ is taken over all possible trajectories generated by following policy ${\pi }^i$, $t_{Dest}$ in the superscript of the summation serves as the terminal time index (not an exponent), indicating that rewards are accumulated from the entry time step $t_{Entry}$ until the aircraft reaches its destination at time step $t_{Dest}$.

Initial State Distribution $\mu$. The initial state distribution $\mu (s_0)$ characterizes the probability distribution over starting configurations. In our terminal maneuvering area scenario, $\mu$ is determined by: (1) Aircraft entry times following a Poisson process with rate $\lambda =52$ aircraft/hour; (2) Entry point selection based on historical origin-destination flow matrices; (3) Initial velocities uniformly distributed within $[v_{\min },v_{\max }]=[200,280]$ knots.

3.2.2. Safety-First Optimization Philosophy

In this paper, we acknowledge that conflict-free operations represent an absolute requirement in air traffic management—safety is non-negotiable. Our optimization framework addresses this through a hierarchical constraint structure:

• Hard Constraints (Safety): Aircraft separation constraints are treated as hard constraints that must be satisfied at all times. The reward function heavily penalizes separation violations (${\mathcal{R}}_{conflict}=-50$ per conflict), and the behavior tree-based execution layer (Section 4.4) provides guaranteed emergency response when the tactical layer fails to resolve conflicts.
• Soft Constraints (Efficiency): Delay minimization and sequence optimization are treated as soft constraints—objectives to be optimized within the safety-guaranteed feasible region.

This formulation reflects operational reality: when multiple conflict-free solutions exist (which is typical in well-designed airspace), optimization selects the most efficient among them. The experimental results (Table 4) demonstrate that our method achieves 97.9% resolution rate, with the remaining conflicts handled by the robust emergency mechanism.

Furthermore, the optimization approach provides several advantages over pure rule-based systems:

• Proactive Conflict Avoidance: By anticipating potential conflicts through forward simulation, the system can take early, minimal interventions rather than reactive emergency maneuvers.
• Resource Efficiency: Among multiple safe trajectories, optimization identifies those minimizing fuel consumption and delays.
• Adaptability: The learned policy adapts to varying traffic patterns without manual rule updates.

3.3. Problem Complexity Analysis

The joint optimization problem formulated in this paper is inherently non-convex due to several factors:

• Discrete Action Space: The maneuver actions (speed and heading adjustments) are discrete, introducing combinatorial complexity.
• Coupled Constraints: The aircraft separation constraints create pairwise coupling between all aircraft, resulting in non-convex feasible regions.
• Dynamic State Evolution: The system transitions depend on the joint actions of all aircraft, creating a multi-agent optimization problem that cannot be decomposed into independent subproblems.
• Temporal Dependencies: The sequential nature of decision-making, where current actions affect future states and constraints, introduces temporal non-convexity.

Given the non-convex nature of this problem, traditional gradient-based optimization methods cannot guarantee global optimality. This motivates our adoption of Multi-Agent Reinforcement Learning (MARL), which handles discrete action spaces naturally through policy learning, captures complex state-action dependencies through neural network approximation, scales to large numbers of agents through decentralized execution with centralized training and provides near-optimal solutions efficiently for real-time applications.

While MARL does not guarantee global optimality, our experimental results (Section 5) demonstrate that the learned policies consistently outperform existing methods across multiple performance metrics.

3.4. Constraints and Objectives

Throughout the overall modeling process, the optimization problem is subject to the following four types of constraints: aircraft separation constraints, weather avoidance constraints, wake separation constraints, and constraints on positional movement limitations. The mathematical definitions for each constraint category are as follows.

Aircraft separation constraints are defined as the spatial separation between any two aircraft within an airspace, as expressed by the following equation:

$${‖p_t^m-p_t^n‖}_2\ge D_{\mathrm{Sep}}^{\mathrm{Aircraft}},\hspace{1em}\forall m,n\in \mathcal{I},m\ne n$$

(6)

where ${‖\cdot ‖}_2$ represents the Euclidean (L2) norm, $D_{\mathrm{Sep}}^{\mathrm{Aircraft}}$ represents the minimum safety separation standard. Specifically, for horizontal separation ${‖p_t^m-p_t^n‖}_2$ refers to the two-dimensional Euclidean distance $\sqrt{({(x_t^m-x_t^n)}^2+{(y_t^m-y_t^n)}^2)}$ with $d_{\min }=5$ nautical miles; for vertical separation, $\left|h_t^m-h_t^n\right|$ utilizes the absolute value (L1 norm in one dimension) with $d_v=1000$ feet.

Weather avoidance constraints require aircraft to maintain a safe distance from hazardous weather units. In addition to the aforementioned aircraft separation constraints, aircraft must also satisfy weather avoidance constraints, defined as follows:

$${‖p_t^i-p_t^{\mathrm{weather}}‖}_2-D^{\mathrm{weather}}\cdot {\rho }_t^{\mathrm{weather}}\ge D_{\mathrm{Sep}}^{\mathrm{Weather}}$$

(7)

where ${‖\cdot ‖}_2$ represents the Euclidean norm measuring the horizontal distance between aircraft position, $D^{\mathrm{weather}}$ is the fixed weather units radius, ${\rho }_t^{\mathrm{weather}}$ is the scaling factor, $D_{\mathrm{Sep}}^{\mathrm{Weather}}=2$ nautical miles.

Wake separation constraints specify the minimum time interval that must be maintained between aircraft arriving consecutively at the same destination airport. In addition to the aforementioned spatial constraints, aircraft must also satisfy temporal constraints (wake separation and positional movement). Wake separation constraints are defined as the following equation:

$$|{\mathrm{RTA}}_t^{m,j}-{\mathrm{RTA}}_t^{n,j}|\ge T\{\mathrm{Sep}(m,n)\},\hspace{1em}\forall m,n\in {\mathcal{I}}_j,|{\mathrm{seq}}_t^{m,j}-{\mathrm{seq}}_t^{n,j}|=1$$

(8)

where $T\{\mathrm{Sep}(m,n)\}$ is determined according to the aircraft weight category based on ICAO standards.

CPM limits the range of positional changes an aircraft can undergo relative to its initial arrival sequence during the scheduling process to ensure scheduling fairness. It is defined as follows:

$$|{\mathrm{seq}}_{t+1}^{ij}-{\mathrm{seq}}_t^{i,j}|\le Z_{CPS}$$

(9)

where $Z_{CPS}=3$ is the maximum allowable sequence shift.

Subject to all the above constraints, the optimization objective is to minimize the system delay, as expressed by the following equation:

$${\min }_{\pi }{\mathbb{E}}_{\pi }[{\displaystyle \sum \{i\in {\mathcal{I}}_j,j\in \mathcal{J}\},T{D}^{i,j}|}\mathcal{T},\mathbb{Q}]$$

(10)

where ${\mathrm{TD}}^{i,j}={\mathrm{RTA}\{\mathrm{t}}_{Dest}^{i,j}\}-{\mathrm{RTA}\{\mathrm{t}}_{Entry}^{i,j}\}>0$ represents the delay time, $\mathcal{T}$ represents the set of conflict-free trajectories, and $\mathbb{Q}$ denotes the set of conflict-free arrival time.

4. Methodology

4.1. Framework Overview

4.1.1. Framework Explanation

The overall architecture of the proposed MSTAGNN-MARL framework is shown in Figure 1. This framework comprises four crucial components: the strategic layer (multi-criteria decision planning), the tactical layer (graph neural network-based multi-agent reinforcement learning solution), the execution layer (trajectory planning), and the inter-layer integration mechanism.

As shown in Figure 1, the strategic layer covers the 30 min preceding the pre-tactical phase, primarily performing multi-criteria decision optimization. This includes aircraft prioritization based on emergency level, flight phase, and conflict involvement; optimal delay allocation targeting minimization of total system delay; and time slot allocation for shared approach fix and runway utilization. This layer employs the MEREC (Method of Effect Removal by Criteria) and MARCOS methods to generate the priority vector $pri$ and the strategic delay allocation scheme $T{D}_{\mathrm{strategic}}$ [35].

The tactical layer operates within the tactical real-time phase, primarily enabling multi-agent collaborative conflict resolution through: dynamic conflict graph construction based on edge features, collaborative action selection adhering to priority policies, and real-time trajectory adjustments accounting for dynamic weather characteristics. This layer employs an architecture combining Graph Attention Networks (GATs) with Deep Q-Network (DQN) to generate maneuvering action commands $a_{t i\in \mathcal{I}}^i$ for all aircraft.

The execution layer operates within a 10 min window for short-term rolling optimization, primarily handling trajectory generation and emergency handling. Its functions include behavior tree-driven action execution based on FAA right-of-way rules, RHC for smooth trajectories, and emergency response and recovery maneuvers for loss-of-control scenarios. This layer employs rolling time-domain control and constraint optimization methods to output an executable sequence of trajectory waypoints ${\mathcal{T}}^i=p_t^i,\dots ,p_{t+10}^i$.

Within the inter-layer integration mechanism, the priority of the strategic layer guides tactical layer action selection through weighted Q-values, tactical layer maneuver decisions provide reference information for trajectory planning at the execution layer, and feedback from the execution layer is used to update the strategic delay estimate for the next optimization period.

4.1.2. Strategic Layer: Multi-Criteria Decision Optimization

The strategic layer employs MEREC (Method based on the Removal Effects of Criteria) for objective weight determination and MARCOS (Measurement of Alternatives and Ranking according to Compromise Solution) for aircraft priority ranking.

MEREC Weight Determination. MEREC determines criterion weights based on the effect of removing each criterion on the overall decision matrix. Given a decision matrix $X={[x_{ij}]}_{\{m\times n\}}$ with $m$ m aircraft and $n$ criteria, the procedure is:

Step 1: Normalize the decision matrix using logarithmic normalization, which is as follows,

$$n_{ij}=\left|\ln (x_{ij})\right|/\left|\ln (\prod _{j=1}^n{x}_{ij}^{\{1/n\}})\right|$$

(11)

Step 2: Calculate the overall performance of alternatives using the following equation,

$$S_j=\ln (1+\left|{\displaystyle \sum _{i=1}^m\ln (x_{ij})}\right|/m)$$

(12)

Step 3: Compute removal effect using the following equation,

$$E_{j^{\prime }}={\displaystyle \sum _{k=1}^n\left|S_k-S_{k^{\prime }}\right|}$$

(13)

Step 4: Determine final weights using the following equation,

$$w_j=E_{j^{\prime }}/{\displaystyle \sum _{k=1}^n{E}_{k^{\prime }}}$$

(14)

In our application, the criteria include: (1) Emergency level, (2) Remaining flight time, (3) Current conflict involvement, (4) Fuel status, (5) Passenger connectivity requirements.

MARCOS Ranking. Using the MEREC-derived weights, MARCOS ranks aircraft by computing utility degrees relative to ideal and anti-ideal solutions:

Step 1: Construct an extended decision matrix with ideal (AI) and anti-ideal (AAI) solutions.

Step 2: Calculate weighted normalized matrix: $V=[v_{ij}]$ where $v_{ij}=w_j\times n_{ij}$.

Step 3: Compute utility degrees using the following equation,

$$K_i^{-}=S_i/S_{\left\{AAI\right\}}$$

(15)

$$K_i^{+}=S_i/S_{\left\{AI\right\}}$$

(16)

where $S_i={\displaystyle \sum _{j=1}^n{v}_{ij}}$.

Step 4: Determine the final utility function using the following equation,

$$f(K_i)=(K_i^{+}+K_i^{-})/(1+(1-f(K_i^{+}))/f(K_i^{+})+(1-f(K_i^{-}))/f(K_i^{-}))$$

(17)

Aircraft are then ranked by $f(K_i)$ values, with higher values indicating higher priority for resource allocation and conflict resolution precedence. This multi-criteria approach enables transparent, explainable priority decisions that incorporate multiple operational factors beyond simple arrival time ordering.

4.2. Multi-Agent Reinforcement Learning

The crucial innovation of the MSTAGNN-MARL framework is the GNN-MARL architecture, which incorporates two innovative mechanisms: explicit edge feature encoding and multi-source uncertainty modeling.

4.2.1. Dynamic Conflict Graph Construction

Unlike existing graph neural network-based ATM methods that only encode node features, the explicit edge feature encoding mechanism proposed in this paper enables the network to:

• Learn directly from conflict characteristics (severity, geometric relationships, temporal properties).
• Distinguish between conflict types requiring different resolution strategies (e.g., head-on conflicts versus overtaking conflicts).
• Incorporate relative priority information to achieve coordinated multi-agent decision-making.

The dynamic social graph constructed at each time step is represented as follows:

$${\mathcal{G}}_t=({\mathcal{V}}_t,{\mathcal{E}}_t,X_t,E_t)$$

(18)

where node ${\mathcal{V}}_t$ corresponds to each aircraft $i\in \mathcal{I}$. Edge ${\mathcal{E}}_t$ represents a bidirectional connection between conflicting aircraft i and j. Node feature $X_t\in {\mathbb{R}}^{\left|\mathcal{I}\right|}\times d_{node}$ contains the characteristics of each aircraft, with dimension $d_{node}=9$. Edge feature $E_t\in {\mathbb{R}}^{\left|{\mathcal{E}}_t\right|}\times d_{edge}$ contains conflict characteristics, with dimension $d_{edge}=8$.

Node feature encoding employs an explicit coding approach, integrating multi-source information into a unified feature representation, defined as follows:

$$x_t^i=\left[{\displaystyle \frac{p_t^i}{[100,100,40000]}},{\displaystyle \frac{v_t^i}{[500,500,1000]}},{\displaystyle \frac{{\mathrm{priority}}^i}{3}},{\sigma }_{pos}^i,w_{\mathrm{impact}}^i\right]$$

(19)

where ${\displaystyle \frac{p_t^i}{[100,100,40000]}}$ is the normalized position vector, ${\displaystyle \frac{v_t^i}{[500,500,1000]}}$ is the normalized velocity vector, ${\displaystyle \frac{{\mathrm{priority}}^i}{3}}$ is the normalized priority, ${\sigma }_{pos}^i$ is the position uncertainty from the sensor, $w_{\mathrm{impact}}^i$ represents the weather effect, computed from the nearby weather unit.

Edge feature encoding achieves this by explicitly modeling conflict relationships between aircraft in airspace, incorporating the following key characteristics: horizontal distance, vertical separation, Time of Closest Point of Approach (TCPA), conflict severity score, conflict type indicator, and relative priority difference. The edge feature vector is defined as follows:

$$e_t^{ij}=\left[{\displaystyle \frac{d_t^{ij}}{10}},{\displaystyle \frac{\Delta h_t^{ij}}{2000}},{\displaystyle \frac{t_{CPA}^{ij}}{10}},{\mathrm{severity}}^{ij},{\mathrm{head}-\mathrm{on}}^{ij},{\mathrm{overtaking}}^{ij},convergin{g}^{ij},{\displaystyle \frac{\Delta {\mathrm{pri}}^{ij}}{3}}\right]$$

(20)

where $d_t^{ij}$ represents the horizontal distance (normalized by dividing by 10 nautical miles), $\Delta h_t^{ij}$ denotes the vertical separation (normalized by dividing by 2000 feet), $t_{CPA}^{ij}$ indicates the time of closest approach (normalized by dividing by 10 min), ${\mathrm{severity}}^{ij}={\displaystyle \frac{D_{\mathrm{Sep}}-d_t^{ij}}{D_{\mathrm{Sep}}}}$ signifies the conflict severity score, ${\mathrm{head}-\mathrm{on}}^{ij}$, ${\mathrm{overtaking}}^{ij}$, and $convergin{g}^{ij}$ serve as conflict type indicators, and $\Delta {\mathrm{pri}}^{ij}$ reflects the relative priority difference.

4.2.2. Image Attention Network Architecture

This paper employs a dual-layer graph attention network architecture with edge features to enhance the ability to represent aircraft states in airspace and characterize conflict features. The forward propagation process of the network is defined as follows:

$$H^{(1)}={\mathrm{GAT}}_1(X_t,{\mathcal{E}}_t,E_t)$$

(21)

$$H^{(2)}={\mathrm{GAT}}_2(H^{(1)},{\mathcal{E}}_t,E_t)$$

(22)

The layer-wise attention network updates the stage representation by calculating attention coefficients and performing weighted fusion, as described by the following equation:

$$h_t^i=\sigma \left({\displaystyle \sum j\in \mathcal{N}(i),{\alpha }_{ij},}W[h_{t-1}^i|h_{t-1}^j|e_t^{ij}]\right)$$

(23)

The attention coefficient ${\alpha }_{ij}$ is calculated using the following equation:

$${\alpha }_{ij}={\displaystyle \frac{\exp (\mathrm{LeakyReLU}(a^T[h_{t-1}^i|h_{t-1}^j|e_t^{ij}]))}{{\displaystyle \sum _k^{\mathcal{N}(i)}\exp (\mathrm{LeakyReLU}(a^T[h_{t-1}^i|h_{t-1}^k|e_t^{ik}]))}}}$$

(24)

After the graph convolution operation in the graph attention network is completed, the policy and value outputs of each agent are obtained as follows:

$$Q_t^i=f_{policy}(h_t^i),\hspace{1em}{V}_t^i=f_{value}(h_t^i)$$

(25)

where $f_{\mathrm{policy}}:{\mathbb{R}}^{d_{\mathrm{hidden}}}\to {\mathbb{R}}^{\left|\mathcal{A}\right|}$ represents the Q-value for each action output, and $f_{\mathrm{value}}:{\mathbb{R}}^{d_{\mathrm{hidden}}}\to \mathbb{R}$ represents the estimated state value.

4.2.3. Multi-Source Uncertainty Modeling

Multi-source uncertainty modeling comprises two submodules: a real-time dynamic weather model and an observational uncertainty model. In a real-time dynamic weather model, weather units are modeled as circular regions with time-varying properties, as shown in the following equation:

$$p_t^{\mathrm{weather}}=p_{t-1}^{\mathrm{weather}}+v_{t-1}^{\mathrm{weather}}\Delta t+{\xi }_t,\hspace{1em}{\xi }_t~\mathcal{N}(0,{\sigma }_{\mathrm{weather}}^{2}I)$$

(26)

$$r_t^{\mathrm{weather}}=D^{\mathrm{weather}}\cdot {\rho }_t^{\mathrm{weather}},\hspace{1em}{\rho }_t^{\mathrm{weather}}\in [0.2,2.0]$$

(27)

where $p_t^{\mathrm{weather}}$ represents the center of mass position of the weather unit, $v_t^{\mathrm{weather}}$ represents the velocity vector, $r_t^{\mathrm{weather}}$ indicates the effective radius, and ${\rho }_t^{\mathrm{weather}}$ represents the scaling factor.

The formulation for calculating the weather impact factor on aircraft i is as follows:

$$w_{\mathrm{impact}}^i={\max }_{\mathrm{cells}}\{{ \mathrm{severity}}^{\mathrm{weather}}\cdot \left(1-{\displaystyle \frac{|p_t^i-p_t^{\mathrm{weather}}|}{r_t^{\mathrm{weather}}}}\right)$$

(28)

For each agent i in the environment, the partial observation it receives is represented as:

$$o_t^i=[o_t^{\mathrm{own}},{\widehat{o}}_t^{\mathrm{detected},1},\dots ,{\widehat{o}}_t^{\mathrm{detected},N},o_t^{\mathrm{weather}}]$$

(29)

where $o_t^{\mathrm{own}}$ represents the self-observation received at time step t, and N represents the number of recently detected aircraft that have been padded/cropped to a fixed dimension.

4.2.4. Priority-Based Collaborative Action Execution Strategy

To accelerate the training process and enhance the interpretability of decisions, a priority-based action execution mechanism is introduced. The priority criterion function is expressed as follows:

$${\mathrm{pri}}^i=q^i+{\displaystyle \frac{d_t^{i,\mathrm{Dest}}}{|p_{\mathrm{Entry}}^i-p_{\mathrm{Dest}}^i|}}+{\displaystyle \frac{h{d}_t^i}{{360}^{\circ }}}$$

(30)

where $q^i$ represents the number of conflicts involving aircraft i, ${\displaystyle \frac{d_t^{i,\mathrm{Dest}}}{|p_{\mathrm{Entry}}^i-p_{\mathrm{Dest}}^i|}}$ denotes the normalized destination distance, and ${\displaystyle \frac{h{d}_t^i}{{360}^{\circ }}}$ indicates the normalized heading deviation.

For conflicting aircraft pairs, the “late-arriving aircraft adjusts first” principle is applied, meaning aircraft with later estimated arrival times receive priority for trajectory adjustments. For conflict groups involving multiple aircraft, sorting is performed based on priority scores, with the highest-priority aircraft maintaining its current trajectory unchanged. This priority strategy embeds domain knowledge into the reinforcement learning framework, effectively improving training efficiency and operational transparency.

4.2.5. Algorithm Implementation Details

To facilitate independent implementation and verification, Algorithm 1 presents the complete training procedure of the MSTAGNN-MARL framework.

Algorithm 1 MSTAGNN-MARL Training Procedure
Require:	Environment $E$ Strategic planner $S$$, \mathrm{Number} \mathrm{of} \mathrm{episodes} N_{ep}$
Ensure:	$\mathrm{Trained} \mathrm{policy} \mathrm{networks} {\left\{{\pi }_i\right\}}_{i\in I}$
1:	Initialize replay buffer $C$$\mathrm{with} \mathrm{capacity} C={10}^6$
2:	Initialize Q-networks $\theta$$\mathrm{and} \mathrm{target} \mathrm{networks} {\theta }^{\prime }$ for all agents
3:	for episode $=1 \mathbf{to} N_{ep}$ do
4:	$\mathrm{Reset} \mathrm{environment}, \mathrm{obtain} \mathrm{initial} \mathrm{states} {\left\{s_i^0\right\}}_{i\in I}$
5:	$\mathrm{Execute} \mathrm{strategic} \mathrm{layer}: pri$$, T{D}_{strategic}\leftarrow MEREC-MARCOS(S)$
6:	$\mathbf{for} \mathrm{t}=0 \mathbf{to} T_{\max }$do
7:	$\mathrm{Construct} \mathrm{dynamic} \mathrm{conflict} \mathrm{graph} G_t=(V_t,E_t,X_t,E_t)$
8:	$\mathbf{for} \mathrm{each} \mathrm{agent} i\in I$do
9:	$\mathrm{Compute} \mathrm{node} \mathrm{embeddings}: H^{(2)}\leftarrow GA{T}_2(GA{T}_1(X_t,E_t,E_t))$
10:	$\mathrm{Select} \mathrm{action}: a_i^t\leftarrow \epsilon -greedy(Q_i(h_i^t))$
11:	end for
12:	$\mathrm{Execute} \mathrm{joint} \mathrm{action} a_t$$, \mathrm{observe} \mathrm{rewards} \left\{r_i^t\right\}$$\mathrm{and} \mathrm{text} \mathrm{states} \left\{s_i^{t+1}\right\}$
13:	Store transition in $D$
14:	Sample minibatch from $D$, update networks using TD-learning
15:	Soft update target networks: ${\theta }^{\prime }\leftarrow \tau \theta +(1-\tau ){\theta }^{\prime }$
16:	end for
17:	end for

The key hyperparameters are summarized in

Table 3. MSTAGNN-MARL Model Hyper Parameter Settings.

Parameter Type	Parameter Name	Value
Network Structure	Attention Layers Number	3
	Attention Head Number	8
	Hidden Layer Dimension	64
	Transformer Encoder Layer	4
	Feedforward Network Dimension	256
	Dropout Rate	0.1
Training-Testing Scenario	Fixed Aircraft Count	52
	Air Traffic Controller	8
	Expert Data Size	1000
	Variable Aircrafts Num	{4, 12, 20, 28, 36, 44, 52, 60, 68, 76, 84, 92}
Adaptive Graph Structure	Similarity Threshold Initial Value	0.5
	Similarity Learning Rate	0.001
	Edge Weight Smoothing Factor	0.1
Training Parameters	Batch Size	64
	Learning Rate	0.001
	Epoch Num	10,000
	Early Stopping Strategy	200
	Loss Function Weight	Prediction loss weight: 0.7

, including network architecture parameters (attention layers, hidden dimensions), training parameters (learning rate, batch size), and environment-specific settings.

4.3. Compliance Automation and Transparency

To address the crucial requirements of human–machine collaboration and decision explainability in safety-critical ATM systems, this paper employs compliance automation and transparency mechanisms to ensure the transparency of the operational preferences and decision-making of air traffic controllers.

4.3.1. Controllers’ Preferred Behavioral Clone Learning

To learn and preserve solution preferences of air traffic controllers, this research collected decision data from eight experienced controllers with distinct solution preferences and styles: Controller 0 preferred heading change strategies with 80% conservatism; Controller 1 favored altitude change strategies with 60% conservatism; Controller 2 favored velocity adjustment strategies with 70% conservatism; Controller 3 favored heading adjustment strategies with 50% aggressiveness; Controller 4 favored altitude change strategies with 90% conservatism; Controller 5 favored combined maneuver strategies with 60% conservatism; Controllers 6 and 7 favored heading and velocity adjustment strategies, respectively, each with varying conservatism levels.

The controller decision generated for each conflict scenario is represented as:

$${\mathrm{Decision}}^{\mathrm{ATCO}}=\{a_{\mathrm{type}},\Delta hd,\Delta h,\Delta v\}$$

(31)

where $a_{\mathrm{type}}\in \{Heading,Height,Veloctiy,Combination\}$ represents the type of change, while $\Delta hd$, $\Delta h$, and $\Delta v$ represent the magnitude of change.

4.3.2. Transparent Dashboard Design

To ensure transparency in system operations, a visual decision support interface has been designed. This interface comprises five crucial components: action value sorting heatmap, conflict feature analysis panel, compliance prediction panel, controller acceptance rate statistics, and decision confidence distribution, as shown in Figure 2.

In the visualization interface, the action value sorting heatmap shows the Q-value distribution for all candidate actions of each agent, highlighting the selected action with a blue box to help controllers understand the rationale behind the decision of “why this action was chosen.” The conflict feature analysis panel presents normalized characteristics of critical conflicts—including distance, vertical separation, time of closest approach, and severity—using grouped bar charts for intuitive comparison.

The compliance prediction panel shows predicted controller actions with confidence levels, color-coded by confidence: green (>80%) indicates high confidence, yellow (60–80%) indicates medium confidence, and red (<60%) indicates low confidence. Predicted maneuver parameters are also listed. Controller acceptance rate statistics show the historical acceptance rate of each controller (via bar charts) to track model performance evolution over time. The decision confidence distribution shows the probability distributions of different action categories via pie charts, aiding assessment of decision certainty.

4.4. Abnormal Scenarios Robustness

4.4.1. Safety Factor and Capacity Reserve

To handle unforeseeable situations, the system maintains operational margins as safety factors:

• Separation Buffer: The working separation standard is set at 110% of the regulatory minimum (5.5 nm instead of 5 nm horizontal, 1100 ft instead of 1000 ft vertical), providing buffer for unexpected trajectory deviations.
• Capacity Reserve: The system operates at 85% of theoretical maximum capacity, reserving 15% capacity for emergency handling. When traffic density exceeds this threshold, ground delay programs are recommended.
• Uncertainty Margins: The Gaussian noise-based perception uncertainty model (Section 4.2.3) explicitly accounts for: (1) Sensor errors: Position uncertainty ${\sigma }_{pos}=0.1$ nm, Velocity uncertainty ${\sigma }_v=0.5$ knots; (2) Trajectory prediction errors: Modeled as increasing uncertainty over the prediction horizon; (3) Communication delays: Assumed 3 s latency in action execution.

4.4.2. Abnormal Scenarios Robustness Formulation

To ensure the framework can handle emergencies beyond normal operating conditions, a rolling time-domain trajectory planning method is employed to uniformly address both normal and emergency scenarios. For the trajectory planning problem, a 10 min rolling optimization is employed. The optimization formulation is expressed as follows:

$${\min }_{p_t,\dots ,p_{t+10}}{\displaystyle \sum _{k=t}^{t+10}\left[|p_k-p_k^{\mathrm{target}}{|}^2+{\lambda }_{\mathrm{smooth}}|\Delta v_k{|}^2\right]}$$

(32)

where ${\lambda }_{\mathrm{smooth}}$ is the smoothing factor, $p_k$ is the aircraft position vector at time k, $p_k^{\mathrm{target}}$ is the target position vector, and $v_k$ is the velocity vector, constrained by the velocity limit $v_{\min }\le ‖v_k‖\le v_{\max }$.

The planning problem is subject to aircraft dynamics constraints. The position relationship between the aircraft at the current time step and the next time step is defined by the following equation:

$$p_{k+1}=p_k+v_k\Delta t$$

(33)

where $v_k$ represents the velocity control vector within the planning time domain.

When detecting aircraft loss of control, the system activates emergency trajectory planning mode. At this point, the target position vector of the aircraft is set to the recovery waypoint output by the behavior tree, ensuring the aircraft can safely return to normal flight status.

5. Experiments

5.1. Experiment Settings

5.1.1. Experiment Scenario

This study employs the terminal maneuvering area of the multi-airport system within the Guangdong-Hong Kong-Macao Greater Bay Area as its experiment scenario. This airspace represents one of the world’s most complex airspace configurations. The system encompasses five major airports: ZGGG (Guangzhou Baiyun), ZGSZ (Shenzhen Baoan), ZGSD (Zhuhai Jinwan), VMMC (Macau), and ZGHZ (Huizhou Pingtan). Aircraft enters the terminal maneuvering area via ten designated approach positioning points: IKAVO, ENVIP, OLPAB, P270, OVGOT, UBDOB, BEKOL, CHALI, LANDA, and LOVTA. Managed by two approach control centers (Guangzhou Approach and Zhuhai Approach), this airspace contains 101 conflict hotspots—the highest density among multi-airport terminal maneuvering areas globally [36].

Traffic flow parameters are derived from actual operational data collected between 1 May 2023 and 7 May 2023, comprising a total of 32,369,828 radar track data points. The dataset reflects real-world operational characteristics, with an average daily arrival rate of 1204 aircraft per day and a peak hourly arrival rate of 52 aircraft. The latter serves as the baseline configuration for training scenarios. Traffic distribution patterns were extracted from historical origin-destination flow matrices, capturing actual flow characteristics from arrival points to destination airports, as shown in Figure 3.

5.1.2. Baseline Model

To comprehensively evaluate the effectiveness of the proposed framework, its performance is compared against five advanced baseline models: (1) DDQN (Double Depth Q-Network) [37], a value-based reinforcement learning approach; (2) DDPG (Deep Deterministic Policy Gradient) [38], an actor-critic method for continuous action spaces; (3) GRL (Graph Reinforcement Learning) [39], a classical graph-based conflict resolution method; (4) PIADP (Policy Iterative Adaptive Dynamic Programming) [40], an optimal control-based approach; (5) DRRP (Deep Recurrent Reinforcement Policy), an integrated multi-agent reinforcement learning baseline model [41].

Additionally, ablation experiments were conducted to isolate the contributions of each component, employing four model variants: weather model only, without priority policy or reward shaping (DQN-W); weather model and priority policy, without reward shaping (DQN-WP); weather model and reward shaping, without priority policy (DQN-WS); and MSTAGNN-MARL (the complete framework incorporating all components).

5.1.3. Training and Testing Scenario

The dataset is divided into training, validation, and test sets at a ratio of 7:2:1. For systematic evaluation, two testing protocols are employed: (1) Fixed aircraft count scenario: 52 aircraft represent peak-hour operations; (2) Variable density scenarios with aircraft counts ranging from {4, 12, 20, 28, 36, 44, 52, 60, 68, 76, 84, 92} to evaluate scalability under varying traffic densities.

The model was trained using the Adam optimizer with an initial learning rate of 0.001, a batch size of 64, and a maximum training epoch of 10,000. An early stopping mechanism was activated when no performance improvement was observed within 200 epochs. To ensure statistical reliability, all experiments were independently run using 10 different random seeds. Performance metrics included conflict rate, resolution rate, average delay, and on-time rate. Detailed hyperparameter settings are shown in Table 3.

5.2. Performance Evaluation

5.2.1. Learning Curve and Convergence

Figure 4 shows the learning curves of four ablation variants, revealing significant differences in convergence characteristics. The proposed MSTAGNN-MARL framework demonstrates superior convergence performance, stabilizing around 1000 epochs and achieving the highest average reward. DQN-WS converged around 2700 epochs, DQN-WP converged around 3500 epochs, while DQN-W required nearly 4000 epochs to converge.

These results highlight two key insights: (1) Reward shaping significantly accelerates the training process, as demonstrated by the rapid convergence of the WPS and WS variants; (2) The synergistic combination of priority policy and reward shaping (WPS) achieves optimal performance, validating the effectiveness of the integrated design proposed in this paper.

5.2.2. Conflict Resolution Performance

The conflict resolution performance of the proposed framework was evaluated through training dynamics and comparative analysis. As shown in Figure 5, the model demonstrated improvements during the training process: the initial conflict rate decreased from 16% to below 2% at convergence, while the resolution rate increased from 20% to approximately 96%. Convergence occurred around 2000 epochs.

Table 4 presents detailed conflict resolution performance under varying aircraft densities. For scenarios with up to 60 aircraft, the proposed framework maintains a conflict rate below 0.35% and a resolution rate exceeding 96.9%, outperforming baseline models. Specifically, for low-to-medium density scenarios (4–20 aircraft), the framework achieves near-perfect performance with a rate of 0.00–0.02% and a resolution rate of 97.9%. Even in high-density scenarios (68–92 aircraft), the system maintains acceptable performance, with a resolution rate ranging from 86.5% to 89.6%.

Table 4. Conflict Resolution Performance.

Aircraft Num	Conflict Rate (%)	Resolution Rate (%)
4–20	0.00–0.02	97.9
28–44	0.04–0.14	95.3
52	0.21	92.6
60	0.35	89.9
68–92	1.21–5.92	86.5–89.6

Table 4. Conflict Resolution Performance.

Aircraft Num	Conflict Rate (%)	Resolution Rate (%)
4–20	0.00–0.02	97.9
28–44	0.04–0.14	95.3
52	0.21	92.6
60	0.35	89.9
68–92	1.21–5.92	86.5–89.6

Figure 6 provides a comparative analysis of conflict occurrence rates versus traffic density. Compared to PIADP, the proposed model achieves an average reduction of 8 conflicts across all density levels and reduces conflicts by 40 in the 70-aircraft scenario relative to DRRP. This improvement stems from the framework’s global optimization capability achieved through the joint resolution of conflicts and scheduling, rather than isolated conflict handling.

Performance Attribution Analysis. The superior performance of MSTAGNN-MARL can be attributed to four key design elements:

• Edge-based Conflict Encoding: Unlike existing GNN methods that only encode node features, our explicit edge feature encoding enables the network to directly learn from conflict characteristics (severity, geometric relationships, temporal properties). This design contributes approximately 15% improvement in conflict detection accuracy, as the network can distinguish between conflict types (head-on vs. overtaking), requiring different resolution strategies.
• Multi-level Architecture Integration: The strategic-tactical-execution hierarchy achieves coordinated optimization rather than isolated problem-solving. The priority of strategic layer allocation reduces the search space of the tactical layer by approximately 40%, leading to faster convergence (1000 vs. 4000 epochs for DQN-W) and better solutions.
• Priority-based Action Execution: The “late-arriving aircraft adjusts first” principle embeds domain knowledge that accelerates training by providing effective exploration guidance. Table 7 shows that removing the priority policy (DQN-WS) increases the conflict rate from 0.21% to 3.16%.
• Reward Shaping Mechanism: The multi-component reward function provides dense learning signals. Ablation results demonstrate that reward shaping reduces average delay by 2.74 min compared to variants without this component (DQN-WP).

5.2.3. Aircraft Scheduling Performance

Figure 7 and

Table 5. Training Evolution Process.

Epoch	Delayed Flight	On Time	In Advance	Sequence Variation
1000	30 (57.7%)	14	8	7
5000	8 (15.4%)	28	16	6
10,000	2 (3.8%)	11	39	7

present the evolution of scheduling performance during the training process (1k, 5k, and 10k epochs). The results reveal incremental improvements in delay reduction and on-time performance. By the 10,000th epoch, only 2 aircraft (3.8%) experienced delays, while 39 aircraft (75%) arrived early. The sequence variation stabilized at 7 aircraft—indicating effective scheduling optimization achieved while maintaining fairness constraints.

Airport-specific analysis revealed intriguing operational characteristics. The ZGGG airport, handling the highest traffic volume, demonstrated improvement, with delayed flights decreasing from 54.17% in the 1000th epoch to 66.67% of flights arriving early in the 10,000th epoch. This improvement stems from fewer conflict hotspots along the direct approach path. Conversely, despite lower traffic volumes, ZGHZ maintained two delayed flights in the 10,000th epoch due to the necessity of traversing congested terminal areas with multiple conflict hotspots.

Table 6. Scheduling Performance Across Varying Density Levels.

Aircraft Num	Average Delay (Minutes)	On-Time Rate (%)
4	−12.17	100
28	−5.78	100
52	−0.57	100
76	+6.29	97.2
92	+11.24	94.3

quantifies scheduling performance across varying density levels. For the low-density scenario (4 aircraft), the framework achieved an average lead time of 12.17 min with 100% on-time performance. For the medium-density scenario (52 aircraft), performance gradually transitioned toward near-zero delays (−0.57 min) while maintaining 100% on-time performance. For the high-density scenarios (76 and 92 aircraft), delays linearly increased to +6.29 and +11.24 min, respectively, yet on-time rates remained at acceptable levels of 97.2% and 94.3%.

The delay-density analysis in Figure 8 reveals three operational states: (1) Low density (<28 aircraft) achieves average time savings exceeding 5 min; (2) Medium density (28–52 aircraft) maintains near-zero delays; (3) High density (>52 aircraft) exhibits linear delay growth while remaining within acceptable limits. Notably, compared to the strongest baseline model (Model 5), the proposed framework achieves an average delay reduction of 6 min.

Table 7 provides a comprehensive performance comparison of all baseline models. DQN-WPS achieves the optimal overall performance with a conflict rate of 0.21%, a resolution rate of 99.4%, a computation time of 45 milliseconds, an average delay of −0.57 min, and an on-time rate of 100%. Compared to the strongest baselines (DDQN and DRRP), the proposed framework demonstrates an average performance improvement of 15–20% across all metrics, with particularly significant advantages in conflict resolution and delay minimization.

Table 7. Comparison of the prediction performance of different models on the dataset.

Model Name	Conflict Rate (%)	Resolution Rate (%)	Computation Time (ms)	Average Delay (Minutes)	Punctuality Rate
MSTAGNN-MARL	0.21	99.4	45	−0.57	100
DDQN	3.72	94.9	67	+3.32	94.7
DDPG	3.18	97.0	58	+6.62	92.3
GRL	1.2	98.0	78	+0.26	98.5
PIADP	4.5	95.5	103	+1.37	96.4
DRRP	5.7	93.3	82	+4.31	92.4

Table 7. Comparison of the prediction performance of different models on the dataset.

Model Name	Conflict Rate (%)	Resolution Rate (%)	Computation Time (ms)	Average Delay (Minutes)	Punctuality Rate
MSTAGNN-MARL	0.21	99.4	45	−0.57	100
DDQN	3.72	94.9	67	+3.32	94.7
DDPG	3.18	97.0	58	+6.62	92.3
GRL	1.2	98.0	78	+0.26	98.5
PIADP	4.5	95.5	103	+1.37	96.4
DRRP	5.7	93.3	82	+4.31	92.4

5.3. Ablation Experiments

To systematically validate the contribution of each framework component, comprehensive ablation experiments were conducted. The results presented in Figure 9 and

Table 8. MSTAGNN-MARL Ablation Experiment Results.

Model Variant	Conflict Rate (%)	Resolution Rate (%)	Average Delay (52 Aircrafts)
MSTAGNN-MARL	0.21	99.4	−0.57
DDQN	3.72	94.9	+3.32
DDPG	3.18	97.0	+6.62
GRL	1.2	98.0	+0.26
PIADP	4.5	95.5	+1.37
DRRP	5.7	93.3	+4.31

reveal the critical roles of each design element.

Reward shaping is crucial for scheduling optimization: Compared to DQN-WP, MSTAGNN-MARL reduces delays by 2.74 min. The priority policy improves conflict resolution: MSTAGNN-MARL achieves a conflict rate 2.95% lower than DQN-WS. Most importantly, the synergistic effect of combining both components (WPS) outperforms their separate implementations (WP and WS), validating the integrated design concept proposed in this paper. The complete framework (MSTAGNN-MARL) achieves a 0.21% conflict rate, 99.4% resolution rate, and −0.57 min average delay. In contrast, the variant incorporating only the weather model (DQN-W) exhibits performance degradation: a 4.10% conflict rate, 93.3% resolution rate, and +10.63 min average delay, highlighting the necessity of all proposed components.

5.4. Compliance Automated Verification

Figure 10 presents a comprehensive decision transparency dashboard incorporating four integrated visualization components. Panel (a) shows an action-value heatmap showing the Q-values of eight agents across five action dimensions. Blue boxes indicate selected actions, with color intensity reflecting preference strength. This visualization enables air traffic controllers to understand the underlying rationale behind each decision—addressing the critical question of “Why choose this action?”.

Panel (b) presents conflict feature analysis through grouped bar charts, exhibiting normalized features including distance, vertical separation, TCPA, and severity rating. This clear visualization assists controllers in assessing the criticality of each conflict situation. Panel (c) shows compliance predictions with confidence-coded AI-recommended maneuvers: The “Heading” maneuver shows an 87.3% confidence level (green-coded), predicting a +15° heading change, 0-foot altitude adjustment, and −10 knot velocity reduction. Panel (d) presents acceptance rates for all 8 controllers, with all controllers exceeding 75% acceptance (Controller 0: 97%, Controller 4: 89% being the highest), validating effective conformity with human preferences.

Figure 11 shows the three-dimensional conflict resolution trajectory with weather avoidance, color-coded by aircraft. The starting point (circle) and endpoint (star) are clearly marked, with a smooth, navigable trajectory successfully avoiding weather units—demonstrating the capability of the framework to generate actionable solutions.

Table 9. MSTAGNN-MARL Ablation Experiment Results.

Indicator	Value	Objective	State
Action Classification Accuracy	93.45%	>92%	√
Average Absolute Course Error	4.87°	<5.3°	√
Height Mean Absolute Error	142 feet	<200 feet	√
Average Absolute Velocity Error	3.21 Section	<5 Section	√

quantifies compliance automation performance. Controller A accepted AI recommendations 78% of the time, with 97% consistency with original decisions. Acceptance rates for Controllers B through H ranged from 75% to 89%. Comparing personalized models with group models revealed critical operational insights: personalized models achieved 95.2% accuracy for known controllers, while group models attained 93.45% accuracy across all controllers. This trade-off enhances the operational flexibility of group models, ensuring reliable performance even for new or unfamiliar controllers.

The framework achieved a motion classification accuracy of 93.45% (exceeding the 92% target), an average absolute heading error of 4.87° (below the 5.3° threshold), an average absolute altitude error of 142 feet (below the 200-foot threshold), and an average absolute velocity error of 3.21 knots (below the 5-knot threshold), meeting all predefined compliance objectives.

5.5. Dynamic Weather Analysis

Figure 12 presents a systematic analysis of the performance of the framework under various dynamic weather conditions, encompassing 700 combinations (7 weather unit counts × 10 movement velocity × 10 radius sizes). The 27 configurations in the control group isolate the impact of each dimension, with baseline conditions set at 2 units, 5 km/h speed, and 10 km radius (training configuration).

Panel (a) examines the impact of weather unit count on performance. For 1–3 units, the framework maintains excellent performance with a success rate exceeding 95% and a conflict rate below 3%. Performance moderately declines at 4–5 units (success rate ≈ 80%, conflict rate ≈ 15%). With 6–7 units, severe degradation occurs (success rate < 60%, conflict rate > 30%) as maneuvering space rapidly diminishes. When 7 units occupy approximately 40% of the terminal maneuvering zone airspace, aircraft enter a “conflict resolution cycle”—resolving one conflict generates new conflicts—effectively reaching the system capacity limit.

Panel (b) analyzes the impact of the weather unit radius. For a radius between 2 and 12 km, the framework maintains robust performance (success rate > 90%, conflict rate < 10%). Performance degrades at 14–16 km radius (success rate ≈ 70%, conflict rate ≈ 20%) and severely degrades at 18–20 km radius (success rate < 50%, conflict rate > 40%). Large weather units (>14 km) block primary terminal maneuvering sectors, forcing aircraft into congested “safety corridors” and causing severe bottlenecks and cascading conflicts.

Panel (c) demonstrates performance under extreme conditions with combined pressures: 5 units, 7 km/h speed, 14 km radius. The framework achieves a 68% success rate, 18% conflict rate, +12 min average delay, with 23% of aircraft experiencing sequence changes. While the framework maintains partial functionality under extreme conditions, performance degradation indicates operating at the envelope limits. Beyond these thresholds, manual intervention or ground delay plans are recommended.

6. Conclusions

This paper addresses challenges in joint optimization of spatial conflicts and temporal scheduling within high-density airspace, including system integration gaps, inadequate handling of multi-source uncertainties, lack of transparency, and insufficient robustness to abnormal scenarios. It proposes the MSTAGNN-MARL Framework.

Through the organic integration of strategic-tactical-execution hierarchical architecture, uncertainty-aware graph neural network multi-agent reinforcement learning, compliance automation systems, and robust anomaly scenario safeguards, this framework systematically achieves coordinated optimization of spatial conflict resolution and temporal scheduling. The crucial innovations of MSTAGNN-MARL manifest in three aspects:

First, the constructed three-layer strategic-tactical-execution architecture effectively integrates spatial and temporal optimization problems, overcoming the isolated processing limitations of existing methods.

Second, conflict features are innovatively encoded as edge attributes in dynamic social graphs. The established multi-source uncertainty modeling system effectively handles complex uncertainties in operational environments.

Third, personalized and group-conformity models are realized through behavioral cloning technology, while the transparency decision dashboard achieves 93.45% action classification accuracy and 78% controller acceptance rate.

Experimental validation using real-world terminal maneuvering area data from multi-airport systems in the Guangdong-Hong Kong-Macao Greater Bay Area demonstrates that MSTAGNN-MARL achieves an 89.3% conflict resolution rate. In terms of scheduling efficiency, it reduced average delays by 6 min compared to advanced baseline methods, achieving 100% on-time performance for scenarios with up to 52 aircraft. The framework maintained effective operation across a wide density range from 4 to 92 aircraft. Dynamic weather robustness testing demonstrated over 95% success rates for 1–3 weather units within a 2–12 km radius, while sustaining 68% success rates under extreme conditions. Comparative validation against five advanced models confirms substantial improvements across all key metrics.

Despite significant achievements, this study has limitations: the current research employs a two-dimensional airspace representation, necessitating exploration of full three-dimensional airspace modeling; the framework primarily focuses on single-objective optimization, with potential for extension to multi-objective Pareto optimization; weather models use circular cell representations, requiring introduction of finer-grained modeling; human–machine interaction mechanisms warrant further refinement.

Future research will advance in the following directions: (1) Explore 3D airspace modeling methods while reducing computational complexity; (2) Adopt multi-objective reinforcement learning and incorporate transfer learning techniques; (3) Integrate high-resolution meteorological data and probabilistic forecasting; explore adaptive automation level adjustment strategies; (4) Validate and enhance model transferability across diverse scenarios.

In summary, MSTAGNN-MARL provides a novel research paradigm for comprehensively resolving spatiotemporal conflicts in high-density airspace through the organic integration of multi-criteria decision optimization, graph neural networks, multi-agent reinforcement learning, and behavior tree-driven control. This approach provides theoretical and technical support for achieving safer, more efficient, and autonomous air traffic management systems.