Hybrid DeepMUSIC-Assisted Cooperative Multi-Agent Deep Reinforcement Learning for Intelligent Spectrum Allocation and Interference Management in Multi-UAV 6G Networks

Abstract

The integration of unmanned aerial vehicles (UAVs) as aerial base stations has emerged as a key enabler for next-generation wireless networks, particularly in disaster recovery, temporary events, and infrastructure-deficient regions. However, multi-UAV deployments introduce severe co-channel interference due to spectrum reuse and overlapping coverage areas, while existing spectrum allocation methods either rely on centralized optimization with limited scalability or on reinforcement learning frameworks that lack spatial awareness of interference sources. To address these challenges, this paper proposes a Hybrid DeepMUSIC-assisted Cooperative Multi-Agent Deep Reinforcement Learning (MADRL) framework for intelligent spectrum allocation and interference management in multi-UAV 6G networks. The proposed framework integrates a hybrid interference localization module, which fuses the classical MUltiple SIgnal Classification (MUSIC) algorithm with a deep neural network to accurately estimate the direction of arrival (DoA) of interference sources, into a DeepMUSIC-enhanced state representation used by cooperative Deep Q-Network (DQN) agents trained under a Centralized Training and Decentralized Execution (CTDE) paradigm, enabling coordinated yet fully distributed spectrum allocation decisions. Extensive simulations demonstrate that the proposed Hybrid DeepMUSIC module reduces the mean DoA estimation error to approximately 0.105°, more than an order of magnitude better than classical MUSIC and standalone DeepMUSIC estimators. Compared with seven baseline algorithms spanning heuristic, optimization-based, single-agent, and cooperative multi-agent reinforcement learning approaches, the proposed framework achieves the highest network throughput, SINR, spectrum efficiency, and energy efficiency, together with the fastest and most stable training convergence, reaching a stable cooperative reward of 76.246 within approximately 371 training epochs. The framework further maintains near-linear computational scaling with the number of UAV agents, confirming its suitability for real-time deployment in dense, AI-native multi-UAV 6G wireless communication systems.

1. Introduction

The rapid evolution of wireless communication technologies is giving rise to unprecedented demands for high data rates, ultra-low latency, and massive connectivity, which are key requirements for future sixth-generation (6G) communication systems. In contrast to previous generations of wireless networks, 6G aims to support intelligent, autonomous, and highly dynamic network environments integrating terrestrial, aerial, and satellite communication infrastructures. In this context, unmanned aerial vehicles (UAVs) have emerged as a promising technology for enabling flexible and adaptive wireless communication architectures due to their mobility, rapid deployment capability, and ability to provide line-of-sight communication links. Consequently, UAV-assisted wireless networks have become an important component of future space–air–ground integrated communication systems designed to support ubiquitous connectivity in next-generation networks [1,2,3,4].

In recent years, UAV-assisted communication has been widely investigated for applications including emergency communications, disaster recovery, temporary network coverage, and Internet of Things (IoT) connectivity. By acting as aerial base stations or relay nodes, UAVs can dynamically adjust their position and coverage area to improve network capacity and connectivity in regions where terrestrial infrastructure is insufficient or unavailable [5,6,7]. The integration of UAVs into wireless communication systems is expected to significantly enhance network coverage, reduce communication latency, and improve system reliability in next-generation wireless networks. But deploying a large scale of UAVs for multiple communication environments creates new challenges, such as radio resource management, spectrum allocation, and interference mitigation [8,9,10].

One of the most critical challenges in UAV-assisted communication systems is efficient spectrum utilization. Due to the increasing number of wireless devices and bandwidth-intensive applications, radio spectrum has become a scarce and valuable resource. In multi-UAV communication networks, spectrum allocation must be carefully managed to avoid severe co-channel interference and ensure efficient resource utilization. Conventional optimization, scheduling, and interference-aware channel assignment techniques have demonstrated considerable effectiveness in wireless resource management, particularly when network topology, channel conditions, and interference relationships can be adequately characterized. Multi-rate multi-channel scheduling and cross-layer resource allocation, for example, can efficiently coordinate transmissions and exploit available channel capacity while mitigating interference. Nevertheless, applying such approaches to highly mobile multi-UAV networks may require frequent reconstruction of interference relationships and repeated optimization as aerial topology, channel conditions, and user associations evolve. This motivates the investigation of adaptive decision-making mechanisms capable of learning spectrum-allocation policies from repeated network interactions rather than replacing conventional optimization methods altogether [11,12,13].

Over the past few years, artificial intelligence (AI) and machine learning (ML) technologies have gained significant attention as effective solutions for addressing complex optimization problems in wireless communication networks. In particular, deep learning-based approaches have demonstrated remarkable potential for enabling intelligent radio resource management, dynamic spectrum access, and interference mitigation in highly dynamic network environments. AI-driven communication systems are expected to play a crucial role in future 6G networks, enabling adaptive and autonomous network control mechanisms capable of optimizing network performance in real time [1,14,15]. Recent studies have shown that deep learning is able to significantly enhance spectrum sensing, channel allocation, and resource optimization in wireless communication systems [16,17,18].

Out of various AI techniques, reinforcement learning (RL) has gained attention as a powerful tool for solving sequential decision-making problems in wireless networks. In reinforcement learning, an agent learns optimal actions by interacting with the environment and receiving feedback in the form of rewards. Deep reinforcement learning (DRL), which integrates deep neural networks with reinforcement learning algorithms, enables intelligent agents to learn complex decision policies in high-dimensional state spaces. DRL has been widely applied in wireless communication systems for dynamic spectrum access, power control, and resource allocation problems [19,20,21].

However, most existing DRL-based approaches focus on single-agent learning frameworks, which may not be suitable for distributed wireless communication environments involving multiple autonomous nodes. In multi-UAV networks, each UAV operates as an independent communication agent that must make decisions regarding spectrum usage while considering interference and resource sharing with neighboring UAVs. Therefore, cooperative multi-agent reinforcement learning (MARL) has been proposed as an effective approach for enabling distributed intelligence in multi-node wireless communication systems. In MARL frameworks, multiple agents interact with the environment and cooperate to achieve global network optimization objectives [22,23,24].

Recent research has demonstrated that cooperative multi-agent deep reinforcement learning can significantly improve spectrum efficiency and network performance in large-scale wireless communication systems. For example, multi-agent learning frameworks have been proposed for intelligent spectrum sharing, UAV trajectory optimization, and interference management in dynamic wireless environments [25,26]. Moreover, advanced AI-based techniques have been applied to optimize resource allocation and improve network capacity in multi-UAV communication systems operating in 6G environments [27]. These approaches highlight the potential of cooperative AI techniques for enabling autonomous wireless networks capable of adapting to complex and dynamic communication scenarios.

Despite the extensive application of DRL to terrestrial wireless resource management, directly transferring these approaches to multi-UAV ad hoc networks is non-trivial because aerial networks exhibit several distinctive characteristics. First, UAV mobility produces a rapid time-varying three-dimensional topology in which neighborhood relationships, link distances, user associations, and interference coupling may change within successive decision intervals. Second, the predominantly line-of-sight propagation of air-to-air and air-to-ground links can extend interference over wider spatial regions, making spectrum decisions strongly dependent on both the power and direction of interfering signals. Third, UAVs operate under constrained onboard energy, computation, and communication resources, such that frequent centralized optimization, global-state exchange, or extensive inter-agent signaling may reduce operational endurance. Fourth, UAV movement must preserve communication continuity while network topology changes. For example, the authors of [28] demonstrated in multi-hop UAV multicasting that UAV transitions must simultaneously consider seamless connectivity, trajectory length, signaling overhead, energy consumption, and communication performance and proposed a resource-efficient transition formation mechanism to address these coupled requirements.

These characteristics create a learning environment fundamentally more dynamic than many conventional ground communication scenarios. From a DRL perspective, UAV mobility continuously changes not only the physical state observed by each agent but also the interference relationships among agents. When multiple UAVs simultaneously move and update their spectrum-selection policies, each agent experiences a doubly non-stationary environment arising from both mobility-induced topology variation and policy-induced changes in neighboring agents’ behavior. Related UAV-swarm studies have similarly demonstrated that multi-agent learning must account for dynamically varying channel conditions, network topology, and inter-agent interactions to maintain reliable communication performance [29]. Furthermore, partial local observations may not reveal the spatial origin of interference, while continuous global information exchange would introduce excessive signaling overhead. Therefore, the challenge of the central research challenge is not merely to apply DRL to spectrum allocation, but to design a cooperative learning architecture that remains interference-aware, scalable, and executable with local observations under rapidly changing aerial topology.

To overcome these challenges, this paper proposes a cooperative multi-agent deep reinforcement learning framework for intelligent spectrum allocation and interference management in multi-UAV 6G networks. We model each UAV as an autonomous agent in the potential domain and use reinforcement learning to selectively decide communication channels based on network state information and interference conditions. Through cooperative learning and distributed decision making, the proposed approach aims to improve spectrum utilization, reduce interference, and enhance overall network performance. Extensive simulations are conducted to evaluate the effectiveness of the proposed approach in comparison with conventional spectrum allocation methods.

The main contributions of this work can be summarized as follows:

• Cooperative sequential spectrum-allocation formulation: The multi-UAV spectrum-allocation problem is formulated as a cooperative sequential decision process that explicitly captures the coupling among channel selection, co-channel interference, UAV interactions, and time-varying network conditions.
• Hybrid DeepMUSIC-assisted spatial interference awareness: A hybrid model-based/data-driven interference-localization mechanism is developed by combining classical MUSIC with DeepMUSIC to provide high-resolution DoA information, thereby extending conventional power- and channel-quality-based interference observations with spatial awareness.
• Spatially enhanced cooperative MADRL under CTDE: The estimated DoA information is embedded directly into the agents’ state representation, enabling cooperative spectrum-selection policies to exploit both spectral and spatial interference characteristics. The CTDE paradigm supports coordinated policy learning while preserving decentralized execution at individual UAVs.
• Comprehensive comparison with conventional and learning-based methods: The integrated framework is evaluated against representative heuristic, interference-aware optimization, single-agent DRL, independent MARL, and cooperative MARL baselines to quantify localization accuracy, communication performance, learning convergence, and computational scalability.

These contributions are designed as interconnected components of a unified research pipeline rather than as independent developments. The first contribution establishes the cooperative sequential spectrum-allocation problem and defines the interaction between channel selection and interference among UAV agents. Building on this formulation, the second contribution provides the spatial interference information required to characterize not only the magnitude but also the directional structure of co-channel interference through Hybrid DeepMUSIC. The third contribution then incorporates this directional information into the enhanced state representation of the cooperative MADRL agents, enabling the learned spectrum-selection policies to exploit both spectral and spatial interference characteristics, while CTDE supports coordinated training and decentralized execution. Finally, the fourth contribution systematically validates the resulting integrated framework against conventional and learning-based baselines in terms of localization accuracy, communication performance, learning convergence, and computational scalability. Therefore, the four contributions collectively form a sequential pipeline of problem formulation, spatial interference localization, cooperative spectrum decision making, and performance validation, thereby achieving the overall research goal of scalable and interference-aware spectrum allocation in dynamic multi-UAV 6G networks.

The remainder of this paper is organized as follows. Section 2 reviews the related work on UAV-assisted wireless communications, intelligent spectrum allocation, and deep reinforcement learning-based resource management in next-generation wireless networks. Section 3 presents the system architecture and interference-aware spectrum allocation model. Section 4 introduces the proposed cooperative multi-agent deep reinforcement learning framework. Section 5 describes the simulation setup and evaluation parameters used to assess the performance of the proposed approach. Section 6 presents and analyzes the simulation results, highlighting improvements in spectrum efficiency, network throughput, and interference mitigation compared with conventional methods. Finally, Section 7 concludes the paper and discusses potential future research directions for intelligent UAV-assisted communication systems in 6G networks.

2. Related Works

Existing studies on spectrum allocation and interference management in multi-UAV networks encompass a continuum of methodologies, ranging from conventional optimization, scheduling, and graph-based resource allocation to learning-based adaptive and cooperative decision-making approaches. Rather than separating these studies solely according to the use or absence of machine learning, this section adopts a problem-oriented methodological progression, allowing different approaches to be compared under the common objective of mitigating interference and improving spectrum utilization in dynamic multi-UAV environments. This organization highlights how increasingly adaptive solution paradigms address the limitations introduced by UAV mobility, time-varying interference relationships, and decentralized spectrum competition, ultimately motivating the cooperative and spatially aware learning framework proposed in this work.

2.1. Spectrum Allocation and Interference Management in Multi-UAV Networks

Efficient spectrum allocation and interference management are fundamental requirements for multi-UAV wireless networks because multiple aerial communication nodes may simultaneously compete for limited radio resources while operating within partially or extensively overlapping coverage regions. In contrast to conventional terrestrial cellular systems, where base stations are generally deployed at fixed locations, UAV-based communication nodes can continuously change their positions according to coverage requirements, user distributions, mission objectives, or environmental conditions. Such mobility causes the spatial relationships among UAVs and ground users to vary over time, resulting in dynamic changes in channel quality, coverage overlap, user association, and interference coupling. Moreover, the predominantly line-of-sight (LoS) characteristics of air-to-ground and air-to-air propagation can improve coverage and communication reliability but may also extend the spatial range over which co-channel transmissions interfere with one another. Consequently, spectrum allocation in multi-UAV networks must simultaneously account for limited spectrum availability, dynamic network topology, and time-varying interference relationships [30,31,32,33,34].

A wide range of conventional spectrum-management mechanisms has been developed to address spectrum scarcity and interference, including fixed and dynamic channel assignment, interference-aware scheduling, graph-based allocation, power control, and cross-layer resource optimization. These approaches have demonstrated considerable effectiveness and should not be regarded as inherently inferior to learning-based methods. In particular, optimization and scheduling mechanisms can provide highly efficient solutions when channel conditions, interference relationships, traffic demands, and network topology can be adequately characterized. For example, Tu [35] developed an efficient wireless multimedia multicast mechanism for multi-rate multi-channel mesh networks in which transmission-rate diversity and multiple channels are jointly exploited to improve resource utilization while controlling interference. The approach demonstrates that carefully coordinated rate and channel scheduling can achieve an effective trade-off between transmission coverage and communication performance. Similarly, Tu [36] investigated efficient resource utilization for multi-flow wireless multicasting through coordinated flow scheduling and channel aggregation. By exploiting residual channel capacity, additional concurrent traffic flows can be accommodated without unnecessarily degrading communication performance. These studies provide important evidence that effective spectrum and interference management does not inherently require machine learning; rather, carefully designed model-based mechanisms can achieve substantial performance improvements when the underlying network relationships are sufficiently known.

Graph-based spectrum allocation provides another important class of conventional interference-management approaches. In such methods, communication nodes or links are generally represented as vertices, while edges characterize potentially harmful interference relationships. Spectrum allocation can then be formulated as a graph-coloring problem in which interfering nodes are assigned different channels whenever possible. This representation provides an intuitive and computationally tractable mechanism for avoiding co-channel interference, particularly when the interference graph remains relatively stable. Similarly, interference-aware scheduling and power-control mechanisms can coordinate simultaneous transmissions according to channel quality, interference constraints, or predefined performance objectives. Collectively, these approaches demonstrate that conventional scheduling, optimization, and graph-based mechanisms remain strong baselines for evaluating emerging intelligent spectrum-management techniques. Nevertheless, their application to highly mobile multi-UAV networks introduces additional challenges that are less prominent in networks with relatively stable infrastructure. UAV movement continuously changes the distances and geometric relationships among aerial nodes and ground users, thereby modifying coverage overlap, channel gains, and interference relationships. An interference graph constructed at one decision instant may therefore become less representative after UAV movement. Graph-based allocation may consequently require repeated graph reconstruction and channel reassignment, while model-based scheduling and optimization may require updated channel and topology information followed by repeated optimization. Although these procedures remain technically feasible, their computational and signaling requirements can increase as the number of UAVs, users, candidate channels, and interference relationships grows.

The distributed nature of multi-UAV operation further complicates spectrum coordination. A spectrum decision made by one UAV does not affect only its own communication performance; selecting a channel already used by a nearby UAV can modify the SINR and achievable throughput of multiple neighboring links. Spectrum allocation is therefore an interference-coupled decision problem, in which individually favorable decisions may not necessarily maximize overall network performance. Centralized optimization can address this coupling by collecting global network information and jointly determining resource assignments. However, continuous centralized coordination may require frequent exchange of channel-state information, UAV locations, user associations, interference measurements, and resource-allocation decisions. Such requirements become increasingly demanding in dynamic UAV networks, particularly when the topology changes on a timescale comparable to that of spectrum decisions. An additional consideration is that conventional interference metrics predominantly characterize the magnitude of interference rather than its spatial origin. Received interference power and SINR provide valuable indicators of communication quality, but two network states with comparable aggregate interference levels may result from interferers located in substantially different directions. This distinction is particularly relevant in multi-UAV networks because strong LoS propagation and three-dimensional UAV mobility can cause the dominant interference direction to change as aerial nodes move. Spatial interference information can therefore provide complementary knowledge for spectrum decisions, particularly in dense deployments where multiple UAV coverage regions overlap. This observation motivates the investigation of interference-localization mechanisms capable of augmenting conventional channel-quality measurements with directional information. To provide a concise comparison of the major spectrum-allocation and interference-management paradigms discussed above, Table 1 summarizes representative non-ML-based and machine learning-based approaches in terms of their interference-handling mechanisms, adaptability to dynamic network conditions, sequential decision-making capability, distributed execution, and spatial DoA awareness.

Table 1. Comparison of representative non-ML-based and learning-based approaches for spectrum allocation and interference management.

Category	Approach	Interference Handling	Dynamic Adaptation	Sequential Decision Making	Distributed Execution	Spatial DoA Awareness
Non-ML	Multi-rate multi-channel scheduling	Interference-aware scheduling	Update required	Limited	Partial	No
Non-ML	Cross-layer multi-flow scheduling	Flow/channel coordination	Re-optimization required	Limited	Partial	No
Non-ML	Graph Coloring	Interference graph	Graph update required	No	Possible	No
ML	DQN	State-based learning	Yes	Yes	Yes	No
ML	VDN/QMIX	Cooperative learning	Yes	Yes	CTDE	No
Proposed	Hybrid DeepMUSIC- assisted Cooperative MADRL	Power- and DoA-aware cooperative allocation	Yes	Yes	CTDE	Yes

The principal methodological distinction among these approaches lies in how spectrum decisions are generated and how interference information is represented. Multi-rate multi-channel and cross-layer scheduling methods determine resource assignments through explicit optimization of known transmission, traffic, and channel relationships, whereas graph-coloring methods abstract interference into pairwise graph constraints and seek conflict-free channel assignments. DQN replaces repeated explicit optimization with a learned sequential state-to-action mapping but generally treats each decision maker independently. Cooperative MARL methods such as VDN and QMIX further extend this paradigm by learning coordinated multi-agent policies under shared network objectives and CTDE. Nevertheless, these learning-based approaches typically characterize interference through aggregate quantities such as SINR, channel gain, or received interference power. In contrast, the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework augments cooperative sequential decision making with explicit DoA-based spatial interference information. Thus, the methodological progression is not simply from non-learning to learning algorithms, but from model-based instantaneous resource coordination to adaptive sequential decision making, cooperative multi-agent optimization, and ultimately spatially aware cooperative spectrum allocation.

Importantly, these considerations do not imply that conventional optimization, scheduling, or graph-based methods are ineffective. Rather, they identify an operating regime in which adaptability, sequential decision making, and distributed coordination become increasingly important. When UAV topology and interference relationships evolve over time, spectrum allocation can be viewed not only as an instantaneous optimization problem but also as a sequential decision process in which current channel selections influence subsequent interference states and network performance. This perspective provides the motivation for investigating learning-based resource-management mechanisms as a complementary approach to established spectrum-management techniques rather than as their universal replacement. Accordingly, the progression from conventional interference-aware resource allocation toward adaptive learning is primarily motivated by the distinctive dynamics of multi-UAV networks. Deep reinforcement learning (DRL) provides a mechanism for learning sequential spectrum-selection policies from repeated interactions with a changing wireless environment, while multi-agent reinforcement learning (MARL) extends this capability to coordinate decisions among multiple autonomous UAVs. However, the effectiveness of such learning mechanisms depends strongly on how the network state and interference environment are represented. This requirement becomes particularly important when spatial interference information is considered in addition to conventional channel and power measurements. These observations establish the basis for the subsequent discussion of UAV-specific adaptive spectrum-management challenges and learning-based cooperative resource allocation.

2.2. UAV-Specific Challenges for Deep Reinforcement Learning

Although DRL has been extensively investigated for spectrum access, power control, routing, and resource allocation in terrestrial wireless networks, its implementation in UAV ad hoc networks introduces a distinct set of challenges arising from the mobility, three-dimensional topology, propagation characteristics, and resource constraints of aerial platforms. These differences are particularly important in multi-UAV systems because the environment experienced by a learning agent changes not only because of communication traffic and channel fluctuations, but also through the physical movement and simultaneous decisions of neighboring UAVs.

Rapid three-dimensional topology variation is the first major challenge. In terrestrial networks, base stations are generally stationary, and many interference relationships evolve primarily because of user mobility or traffic variations. In contrast, UAV base stations themselves are mobile. Consequently, UAV movement simultaneously modifies link distances, coverage overlap, user associations, channel gains, and interference relationships. The transition dynamics observed by a DRL agent therefore become substantially more complex, while previously learned state–action relationships may become obsolete as the aerial topology evolves.

A second challenge is maintaining seamless communication during UAV mobility. Mobility is not merely an environmental disturbance but an intrinsic network operation that must occur without disrupting active communication services. Maintaining seamless communication during UAV mobility represents another distinctive challenge. Tu [28] demonstrated that UAV transitions require the joint consideration of communication continuity, trajectory length, energy consumption, signaling overhead, and network performance. These findings illustrate an important distinction between UAV and conventional terrestrial networks: mobility decisions and communication-resource decisions are tightly coupled, and maintaining connectivity itself consumes limited aerial resources.

Third, aerial propagation creates distinctive interference characteristics. Elevated UAVs frequently establish strong LoS links with both intended users and unintended receivers. Although LoS propagation improves coverage, it may also increase the spatial extent of co-channel interference. As UAV positions change, dominant interference directions can change rapidly even when aggregate received interference power remains similar. Conventional DRL states based solely on SINR, channel gain, or received interference power may therefore fail to capture this directional structure. This observation directly motivates the Hybrid DeepMUSIC component of the present work, which augments the learning state with high-resolution DoA information.

Fourth, UAVs face stringent energy and computational constraints. Unlike terrestrial base stations connected to persistent power supplies, UAVs must expend limited onboard energy for propulsion, communication, sensing, and computation simultaneously. The UAV transition study in [28] further demonstrates the importance of controlling trajectory length, processing complexity, and additional signaling because these operations directly affect aerial energy consumption and communication performance. Consequently, a practical DRL architecture should avoid computationally intensive global optimization during online operation and minimize continuous inter-UAV information exchange.

Fifth, multi-UAV learning introduces coupled non-stationarity and partial observability. In conventional single-agent DRL, environmental dynamics are typically treated as stationary with respect to the agent’s policy. In multi-UAV networks, however, each UAV simultaneously updates its policy while its neighbors move and make their own resource-allocation decisions. Consequently, the state-transition distribution observed by one UAV varies because of both physical mobility and policy evolution. This dual source of non-stationarity can destabilize independent DRL training and motivates cooperative multi-agent learning.

Finally, scalability and signaling overhead become increasingly important as the UAV swarm grows. Fully centralized DRL may exploit global network information but requires continuous collection of UAV states, channel measurements, interference observations, and actions. Such signaling is undesirable in rapidly changing aerial networks, particularly when backhaul connectivity is intermittent or bandwidth-constrained. Conversely, fully independent learning reduces communication overhead but may lead to conflicting channel decisions and unstable learning. CTDE therefore provides a suitable compromise: global information can be exploited during offline or centralized training, whereas individual UAVs execute the learned policy locally during deployment. These UAV-specific challenges provide the central motivation for the proposed framework. Rather than applying a conventional terrestrial DRL architecture directly to an aerial network, this study combines three complementary mechanisms: Hybrid DeepMUSIC to expose the spatial structure of dynamically changing interference, cooperative MADRL to capture the coupled spectrum decisions of multiple UAVs, and CTDE to stabilize cooperative learning while preserving decentralized execution. Accordingly, the novelty lies in jointly addressing mobility-induced interference dynamics, spatial interference awareness, multi-agent non-stationarity, and distributed execution within a unified spectrum-allocation framework. Table 2 highlights that the motivation for MADRL in the present study arises from the interaction of several UAV-specific factors rather than from spectrum allocation alone. In particular, mobility-induced topology variation changes the interference graph itself, while strong aerial LoS propagation introduces spatial interference patterns that cannot be fully characterized by received power measurements. Meanwhile, battery limitations and dynamic aerial links restrict the feasibility of continuous centralized coordination. The proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework is therefore designed as a direct response to these characteristics rather than as a straightforward transfer of terrestrial DRL to an aerial communication scenario.

Table 2. Comparison of DRL-related challenges in terrestrial and multi-UAV wireless networks.

Key Aspect	Terrestrial Networks	Multi-UAV Networks	Key Challenge for DRL	Proposed Solution
Network topology	Mostly stationary infrastructure	Rapidly changing 3D topology	Time-varying state transitions	Adaptive cooperative MADRL
Interference	Relatively predictable interference patterns	Dynamic LoS and spatial interference	Conventional power-based observations may be insufficient	Hybrid DeepMUSIC-based DoA awareness
Mobility and connectivity	Mobility mainly occurs at user terminals	UAVs themselves move while providing communication services	Maintaining connectivity during topology changes	Cooperative state-aware decision making
Resource constraints	Stable power and computing resources	Limited onboard energy and computation	Learning and signaling overhead	Lightweight decentralized execution
Multi-agent interaction	Relatively stable network environment	Mobility and simultaneous agent decisions	Mobility and policy induced non- stationarity	Cooperative MADRL with CTDE
Scalability and coordination	Reliable infrastructure/backhaul	Dynamic swarm and inter-UAV links	Centralized control becomes increasingly costly	Centralized training and decentralized execution

2.3. DRL and Multi-Agent Learning for Spectrum Allocation

Reinforcement learning (RL) provides a framework for sequential spectrum-allocation problems in which communication nodes repeatedly observe network conditions, select spectrum resources, and receive feedback reflecting communication performance. Deep reinforcement learning (DRL) extends this capability by employing deep neural networks to approximate policies or value functions over high-dimensional network states and has consequently been investigated for dynamic spectrum access, power control, and wireless resource allocation [19,20,21,37,38]. For spectrum allocation, a DRL agent can learn long-term channel-selection policies from repeated interaction with the wireless environment instead of independently solving a new optimization problem at every decision epoch. This characteristic is particularly relevant when channel quality, user associations, and interference relationships evolve over time. However, many conventional DRL formulations adopt a single-agent perspective, limiting their ability to explicitly represent interactions among multiple spectrum-sharing communication nodes.

In multi-UAV networks, spectrum allocation naturally becomes a multi-agent decision problem because the channel selected by one UAV directly influences the interference experienced by neighboring UAVs. Multi-agent reinforcement learning (MARL) addresses this coupling by enabling multiple autonomous agents to learn coordinated policies from shared environmental feedback [39,40]. Cooperative reward mechanisms can encourage UAVs to optimize network-wide performance rather than selfishly maximizing individual throughput. Nevertheless, independent MARL introduces non-stationarity because each agent’s environment changes as neighboring agents simultaneously update their policies. Cooperative learning architectures such as Value Decomposition Networks (VDNs) and QMIX address this issue by exploiting global information during training while retaining decentralized policy execution. The Centralized Training and Decentralized Execution (CTDE) paradigm is particularly relevant to multi-UAV networks because it allows network-wide information to stabilize cooperative learning while avoiding continuous centralized coordination during deployment. A closely related UAV-specific application of multi-agent learning was investigated by Lv et al. [29], who proposed a multi-agent reinforcement-learning-based communication scheme for UAV swarms operating under jamming. Their framework jointly optimizes relay selection and power allocation by exploiting network topology, channel states, previous communication performance, and observations shared by neighboring UAVs. Shared swarm experiences and transfer learning were further employed to accelerate policy learning, while a deep reinforcement-learning extension was introduced to mitigate state-quantization errors under rapidly varying channel conditions associated with high UAV mobility. These results demonstrate the importance of cooperative information exchange and adaptive multi-agent learning for maintaining communication performance in highly dynamic UAV environments. However, the study primarily addresses anti-jamming relay and power-control decisions rather than co-channel spectrum allocation, and it does not explicitly incorporate high-resolution spatial localization of interference sources into the agents’ state representation. In contrast, the present framework integrates Hybrid DeepMUSIC-based DoA estimation with cooperative MADRL, allowing spectrum-selection decisions to exploit both conventional radio measurements and the spatial direction of dominant interferers.

Despite these advances, existing DRL and MARL spectrum-allocation approaches commonly characterize interference through aggregate measurements such as received interference power, SINR, or channel-state information. Such representations indicate the magnitude of interference but generally provide limited information regarding its spatial origin. This limitation becomes particularly relevant in aerial networks, where strong LoS propagation and UAV mobility can produce rapidly changing directional interference patterns.

Direction-of-arrival (DoA) estimation offers a mechanism for introducing this spatial information. Classical MUSIC provides high-resolution source localization through signal- and noise-subspace decomposition [41,42,43], while DeepMUSIC uses deep learning to improve DoA estimation under challenging signal conditions [44]. However, these localization techniques have generally been investigated independently from cooperative spectrum decision making. Their integration with MARL therefore provides an opportunity to construct spectrum-allocation policies that exploit not only channel quality and interference magnitude but also the directional structure of interfering signals.

2.4. Research Gap and Motivation

The reviewed literature reveals a methodological progression from conventional interference-aware scheduling and optimization toward adaptive DRL and cooperative MARL. Conventional scheduling, graph-based allocation, and cross-layer optimization can achieve effective spectrum utilization when network states and interference relationships can be accurately modeled. However, rapidly changing UAV topology may require repeated scheduling, interference-graph reconstruction, or optimization.

DRL provides sequential adaptation to changing network conditions, while MARL extends this capability to interference-coupled decisions among multiple autonomous UAVs. Nevertheless, existing learning-based spectrum-allocation approaches predominantly represent interference through aggregate measurements and generally do not explicitly exploit high-resolution directional information regarding interference sources. Consequently, an important research gap remains at the intersection of spatial interference localization and cooperative sequential spectrum allocation. Specifically, limited attention has been given to integrating high-resolution DoA estimation directly into the state representation of cooperative UAV agents while maintaining decentralized execution suitable for dynamic multi-UAV networks.

Accordingly, the remaining challenge is not merely to introduce additional interference measurements into a MARL state, but to establish an explicit and reliability-aware coupling between spatial interference estimation and cooperative sequential spectrum decision making. To address this gap, the present study proposes a Hybrid DeepMUSIC-assisted Cooperative MADRL framework that integrates model-based and data-driven interference localization with cooperative spectrum decision making. Unlike conventional MARL approaches that rely primarily on aggregate interference measurements, the proposed framework embeds both fused DoA estimates and localization-confidence information into the agents’ state representation, thereby enabling reliability-aware spatial interference perception. These spatial features are integrated with cooperative policy learning under the CTDE paradigm, which enables coordinated training while preserving decentralized execution based on locally available information. The resulting framework addresses four interconnected requirements of dynamic multi-UAV spectrum management: dynamic adaptation, cooperative spectrum allocation, spatial interference awareness, and scalable decentralized execution.

3. System Architecture and Interference-Aware Spectrum Allocation Model for Multi-UAV Wireless Networks

This section presents the system architecture and network model considered in this study. The proposed framework focuses on cooperative spectrum allocation and interference management in a multi-UAV wireless communication network. The overall system model describes the network structure, communication assumptions, and interaction mechanisms among UAV agents operating in a shared wireless environment.

3.1. System Architecture

The system architecture of the multi-UAV wireless communication network under consideration in this research is depicted in Figure 1. In this architecture, multiple unmanned aerial vehicles (UAVs) operate as aerial base stations (UAV BSs) to provide wireless communication services to ground users within a designated coverage area. Wireless backhaul links connect the UAV base stations to a central terrestrial infrastructure, as illustrated by the Macro Base Station (Macro BS). This Macro BS serves as the gateway that connects the UAV communication network to the core network. To provide the wireless connectivity for distributed ground users, we place five UAV base stations (UAV BS 1–UAV BS 5) in the target area. Each UAV provides communication coverage to a group of users located within its service region. Due to the elevated deployment of UAVs, the communication links between UAVs and ground users typically exhibit strong line-of-sight (LoS) characteristics, which can improve signal strength and transmission reliability compared with conventional terrestrial communication systems.

Figure 1. System architecture of the proposed Hybrid DeepMUSIC-assisted cooperative multi-agent deep reinforcement learning framework for multi-UAV 6G networks. UAV base stations provide wireless services to ground users through shared spectrum resources and wireless backhaul connections to the macro base station. Overlapping coverage areas may generate co-channel interference, which is identified through DeepMUSIC-based interference localization. The estimated interference information is incorporated into the MARL framework to support intelligent spectrum allocation, interference mitigation, and adaptive resource management in dynamic wireless environments (generated using AI-assisted visualization tools and modified by the authors.).

The colored areas in the figure display the coverage regions of UAV base stations. The green areas are the non-overlapping coverage areas of UAV base stations, and only one user is served by a single UAV base station without significant interference from neighboring UAVs. The red parts denote overlapping coverage areas, i.e., some UAVs are covering the same area. Under these overlapping area conditions, the co-channel interference occurs when two neighbor UAVs transmit on the same spectrum channel. The brightness of the red color represents coverage overlap, and bad phase conditions are depicted by regions with darker reds. Since UAV base stations compete for finite wireless spectrum resources, interference management is a key issue in overlapping coverage region. In cases where multiple UAVs choose the same communication frequency, co-channel interference can severely hinder the signal quality received by ground users. Hence, efficient spectrum assignment strategies are needed to allow for the cooperation of multiple UAV agents and reduce interference with nearby UAVs. The dashed blue lines in the figure represent the backhaul communication links between UAV base stations and the Macro BS. These backhaul links enable the exchange of network information and support coordination among UAV nodes. However, the spectrum allocation decisions for user communication are assumed to be performed in a decentralized manner by individual UAV agents.

In the examined system framework, every UAV functions as an autonomous agent for decision making capable of intelligently choosing spectrum channels to serve their respective end users. In a distributed decision-making environment, as multiple UAV agents interact with each other, they need to consider their own communication performance and the interference effects generated by neighboring UAVs. Thus, in the multi-UAV network, the spectrum allocation problem can be modeled as a cooperative multi-agent learning task. Based on this system architecture, the multi-agent deep reinforcement learning techniques for cooperative spectrum allocation and interference management in UAV-assisted wireless communication systems can be applied.

3.2. Spectrum Resource Model

Efficient spectrum management is a critical challenge in UAV-assisted wireless communication systems, particularly in dense network deployments where multiple aerial base stations operate within a shared radio environment. In the considered multi-UAV communication architecture, spectrum resources are limited and must be dynamically allocated among multiple UAV agents to ensure reliable communication with ground users while minimizing co-channel interference.

Before spectrum allocation is performed, the serving UAV of each ground user is dynamically determined according to the current network geometry and large-scale channel conditions. Let $x_{i,u}\left(t\right)\in \left\{0,1\right\}$ denote the association indicator, where $x_{i,u}\left(t\right)=1$ if ground user $u$ is associated with UAV $i$ at decision epoch $t$, and $x_{i,u}\left(t\right)=0$ otherwise. Each user is associated with at most one UAV according to ${\displaystyle \sum _{i=1}^N{x}_{i,u}\left(t\right)}=1; \forall u$, provided that at least one UAV satisfies the coverage requirement. Among the candidate UAVs covering user $u$, the serving UAV is selected according to the strongest large-scale channel gain

$$i_u^{\ast }\left(t\right)=\arg \underset{i\in {\mathcal{B}}_u\left(t\right)}{\max }{\overline{h}}_{i,u}\left(t\right),$$

(1)

where ${\mathcal{B}}_u\left(t\right)$ denotes the set of UAVs whose coverage regions include user $u$ and ${\overline{h}}_{i,u}\left(t\right)$ the large-scale UAV–user channel gain incorporating distance-dependent path loss and shadow fading. Accordingly, $x_{i,u}\left(t\right)=1$ for $i=i_u^{\ast }\left(t\right)$, and the set of users served by UAV $i$ is defined as $u_i\left(t\right)=\left\{u: x_{i,u}\left(t\right)=1\right\}.$ The association is therefore dynamically updated as UAV positions and user locations change. In the considered simulations, ground-user mobility follows the Random Waypoint model, while UAV mobility changes the corresponding three-dimensional link geometry. Instantaneous small-scale fading and co-channel interference are not directly used as association triggers; instead, they affect the subsequent SINR and achievable-rate calculations. This separation avoids excessively frequent association switching while allowing the serving UAV to adapt to mobility-induced changes in network geometry and large-scale link quality.

The available spectrum is divided into $K$ orthogonal channels, each with bandwidth $B_c.$ At each decision epoch $t$, each UAV agent selects one channel from the available spectrum pool according to its local observation and learned spectrum-selection policy. The simultaneous channel selections of all $N$ UAV agents constitute the joint spectrum-allocation decision. When neighboring UAVs with overlapping coverage select the same channel, co-channel interference arises and affects the achievable communication performance.

Because UAV agents make spectrum decisions independently in a distributed environment, multiple UAVs may select the same channel simultaneously, which may lead to interference in overlapping coverage regions. In contrast, the conventional cellular networks where spectrum allocation is centrally coordinated by a base station controller, the considered UAV-assisted network operates under a distributed spectrum sharing framework. In this framework, UAV agents dynamically select channels based on locally observed network conditions. Two primary scenarios may occur during spectrum allocation: (1) if neighboring UAVs select different channels, spectrum resources are used efficiently without causing mutual interference. This situation typically occurs in non-overlapping coverage regions where UAV transmissions are spatially separated. (2) When multiple UAVs select the same channel while operating within overlapping coverage areas, co-channel interference may occur. This interference may degrade signal quality and reduce the achievable communication throughput of affected users. Therefore, efficient spectrum allocation strategies must enable UAV agents to coordinate channel selection decisions to reduce channel conflicts and improve spectrum utilization efficiency.

Efficient spectrum utilization is a key performance objective in multi-UAV wireless communication networks. Proper spectrum allocation enables multiple UAVs to simultaneously serve ground users while mitigating co-channel interference and improving the overall network throughput. Given the dynamically updated user association and the channel selected by each UAV, the achievable rate of an associated UAV–user link is determined by its instantaneous signal-to-interference-plus-noise ratio (SINR). Let $a_i\left(t\right)$ denote the channel selected by UAV $i$, $P_i$ its transmit power, and $h_{i,u}\left(t\right)$ the instantaneous channel power gain between UAV $i$ and user $u$ incorporates the adopted large-scale path-loss and shadow-fading components together with small-scale fading, consistent with the channel model specified in Section 5.2. The SINR of user associated with UAV is expressed as

$${\gamma }_{i,u}\left(t\right)={\displaystyle \frac{P_i{h}_{i,u}\left(t\right)}{{\displaystyle \sum _{\begin{array}{l}j=1\\ j\ne i\end{array}}^{N}1\left[a_j\left(t\right)=a_i\left(t\right)\right]P_j{h}_{j,u}\left(t\right)+{\sigma }^2}}}$$

(2)

where $1\left[\cdot \right]$ is the indicator function, which equals one when UAV $j$ selects the same channel as that serving UAV $i$ and zero otherwise, and ${\sigma }^2$ denotes the receiver noise power. Thus, only UAVs simultaneously using the same channel contribute to the co-channel interference term.

$$R_{i,u}\left(t\right)=x_{i,u}\left(t\right)B_c{\log }_2\left(1+{\gamma }_{i,u}\left(t\right)\right)$$

(3)

$$R_{total}\left(t\right)={\displaystyle \sum _{i=1}^N{\displaystyle \sum _{u\in u_i\left(t\right)}{R}_{i,u}\left(t\right).}}$$

(4)

where $N$ denotes the number of UAVs, $u_i\left(t\right)$ represents the set of ground users served by UAV $i$ at decision epoch $t$, $B_c$ is the bandwidth of the channel allocated to the corresponding link, and ${\gamma }_{i,u}\left(t\right)$ represents the instantaneous SINR between UAV $i$ and its associated user $u$. This formulation directly couples spectrum allocation with communication performance because simultaneous use of the same channel by neighboring UAVs may increase co-channel interference, thereby reducing the SINR and achievable throughput of the affected links.

3.3. Multi-Agent Interaction Model

The proposed system considers each UAV as an independent learning agent that interacts with the wireless environment and learns its spectrum allocation strategy. Therefore, the multi-UAV communication system can be modeled as a multi-agent decision process, where multiple agents simultaneously learn how to optimize their actions by receiving feedback from the environment. All the UAVs observe the current network state at every decision based on information including channel conditions, the interference level and previously selected channels. Considering this, the UAV chooses an action with respect to a given spectrum channel. Thereafter, the environment is updated according to the integrated actions of all UAV agents, and each agent obtains a reward indicating the communication performance under selected spectrum configuration. As an action of one UAV may affect another UAV’s communication performance through jamming effects, the learning environment becomes interactive and dynamic. This necessitates the need for cooperative learning mechanisms that allow UAV agents to collaboratively learn spectrum allocation strategies in a way that improves the global efficiency of network services.

The objective of the proposed system is to optimize spectrum allocation among multiple UAV agents to maximize network communication performance while minimizing interference among UAV transmissions. Consider that the performance of the network can be measured, e.g., by spectrum utilization efficiency and communication reliability. Hence, the optimization problem can be given as maximizing the cumulative network reward that UAV agents achieve by cooperatively deciding spectrum allocation. Each UAV agent learns a spectrum-selection policy that maximizes the expected long-term cumulative reward over successive decision epochs. Future rewards are discounted by the factor $\gamma \in \left[0,1\right]$, allowing the learned policy to balance immediate spectrum-allocation performance against its long-term effect on network interference and throughput.

Due to the dynamic and multi-agent nature of the network environment, solving this optimization problem using conventional analytical approaches becomes extremely challenging. The limitations of conventional spectrum allocation approaches and the challenges associated with distributed decision-making highlight the need for intelligent and adaptive resource management frameworks. In particular, multi-agent deep reinforcement learning provides a powerful solution for enabling UAV agents to learn cooperative spectrum allocation strategies in dynamic wireless environments. By integrating deep learning with reinforcement learning, UAV agents can effectively process high-dimensional network state information and learn optimal decision policies through experience. This approach enables scalable and distributed spectrum management without relying on centralized control mechanisms. Therefore, this study adopts a cooperative multi-agent deep reinforcement learning framework to enable UAV agents to autonomously learn efficient spectrum allocation policies through repeated interactions with the wireless environment.

3.4. Multi-Agent Reinforcement Learning State–Action–Reward Formulation

To enable cooperative spectrum allocation in the multi-UAV communication network, the system is formulated as a multi-agent reinforcement learning (MARL) problem. In this framework, each UAV operates as an autonomous learning agent that interacts with the wireless environment and learns an optimal spectrum selection policy through repeated observations and feedback.

Let $N$ denote the number of UAV agents in the network and $C$ represent the number of available spectrum channels.

At each discrete time step $t$, every UAV agent observes the network state, selects a spectrum channel as its action, and receives a reward based on the resulting communication performance.

3.4.1. State Space

The state representation is designed to provide each UAV agent with sufficient information to characterize its local communication, mobility, and traffic conditions while preserving decentralized execution with limited inter-agent signaling. At decision epoch $t$, the local state observed by UAV agent $i$ is defined as

$$s_i\left(t\right)=\left[I_i\left(t\right),\hspace{0.33em}{q}_i\left(t\right),\hspace{0.33em}{a}_i\left(t-1\right),p_i\left(t\right),{\rho }_i\left(t\right)\right]$$

(5)

where $I_i\left(t\right)$ denotes the locally observed interference level, $q_i\left(t\right)$ represents the channel-quality indicators of the available spectrum channels, $a_i\left(t-1\right)$ is the channel selected by UAV at the previous decision epoch, $p_i\left(t\right)=\left[x_i\left(t\right),y_i\left(t\right),z_i\left(t\right)\right]$ denotes the current three-dimensional UAV position, and ${\rho }_i\left(t\right)$ represents a compact descriptor of the associated-user distribution or traffic load within the coverage region of UAV $i$. In this study, ${\rho }_i\left(t\right)$ is compactly represented by the number of users currently associated with the UAV, together with the centroid of their horizontal positions. This representation captures both the instantaneous service load and the spatial distribution of the served users without introducing the individual coordinates of all users into the learning state. The associated-user information is dynamically updated according to the user–UAV association mechanism defined in Section 3.2, thereby allowing the state representation to reflect changes in user distribution caused by both UAV and ground-user mobility.

The UAV position is included because changes in three-dimensional geometry directly affect coverage overlap, propagation conditions, and interference coupling with neighboring UAVs. Similarly, the associated-user descriptor provides information on the instantaneous service load and spatial distribution of the users served by each UAV. These variables complement the channel-quality and interference measurements and provide a more informative representation of the dynamic aerial communication environment. The inclusion of mobility- and user-related information is also consistent with previous UAV-swarm MARL studies demonstrating that network topology, time-varying channel states, and inter-agent information can significantly affect cooperative communication decisions [29]. The instantaneous actions of neighboring UAV agents are deliberately excluded from the local execution state because requiring such information during deployment would introduce continuous inter-UAV signaling and reduce the scalability of decentralized execution. Instead, their effects are captured indirectly through locally measured interference and are subsequently characterized spatially through the Hybrid DeepMUSIC-based DoA information introduced in Section 3.5. During centralized training, joint experiences and network-wide information are available to the trainer under the CTDE paradigm described in Section 4.5, enabling inter-agent dependencies to be learned without requiring explicit neighboring-agent actions during decentralized execution. Accordingly, the local state defined in this subsection serves as the baseline communication, mobility, and user-distribution representation and is subsequently augmented with directional interference information and localization confidence obtained from the Hybrid DeepMUSIC module.

3.4.2. Action Space

The action of each UAV agent corresponds to the selection of a spectrum channel for data transmission from the available channel set. At each decision epoch $t$, UAV $i$ selects one channel according to its current local state and learned spectrum-selection policy. The simultaneous channel selections of all UAV agents collectively constitute the joint spectrum-allocation decision of the multi-UAV network. Since neighboring UAVs operate autonomously, two or more UAVs may select the same channel, potentially resulting in co-channel interference within overlapping coverage regions. The effect of these simultaneous channel-selection decisions is subsequently reflected in the observed SINR, achievable throughput, and cooperative reward, thereby enabling the agents to learn interference-aware spectrum-allocation policies through repeated interaction with the network environment.

3.4.3. Reward Function

To align individual spectrum decisions with the overall communication objective, the reward function is designed to jointly account for useful data transmission, co-channel interference, and the reliability of the spatial interference estimate. The instantaneous reward of UAV agent $i$ is defined as

$$R_i\left(t\right)=\alpha T_i\left(t\right)-\beta I_i\left(t\right)+\delta L_i\left(t\right)$$

(6)

where $T_i\left(t\right)$ is achieved communication throughput, $I_i\left(t\right)$ is experienced interference level, $\alpha$ is throughput reward weight, $\beta$ is the interference penalty factor, and $\delta \ge 0$ is a weighting coefficient controlling the contribution of localization reliability to the overall reward. $L_i\left(t\right)\in \left[0,1\right]$ denotes the DeepMUSIC localization confidence score of UAV agent $i$ at time $t$. The confidence score quantifies the reliability of the fused DoA estimate ${\theta }_i^{\ast }\left(t\right)$ obtained from the hybrid DeepMUSIC module and is defined as $L_i\left(t\right)=1-{\displaystyle \frac{\left|{\theta }_{MUSIC,i}\left(t\right)-{\theta }_{DL,i}\left(t\right)\right|}{{\theta }_{\Delta }}},$ and ${\theta }_{MUSIC,i}\left(t\right)$ and ${\theta }_{DL,i}\left(t\right)$ are the MUSIC-based and deep-learning-based DoA estimates of agent $i$ and ${\theta }_{\Delta }$ is a normalization constant corresponding to the maximum admissible angular deviation between the two estimates. When the two estimates converge, $\left({\theta }_{MUSIC,i}\left(t\right)\approx {\theta }_{DL,i}\left(t\right)\right)$, $L_i\left(t\right)\to 1,$ indicating high confidence in ${\theta }_i^{\ast }\left(t\right);$ conversely, substantial disagreement between the estimates yields $L_i\left(t\right)\to 0$. The confidence term serves as an auxiliary reliability signal rather than an independent spectrum-selection objective. At decision epoch $t$, $c_i\left(t\right)$ is computed from the agreement between the MUSIC-based and DeepMUSIC-based DoA estimates obtained from the interference observations available before action $a_i\left(t\right)$ is executed. It therefore modulates the learning signal according to the reliability of the spatial interference information supplied to the agent. A high confidence score reinforces decisions made under reliable DoA information, whereas a low confidence score reduces the contribution of localization reliability to the reward. The relatively small weighting coefficient ${\lambda }_c$ ensures that this auxiliary term does not dominate the primary throughput-maximization and interference-mitigation objectives. Overall, this reward structure encourages the UAV agents to maximize communication throughput while mitigating co-channel interference and accounting for the reliability of the available spatial interference information. The global reward used for cooperative learning is then expressed as

$$R\left(t\right)={\displaystyle \sum _{i=1}^N{R}_i\left(t\right)}$$

(7)

which encourages agents to optimize the overall network performance rather than acting selfishly.

3.4.4. Policy Objective

The objective of each UAV agent is to learn a spectrum-selection policy that maximizes the expected long-term cumulative reward defined by the reward mechanism in Section 3.4.3. Future rewards are discounted by a factor $\gamma \in \left[0,1\right]$, enabling the learned policy to balance immediate spectrum-allocation performance with the long-term consequences of channel-selection decisions on network throughput and co-channel interference. Through repeated interactions with the wireless environment and cooperative feedback, the UAV agents progressively learn coordinated spectrum-allocation policies that improve network-wide communication performance while mitigating interference.

3.5. Hybrid DeepMUSIC-Based Interference Localization

In multi-UAV wireless communication networks, accurate localization of interference sources plays a crucial role in improving spectrum allocation efficiency and mitigating co-channel interference. Existing algorithms for interference estimation based either on signal strength measurement or statistical channel models cannot provide the detailed spatial resolution required in complicated wireless environments. To overcome this limitation, a hybrid DeepMUSIC-based interference localization module for the proposed system model is integrated in this study. The proposed hybrid approach combines the classical MUltiple SIgnal Classification (MUSIC) algorithm with deep learning techniques to enhance the accuracy and robustness of direction-of-arrival (DoA) estimation for interference signals. This integration enables UAV agents to obtain high-resolution spatial information about interfering sources, which can be used to improve spectrum allocation decisions in the multi-agent learning framework.

3.5.1. Signal Model for Interference Localization

Consider a UAV equipped with an $M$-element antenna array receiving signals from $L$ co-channel interference sources. To account for the propagation effects considered in the network model, the received array signal at snapshot $t$ is expressed as [43,44]

$$x\left(t\right)={\displaystyle {\displaystyle \sum _{l=1}^L}{\alpha }_l\left(t\right)}A\left({\theta }_l\right)s_l\left(t\right)+n\left(t\right),$$

(8)

where $x\left(t\right)\in {ℂ}^{M\times 1}$ represents the received signal vector, $A\left({\theta }_l\right)$ is a steering matrix corresponding to DoA, ${\theta }_l$ $s_l\left(t\right)$ represents its transmitted interference signal, and $n\left(t\right)$ is the additive noise. The complex coefficient ${\alpha }_l\left(t\right)$ represents the composite propagation gain of the interference path and incorporates the effects of distance-dependent path loss, log-normal shadow fading, and small-scale fading. In the simulations, the large-scale component of ${\alpha }_l\left(t\right)$ follows the 3GPP-compliant path-loss and shadow-fading assumptions specified in Section 5.2, whereas the small-scale component is modeled according to the adopted Rayleigh fading process. Consequently, UAV mobility changes the large-scale received interference strength through the time-varying propagation geometry, while small-scale fading introduces instantaneous amplitude and phase fluctuations across the received snapshots.

The covariance matrix of the received signal is given by

$$R_x=E\left[x\left(t\right)x^H\left(t\right)\right]$$

(9)

where $x^H\left(t\right)$ is the Hermitian transpose of the received signal vector. Since $x\left(t\right)$ already contains the composite propagation coefficients ${\alpha }_l\left(t\right)$, the covariance matrix inherently captures the power variations and phase perturbations introduced by both large-scale and small-scale fading. This covariance representation is subsequently used by the classical MUSIC estimator and the DeepMUSIC network, ensuring that both localization components operate on the same faded interference observations rather than on an idealized unfaded signal model.

3.5.2. Classical MUSIC Algorithm

The MUSIC algorithm estimates the direction of arrival of signals by exploiting the orthogonality between the signal subspace and noise subspace. The pseudo-spectrum is computed as [41]:

$$P_{MUSIC}\left(\theta \right)={\displaystyle \frac{1}{a^H\left(\theta \right)E_n{E}_n^{H}a\left(\theta \right)}}$$

(10)

where $E_n$ and $E_n^H$ denote the noise subspace eigenvectors and their Hermitian transpose, respectively. Also, $a\left(\theta \right)$ is a steering vector as follows:

$$a\left(\theta \right)={\left[1 e^{-jkd\sin \theta } e^{-j2kd\sin \theta } \cdots e^{-j\left(M-1\right)kd\sin \theta }\right]}^T$$

(11)

where $M$ is defined as the number of antenna elements, $d$ corresponds to the inter-element spacing, $k={\displaystyle \frac{2\pi }{\lambda }}$ denotes the wave number and $\lambda$ represents the free-space wavelength. The peaks of $P_{MUSIC}\left(\theta \right)$ correspond to the estimated directions of interference sources. Although MUSIC provides high-resolution DoA estimation, its performance may degrade in low signal-to-noise ratio (SNR) conditions or when the number of snapshots is limited.

3.5.3. Deep Learning Enhancement

In parallel with the classical MUSIC estimator, the DeepMUSIC network receives the covariance-based representation as input and learns a nonlinear mapping to the DoA of the dominant interference sources. The network is trained using labeled covariance–DoA pairs, enabling it to compensate for estimation degradation that may occur under noisy or snapshot-limited conditions [42].

3.5.4. Hybrid DeepMUSIC Framework

The proposed hybrid framework is designed to complementarily integrate the complementary strengths of both the MUSIC algorithm and deep learning approaches. In this framework, MUSIC first provides an initial high-resolution spatial spectrum, which serves as a structured and physically interpretable input for subsequent processing. A deep learning model then refines the DoA estimation by capturing nonlinear patterns and complex dependencies that conventional spectral methods are unable to resolve. The outputs from both components are subsequently fused to yield a robust and accurate interference localization result. The final DoA estimate is accordingly obtained as

$${\theta }^{\ast }=\lambda {\theta }_{MUSIC}+\left(1-\lambda \right){\theta }_{DL}$$

(12)

where $\lambda \in \left[0, 1\right]$ is defined the weighting factor and ${\theta }_{DL}$ represents the deep learning estimate. This hybrid approach improves robustness against noise and enhances localization accuracy in dynamic environments.

3.5.5. Integration with MARL Framework

The key integration between interference localization and cooperative spectrum allocation occurs at the state-representation level. Specifically, the conventional communication state is augmented with the fused DoA estimate and its associated localization confidence, yielding the spatially enhanced state

$${\tilde{s}}_i\left(t\right)=\left[s_i\left(t\right),{\mathsf{\Theta }}_i\left(t\right),c_i\left(t\right)\right],$$

(13)

where ${\mathsf{\Theta }}_i\left(t\right)$ is estimated interference directions.

Accordingly, the enhanced state combines communication conditions, mobility and user-distribution information, historical spectrum decisions, and spatial interference awareness within a unified representation for decentralized spectrum selection. The interaction between interference localization and spectrum selection follows a sequential decision process. At decision epoch $t$, the Hybrid DeepMUSIC module first estimates the dominant interference directions and the corresponding localization confidence from the currently observed interference signals. These quantities augment the local state and are supplied to the MADRL agent before channel selection. The agent then selects $a_i\left(t\right)$, after which the action is executed in a wireless environment. Consequently, the current action does not directly alter the DoA estimate or confidence score used for that same decision. Instead, the selected spectrum allocation changes the resulting co-channel interference pattern and received observations, which are used to update the localization estimate and confidence at the subsequent decision epoch $t+1$. Thus, the framework implements a closed-loop interaction in which localization informs the current spectrum decision, while spectrum decisions can indirectly influence future localization conditions through changes in the interference environment. Importantly, this architecture differs from a simple cascade of a DeepMUSIC localization module followed by an independent MADRL spectrum allocator. In a cascaded design, the localization output would function merely as an external preprocessing result without explicitly influencing the internal learning objective or subsequent localization conditions. In the proposed framework, however, spatial interference information is coupled to spectrum decision making through three complementary mechanisms: the fused DoA and localization-confidence information are embedded directly into the agent state, localization confidence contributes an auxiliary reliability signal during policy learning, and the executed joint spectrum decisions modify the interference observations from which subsequent DoA estimates are obtained. In addition, these spatially enriched experiences are incorporated into cooperative policy optimization under CTDE. The proposed framework therefore establishes a closed-loop, spatially aware cooperative decision process rather than a one-way concatenation of localization and spectrum-allocation modules.

4. Cooperative Multi-Agent Deep Reinforcement Learning Framework

Building upon the system model and the Hybrid DeepMUSIC-based interference localization mechanism presented in Section 3, this section proposes a Cooperative Multi-Agent Deep Reinforcement Learning (MADRL) framework for dynamic spectrum allocation in UAV-assisted wireless networks.

Conventional spectrum allocation approaches rely primarily on interference power measurements or local observations, limiting their ability to respond proactively to spatially distributed interference. To overcome this limitation, the proposed framework integrates high-resolution directional interference information derived from the Hybrid DeepMUSIC model into the agents’ decision-making process. Specifically, estimated directions of arrival (DoAs) are incorporated into a DeepMUSIC-enhanced state representation, enabling individual UAV agents to identify interference-prone channels and coordinate spectrum access decisions with greater spatial awareness.

To achieve efficient and scalable operation, the framework adopts a cooperative multi-agent learning paradigm in which multiple UAV agents jointly optimize spectrum allocation policies while retaining fully distributed execution capabilities. This design supports adaptive and intelligent spectrum management in dynamic, interference-prone environments without requiring centralized control during deployment.

4.1. Framework Overview

The overall architecture of the proposed DeepMUSIC-assisted cooperative multi-agent deep reinforcement learning (MADRL) framework is illustrated in Figure 2. The framework consists of six major components, namely signal acquisition, hybrid DeepMUSIC-based interference localization, state construction, cooperative multi-agent learning, wireless network environment, and feedback-driven policy optimization. The objective of the proposed framework is to jointly perform interference localization, spectrum allocation, and interference management in a distributed multi-UAV communication environment.

Figure 2. Architecture of the proposed Hybrid DeepMUSIC-assisted cooperative MADRL framework. Hybrid DeepMUSIC first performs high-resolution interference localization and DoA estimation from the received signals. The extracted spatial interference information is then integrated into the MARL state space, allowing UAV agents to make interference-aware spectrum allocation decisions. Using a centralized training and decentralized execution (CTDE) paradigm, the cooperative learning framework enables distributed spectrum management, interference mitigation, and adaptive resource allocation, thereby improving throughput, spectrum efficiency, and communication reliability in multi-UAV 6G networks. Ellipses (…) indicate omitted repeated elements. Colors distinguish functional modules, while line styles and symbols represent different communication links, interference sources, and ground users (generated using AI-assisted visualization tools and modified by the authors).

Initially, each UAV is equipped with an antenna array that continuously receives wireless signals from ground users and surrounding interference sources, as shown in Block A. The received signal samples are collected and used to estimate the covariance matrix of the received signals. This covariance matrix serves as the primary input to the interference localization module. Subsequently, Block B performs Hybrid DeepMUSIC-Based Interference Localization. First, the classical MUSIC algorithm computes the spatial pseudo-spectrum through eigenvalue decomposition of the covariance matrix and extracts the corresponding noise subspace. The resulting pseudo-spectrum provides high-resolution estimates of the direction-of-arrival (DoA) of interference signals. However, since the performance of classical MUSIC may deteriorate under low signal-to-noise ratio (SNR) conditions or limited observation snapshots, a DeepMUSIC enhancement module is incorporated to learn nonlinear spatial features directly from the covariance matrix. The outputs of the classical MUSIC estimator and the deep learning model are then combined through a hybrid fusion mechanism to generate a robust estimate of the interference direction set ${\mathsf{\Theta }}_i\left(t\right)$.

The interference in localization information is subsequently integrated into the state construction module (Block C). Unlike conventional spectrum allocation frameworks that rely solely on channel state information and interference power measurements, the proposed framework augments the state vector with spatial interference awareness obtained from DeepMUSIC. Consequently, the state observed by UAV agent $i$ can be represented as Equation (16). Block D illustrates the cooperative multi-agent deep reinforcement learning module. Each UAV operates as an independent learning agent that receives the constructed state vector and selects a spectrum allocation action. The agents are trained under a cooperative learning paradigm in which network-wide performance is optimized through information sharing and global reward coordination. This design enables decentralized execution while maintaining cooperative behavior among UAVs. The selected spectrum allocation actions are applied to the multi-UAV 6G network environment shown in Block E. The environment includes multiple UAV base stations, ground users, wireless communication links, and interference sources. Based on the collective actions of all UAV agents, the environment generates communication outcomes such as throughput, spectrum utilization efficiency, fairness, and interference levels.

Finally, Block F performs feedback and policy optimization. The observed rewards and state transitions are stored in replay buffers and used to update the neural network parameters through mini-batch learning. The reward function jointly considers throughput maximization, interference reduction, spectrum efficiency, and network fairness. Through continuous interactions with the environment, UAV agents gradually learn cooperative spectrum allocation policies that maximize long-term network performance.

Overall, the proposed Hybrid DeepMUSIC-assisted MADRL framework establishes a closed-loop architecture that tightly couples spatial interference localization with cooperative spectrum allocation. Unlike conventional MARL schemes that merely append interference-related measurements to the agent state, the proposed framework combines: (i) MUSIC–DeepMUSIC fusion for spatial interference localization; (ii) reliability-aware state construction using fused DoA and localization confidence; (iii) localization reliability as an auxiliary signal for cooperative policy learning; and (iv) temporal feedback whereby current spectrum decisions modify the interference observations used for localization at the subsequent decision epoch. These mechanisms are integrated within CTDE, enabling spatially aware cooperative learning while preserving decentralized execution based on locally available information.

4.2. Cooperative Learning Architecture

The proposed framework models the UAV network as a cooperative multi-agent system in which each UAV operates as an autonomous decision-making agent. Since the actions of one UAV directly affect the communication performance of its neighbors through co-channel interference, isolated decision-making leads to suboptimal spectrum utilization. Cooperative learning is therefore introduced to encourage coordinated behavior and improve overall network performance. The cooperative spectrum-allocation process is modeled as a Markov game characterized by the UAV agents, state and action spaces, state-transition dynamics, cooperative reward, and discount factor. A key characteristic of the proposed architecture is the use of a shared network objective. Rather than maximizing individual throughput alone, each agent optimizes collective network performance through the global reward mechanism defined in Section 3.4, where rewards reflect both local throughput and interference imposed on neighboring UAVs. This design encourages agents to balance local requirements with global network efficiency.

The cooperative learning process follows four sequential stages: (1) each UAV collects local observations, including channel quality indicators, interference measurements, previous channel selections, and DoA estimates from the Hybrid DeepMUSIC module; (2) the observations are transformed into a DeepMUSIC-enhanced state representation capturing both spectral and spatial characteristics; (3) each UAV selects a spectrum channel based on its learned policy; and (4) the environment returns a cooperative reward and transitions to the next state according to the joint actions of all agents. Through repeated interactions, agents gradually converge toward coordinated allocation strategies that maximize long-term network reward.

To address the non-stationarity problem inherent in multi-agent environments-where simultaneous policy updates cause the environment to appear non-stationary from each agent’s perspective-agents are trained using shared environmental feedback and cooperative reward signals. This strategy promotes complementary rather than competitive behavior, thereby improving learning stability and convergence.

The architecture further integrates spatial interference awareness through Hybrid DeepMUSIC-based DoA information, enabling agents to proactively identify dominant interference sources and avoid interference-prone channels. This spatial enrichment significantly improves decision quality beyond what interference power measurements alone can provide. During deployment, each UAV executes spectrum allocation decisions independently using locally observed information, eliminating the need for continuous centralized coordination and reducing communication overhead-a critical property for scalable UAV-assisted 6G network operation.

Figure 3 illustrates the overall operational flow of the proposed Cooperative MADRL framework. The process begins with the Hybrid DeepMUSIC module, which estimates the directions of arrival (DoAs) of dominant interference sources from received array signals. The resulting interference localization output is subsequently used to construct a DeepMUSIC-enhanced state representation that encodes both spectral and spatial characteristics of the wireless environment. Based on this enriched state, each UAV agent independently selects a spectrum channel according to its local policy and simultaneously interacts with the shared wireless environment. The collective actions of all UAV agents produce a global cooperative reward signal that reflects both individual throughput and co-channel interference imposed on neighboring agents. This reward is then used to update each agent’s policy through deep reinforcement learning optimization. The process repeats iteratively, allowing agents to progressively refine their spectrum allocation strategies toward coordinated and interference-aware decisions.

Figure 3. Operational flowchart of the proposed Hybrid DeepMUSIC-assisted Cooperative Multi-Agent Deep Reinforcement Learning (MADRL) framework for dynamic spectrum allocation in UAV-assisted wireless networks. The framework integrates DoA-based interference localization with cooperative multi-agent learning, where each UAV agent independently executes spectrum allocation decisions based on a spatially enriched state representation while collectively optimizing a global cooperative reward through iterative policy updates.

4.3. DeepMUSIC-Enhanced State Representation in the Learning Pipeline

The enhanced state representation ${\tilde{s}}_i\left(t\right)=\left\{I_i\left(t\right),\hspace{0.33em}{H}_i\left(t\right),\hspace{0.33em}{C}_i\left(t\right),{\mathsf{\Theta }}_i\left(t\right)\right\}$, formulated in Section 3.5.5, serves as the direct input to the deep reinforcement learning network of each UAV agent within the proposed cooperative MADRL framework. While the conventional components $I_i\left(t\right),\hspace{0.33em}{H}_i\left(t\right),$ and $C_i\left(t\right)$ capture the instantaneous spectral condition observed locally, the additional term ${\mathsf{\Theta }}_i\left(t\right).$—the set of $K$ estimated DoAs obtained from the Hybrid DeepMUSIC module—embeds spatial awareness directly into the learning process. From an architectural standpoint, this integration has two practical implications for the network design described in Section 4.4. First, the input dimensionality of each agent’s policy network increases from $3$ to $3+K,$ requiring the input layer to be sized accordingly to accommodate the DoA vector alongside the spectral features. Second, because ${\mathsf{\Theta }}_i\left(t\right).$ and the spectral terms $\left\{I_i\left(t\right),\hspace{0.33em}{H}_i\left(t\right),\hspace{0.33em}{C}_i\left(t\right)\right\}$ differ substantially in scale and physical units (angular degrees versus power and rate metrics), both components are normalized independently prior to concatenation, ensuring that spatial information contributes proportionally during gradient-based policy updates.

This design allows each UAV agent to map directional interference patterns to specific channel-selection outcomes during training, rather than treating ${\mathsf{\Theta }}_i\left(t\right).$ as auxiliary metadata. Consequently, the policy network is able to associate recurring DoA patterns with channels that are likely to experience persistent co-channel interference, enabling the proactive channel-avoidance behavior discussed in Section 4.2.

4.4. Deep Reinforcement Learning Network Design

Effectively exploiting the high-dimensional, DeepMUSIC-enhanced state representation requires a learning mechanism capable of capturing complex nonlinear relationships between network conditions and spectrum allocation decisions. To this end, each UAV agent is equipped with a Deep Q-Network (DQN) that approximates the action-value function

$$Q\left(s, a; \theta \right)=\mathrm{E}\left[{\displaystyle \sum _{k=0}^{\infty }\left.{\gamma }^k{r}_{t+k}\right|s_t=s, a_t=a}\right],$$

(14)

where $\theta$ denotes the trainable network parameters, $r_t$ the immediate reward, and $\gamma$ the discount factor. Unlike tabular reinforcement learning, the DQN allows agents to generalize across similar network conditions, which is essential as the state space grows with the number of agents, channels, and interference sources.

Network architecture: the input layer receives the enhanced state vector ${\tilde{s}}_i\left(t\right)$ defined in Section 4.3, integrating spectral information $\left(I_i\left(t\right),\hspace{0.33em}{H}_i\left(t\right),\hspace{0.33em}{C}_i\left(t\right)\right)$ with the DoA vector ${\mathsf{\Theta }}_i\left(t\right).$ The hidden layers consist of multiple fully connected layers with rectified linear unit (ReLU) activations, where the output of the $l$-th layer is given by

$$h^{\left(l\right)}=\sigma \left(w^{\left(l\right)}{h}^{\left(l-1\right)}+b^{\left(l\right)}\right),$$

(15)

with $w^{\left(l\right)}$ and $b^{\left(l\right)}$ denoting the layer weights and biases. These layers extract latent features capturing the nonlinear interactions among channel quality, interference level, and spatial interference distribution. The output layer contains $C$ neurons, each estimating the Q-value of a candidate channel; the selected action is

$$a_i^{\ast }=\arg \underset{a\in C}{\max } Q\left(s_i, a; \theta \right).$$

(16)

Exploration strategy: an $\epsilon$-greedy policy balances exploration and exploitation: with probability $\epsilon ,$ the agent selects a random channel to explore alternative allocation strategies, and otherwise selects $a_i^{\ast }.$ This prevents premature convergence to locally optimal policies.

Training stability: A target network with parameters ${\theta }^{-}$ is used to compute the target Q-value

$$y_t=r_t+\gamma \underset{a^{\prime }}{\max } Q\left(s_{t+1}, a^{\prime }; {\theta }^{-}\right),$$

(17)

and the online network parameters $\theta$ are updated by minimizing the temporal-difference loss

$$L\left(\theta \right)=\mathrm{E}\left[{\left(y_t-Q\left(s_t, a_t; \theta \right)\right)}^2\right] .$$

(18)

Periodically synchronizing ${\theta }^{-}$ with $\theta$ reduces oscillations and improves convergence stability during training. By embedding the DoA vector ${\mathsf{\Theta }}_i\left(t\right).$ directly into the input layer, the network learns to associate recurring spatial interference patterns with their corresponding channel allocation outcomes during the feature extraction process. This allows the resulting policy to anticipate interference-prone channels before degradation occurs, providing the architectural basis for the proactive channel-avoidance behavior discussed in Section 4.2.

4.5. Centralized Training and Decentralized Execution

To mitigate the non-stationarity problem discussed in Section 4.2, the proposed framework adopts the Centralized Training and Decentralized Execution (CTDE) paradigm, which combines the convergence benefits of centralized learning with the scalability of distributed decision-making. In the decentralized execution, each UAV agent independently selects a spectrum channel based only on its locally available enhanced state, without requiring access to the complete global network state. Neighboring-agent actions and global network information are available only during centralized training and are not required during decentralized execution; their local effects are reflected through interference measurements and Hybrid DeepMUSIC-derived spatial interference features. This design reduces inter-UAV signaling overhead during deployment while preserving the cooperative behavior learned through centralized training.

In the centralized training phase, a training server aggregates experiences from all UAV agents—including joint actions, network-wide throughput, and DoA estimates from multiple UAVs—to construct a global interference map and update the agents’ policies. Let the global state and joint action be defined as using this global information; the centralized trainer optimizes the joint policy to maximize the expected cumulative global reward $\mathrm{E}\left[{\displaystyle \sum _{t=0}^{\infty }{\gamma }^{t}R\left(t\right)}\right] ,$ where $R\left(t\right)$ is the cooperative reward defined in Equation (4). Because the trainer observes the joint behavior of all agents, it can model environment dynamics more accurately than a fully decentralized learner, resulting in more stable policy updates and faster convergence.

After training, the learned policies $Q\left(s_i, a ; \theta \right)$ (Section 4.4) are deployed independently to each UAV, which selects its spectrum allocation action using only its local enhanced state—without requiring continuous communication with the training server. This decentralized execution offers three practical advantages for large-scale UAV networks: reduced signaling overhead, since agents do not exchange complete state information during operation; improved scalability, as new agents can join without increasing coordination complexity; and enhanced robustness, by eliminating the single point of failure inherent to centralized control.

The CTDE paradigm is particularly compatible with the Hybrid DeepMUSIC module: during training, DoA information from multiple UAVs is aggregated into a network-wide spatial interference map that improves policy learning, while during execution each agent relies only on its locally estimated ${\mathsf{\Theta }}_i\left(t\right),$ preserving operational independence while retaining the benefits of centralized training.

4.6. Experience Replay and Policy Update

In cooperative multi-agent environments, sequentially collected experiences are strongly temporally correlated, which degrades gradient-based optimization and may prevent convergence. To address this, the proposed framework incorporates an experience replay mechanism in which each UAV agent $i$ stores interaction tuples

$$e_i\left(t\right)=\left(s_i\left(t\right),a_i\left(t\right),r_i\left(t\right),s_i\left(t+1\right)\right)$$

(19)

into a shared replay buffer $D=\left\{e_1,e_2,\dots ,e_M\right\}$ of capacity $M,$ where $s_i\left(t\right)$ is the enhanced state (Section 4.3), $a_i\left(t\right)$ the selected channel, $r_i\left(t\right)$ the reward from Equation (6), and $s_i\left(t+1\right)$ the subsequent state. At each learning iteration, a mini-batch $B=\left\{e^{\left(1\right)},e^{\left(2\right)},\dots ,e^{\left(B\right)}\right\}$ of size $B$ is drawn uniformly at random from $D.$ This random sampling breaks temporal correlations among consecutive observations, reducing gradient variance and improving convergence stability.

The DQN parameters $\theta$ are updated by minimizing the mean TD loss over the mini-batch

$$L\left(\theta \right)={\displaystyle \frac{1}{B}}{\displaystyle \sum _{j=1}^B{\left(y_j-Q\left(s_j,a_j;\theta \right)\right)}^2,}$$

(20)

where $y_j=r_j+\gamma \underset{a^{\prime }}{\max }Q\left(s_{j+1},a^{\prime };{\theta }^{-}\right)$ is the target Q-value computed using the target network ${\theta }^{-}$ introduced in Section 4.4. The gradient step follows $\theta \leftarrow \theta -\eta {\nabla }_{\theta }L\left(\theta \right),$ with learning rate $\eta$, using the Adam optimizer. The target network is synchronized every $C$ iterations $\left({\theta }^{-}\leftarrow \theta \right),$ preventing rapid fluctuations in target Q-values that would otherwise destabilize multi-agent training. To maintain the $\epsilon$-greedy exploration schedule introduced in Section 4.4, $\epsilon$ decays exponentially as

$$\epsilon =\max \left({\epsilon }_{\min },{\epsilon }_0{e}^{-kt}\right),$$

(21)

where ${\epsilon }_0$ is the initial exploration rate, ${\epsilon }_{\min }$ the minimum floor, and $k$ the decay constant. This schedule ensures broad exploration of spectrum allocation strategies early in training while progressively concentrating on exploitation as the policy matures. Because each experience tuple contains the full enhanced state ${\tilde{s}}_i\left(t\right)$—including the DoA vector ${\mathsf{\Theta }}_i\left(t\right)$ from the Hybrid DeepMUSIC module—the replay buffer preserves directional interference patterns across diverse network conditions encountered during centralized training (Section 4.5). This enriched experience dataset strengthens the correlation between interference localization and channel-selection outcomes learned by the DQN, improving both policy quality and generalization to unseen interference scenarios.

4.7. Proposed Hybrid DeepMUSIC-Assisted Cooperative MADRL Algorithm

To consolidate the individual components described above into a reproducible procedure, Algorithm 1 summarizes the complete training and execution process of the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework. The algorithm establishes a closed-loop relationship between spatial interference localization and cooperative spectrum decision making. At each decision epoch, received array signals are first processed by the Hybrid DeepMUSIC module to estimate the directions of dominant interference sources. These estimates are incorporated into the local state of each UAV, after which decentralized DQN agents select spectrum channels. The resulting joint allocation modifies the interference environment and generates local and global rewards. During centralized training, the resulting transitions are stored in the replay buffer and used to update the shared policy, whereas during deployment only the state-construction and decentralized action-selection stages are required.

Algorithm 1. Hybrid DeepMUSIC-Assisted Cooperative MADRL for Interference-Aware Spectrum Allocation

Input: Number of UAVs $N$; available channels $K$; trained/initialized DeepMUSIC parameters; DQN parameters $\theta$; target-network parameters ${\theta }^{-}$; replay buffer $D$. Output: Cooperative spectrum-allocation policy ${\pi }^{\ast }$. Initialize UAV network environment, DQN parameters $\theta$, target network ${\theta }^{-}$, and replay buffer $D$. For each training episode do Reset UAV and user positions and channel conditions and update the user–UAV associations according to the dynamic large-scale channel-gain criterion defined in Section 3.2. For each decision epoch $t$ do For each UAV agent $i$: Acquire received array-signal snapshots. Estimate the covariance matrix of the received signals. Compute the MUSIC spatial spectrum and initial DoA estimate. Obtain the DeepMUSIC DoA estimate. Fuse the MUSIC and DeepMUSIC estimates to obtain ${\widehat{\theta }}_{i,t}$. Measure local channel quality and interference conditions. Construct the DeepMUSIC-enhanced local state $s_{i,t}$. Select channel $a_{i,t}$ using the $\epsilon$-greedy DQN policy. End for Execute the joint channel allocation in the multi-UAV environment. Compute achieved throughput and co-channel interference. Compute individual rewards and the global cooperative reward. Observe the next enhanced states. Store transitions in replay buffer D. Sample a mini-batch from D. Update DQN parameters by minimizing the TD loss. Periodically update target-network parameters. Decay exploration rate $\epsilon$. End for End for Deploy the learned policy to individual UAVs for decentralized spectrum selection using local DeepMUSIC-enhanced states.

To illustrate the operation of the proposed framework, consider a simplified scenario consisting of three UAV agents, $U_1$, $U_2$, and $U_3$, sharing three orthogonal spectrum channels, $C_1$, $C_2$, and $C_3$. Assume that $U_1$ and $U_2$ have partially overlapping coverage areas and initially operate on $C_1$, whereas $U_3$ operates on $C_2$. Consequently, the simultaneous use of $C_1$ by $U_1$ and $U_2$ produces significant co-channel interference in their overlapping service region.

At the next decision epoch, the antenna array of $U_1$ captures the received signal snapshots, from which the Hybrid DeepMUSIC module estimates the dominant interference direction. Suppose that a dominant interferer is detected at approximately $35°$, corresponding to the spatial direction of $U_2$. Unlike a conventional state containing only interference power and channel-quality information, the proposed enhanced state therefore informs $U_1$ that the degradation observed on $C_1$ is associated with a dominant spatial interferer in the direction of $U_2$.

The enhanced state is supplied to the trained DQN policy of $U_1$. Based on the Q-values associated with the three candidate channels, assume that $C_3$ receives the highest action value, while $U_2$ remains on $C_1$ and $U_3$ remains on $C_2$. The resulting joint allocation therefore changes from $\left\{C_1,C_2,C_3\right\}$ to $\left\{C_3,C_1,C_2\right\}$. This decision removes the channel conflict between $U_1$ and $U_2$, thereby reducing co-channel interference and improving the corresponding SINR and achievable throughput as shown in Table 3.

Table 3. Illustrative spectrum-allocation example for three UAV agents.

Stage	UAV 1	UAV 2	UAV 3	Network Effect
Initial allocation	$C_1$	$C_1$	$C_2$	Strong $U_1$ − $U_2$ co-channel interference
Hybrid DeepMUSIC observation	Interferer at $\sim 35°$	-	-	Spatial origin of interference identified
Enhanced-state decision	Select $C_3$	Keep $C_1$	Keep $C_2$	Channel conflict removed
Updated allocation	$C_3$	$C_1$	$C_2$	Lower interference, higher SINR/throughput
Learning feedback	Positive reward	Positive reward	Positive reward	Cooperative policy reinforced

The environment subsequently evaluates the new allocation and generates the reward according to the throughput, interference, and localization-confidence terms. Because the new allocation increases useful throughput while reducing interference, the resulting cooperative reward increases. During training, this transition is stored in the replay buffer and reinforces the association between the observed directional interference pattern and the interference-avoiding channel-selection action. Through repeated interactions with different UAV positions and interference configurations, the cooperative agents learn policies that generalize this behavior beyond the illustrative three-UAV example.

4.8. Computational Complexity Analysis

The overall computational cost of the proposed framework is primarily determined by two stages: (i) Hybrid DeepMUSIC-assisted interference localization and centralized policy training, and (ii) decentralized online spectrum decision making. For an $M$ element antenna array with $T$ signal snapshots, the Hybrid DeepMUSIC module requires covariance-matrix construction with complexity $O\left(M^{2}T\right)$, eigenvalue decomposition of the covariance matrix with complexity $O\left(M^3\right)$, MUSIC pseudo-spectrum evaluation over $G$ candidate directions with complexity $O\left(G{M}^2\right)$, and a forward pass through the DeepMUSIC network with $L$ hidden layers of width $H$, contributing approximately $O\left(L{H}^2\right).$ For the DQN component, a mini-batch update of size $B$, with input dimension $d_s$, hidden layer width $H$ and $C$ candidate output channels, requires approximately $O\left(B\left(d_{s}H+H^2+HC\right)\right).$ Accordingly, the dominant computational complexity of one training update can be summarized as

$$O\left(M^{2}T+M^3+G{M}^2+L{H}^2+NB\left(d_{s}H+H^2+HC\right)\right).$$

(22)

This expression shows that the computationally intensive operations are concentrated mainly in the offline/centralized training stage. In particular, the $O\left(M^3\right)$ eigenvalue decomposition can dominate the interference-localization cost for large antenna arrays, whereas DQN training scales with the mini-batch size and network dimensions. These operations do not need to be repeatedly performed as global optimization during decentralized spectrum selection. Under the CTDE paradigm, centralized information is exploited during training to learn cooperative spectrum-allocation policies, whereas each UAV subsequently executes its learned policy using locally available state information. During online decentralized execution, each of the $N$ UAV agents independently evaluates its policy to select a spectrum channel. The resulting online decision complexity therefore scales as

$$O\left(N\left(C_{HDM}+d_{s}H+H^2+HC\right)\right).$$

(23)

where $C_{HDM}$ denotes the computational cost of the Hybrid DeepMUSIC interference-localization stage required for constructing the spatially enhanced state.

Hence, the online policy-execution cost increases linearly with the number of UAV agents $N$ for a fixed network architecture. This property is particularly important for multi-UAV deployments because adding UAV agents does not require solving a new centralized combinatorial spectrum-allocation problem at every decision epoch. Instead, individual UAVs perform decentralized inference using the learned policy, while the cooperative behavior acquired during centralized training is preserved through the CTDE framework. Consequently, the proposed architecture shifts the major computational burden toward the training stage while maintaining comparatively lightweight and scalable decentralized execution. It should be noted that the overall runtime is influenced not only by the number of UAV agents but also by the antenna-array size, DoA search resolution, DeepMUSIC architecture, DQN dimensions, and mini-batch size. Therefore, the theoretical complexity analysis is complemented by the runtime and scalability experiments in Section 5, where computational performance is evaluated as the number of UAV agents increases. Together, the analytical and empirical results provide a more comprehensive assessment of the computational feasibility of the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework for dense multi-UAV spectrum-allocation scenarios.

5. Performance Evaluation

Having established the system architecture, the Hybrid DeepMUSIC-based interference localization mechanism, and the cooperative MADRL framework in Section 3 and Section 4, this section presents the experimental methodology and configuration adopted to empirically validate the proposed approach. Rather than relying solely on the theoretical complexity analysis of Section 4.8, extensive simulations are conducted under realistic multi-UAV network conditions to assess whether the integration of spatial interference awareness with cooperative deep reinforcement learning translates into measurable gains in spectrum efficiency, interference mitigation, and communication reliability.

The evaluation methodology is organized into six complementary aspects to provide a systematic assessment of the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework. Section 5.1 describes the software and hardware environment used to implement the Hybrid DeepMUSIC module and the DQN-based cooperative agents, together with the Monte Carlo procedure adopted to ensure statistically reliable results. Section 5.2 specifies the multi-UAV network configuration, including UAV deployment, user mobility, wireless channel modeling, and spectrum resource settings, which collectively define the common simulation environment for all evaluated methods. Section 5.3 presents the configuration of the Hybrid DeepMUSIC interference-localization module, whereas Section 5.4 details the training configuration and hyperparameters of the cooperative MADRL framework, including the reward-weighting coefficients governing the trade-off among throughput maximization, interference mitigation, and localization reliability. Section 5.5 describes the baseline algorithms and the rationale for their selection, covering non-learning heuristic and interference-aware methods, single-agent DRL, independent multi-agent learning, and cooperative MARL approaches. Finally, Section 5.6 defines and interprets the performance metrics used to evaluate the framework from four complementary perspectives: communication and spectrum performance, interference-localization accuracy, learning behavior, and computational scalability. The results obtained under this unified evaluation methodology are subsequently presented and discussed in Section 6.

5.1. Simulation Environment

The proposed framework is implemented and evaluated entirely in MATLAB, leveraging its Deep Learning Toolbox and Reinforcement Learning Toolbox to construct the DQN-based agents described in Section 4.4, while the classical MUSIC algorithm and the DeepMUSIC fusion mechanism of Section 3.5 are implemented through custom MATLAB routines to allow for precise control over the covariance matrix estimation, eigenvalue decomposition, and hybrid DoA fusion process. All experiments are executed on a workstation equipped with a high-performance CPU, sufficient system memory, and a dedicated GPU to accelerate the training of the DQN policy networks and the DeepMUSIC neural module. To ensure the statistical reliability of the reported results and to mitigate the effect of stochastic variations arising from random UAV–user placements, channel realizations, and network weight initialization, each configuration is evaluated over multiple independent Monte Carlo runs using distinct random seeds, with performance metrics averaged across runs. Table 4 summarizes the simulation environment.

Table 4. Simulation environment and computational setup.

Parameter	Value
Simulator	MATLAB R2025b
OS	Windows 11 Pro 64-bit
CPU	Intel Core i9-14900K
RAM	64 GB
GPU	NVIDIA RTX 4090 24 GB
DL Toolbox	Deep Learning Toolbox
RL Toolbox	Reinforcement Learning Toolbox
MUSIC implementation	Custom MATLAB
Monte Carlo runs	30
Random seeds	30 different seeds

5.2. Network Parameters

The simulated network follows the multi-UAV system architecture described in Section 3.1, in which multiple UAV base stations provide wireless coverage to ground users distributed within a shared service area, with overlapping coverage regions introducing co-channel interference as illustrated in Figure 1. The UAV, user, wireless channel, and spectrum resource parameters adopted in the simulation are configured to reflect realistic aerial base station deployments consistent with 3GPP-compliant path loss modeling, thereby providing a representative testbed for evaluating spectrum allocation and interference management performance. Table 5 lists the UAV-related configuration, including deployment altitude, mobility, and antenna array parameters required by the interference localization module of Section 3.5.1. Table 6 specifies the user-side mobility model, Table 7 details the wireless channel model used to compute the achievable data rate, and Table 8 presents the spectrum resource configuration adopted in the proposed network model. User–UAV association is dynamically updated at each decision epoch using the large-scale channel-gain criterion defined in Section 3.2, thereby accounting for changes in UAV and user positions while avoiding association fluctuations caused by instantaneous small-scale fading.

Table 5. UAV deployment and antenna array parameters.

Parameter	Value
Number of UAVs	5
UAV altitude	120 m
UAV speed	15–25 m/s
Coverage radius	300 m
Flight area	1000 × 1000 m2
Antenna array	8-element ULA
Carrier frequency	3.5 GHz
Bandwidth	20 MHz
Tx Power	30 dBm

Table 6. Ground user distribution and mobility parameters.

Parameter	Value
Number of users	100
User mobility	Random Waypoint
Speed	1–5 m/s

Table 7. Wireless channel model parameters.

Parameter	Value
Path loss	3GPP TR 38.901
Shadow fading	4 dB
Noise power	−174 dBm/Hz
Small-scale fading	Rayleigh
SNR	−10~20 dB

Table 8. Spectrum resource configuration.

Parameter	Value
Number of channels	12
Channel bandwidth	1.67 MHz
Frequency reuse	Dynamic

5.3. Hybrid DeepMUSIC Parameters

The Hybrid DeepMUSIC-based interference localization module described in Section 3.5 requires configuration of both the classical MUSIC estimator and the deep learning enhancement component. The antenna array geometry and snapshot settings determine the resolution and reliability of the pseudo-spectrum computed in Equation (10), while the deep neural network configuration governs the accuracy with which the DoA mapping is learned. The angular search range and resolution define the granularity at which candidate interference directions are evaluated, directly affecting the precision of the fused DoA estimate obtained via the hybrid fusion rule in Equation (12) and, consequently, the localization confidence score $L_i\left(t\right)$ used in the reward function of Equation (6). The received signal snapshots used for both MUSIC processing and DeepMUSIC training/evaluation are generated under the propagation model specified in Section 5.2, including distance-dependent path loss, log-normal shadowing, Rayleigh small-scale fading, and additive noise. Consequently, the reported localization performance reflects non-ideal fading conditions rather than an unfaded array-processing scenario. Table 9 summarizes the parameters adopted for the Hybrid DeepMUSIC module.

Table 9. Configuration parameters of the Hybrid DeepMUSIC interference localization module.

Parameter	Value
Antenna elements	8
Inter-element spacing	λ/2
Snapshots	200
Number of signals	3
Angular search	−90–+90°
Resolution	0.5°
Deep network	Fully Connected
Hidden layers	4
Hidden neurons	256-128-64-32
Activation	ReLU
Output	DoA
Optimizer	Adam
Learning rate	0.0005
Epochs	200
Batch size	64
Loss	MSE

5.4. MADRL Hyperparameters

The training configuration of the cooperative MADRL framework follows the DQN architecture and CTDE paradigm formulated in Section 4.4 and Section 4.5. Each UAV agent is equipped with a fully connected Q-network whose input, hidden-layer, and output dimensions correspond to the enhanced state representation ${\tilde{s}}_i\left(t\right)$ and action space defined in Equation (13), while the experience replays and target network update mechanisms follow the formulation in Section 4.6. Beyond the individual agent configuration, the cooperative behavior of the framework is governed by the multi-agent coordination settings established in Section 4.2, namely the number of participating agents, the CTDE training mode, and the reward and policy-sharing mechanisms that enable UAV agents to jointly optimize the global network reward $R\left(t\right)$ defined in Equation (7). Table 10 summarizes the network architecture, training procedure, exploration schedule, and cooperative configuration adopted for all UAV agents.

Table 10. MADRL training hyperparameters, including Q-network architecture and cooperative multi-agent configuration.

Category	Parameter	Value
Network Architecture	Algorithm	DQN
	Hidden layers	3
	Hidden neurons	256
	Activation function	ReLU
Training Configuration	Replay Buffer	100,000
	Batch Size	64
	Learning Rate	0.0001
	Discount Factor	0.99
	Target Update	every 500 steps
	Optimizer	Adam
	Episodes	1000
	Max steps	500
Exploration Schedule	Initial exploration rate $\left({\epsilon }_0\right)$	1
	Final exploration rate $\left({\epsilon }_{\min }\right)$	0.05
	Exploration decay rate	0.995
Cooperative Configuration	Number of agents	5
	Training paradigm	CTDE
	Reward sharing	Global
	Policy structure	Shared

The reward weighting coefficients are configured as $\alpha =1.0, \beta =0.8,$ and $\delta =0.3$. The throughput weight $\alpha$ is normalized to unity to serve as the reference scale against which the remaining terms are calibrated. The interference penalty factor $\beta$ is set below $\alpha$ to ensure that the reward remains predominantly throughput-driven while still imposing a meaningful penalty on co-channel interference, preventing UAV agents from converging toward overly conservative channel selection strategies that sacrifice throughput to avoid interference altogether. The localization confidence weighting $\delta$ is set to a comparatively small value, reflecting its role as an auxiliary term that reinforces reliance on directionally trustworthy DeepMUSIC estimates without dominating the throughput–interference trade-off that primarily governs spectrum allocation decisions.

5.5. Baseline Algorithms

To provide a systematic assessment of the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework, seven representative baseline algorithms are considered. The baselines are selected to progressively represent different levels of decision-making capability, ranging from non-learning and locally greedy spectrum selection to interference-aware conventional allocation, single-agent reinforcement learning, independent multi-agent learning, and cooperative MARL. This progressive comparison enables the performance gains associated with adaptive learning, inter-agent cooperation, CTDE, and DoA-based spatial interference awareness to be distinguished rather than attributing all improvements solely to the use of deep reinforcement learning.

Random Spectrum Allocation (RSA) serves as the lower performance bound of the comparison. Each UAV independently selects one of the $C$ available channels defined with equal probability, without regard to channel quality, interference conditions, or neighboring UAV decisions. The absence of any decision-making mechanism typically results in severe co-channel interference and poor spectrum utilization, particularly in dense multi-UAV deployments.

Greedy Spectrum Allocation (GSA) improves upon RSA by selecting, at each decision interval, the channel with the highest instantaneous channel quality indicator $H_i\left(t\right)$ or received SINR. While this exploits locally observed channel information, decisions remain independent across UAVs and disregard the future impact of current actions, which frequently leads to excessive channel contention and unstable performance under the dynamic mobility conditions described in Section 3.1.

Graph Coloring-Based Spectrum Allocation (GC) represents a classical optimization-based interference management approach. UAVs are modeled as vertices of an interference graph, with edges denoting potential co-channel interference; channels are then assigned such that adjacent UAVs receive different channels whenever feasible. Although effective under static network topologies, the computational complexity of graph coloring grows rapidly with network size, and the method lacks the adaptability required by the highly dynamic UAV mobility considered in this study.

Classical Deep Q-Network (DQN) corresponds to the single-agent variant of the DQN architecture formulated in Section 4.4, in which each UAV independently learns a spectrum allocation policy by approximating the action-value function of Equation (14) from its own local state, without access to the state or actions of neighboring agents. While DQN outperforms the heuristic baselines above, the absence of inter-agent cooperation limits its effectiveness once strong interference coupling exists among UAVs.

Independent Multi-Agent DQN (MADQN) extends the single-agent DQN to a multi-agent setting by assigning each UAV its own independently trained network, following the same architecture as Section 4.4 but without the shared reward or centralized training introduced in Section 4.5. Because each agent treats the other UAVs as part of a non-stationary environment rather than as cooperating agents, learning instability during training frequently degrades convergence, especially in dense deployments.

Value Decomposition Network (VDN) introduces cooperative learning by decomposing the global action-value function into a summation of individual per-agent value functions, trained under a shared team reward while retaining decentralized execution. Although this represents an improvement over independent learning, the linear decomposition assumption limits VDN’s ability to capture the complex, spatially coupled interference interactions present in multi-UAV wireless environments.

QMIX extends VDN by replacing the linear value decomposition with a trainable, nonlinear monotonic mixing network, allowing for a more expressive approximation of the global action-value function while preserving decentralized execution. Owing to its strong coordination capability, QMIX is widely adopted as a benchmark for cooperative resource allocation problems and therefore constitutes the strongest MARL baseline considered in this study.

Proposed Hybrid DeepMUSIC-assisted Cooperative MADRL differs from all baselines above in that it augments the cooperative MARL state representation with direction-of-arrival information obtained from the Hybrid DeepMUSIC module of Section 3.5, as formalized in the enhanced state ${\tilde{s}}_i\left(t\right)$ of Equation (13). Rather than relying solely on channel quality and interference power measurements, UAV agents exploit this spatial awareness to proactively identify interference-prone channels before communication degradation occurs, while the CTDE paradigm of Section 4.5 preserves fully decentralized execution during deployment.

Together, these baselines provide a progressive comparison from uncoordinated and interference-aware conventional spectrum allocation to adaptive single-agent and cooperative multi-agent learning. This evaluation design enables the individual benefits of adaptive learning, multi-agent cooperation, CTDE, and spatial interference awareness to be examined systematically. The performance criteria used for this comparison are defined in Section 5.6.

5.6. Evaluation Metrics

The performance of the proposed framework is evaluated using complementary metrics designed to assess not only the resulting communication performance but also the accuracy of spatial interference localization, the learning behavior of the cooperative agents, and the computational scalability of the overall framework. The metrics are therefore organized into four evaluation dimensions: communication and spectrum performance, interference-localization performance, learning performance, and computational performance. This multi-dimensional evaluation is necessary because improvements in spectrum allocation alone do not demonstrate whether the spatial localization module is accurate, whether the cooperative learning process converges reliably, or whether the resulting framework remains computationally feasible as the network size increases.

Communication and spectrum performance:

Average throughput measures the successfully achievable data rate across the UAV-served communication links and directly evaluates whether the spectrum-allocation strategy increases useful network capacity. Average SINR quantifies received signal quality relative to interference and noise and therefore serves as a direct indicator of the effectiveness of co-channel interference mitigation. Spectrum efficiency, expressed in bit/s/Hz, measures the amount of useful information delivered per unit bandwidth and evaluates how effectively the available spectrum resources are utilized. Energy efficiency, expressed in bit/J, measures the amount of successfully delivered information relative to the energy consumed and captures the energy cost associated with inefficient transmissions and retransmissions. Finally, end-to-end latency measures the communication delay experienced by transmitted data and is used to determine whether improved interference management translates into lower transmission and retransmission delays under increasingly dense UAV deployments.

Interference-localization performance:

The interference-localization capability of the Hybrid DeepMUSIC module is evaluated separately to determine whether the spatial information supplied to the MADRL agents is sufficiently accurate. DoA estimation error measures the absolute angular deviation between the estimated and ground-truth interference directions. Localization RMSE summarizes the overall angular estimation error across multiple interference sources and simulation realizations, while localization accuracy indicates the proportion of interference sources whose estimated DoAs fall within a predefined angular tolerance of the ground truth. These metrics enable a direct comparison among Classical MUSIC, standalone DeepMUSIC, and the proposed Hybrid DeepMUSIC estimator before the resulting spatial information is incorporated into cooperative spectrum allocation.

Learning performance:

The learning behavior of the reinforcement-learning-based approaches is evaluated using average cooperative reward, training loss, and convergence behavior. The average cooperative reward reflects the quality of the learned spectrum-allocation policy with respect to the network-wide objective, while the training loss indicates the stability of value-function optimization during policy learning. Convergence is assessed by the number of training epochs required for the reward trajectory to reach a stable operating region. These metrics distinguish improvements arising from more effective cooperative learning from those attributable solely to favorable instantaneous channel conditions.

Computational performance and scalability:

Computational feasibility is evaluated through average inference/processing time as the number of UAV agents increases. This metric quantifies the computational overhead introduced by Hybrid DeepMUSIC localization and cooperative policy execution and complements the theoretical complexity analysis of Section 4.8. In addition to absolute processing time, the scaling trend with the number of UAV agents is examined to determine whether the decentralized execution mechanism remains practical for increasingly dense multi-UAV deployments.

Together, these four groups of metrics provide a comprehensive evaluation of the proposed framework. Communication and spectrum metrics assess the primary resource-allocation objective; localization metrics isolate the contribution of Hybrid DeepMUSIC; learning metrics assess the effectiveness of cooperative MADRL; and computational metrics quantify the cost and scalability of integrating these components. Unless otherwise specified, stochastic performance results are averaged over 30 independent Monte Carlo runs, and 95% confidence intervals are reported to characterize statistical variability. Table 11 summarizes the metrics, their units, and their corresponding evaluation purposes.

Table 11. Summary of performance metrics used to evaluate communication efficiency, interference localization, learning behavior, and computational scalability.

Evaluation Dimension	Metric	Unit	Evaluation Purpose
Communication	Average Throughput	Mbps	Measures achievable network data rate
Communication	Average SINR	dB	Evaluates link quality and interference mitigation
Spectrum	Spectrum Efficiency	bit/s/Hz	Measures utilization of available spectrum
Energy	Energy Efficiency	bit/J	Evaluates delivered information per consumed energy
Delay	End-to-End Latency	ms	Measures communication delay
Localization	DoA Estimation Error	degree	Measures angular deviation from ground-truth DoA
Localization	Localization RMSE	degree	Quantifies overall DoA estimation error
Localization	Localization Accuracy	%	Measures successful localization within the defined tolerance
Learning	Cooperative Reward	—	Evaluates quality of the learned cooperative policy
Learning	Training Loss	—	Assesses learning stability
Learning	Convergence	epoch	Measures learning speed
Computational	Processing/Inference Time	ms	Quantifies computational overhead
Scalability	Runtime Scaling	—	Assesses sensitivity to the number of UAV agents

6. Results and Discussion

This section presents and analyzes the simulation results obtained under the evaluation methodology described in Section 5. Section 6.1 first examines the localization accuracy of the proposed Hybrid DeepMUSIC module, followed by the spectrum allocation performance in Section 6.2, the learning convergence behavior of the cooperative MADRL framework in Section 6.3, and the computational scalability analysis in Section 6.4.

6.1. Performance of Hybrid DeepMUSIC-Based Interference Localization

This subsection evaluates the localization accuracy of the proposed Hybrid DeepMUSIC module described in Section 3.5, which underlies the spatially aware state representation used by the cooperative MADRL framework. The interference localization performance is first examined qualitatively through the estimated spatial spectrum and then quantitatively through a direct comparison of the estimated directions of arrival (DoAs) against the classical MUSIC algorithm and a standalone deep-learning-based DoA estimator (DeepMUSIC) as depicted in Table 12. As shown in Figure 4, the proposed Hybrid DeepMUSIC module produces ten well-separated spectral peaks that align closely with the true angular positions of the interference sources spanning the −70° to 70° field of view. The absence of spurious peaks or spectral leakage between closely spaced sources indicates that the fusion of the physically interpretable MUSIC pseudo-spectrum with the learned spatial features of the deep network preserves high angular resolution even when interference sources are separated by as little as 15°. This behavior contrasts with the known degradation of classical subspace-based methods at low SNR or with a limited number of snapshots and confirms that the hybrid design retains the resolution benefits of MUSIC while compensating for its sensitivity to noise.

Table 12. Comparison of baseline spectrum allocation and resource management algorithms.

Algorithm	Learning Type	Cooperation	Interference Awareness	Deep Learning	DeepMUSIC	CTDE
Random Allocation	No	No	No	No	No	No
Greedy Allocation	Heuristic	No	Partial	No	No	No
Graph Coloring	Optimization	Partial	Yes	No	No	No
DQN	Single-Agent RL	No	Partial	Yes	No	No
Independent MADQN	MARL	No	Partial	Yes	No	No
VDN	Cooperative MARL	Yes	Partial	Yes	No	Yes
QMIX	Cooperative MARL	Yes	Partial	Yes	No	Yes
Proposed Hybrid DeepMUSIC + MADRL	Cooperative MARL	Yes	Yes (DoA-based)	Yes	Yes	Yes

Figure 4. Normalized spatial spectrum generated by the proposed Hybrid DeepMUSIC-based interference localization module for estimating the directions of arrival (DoAs) of ten simultaneous interference sources. The colored dashed lines denote the estimated DoAs, whereas the circular markers identify the corresponding interference source angles. The proposed algorithm accurately resolves all interference sources across the entire angular range from −70° to 70°, producing sharp spectral peaks with high spatial resolution and minimal estimation error. These localization results provide reliable spatial information for the subsequent cooperative multi-agent deep reinforcement learning framework, enabling interference-aware spectrum allocation and improving communication reliability in dense multi-UAV AI-native 6G wireless networks.

Table 13 quantifies these qualitative observations. Averaged over the ten interference sources, the classical MUSIC algorithm exhibits a mean absolute DoA estimation error of approximately 2.24°, with the largest deviation (4.05°) occurring at the extreme angle of 70°, reflecting the well-known degradation of subspace-based estimators near the edges of the array’s field of view. The standalone DeepMUSIC network reduces the mean error to approximately 1.65°; however, it exhibits comparatively larger errors (2.1–2.3°) for the closely spaced source pair at ±10°, suggesting that a purely data-driven estimator can struggle to resolve angularly adjacent sources without the structured subspace information exploited by MUSIC. The proposed Hybrid DeepMUSIC algorithm overcomes both limitations simultaneously, achieving a mean absolute error of only 0.105°, representing an approximately 15-fold improvement over the standalone DeepMUSIC network and a 21-fold improvement over classical MUSIC, with a worst-case error of just 0.24°. These results confirm that fusing the model-based and data-driven estimates yields a localization module that is both highly accurate and robust across the full angular range, providing the reliable directional information required to compute the localization confidence score and, in turn, to support interference-aware spectrum allocation decisions in the cooperative MADRL framework.

Table 13. Comparison of direction-of-arrival (DoA) estimation results for ten interference sources using Classical MUSIC, DeepMUSIC, and the proposed Hybrid DeepMUSIC algorithms. The proposed Hybrid DeepMUSIC achieves the smallest angular estimation errors, demonstrating improved localization accuracy and robustness.

Algorithm	Direction-of-Arrival Estimation (Degree)
True DOA	−70.00	−55.00	−40.00	−25.00	−10.00	10.00	25.00	40.00	55.00	70.00
MUSIC	−71.80	−57.50	−43.00	−23.67	−9.00	11.20	28.00	38.50	58.00	74.05
Deep MUSIC	−70.94	−54.39	−38.98	−23.72	−12.31	7.92	23.34	41.49	53.07	73.17
Hybrid Deep MUSIC	−70.06	−54.89	−39.95	−24.76	−10.09	9.88	25.10	40.05	54.92	70.15

6.2. Spectrum Allocation Performance

Building on the accurate interference localization established in Section 6.1, this subsection evaluates how effectively the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework translates spatial interference awareness into improved spectrum allocation performance. As illustrated in Figure 5, average network throughput declines for all algorithms as the number of UAV agents increases, which is an expected consequence of intensified spectrum contention and co-channel interference over a fixed pool of twelve orthogonal channels. However, the rate of decline differs markedly across algorithms: heuristic approaches such as Random and Greedy allocation degrade rapidly because they lack any mechanism for coordinating channel choices among neighboring UAVs, whereas the cooperative MARL baselines (VDN, QMIX) degrade more gracefully due to their shared reward structure. The proposed framework consistently achieves the highest throughput across all evaluated network sizes, and its throughput advantage over QMIX, the strongest baseline, widens as the number of UAVs grows. This trend indicates that the directional interference awareness supplied by Hybrid DeepMUSIC becomes increasingly valuable precisely when spectrum contention is most severe, allowing UAV agents to proactively steer away from directionally congested channels rather than merely reacting to already-degraded link quality.

Figure 5. Average network throughput achieved by different spectrum allocation algorithms under varying numbers of UAV agents. Error bars represent the 95% confidence intervals calculated from 30 independent Monte Carlo simulation runs using Student’s t-distribution. As the number of UAV agents increases, the average throughput gradually decreases due to intensified spectrum contention and co-channel interference. Nevertheless, the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework consistently achieves the highest throughput across all network sizes by utilizing interference-aware spectrum allocation and cooperative multi-agent learning, demonstrating superior scalability and communication efficiency compared with conventional heuristic and state-of-the-art learning approaches.

A consistent pattern emerges in the SINR results of Figure 6. Because SINR is directly and adversely affected by co-channel interference, it is the metric most sensitive to the quality of interference management. All algorithms experience SINR degradation as network density increases; however, the proposed framework exhibits the slowest degradation among all compared methods. This behavior is attributable to the DoA information embedded in the enhanced state representation of Equation (13), which allows UAV agents to anticipate which channels are exposed to strong directional interference before selecting them, rather than discovering the resulting SINR loss only after transmission. Baselines lacking any spatial awareness, including DQN and Independent MADQN, show comparatively steeper SINR decline as the number of agents increases, confirming that power- or quality-based state information alone is insufficient to anticipate spatially correlated interference in dense deployments.

Figure 6. Average SINR performance of different spectrum allocation algorithms as the number of UAV agents increases. Error bars indicate the 95% confidence intervals computed from 30 independent Monte Carlo simulation runs. Increasing the number of UAV agents intensifies co-channel interference and spectrum competition, causing the average SINR to decrease for all compared algorithms. However, the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework exhibits the slowest SINR degradation by accurately identifying interference sources through Hybrid DeepMUSIC and proactively coordinating channel allocation among neighboring UAV agents. Consequently, the proposed framework maintains superior communication reliability and interference resilience in dense multi-UAV 6G communication networks.

Spectrum efficiency, shown in Figure 7, follows a similar trend, declining across all methods as the number of UAV agents increases due to intensified channel contention. The proposed framework again exhibits the slowest efficiency degradation, which follows directly from its combination of accurate interference localization and cooperative decision-making: by proactively avoiding interference-prone channels, UAV agents reduce the frequency of colliding channel selections that would otherwise waste spectral resources. The gap between the proposed framework and the strongest cooperative MARL baseline (QMIX) widens at higher UAV counts, reinforcing the observation from the throughput results in Figure 5 that the benefit of spatial interference awareness scales favorably with network density rather than diminishing.

Figure 7. Spectrum efficiency performance of different spectrum allocation algorithms as the number of UAV agents increases. Error bars indicate the 95% confidence intervals computed from 30 independent Monte Carlo simulation runs. Increasing the number of UAV agents intensifies spectrum competition and mutual interference, leading to a gradual decline in spectrum utilization efficiency for all compared methods. However, the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework exhibits the slowest performance degradation by integrating accurate interference localization with cooperative multi-agent decision making, enabling proactive channel selection, reduced channel collisions, and more effective utilization of the available spectrum resources. Consequently, the proposed framework achieves superior spectrum efficiency and demonstrates excellent scalability for dense AI-native multi-UAV 6G wireless communication networks.

The energy efficiency results in Figure 8 provide a complementary perspective on the same underlying mechanism. Because energy efficiency captures the ratio of successfully delivered data to energy expended, it penalizes retransmissions caused by interference-induced packet loss more heavily than throughput alone. The proposed framework maintains the highest energy efficiency as network density increases, indicating that the reduction in channel collisions achieved through interference-aware spectrum allocation not only improves raw throughput but also reduces the retransmission overhead that would otherwise erode energy efficiency in dense multi-UAV deployments.

Figure 8. Energy efficiency is achieved by different spectrum allocation algorithms under varying numbers of UAV agents. Error bars indicate the 95% confidence intervals calculated from 30 independent Monte Carlo simulation runs using Student’s t-distribution. The proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework maintains higher energy efficiency as network density increases owing to improved interference localization, reduced channel collisions, and fewer retransmissions.

Finally, Figure 9 examines end-to-end latency. Although the proposed framework introduces additional inference overhead relative to lightweight heuristics, as quantified in Section 6.4, this overhead is more than offset at the system level: by reducing channel collisions and the resulting retransmissions, the proposed framework achieves lower overall latency than the baseline algorithms under dense multi-UAV conditions. Taken together, the results in Figure 5, Figure 6, Figure 7, Figure 8 and Figure 9 demonstrate that the gains delivered by Hybrid DeepMUSIC-assisted spatial awareness are not confined to a single performance dimension but consistently improve throughput, SINR, spectrum efficiency, energy efficiency, and latency simultaneously, particularly as network density increases.

Figure 9. Average end-to-end latency of different spectrum allocation algorithms under varying numbers of UAV agents. Error bars represent the 95% confidence intervals obtained from 30 independent Monte Carlo simulation runs. Although the proposed framework introduces additional inference processing, its interference-aware cooperative allocation mechanism reduces queuing and retransmission delays, resulting in lower overall latency under dense multi-UAV conditions.

Ablation Study on the Contribution of Hybrid DeepMUSIC

To isolate the contribution of spatial interference localization from that of cooperative multi-agent learning, an additional ablation experiment was conducted by removing the Hybrid DeepMUSIC module from the proposed framework. The resulting variant, denoted by “Proposed w/o Hybrid DeepMUSIC,” retains the same cooperative MADRL architecture, CTDE paradigm, DQN configuration, dynamic user–UAV association mechanism, channel model, training hyperparameters, and Monte Carlo realizations as the complete framework. The only difference is that the DoA estimates and localization-confidence information derived from Hybrid DeepMUSIC are excluded from the agent state representation. Correspondingly, the localization-confidence component is omitted from the reward, while the throughput and interference terms remain unchanged.

This controlled configuration enables the incremental contribution of spatial interference awareness to be quantified independently of the benefits arising from cooperative learning and centralized training. The ablated agents therefore continue to observe the communication, mobility, user-distribution, channel-quality, interference-level, and historical channel-selection variables defined in Section 3.4.1, but they must infer spectrum conflicts without explicit information regarding the direction of dominant interference sources. Both configurations are evaluated under identical network conditions for varying numbers of UAV agents using 30 independent Monte Carlo runs and the same 95% confidence-interval procedure adopted in the main experiments.

The ablation study quantifies the contribution of the Hybrid DeepMUSIC module by comparing the full proposed framework with an otherwise identical cooperative MADRL configuration without Hybrid DeepMUSIC. All other network settings, state variables, learning parameters, and CTDE mechanisms are maintained unchanged, allowing the performance differences to primarily reflect the contribution of DoA-based spatial interference information. As shown in Table 14 and Table 15, Hybrid DeepMUSIC consistently improves all evaluated metrics, with its contribution becoming more pronounced as UAV density increases. For three UAVs, the full framework improves throughput by 2.10%, spectrum efficiency by 3.80%, and energy efficiency by 3.79%, provides an absolute SINR gain of 0.7 dB, and reduces latency by 4.97%. At 15 UAVs, these gains increase substantially to 13.88% in throughput, 14.96% in spectrum efficiency, and 12.86% in energy efficiency, together with a 3.2 dB SINR gain and a 12.26% latency reduction. The progressively increasing performance gap indicates that spatial interference information becomes more valuable as coverage overlaps and co-channel interference intensifies. These results confirm that the performance gains of the full framework cannot be attributed solely to cooperative MADRL or CTDE. While the ablated framework relies on conventional communication and interference observations, Hybrid DeepMUSIC additionally provides DoA and localization-confidence information, enabling the agents to better distinguish the spatial characteristics of dominant interferers. The increase in throughput improvement from 2.10% to 13.88% and in absolute SINR gain from 0.7 to 3.2 dB as the UAV density increases provides quantitative evidence that spatial interference awareness contributes increasingly to effective spectrum allocation in dense multi-UAV networks.

Table 14. Ablation analysis of the contribution of Hybrid DeepMUSIC to spectrum-allocation performance under varying UAV densities.

UAVs	Framework	Throughput (Mbps)	SINR (dB)	Spectrum Efficiency (bit/s/Hz)	Energy Efficiency (bit/J)	Latency (ms)
3	Proposed w/o Hybrid DeepMUSIC	90.6	18.6	7.11	4.75	18.1
3	Full Proposed	92.5	19.3	7.38	4.93	17.2
5	Proposed w/o Hybrid DeepMUSIC	84.1	16.7	6.62	4.37	20.9
5	Full Proposed	87.9	17.8	7.08	4.67	19.8
7	Proposed w/o Hybrid DeepMUSIC	76.4	14.4	6.02	4.04	25.1
7	Full Proposed	81.8	16.1	6.54	4.33	22.9
10	Proposed w/o Hybrid DeepMUSIC	71.5	12.2	5.31	3.65	31.5
10	Full Proposed	78.4	14.9	5.87	4.01	28.1
15	Proposed w/o Hybrid DeepMUSIC	59.8	10.0	4.48	3.11	42.4
15	Full Proposed	68.1	13.2	5.15	3.51	37.2

Table 15. Ablation-based quantification of the performance gains contributed by the Hybrid DeepMUSIC module under varying UAV densities.

UAVs	Throughput Improvement	SINR Improvement	Spectrum Efficiency Improvement	Energy Efficiency Improvement	Latency Reduction
3	2.10%	+0.7 dB	3.80%	3.79%	4.97%
5	4.52%	+1.1 dB	6.95%	6.86%	5.26%
7	7.07%	+1.7 dB	8.64%	7.18%	8.76%
10	9.65%	+2.7 dB	10.35%	9.86%	10.79%
15	13.88%	+3.2 dB	14.96%	12.86%	12.26%

6.3. Learning Performance Analysis

Beyond steady-state communication performance, the learning behavior of the cooperative MADRL framework itself is an important indicator of practical viability. This subsection examines the training convergence characteristics of the proposed framework relative to the learning-based baselines in terms of cooperative reward accumulation and training loss reduction.

Figure 10 shows the evolution of the average cooperative reward during training for all learning-based algorithms. The Random and Greedy baselines, which involve no learning mechanism, are included only as static reference lines and as expected, never improve over the training horizon. Among the learning-based methods, all algorithms exhibit the expected pattern of gradually increasing reward as training progresses; however, the proposed Hybrid DeepMUSIC-assisted framework converges noticeably faster and reaches a visibly higher stable reward plateau than DQN, Independent MADQN, and QMIX.

Figure 10. Training convergence of the average cooperative reward for different spectrum allocation algorithms. As training progresses, all learning-based methods gradually improve their cooperative policies; however, the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework exhibits the fastest convergence rate and achieves the highest stable reward. The improved convergence behavior is attributed to the integration of DeepMUSIC-enhanced state representation, cooperative multi-agent learning, and the CTDE strategy, which together enable more efficient interference-aware spectrum allocation in dense multi-UAV AI-native 6G networks.

Table 16 quantifies these observations. The proposed framework achieves a final average cooperative reward of 76.246 ± 0.443 over the last 100 training epochs, exceeding QMIX (57.819 ± 0.608) by approximately 32%, Independent MADQN (41.718 ± 0.627) by approximately 83%, and the single-agent DQN baseline (27.138 ± 0.584) by nearly a factor of three. Graph Coloring, despite being an optimization-based method, converges to a comparatively low and still-negative reward of −9.587 ± 0.644, reflecting the fact that its static interference-graph formulation cannot adapt to the time-varying mobility and channel conditions of the simulated environment; meanwhile, the Random and Greedy heuristics never converge, remaining at strongly negative average rewards of −37.716 ± 0.588 and −21.191 ± 0.619, respectively, due to their complete lack of coordination among UAV agents. In terms of convergence speed, the proposed framework reaches its stable policy in approximately 371 epochs, which is 19.5% faster than QMIX (461 epochs), 30.7% faster than Independent MADQN (535 epochs), and 41.7% faster than classical DQN (636 epochs). The proposed framework also exhibits the tightest confidence interval among all learning-based methods (±0.443), indicating more consistent convergence behavior across the 30 independent Monte Carlo runs. These improvements are attributable to the DeepMUSIC-enhanced state representation, which supplies UAV agents with directionally informative features that reduce the exploration burden otherwise required to discover interference-avoiding policies from spectral information alone, together with the CTDE paradigm of Section 4.5, which stabilizes learning by exposing the centralized trainer to global network dynamics during training.

Table 16. Comparison of the average cooperative reward over the final 100 training epochs and the estimated convergence epochs for different spectrum allocation algorithms. Results are reported as mean ± 95% confidence interval (CI).

Algorithm	Average Reward over the Final 100 Epochs ± 95% CI	Estimated Convergence Epochs
Random	−37.716 ± 0.588	Not converged
Greedy	−21.191 ± 0.619	Not converged
Graph Coloring	−9.587 ± 0.644	920
DQN	27.138 ± 0.584	636
Independent MADQN	41.718 ± 0.627	535
QMIX	57.819 ± 0.608	461
Proposed Hybrid DeepMUSIC + MADRL	76.246 ± 0.443	371

The training-loss curves in Figure 11 corroborate the reward-based convergence analysis. The proposed framework exhibits both the fastest initial loss reduction and the lowest stabilized training loss among the learning-based methods, with narrower confidence bands than the baselines throughout most of the 1000-episode training horizon. The comparatively wider confidence intervals observed for DQN and Independent MADQN reflect the instability introduced by the non-stationarity problem discussed in Section 4.2, wherein each agent’s learning target shifts as other agents simultaneously update their own policies; the shared cooperative reward and DoA-enhanced state of the proposed framework mitigate this effect and yield more stable gradient-based updates.

Figure 11. Training-loss convergence of the learning-based spectrum allocation algorithms over 1000 episodes. The solid curves represent the mean training losses obtained from 30 independent runs, while the shaded regions indicate the corresponding 95% confidence intervals calculated using Student’s t-distribution. The proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework exhibits the fastest loss reduction and the lowest stabilized training loss, demonstrating improved learning stability and convergence efficiency.

6.4. Computational Scalability Analysis

While Section 6.1, Section 6.2 and Section 6.3 established the accuracy, communication performance, and learning efficiency of the proposed framework, its practical deploy-ability in real-time multi-UAV systems ultimately depends on its computational scalability. This subsection therefore analyzes the inference-time and processing-time behavior of the proposed framework and the baseline algorithms as the number of UAV agents increases, in relation to the theoretical complexity analysis of Section 4.8.

Table 17 reports the average per-step inference time of all algorithms for network sizes ranging from 3 to 15 UAVs. As expected, the lightweight heuristics (Random, Greedy) incur the lowest absolute inference times, while learning-based methods require additional computation for neural network forward passes, and the proposed framework, which additionally performs DeepMUSIC-based interference localization, has the highest absolute inference time at every network size (6.35 ms at 3 UAVs, rising to 23.91 ms at 15 UAVs). However, examining the scaling behavior rather than absolute values reveals a more favorable picture: as the number of UAVs increases fivefold from 3 to 15, the inference time of the proposed framework increases by a factor of only 3.76×, the smallest scaling factor among all compared algorithms, compared with 3.96× for QMIX, 4.52× for classical DQN, and 5.56× for Graph Coloring. The relatively steep scaling of Graph Coloring reflects the combinatorial growth of its underlying interference-graph optimization as the number of vertices and edges increases, whereas the near-linear scaling of the proposed framework is consistent with the theoretical complexity analysis in Equation (23), which shows that online execution complexity under the CTDE paradigm grows linearly with the number of agents. Consequently, the relative overhead of the proposed framework over QMIX actually narrows as the network grows, from approximately 20.5% at 3 UAVs to 14.6% at 15 UAVs, indicating that the additional cost of interference localization becomes proportionally less significant in denser UAV deployments.

Table 17. Computational average inference time of different spectrum allocation algorithms under varying numbers of UAV agents. The reported values are averaged over 30 independent Monte Carlo simulation runs.

Number of UAVs	Random (ms)	Greedy (ms)	Graph Coloring (ms)	DQN (ms)	Independent MADQN (ms)	QMIX (ms)	Proposed Hybrid DeepMUSIC + MADRL (ms)
3	0.41	0.72	1.36	2.94	3.82	5.27	6.35
5	0.62	1.05	2.14	4.08	5.41	7.16	8.42
7	0.88	1.46	3.18	5.76	7.54	9.62	11.24
10	1.34	2.11	4.83	8.64	10.87	13.95	15.83
15	2.07	3.35	7.56	13.28	16.74	20.86	23.91

Table 18 decomposes the total processing time of the proposed framework into its three constituent components: Hybrid DeepMUSIC localization, MARL decision-making, and CTDE update overhead, evaluated on both CPU and GPU. At the smallest network size (3 UAVs), the three components contribute comparably to the total CPU time (36.4%, 29.0%, and 34.6%, respectively, for DeepMUSIC, MARL decision, and CTDE update). As the network scales up to 15 UAVs, the relative contribution of the CTDE update component grows to 43.3% of total processing time, while the share attributable to DeepMUSIC localization, which is computed independently per agent, correspondingly shrinks to 25.2%. This shift confirms that the dominant scalability bottleneck of the proposed framework lies in multi-agent coordination overhead during centralized training rather than in the interference localization module itself, consistent with the complexity analysis of Section 4.8, which identifies eigenvalue decomposition and DQN training as the dominant offline costs while online per-agent execution remains lightweight. Comparing CPU and GPU execution further shows that GPU acceleration provides an approximately consistent 3.1×–3.2× speedup across all evaluated network sizes (from 3.11× at 3 UAVs to 3.21× at 15 UAVs), demonstrating that the proposed framework benefits proportionally from hardware acceleration regardless of network density and supporting its feasibility for deployment on GPU-enabled edge computing platforms or onboard AI accelerators, as noted in Section 4.8.

Table 18. Computational processing time of the proposed Hybrid DeepMUSIC-assisted cooperative MADRL framework.

Number of UAVs	DeepMUSIC (ms)	MARL Decision (ms)	CTDE Update (ms)	Total CPU Time (ms)	Total GPU Time (ms)
3	2.31	1.84	2.20	6.35	2.04
5	2.64	2.68	3.10	8.42	2.66
7	3.12	3.41	4.71	11.24	3.48
10	4.27	4.95	6.61	15.83	4.92
15	6.03	7.54	10.34	23.91	7.45

Figure 12 presents these results in normalized form, further illustrating that although the proposed framework introduces additional computational overhead relative to simpler baselines, this overhead grows predictably and near-linearly with the number of UAV agents rather than exponentially, in agreement with Table 18 and the theoretical complexity results of Section 4.8. Overall, the computational scalability analysis indicates that the performance gains in interference localization accuracy, spectrum allocation efficiency, and learning convergence documented in Section 6.1, Section 6.2 and Section 6.3 are achieved without sacrificing real-time feasibility, supporting the suitability of the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework for dense, AI-native multi-UAV 6G network deployments.

Figure 12. Normalized computational complexity of different spectrum allocation algorithms under varying numbers of UAV agents. Error bars represent the 95% confidence intervals computed from 30 independent Monte Carlo simulation runs using Student’s t-distribution. Although the proposed Hybrid DeepMUSIC-assisted Cooperative MADRL framework introduces additional computational overhead due to interference localization and cooperative learning, its computational complexity increases in a predictable manner with network size, demonstrating good scalability and suitability for real-time multi-UAV 6G communication systems.

7. Conclusions

This paper addressed the problem of interference-aware spectrum allocation in multi-UAV 6G wireless networks, where overlapping coverage areas and shared spectrum resources give rise to severe co-channel interference that conventional centralized allocation schemes struggle to manage in a scalable and adaptive manner. To this end, a Hybrid DeepMUSIC-assisted Cooperative Multi-Agent Deep Reinforcement Learning (MADRL) framework was proposed, in which a hybrid interference localization module fusing the classical MUSIC algorithm with a deep neural network estimates the directions of arrival of interference sources with high spatial resolution. This directional information is embedded into a DeepMUSIC-enhanced state representation used by cooperative DQN-based UAV agents trained under a Centralized Training and Decentralized Execution (CTDE) paradigm, allowing for coordinated spectrum allocation decisions while preserving fully distributed execution during deployment.

Extensive Monte Carlo simulations demonstrated that the proposed Hybrid DeepMUSIC module reduces the mean DoA estimation error to approximately 0.105°, representing more than a tenfold improvement over classical MUSIC and standalone DeepMUSIC estimators. When compared against seven representative baseline algorithms spanning heuristic, optimization-based, single-agent, and cooperative multi-agent reinforcement learning approaches, the proposed framework consistently achieved the highest network throughput, SINR, spectrum efficiency, and energy efficiency, together with the lowest end-to-end latency, particularly as the number of UAV agents increased. In terms of learning behavior, the proposed framework converged to the highest stable cooperative reward (76.246 ± 0.443) in only 371 training epochs, faster and more consistently than QMIX, Independent MADQN, and classical DQN. Furthermore, the computational scalability analysis confirmed that the inference-time overhead introduced by interference localization and cooperative decision-making grows near-linearly with the number of UAV agents (3.76× over a fivefold increase in network size), the most favorable scaling among all compared methods, and benefits proportionally from GPU acceleration, supporting the practical feasibility of the proposed framework for real-time, dense multi-UAV 6G deployments.

While the results confirm the effectiveness of integrating spatial interference awareness with cooperative multi-agent learning, the present study was evaluated under a simulated single-layer multi-UAV network with a fixed number of orthogonal channels and idealized antenna array assumptions. Future work will therefore extend the proposed framework to joint three-dimensional UAV trajectory optimization and spectrum allocation, incorporate realistic hardware impairments and imperfect channel state information into the DeepMUSIC localization module, and investigate scalability to larger and heterogeneous multi-UAV swarms operating alongside the terrestrial and satellite segments of future space-air-ground integrated 6G networks. Extending the cooperative learning architecture to support dynamic agent entry and exit, as well as validating the framework through field trials or hardware-in-the-loop testbeds, also represent promising directions for future research.