SAC-MS: Joint Slice Resource Allocation, User Association and UAV Trajectory Optimization with No-Fly Zone Constraints

Abstract

With the rapid growth of user service demands, space–air–ground integrated networks (SAGINs) face challenges such as limited resources, complex connectivity, diverse service requirements, and no-fly zone (NFZ) constraints. To address these issues, this paper proposes a joint optimization approach under NFZ constraints, maximizing system utility by simultaneously optimizing user association, unmanned aerial vehicle (UAV) trajectory, and slice resource allocation. Due to the problem’s non-convexity, it is decomposed into three subproblems: user association, UAV trajectory optimization, and slice resource allocation. To solve them efficiently, we design the iterative SAC-MS algorithm, which combines matching game theory for user association, sequential convex approximation (SCA) for UAV trajectory, and soft actor–critic (SAC) reinforcement learning for slice resource allocation. Simulation results show that SAC-MS outperforms TD3-MS, DDPG-MS, DQN-MS, and hard slicing, improving system utility by 10.53%, 13.17%, 31.25%, and 45.38%, respectively.

1. Introduction

With the advent of the sixth-generation (6G) wireless communication era, the paradigm of ubiquitous connectivity and support for diverse vertical industries has placed unprecedented demands on network capabilities. To meet these heterogeneous service demands, network slicing has emerged as a promising solution. It enables the partitioning of a shared physical network into multiple virtual networks based on different service types, with each network independently optimizing its resources and control strategies [1,2]. Driven by the rapid proliferation of high-bandwidth applications such as Internet of Things (IoT) devices and intelligent terminals, global mobile data traffic has increased dramatically. Traditional terrestrial networks are increasingly unable to meet the needs of wide-area coverage, high-speed access, and ultra-low latency, especially in remote or disaster-stricken regions. In these areas, terrestrial base stations are limited by geographical constraints and high deployment costs, resulting in significant coverage gaps. Furthermore, in urban hotspots or large-scale event scenarios, severe network congestion can significantly degrade the quality of service.

To address these challenges, the space–air–ground integrated network (SAGIN) has emerged as a promising architecture for next-generation wireless networks [3,4,5]. Specifically, low earth orbit (LEO) satellites can offer seamless connectivity in remote areas; however, the long communication distances between satellites and users hinder their ability to meet low-latency requirements [6]. In contrast, UAVs, acting as aerial base stations, offer flexible and rapid deployment and benefit from line-of-sight (LoS) communication, making them well-suited to complement terrestrial infrastructures [7,8]. By integrating satellites, unmanned aerial vehicles UAVs, and terrestrial networks, SAGIN enhances edge computing capabilities and improves network resilience, thereby alleviating the burden on traditional networks. This architecture is particularly well-suited for scenarios involving emergency response, coverage of remote areas, and dynamic service demands.

In practical deployment scenarios, UAVs, acting as mobile aerial base stations, provide communication and computing services to ground users. The design of UAV flight trajectories plays a critical role in determining the overall system performance. To achieve wide-area coverage and efficient service provisioning for ground users, it is essential to plan UAV flight paths carefully. A substantial body of research has been devoted to UAV trajectory optimization in multiple access systems [9,10,11], multi-UAV cooperative networks [12], and relay-based communication systems [13], yielding valuable insights.

However, in real-world environments, UAV trajectories are strictly constrained by regulatory and geographical factors. High-risk areas such as airports, power transmission lines, and military zones are often designated as no-fly zones (NFZs) [14], which must be explicitly considered in UAV trajectory planning. In recent years, several studies have begun incorporating NFZ constraints into trajectory optimization frameworks to enhance the safety and feasibility of UAV operations [15,16]. On the other hand, within the architecture of space–air–ground integrated networks, the way ground users select the most suitable access point—whether a terrestrial base station, UAV, or satellite—significantly impacts overall network performance. Moreover, given the limited communication and computing resources available at each access node, efficiently allocating these resources to meet heterogeneous service demands remains a key challenge requiring urgent attention.

Given the aforementioned challenges, this study develops a space–air–ground integrated network architecture with NFZ constraints. The proposed framework jointly optimizes user association, slice resource allocation, and UAV trajectory planning to meet the diverse service requirements of ground users. The main contributions of this work are summarized as follows:

• To meet the customized demands of diverse services under resource-constrained conditions, this paper introduces a dynamic radio access network slicing mechanism within the architecture of a SAGIN, subject to NFZ constraints. This design aims to enable on-demand allocation and efficient management of communication resources.
• We formulate a joint optimization problem that integrates user association, slice resource allocation, and UAV trajectory optimization, aiming to maximize the system utility, defined as the difference between the total system gains and the system costs. It is then decomposed into three subproblems: user association, UAV trajectory optimization and slice resource allocation.
• We propose the SAC-MS algorithm to solve the joint optimization problem. Specifically, a many-to-one matching game is adopted to achieve stable user–base station association, the SCA method is employed to transform the non-convex UAV trajectory optimization problem into convex subproblems, and a deep reinforcement learning-based algorithm is introduced for adaptive slice resource allocation in dynamic networks.
• Simulation results show that the proposed algorithm improves system utility by 10.53%, 13.17%, 31.25%, and 45.38% compared to the benchmark algorithms (TD3-MS, DDPG-MS, DQN-MS, and Hard_slicing), respectively.

The remainder of this paper is organized as follows. Section 2 provides a comprehensive review of related works on SAGIN, UAV trajectory optimization, user association, and slice resource allocation. Section 3 describes the system model, including the network model, UAV mobility model, NFZ model, communication model, and computation model. Section 4 formulates the joint optimization problem. Section 5 presents the proposed joint algorithm for user association, UAV trajectory optimization, and resource allocation. Section 6 analyzes the simulation results. Finally, Section 7 concludes the paper. To ensure clarity, we summarize all symbols and their definitions in

Table 1. Summary of main notations.

Notation	Definition
$L$	The set of LEO satellite
$B$	The set of terrestrial base stations
$U$	The set of unmanned aerial vehicle
$I_n$	The set of users associated with slice
$J$	The set of all base stations
$N$	The set of network slice
$K$	The set of No-Fly Zone
$W$	The set of bandwidth resources
$F$	The set of computing resources
$x_{n_i,j}\left(t\right)$	Association variable between user $i$ and base station $j$ on slice $n$
$b_{n,j}\left(t\right)$	Bandwidth allocation ratio for slice $n$
$f_{n,j}\left(t\right)$	Computation resource allocation ratio for slice $n$
$y_{j,n_i}\left(t\right)$	Bandwidth allocation ratio from base station $j$ to user $i$ on slice $n$
${\xi }_1,{\xi }_2,{\xi }_3$	Weight coefficients
$r_{NFZ}$	No-fly zone radius
$\alpha$	Path loss exponent
$\tilde{\alpha }$	Temperature coefficient
${\phi }_n^{comp}$	Latency tolerance threshold of slice $n$
$R_e$	Minimum transmission rate threshold

and provide the explanations of Acronyms in

Table 2. Summary of Acronyms.

Acronyms	Definition
SAGIN	Space–Air–Ground Integrated Network
UAV	Unmanned Aerial Vehicle
LEO	Low Earth Orbit
TBS	Terrestrial Base Station
MBS	Macro Base Station
SBS	Small Base Station
NFZ	No-Fly Zone
SLS	Latency-Sensitive Slice
SLT	Latency-Tolerant Slice
SR	High-Data-Rate Slice
UA	User Association
TO	Trajectory Optimization
RA	Resource Allocation
SAC	Soft Actor–Critic
TD3	No-fly zone radius
DDPG	Deep Deterministic Policy Gradient
DQN	Deep Q-Network
SAC-MS	Matching game, Sequential Convex Approximation, and Soft Actor–Critic-based Multi-Slice optimization algorithm

.

2. Related Works

To meet the growing demand for wireless services, the space–air–ground integrated network (SAGIN) has emerged as a pivotal architecture for enhancing connectivity and service quality by integrating satellites, UAVs, and terrestrial infrastructures. While extensive research has been conducted on its individual components, a significant gap remains in holistically integrating these elements under practical constraints [17].

Regarding SAGIN architectures, studies have made progress in supporting seamless coverage and massive connectivity. For instance, Mao et al. [18] introduced a space–aerial-assisted hybrid cloud–edge computing framework to minimize computation delay, while Cheng et al. [19] developed a SAGIN-based computing architecture for offloading. Others have focused on service function chaining [20] and dynamic resource allocation [21]. However, a common limitation across these works is their treatment of the physical network as a monolithic entity. They struggle to meet the diverse and customized needs of different users simultaneously. Reference [22] analyzed how dynamic network slice allocation delays affect latency, bandwidth, and service continuity, showing that reducing allocation time enhances performance for mobile users in 5G networks. While RAN slicing is a promising solution to this problem, its application in the complex SAGIN context remains strikingly underexplored, creating a clear research opportunity.

In the domain of UAV trajectory optimization, the focus has been on leveraging UAVs’ flexibility and LoS links to improve performance. Works like [23] jointly optimized trajectory and power control, and [24,25] aimed to minimize delay through integrated trajectory and resource planning. Ref. [26] applied the Successive Convex Approximation (SCA) method to solve the subproblems of computation resource and bandwidth allocation as well as UAV trajectory optimization. Notwithstanding these contributions, the vast majority of these studies operate in idealized environments. A critical oversight is the prevalent neglect of No-Fly Zone (NFZ) constraints, which are non-negotiable for real-world UAV deployment around airports, power lines, or military zones. Consequently, many proposed trajectories are academically interesting but practically infeasible, highlighting the need for trajectory designs that explicitly incorporate such safety-critical constraints.

On the front of user association and slice resource allocation, References [27,28] investigated the user association problem using evolutionary game theory and the Lagrangian dual method, respectively. The joint optimization of user association and resource allocation has been extensively studied in [29,30,31]. Ref. [29] used a D3QN-based method for resource allocation, and [31] developed a MADDPG framework for joint user association and allocation. Ref. [32] proposed a fractional programming approach using quadratic transform, weighted MMSE, and SCA for beamforming, offloading, and resource allocation. Ref. [33] proposed an energy-efficient user selection algorithm with dynamic caching. In addition, ref. [34] introduced a multi-agent deep reinforcement learning-based strategy for resource allocation in vehicular networks. Despite their sophistication, these approaches are often myopic. They predominantly focus on single-dimension optimization within a static or ground-based network topology. Crucially, they fail to account for the dynamic network topology introduced by mobile UAVs and the intricate coupling between association decisions, resource slicing policies, and UAV trajectories. Furthermore, many DRL solutions rely on algorithms like DQN, DDPG, or TD3, which are known to suffer from training instability and inefficiency in high-dimensional continuous action spaces, precisely the nature of the resource allocation problem here.

Therefore, in contrast to the existing literature, our work proposes a comprehensive solution that bridges these gaps. We introduce a joint optimization framework that (1) incorporates RAN slicing into SAGIN for customized service provisioning, (2) rigorously embeds NFZ constraints into UAV trajectory planning to ensure practicality, and (3) simultaneously solves user association, slice resource allocation, and trajectory optimization. We employ the Soft Actor–Critic (SAC) algorithm, which is specifically designed for stability and efficiency in high-dimensional continuous action spaces, thereby addressing the limitations of prior DRL approaches used in this domain.

To clearly highlight the differences between this study and existing literature and emphasize its novelty, we compared it with previous works, as shown in

Table 3. Literature comparison.

Related Paper	Objective	UAV	Network Slicing	NFZ	UAV Trajectory Optimization	User Association	Resource Allocation
[18]	minimize the maximum computation delay	√	×	×	√	×	√
[19]	Minimize the total system cost	√	×	×	×	×	√
[21]	Maximize the long term network utility	×	×	×	×	×	√
[23]	Maximize the minimum throughput	√	×	×	√	√	×
[24]	Minimize the system latency	√	×	×	√	×	√
[25]	Maximize the system EE	√	×	×	√	×	√
[26]	Minimize the system’s weighted energy consumption	√	×	×	×	×	√
[27]	SINR, average rate, and mobility-induced time overhead	×	×	×	×	√	×
[28]	Maximizing QoE of users	√	×	×	√	√	√
[29]	Maximum long-term overall network utility	×	×	×	×	√	√
[30]	Maximize the network utility reflecting proportional fairness	×	×	×	×	√	√
[31]	Weighting of maximized throughput, SINR and minimized delay	√	√	×	×	√	√
[32]	Minimize the weighted energy consumption	√	×	×	√	√	√
[33]	Maximize the residual energy of the satellite	√	×	×	×	√	×
Our work	Maximum long-term overall system utility	√	√	√	√	√	√

.

3. System Model

3.1. Network Model

As shown in Figure 1, we study a SAGIN consisting of $B$ terrestrial base stations (TBSs, including macro base stations (MBSs) and small base stations (SBSs)), $I$ ground users, $U$ UAVs, and a constellation of low earth orbit (LEO) satellites. The coverage area of each MBS contains multiple SBSs. UAVs act as aerial base stations and fly from a starting point $l_0$ to a destination point $l_e$ at a constant power to provide services to ground users. The UAV service area is denoted as the region $x_{\max } \times y_{\max }$, within which several no-fly zones (NFZs) are defined to prohibit UAV traversal. Due to the limited coverage of terrestrial base stations and their potential damage to these stations in disaster areas, which can disrupt communications, UAVs and LEO satellites are introduced to achieve seamless coverage across the entire cell. This ensures continuous communication services for ground users. Each user can establish a communication link with terrestrial base stations, UAVs, or LEO satellites. We assume that sets $W = \{W_0,\dots , W_{B+U}\}$ and $F = \{F_0,\dots , F_{B+U}\}$ represent the total virtual bandwidth and computing resources of all base stations, respectively. The bandwidth and computing resources of each terrestrial base station and UAV are partitioned into three service slices: SLS, SLT, and SR, which correspond to latency-sensitive, latency-tolerant, and high-data-rate tasks, respectively. Accordingly, the set of all slices is denoted by $n\in N = \{SLS, SLT, SR\}$. This resource slicing scheme can be applied to any number of service slices [35].

In addition, we define the sets of LEO satellites, TBSs, and UAVs as $L = \left\{1, 2,\dots , L\right\}, B = \left\{1, 2,\dots , B\right\}$, and $U = \{1, 2,\dots , U\}$, respectively. The set of all base stations is denoted as $J = L\cup B\cup U$. The set of users associated with slice $n$ is denoted as $I_n = \{1,\dots , M_1, \dots , M_1 + M_2,\dots , M_1 + M_2 + M_3\}$, while $I_{M_1} = \{1, 2,\dots , M_1\}, I_{M_2} = \{1, 2,\dots , M_2\}$, and $I_{M_3} = \{1, 2,\dots , M_3\}$ represent the sets of users served by the SLS, SLT, and SR slices, respectively. We also define a binary association variable $x_{n_i,j}\left(t\right)$ to indicate whether user $i$ on slice $n$ is associated with base station $j$, where $x_{n_i,j}\left(t\right) = 1$ if associated, otherwise, $x_{n_i,j}\left(t\right) = 0$. At each time slot $t$, each user $i$ on slice $n$ can be associated with only one base station, i.e., ${\displaystyle \sum _{j\in J}{x}_{n_i,j}\left(t\right)} = 1$. The bandwidth and computing resources allocated by the $j$th base station to slice $n$ are denoted by $W_n = b_{n,j}\left(t\right)W_j$ and $F_n = f_{n,j}\left(t\right)F_j$, respectively. $b_{n,j}\left(t\right)$ and $f_{n,j}\left(t\right)$ represent the allocation ratios of bandwidth and computing resources assigned to slice $n$, respectively, and $b_{n,j}\left(t\right)\in \left[0, 1\right], f_{n,j}\left(t\right)\in \left[0, 1\right]$.

3.2. UAV Mobility Model

To simplify the modeling, we assume that all UAVs fly at a fixed altitude $H$. Based on the 3D Cartesian coordinate system, the position of the $u$th UAV at time slot $t$ is denoted by $l_u(t) = \left\{x_u^{UAV}\left(t\right), y_u^{UAV}\left(t\right), H\right\}$, and its projection onto the horizontal plane is denoted by $l_u^{\prime }(t) = \left\{x_u^{UAV}\left(t\right), y_u^{UAV}\left(t\right)\right\}$. The UAV mobility is subject to a set of constraints, including its initial and final positions, which are expressed as follows.

$$l_u^{\prime }(0)=l_0,l_u^{\prime }(T)=l_e$$

(1)

In addition, the UAV’s flight speed is denoted as $V_u\left(t\right) = {\displaystyle \frac{l_u^{\prime }(t + 1) - l_u^{\prime }(t)}{\tau }}$ [36]. Furthermore, considering the constraints of the service area, as well as the UAV’s minimum and maximum speed requirements, denoted by $V_{\min }$ and $V_{\max }$, respectively, the UAV’s trajectory and flight speed should satisfy the following conditions.

$$0\le x_u^{UAV}\left(t\right)\le x_{\max }$$

(2)

$$0\le y_u^{UAV}\left(t\right)\le y_{\max }$$

(3)

$$V_{\min }^2\le {\Vert V_u(t)\Vert }^2\le V_{\max }^2$$

(4)

Due to limitations in horizontal flight speed, the UAV’s flight distance is also constrained and must satisfy the following condition.

$$\Delta l_u^{\prime }(t)=\Vert l_u^{\prime }(t+1)-l_u^{\prime }(t)\Vert \le L_{\max }^x$$

(5)

where $\Delta l_u^{\prime }(t)$ denotes the UAV’s horizontal travel distance, and $L_{\max }^x$ represents the maximum allowable horizontal distance.

3.3. NFZ Model

In certain specific areas, there are $K$ randomly distributed but non-overlapping NFZs that prohibit UAVs from flying through [37], we model each NFZ as a cylinder of infinite height [14], which is a common simplification for conservative trajectory planning. This implies that a UAV violates the NFZ constraint if its horizontal projection enters the circular area of the NFZ, regardless of its altitude. Moreover, we consider NFZs with a radius of $r_{NFZ}$ and a sufficiently high altitude. The set of NFZs is denoted by $K = \left\{1, 2,\dots , K\right\}$, and the center of the $k$th NFZ is represented by $p_k = \left({\overline{x}}_k, {\overline{y}}_k\right)$, as shown in Figure 2. The NFZ constraint is formulated solely based on the horizontal distance.

$${\Vert l_u^{\prime }\left(t\right)-p_k\Vert }^2\ge r_{NFZ}^2,\forall u\in U,k\in K$$

(6)

The radius $r_{NFZ}$ is usually set proportionally to the size of the simulation scenario, reflecting a typical safety margin around sensitive areas. The assumption of infinite height ($H_{NFZ} \to \infty$) is valid as long as the UAV’s operational altitude $H$ is significantly lower than any realistic NFZ ceiling, which holds true for our scenario.

3.4. Communication Model

In wireless communication, the distance between the sender and the receiver is an important factor to consider. For convenience, we assume the geographical position of the LEO satellite is $l_l\left(t\right) = \left\{x_l^{LEO}\left(t\right), y_l^{LEO}\left(t\right), z_l^{LEO}\left(t\right)\right\}$, the location of the $b$th TBS is defined as $l_b\left(t\right) = \left\{x_b^{TBS}\left(t\right), y_b^{TBS}\left(t\right), z_b^{TBS}\left(t\right)\right\}$, and the coordinates of user $i$ on slice $n$ are given by $l_{n_i}\left(t\right)=\left\{x_{n_i}\left(t\right),y_{n_i}\left(t\right),0\right\}$. Considering a 3D Cartesian coordinate system, at any time $t$, the projected positions of LEO $l$, TBS $b$, and user $i$ on the horizontal plane are denoted by $l_l^{\prime }\left(t\right)=\left\{x_l^{LEO}\left(t\right), y_l^{LEO}\left(t\right)\right\},l_b^{\prime }\left(t\right)=\left\{x_b^{TBS}\left(t\right),y_b^{TBS}\left(t\right)\right\}$, and $l_{n_i}^{\prime }\left(t\right)=\left\{x_{n_i}\left(t\right),y_{n_i}\left(t\right)\right\}$, respectively. Accordingly, the distances between LEO $l$, TBS $b$, UAV $u$ and user $i$ on slice $n$ are denoted as $d_{n_i,l}\left(t\right)=\sqrt{{\left(z_l^{LEO}(t)\right)}^2+{‖l_l^{\prime }\left(t\right)-l_{n_i}^{\prime }\left(t\right)‖}^2}$, $d_{n_i,b}\left(t\right)=\sqrt{{\left(z_b^{TBS}(t)\right)}^2+{‖l_b^{\prime }\left(t\right)-l_{n_i}^{\prime }\left(t\right)‖}^2}$ and $d_{n_i,u}\left(t\right)=\sqrt{H^2+{‖l_u^{\prime }(t)-l_{n_i}^{\prime }\left(t\right)‖}^2}$, respectively.

3.4.1. LEO-User Communication Model

With the aid of LEO satellite downlinks, seamless coverage can be provided to users. Since the distance between the satellite and the user is relatively large, the impact of user mobility on the channel gain can be neglected. The channel coefficient between user $i$ on slice $n$ and LEO $l$ is denoted as

$$g_{n_i,l}^{down}\left(t\right)=h_{ni,l}\left(t\right){\left[d_{ni,l}\left(t\right)\right]}^{-\alpha }={\left({\displaystyle \frac{c}{4\pi f_c}}\right)}^2{\left[d_{ni,l}\left(t\right)\right]}^{-\alpha }$$

(7)

where $h_{n_i,l}\left(t\right)$ denotes the unit radio propagation loss of the satellite link due to free-space path loss [12]. $c$ represents the speed of light, $f_c$ denotes the carrier frequency, $d_{n_i,l}\left(t\right)$ is the distance between LEO $l$ and user $i$, and $\alpha$ is the path loss exponent. Therefore, at time slot $t$, the downlink transmission rate of user $i$ on slice $n$ associated with LEO $l$ is

$$r_{n_i,l}^{down}\left(t\right)=x_{ni,l}\left(t\right)y_{l,ni}\left(t\right)W_l\log \left(1+{\displaystyle \frac{p_{l,n_i}^{down}\left(t\right)g_{n_i,l}^{down}\left(t\right)}{{\sigma }^2}}\right)$$

(8)

where the transmission power of LEO $l$ to user $i$ on slice $n$ is denoted by $p_{l,n_i}^{down}\left(t\right)$, and $x_{n_i,l}\left(t\right)$ is a binary variable indicating whether user $i$ is associated with LEO $l$. ${\sigma }^2$ represents the noise variance, and $y_{l,ni}\left(t\right)$ denotes the proportion of bandwidth resources allocated to user $i$ on slice $n$, associated with LEO $l$.

For users on both latency-tolerant and latency-sensitive slices, computing tasks need to be offloaded to MEC servers for processing. Accordingly, the uplink data transmission rate of user $i$ on slice $n$ when offloading tasks to the MEC server deployed on LEO satellite $l$ is

$$r_{n_i,l}^{up}\left(t\right)=x_{ni,l}\left(t\right)y_{l,ni}\left(t\right)W_l\log \left(1+{\displaystyle \frac{p_{n_i,l}^{up}\left(t\right)g_{n_i,l}^{up}\left(t\right)}{{\sigma }^2}}\right)$$

(9)

3.4.2. UAV–User Communication Model

In the UAV–user communication model, the UAV acts as an aerial base station providing different types of services to users via the downlink. Similarly, at time slot $t$, the channel gain between UAV $u$ and user $i$ on slice $n$ can be expressed as

$$g_{n_i,u}^{down}\left(t\right)=h_{ni,u}\left(t\right){\left[d_{ni,u}\left(t\right)\right]}^{-\alpha }$$

(10)

It is assumed that the small-scale fading component $h_{n_i,u}\left(t\right)$ follows a Rayleigh fading channel model and is given by $h_{n_i,u}\left(t\right)=h_0\left[{\displaystyle \frac{R}{R+1}}{\widehat{h}}_{n_i,u}\left(t\right)+{\displaystyle \frac{1}{R+1}}{\tilde{h}}_{n_i,u}\left(t\right)\right]$, where $h_0$ denotes the reference channel gain at a distance of 1 m, $R$ represents the Rician factor, ${\widehat{h}}_{n_i,u}\left(t\right)$ is the Line-of-Sight (LoS) component, ${\tilde{h}}_{n_i,u}\left(t\right)$ is the Non-Line-of-Sight (NLoS) component, and ${\tilde{h}}_{n_i,u}\left(t\right)~\mathcal{C}\mathcal{N}(0, 1)$. Accordingly, the data transmission rate between UAV $u$ and user $i$ on slice $n$ at time slot $t$ can be expressed as follows.

$$r_{n_i,u}^{\mathrm{down}}(t)=x_{n_i,u}(t)\cdot y_{u,n_i}(t)\cdot W_u{\log }_2\left(1+{\displaystyle \frac{p_{u,n_i}^{\mathrm{down}}(t)\cdot g_{n_i,u}^{\mathrm{down}}(t)}{{\displaystyle \sum _{u^{\prime }\in \mathcal{U}\setminus \left\{u\right\}}{p}_{u^{\prime },n_i}^{\mathrm{down}}}(t)\cdot g_{n_i,u^{\prime }}^{\mathrm{down}}(t)+{\sigma }^2}}\right)$$

(11)

Similarly, the uplink data transmission rate for user $i$ offloading tasks to the MEC server equipped on UAV $u$ is

$$r_{n_i,u}^{\mathrm{up}}(t)=x_{n_i,u}(t)\cdot y_{u,n_i}(t)\cdot W_u\cdot \log \left(1+{\displaystyle \frac{p_{n_i,u}^{\mathrm{up}}(t)\cdot g_{n_i,u}^{\mathrm{up}}(t)}{{\sum }_{u^{\prime }\in \mathcal{U}\backslash \left\{u\right\}}{p}_{n_i,u^{\prime }}^{\mathrm{up}}(t)\cdot g_{n_i,u^{\prime }}^{\mathrm{up}}(t)+{\sigma }^2}}\right)$$

(12)

3.4.3. TBS-User Communication Model

In the communication model between TBSs and users, each TBS provides customized services to multiple users. At time slot $t$, the channel gain between user $i$ and TBS $b$ is defined as $g_{n_i,b}^{down}\left(t\right)=h_{n_i,b}\left(t\right){\left[d_{n_i,b}\left(t\right)\right]}^{-\alpha }$, where $h_{n_i,b}\left(t\right)$ follows an exponential distribution with unit mean [5]. Therefore, at time slot $t$, the downlink data transmission rate between terrestrial base station $b$ and user $i$ on slice $n$ is

$$r_{n_i,b}^{\mathrm{down}}(t)=x_{n_i,b}(t)\cdot y_{b,n_i}(t)\cdot W_b\cdot \log \left(1+{\displaystyle \frac{p_{b,n_i}^{\mathrm{down}}(t)\cdot g_{n_i,b}^{\mathrm{down}}(t)}{{\displaystyle \sum _{b^{\prime }\in \mathcal{B}\setminus \left\{b\right\}}{p}_{b^{\prime },n_i}^{\mathrm{down}}}(t)\cdot g_{n_i,b^{\prime }}^{\mathrm{down}}(t)+{\sigma }^2}}\right)$$

(13)

Similarly to the satellite and UAV cases, when user $i$ offloads a task to the MEC server equipped at terrestrial base station $b$, the uplink data transmission rate is

$$r_{n_i,u}^{\mathrm{up}}(t)=x_{n_i,u}(t)\cdot y_{u,n_i}(t)\cdot W_u\cdot \log \left(1+{\displaystyle \frac{p_{n_i,u}^{\mathrm{up}}(t)\cdot g_{n_i,u}^{\mathrm{up}}(t)}{{\sum }_{u^{\prime }\in \mathcal{U}\backslash \left\{u\right\}}{p}_{n_i,u^{\prime }}^{\mathrm{up}}(t)\cdot g_{n_i,u^{\prime }}^{\mathrm{up}}(t)+{\sigma }^2}}\right)$$

(14)

3.4.4. Computation Model

Each user $i$ is required to process a computation task at each time slot $t$, which can be represented by a tuple ${\Omega }_{n_i}\left(t\right)=\left\{L_{n_i}\left(t\right),C_{n_i}\left(t\right),{\phi }_n^{comp}\right\}$. $L_{n_i}\left(t\right)$ denotes the data size, $C_{n_i}\left(t\right)$ is the required number of CPU cycles per bit, and ${\phi }_n^{comp}$ represents the maximum tolerable delay threshold allowed for slice $n$. Considering that the amount of data generated by computation is relatively small compared to the total task data, the download time can be neglected. Therefore, the total service delay experienced by the user mainly consists of two components: transmission delay and computation delay. The transmission delay can be expressed as

$$T_{n_i,j}^{up}\left(t\right)={\displaystyle \frac{L_{n_i}\left(t\right)}{r_{n_i,j}^{up}\left(t\right)}}$$

(15)

Let $f_{n_i,j}\left(t\right)$ (CPU cycles per second) denote the amount of computing resources allocated to user $i$ on slice $n$. Then, the computation delay for processing the task on slice $n$ can be expressed as

$$T_{n_i,j}^{com}\left(t\right)={\displaystyle \frac{C_{n_i}\left(t\right)}{f_{n_i,j}\left(t\right)}}$$

(16)

Therefore, for user $i$ associated with base station $j$, the total delay can be expressed as

$$T_{n_i,j}\left(t\right)=T_{n_i,j}^{up}\left(t\right)+T_{n_i,j}^{com}\left(t\right)$$

(17)

In the above expression, the two terms represent the transmission delay and the processing delay of the task, respectively. Moreover, the computational capabilities vary across different types of base stations. The SLT slice is designed for applications that can tolerate a certain degree of delay but have requirements for high reliability and substantial data volume. Therefore, for a user ii associated with the SLT slice, the constraint is defined by a maximum tolerable latency threshold ${\phi }_{\mathrm{SLT}}^{comp}$, The constraint is formalized as:

$$T_{n,j}\left(t\right)\le {\phi }_{\mathrm{SLT}}^{comp}, \forall n=\mathrm{SLT}$$

(18)

For users in the SLS slice, which caters to ultra-reliable low-latency communication (URLLC) services such as autonomous driving and industrial control, the QoS requirement is the most stringent. The total service delay must not exceed a very small maximum tolerable latency threshold, and the service must be highly reliable. Therefore, the QoS constraint for a user ii associated with the SLS slice is defined as follows:

$$T_{n,j}\left(t\right)\le {\phi }_{\mathrm{SLS}}^{comp}, \forall n=\mathrm{SLS}$$

(19)

where ${\phi }_{\mathrm{SLS}}^{comp}$ is the maximum end-to-end latency threshold for the SLS slice, and its value is typically much smaller than that of the SLT slice (i.e., ${\phi }_{\mathrm{SLS}}^{comp}\le {\phi }_{\mathrm{SLT}}^{comp}$).

For the SR slice, the primary concern is the transmission rate of users. Therefore, the transmission rate of a user assigned to the SR slice should exceed the minimum rate threshold defined for the slice, i.e.,

$$r_{n_i,j}\left(t\right)⩾R_e$$

(20)

4. Problem Formulation

In this section, our objective is to minimize the overall system cost while maximizing system utility, subject to the service requirements of users across different network slices.

4.1. System Cost

The total system cost of network slicing consists of the operational cost and the reconfiguration cost of the slices.

4.1.1. Operational Cost

The operational cost of a slice depends on the bandwidth and computational resources allocated to it by the base stations. Thus, at time slot $t$, the operational cost can be expressed as

$$U_o(t)={\displaystyle \sum _{n\in \mathcal{N}}{\displaystyle \sum _{i\in {\mathcal{I}}_n}{\displaystyle \sum _{j\in \mathcal{J}}\left({\zeta }_{o,j,b}{b}_{n,j}(t)W_j+{\zeta }_{o,j,f}{f}_{n,j}(t)F_j\right)}}}$$

(21)

where ${\zeta }_{o,j,b}$ and ${\zeta }_{o,j,f}$ denote the unit costs of bandwidth and computational resources at base station $j$, respectively.

4.1.2. Reconfiguration Cost

At different time slots $t$, the service requests of user $i$ may vary dynamically, necessitating adjustments in resource allocation strategies. Moreover, the reconfiguration cost of a slice is closely related to the type of user service. The reconfiguration cost at time slot $t$ is defined as follows

$$U_r(t)={\displaystyle \sum _{n\in \mathcal{N}}{\displaystyle \sum _{i\in {\mathcal{I}}_n}{\displaystyle \sum _{j\in \mathcal{J}}\left({\zeta }_{r,j,b}\cdot C_1+{\zeta }_{r,j,f}\cdot C_2\right)}}}$$

(22)

For users in the SR slice:

$$C_1={\left[b_{n,j}\left(t\right)W_j-b_{n,j}\left(t-1\right)W_j\right]}^{+}$$

(23)

$$C_1=\left\{\begin{array}{ll}{b}_{n,j}(t)W_j-b_{n,j}(t-1)W_j,\hfill & \mathrm{if} b_{n,j}(t)W_j>b_{n,j}(t-1)W_j\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(24)

For users in computation offloading slices:

$$C_2={\left[f_{n,j}\left(t\right)F_j-f_{n,j}\left(t-1\right)F_j\right]}^{+}$$

(25)

$$C_2=\left\{\begin{array}{ll}{f}_{n,j}\left(t\right)F_j-f_{n,j}\left(t-1\right)F_j,& \mathrm{if} f_{n,j}\left(t\right)F_j>f_{n,j}\left(t-1\right)F_j\\ 0& \hspace{1em}\mathrm{otherwise}\end{array}\right.$$

(26)

Based on the above analysis, the total system cost of network slicing can be expressed as follows

$$U_{\mathrm{cost}}\left(t\right)=U_o\left(t\right)+U_r\left(t\right)$$

(27)

4.2. System Revenue

In this paper, the system revenue is defined as consisting of two components: the communication rate utility and the computational efficiency utility. Therefore, the total system revenue can be expressed as follows

$$U_{\mathrm{renu}}(t)={\displaystyle \sum _{n\in \mathcal{N}}{\displaystyle \sum _{i\in {\mathcal{I}}_n}{\displaystyle \sum _{j\in \mathcal{J}}{\omega }_1}}}{r}_{n_i,j}(t)+{\omega }_2{\displaystyle \frac{C_{n_i}(t)}{T_{n_i,j}^{\mathrm{com}}(t)}}$$

(28)

The joint problem of user association, slice resource allocation, and UAV trajectory optimization is formulated with the objective of maximizing the overall system utility while minimizing the total system cost. Therefore, the system optimization objective is defined as follows

$$P1:\underset{x,b,f,l}{\max }{\displaystyle \sum _{t\in \mathcal{T}}\left(U_{\mathrm{renu}}(t)-U_{\mathrm{cost}}(t)\right)}$$

(29)

$$C1:0\le x_u^{\mathrm{UAV}}(t)\le x_{\max },\forall u\in \mathcal{U}$$

(29a)

$$C2:0\le y_u^{UAV}(t)\le y_{\max },\forall u\in \mathcal{U}$$

(29b)

$$C3: V_{\min }^2\le {‖V_u(t)‖}^2\le V_{\max }^2,\forall u\in \mathcal{U}$$

(29c)

$$C4:\Delta l_u^{\prime }(t)=\Vert l_u^{\prime }(t+1)-l_u^{\prime }(t)\Vert \le L_{\max }^x,\forall u\in \mathcal{U}$$

(29d)

$$C5:\Vert l_u^{\prime }(t)-p_k{\Vert }^2\ge r_{NFZ}^2,\forall u\in \mathcal{U},k\in \mathcal{K}$$

(29e)

$$C6:T_{n_i,j}(t)\le {\phi }_n^{\mathrm{comp}},n\in \left\{SLS,SLT\right\},i\in {\mathcal{I}}_n,j\in \mathcal{J}$$

(29f)

$$C7:r_{n_i,j}\left(t\right)\ge R_e,\forall \mathrm{n}\in \mathcal{N},i\in \mathcal{I},j\in \mathcal{J}$$

(29g)

$$\mathrm{C}8: b_{n,j}\left(t\right)\in [0,1],\forall \mathrm{n}\in \mathcal{N},j\in \mathcal{J}$$

(29h)

$$\mathrm{C}9: f_{n,j}\left(t\right)\in [0,1],\forall \mathrm{n}\in \mathcal{N},j\in \mathcal{J}$$

(29i)

$$\mathrm{C}10: {\displaystyle \sum _{n\in \mathcal{N}}{b}_{n,j}\left(t\right)=1,}\forall \mathrm{n}\in \mathcal{N},$$

(29j)

$$\mathrm{C}11:{\displaystyle \sum _{n\in \mathcal{N}}{f}_{n,j}\left(t\right)=1,}\forall \mathrm{n}\in \mathcal{N}$$

(29k)

$$\mathrm{C}12:x_{n_i,j}\left(t\right)\in \left\{0,1\right\}, \forall \mathrm{n}\in \mathcal{N},i\in {\mathcal{I}}_n,j\in \mathcal{J}$$

(29l)

$$\mathrm{C}13:{\displaystyle \sum _{j\in \mathcal{J}}{x}_{n_i,j}\left(t\right)}\le 1,\forall \mathrm{n}\in \mathcal{N},i\in {\mathcal{I}}_n$$

(29m)

$$\mathrm{C}14:{\displaystyle \sum _{n\in \mathcal{N}}{B}_n}\le W_j, \forall j\in \mathcal{J}$$

(29n)

$$\mathrm{C}15: {\displaystyle \sum _{n\in \mathcal{N}}{F}_n}\le F_j, \forall j\in \mathcal{J}$$

(29o)

where $\mathrm{x}=\left\{x_{n_i,j}\left(t\right)\right\}$ denotes the set of user-associated variables, while $\mathrm{b}=\left\{b_{n,j}\left(t\right)\right\}$ and $\mathrm{f}=\left\{f_{n,j}\left(t\right)\right\}$ represents the set of bandwidth resource allocation ratios and the set of computing resource allocation ratios. The $\mathrm{l}=\left\{l_u(t)\right\}$ denotes the set of UAV trajectories. Constraints C1–C4 define the limitations on UAV flight trajectory and velocity; C5 represents the NFZ constraint; C6–C7 impose requirements on the data transmission rate and service delay for slice $n$; C8–C9 specify the proportions of bandwidth and computing resources that base station $j$ allocates to user $i$ on slice $n$; C10–C11 indicate that the total available bandwidth and computing resources of the base station are distributed among all users; C12 defines the user association as a binary variable, when $x_{n_i,j}\left(t\right) = 1$, it implies that user $i$ on slice $n$ is associated with base station $j$, otherwise $x_{n_i,j}\left(t\right) = 0$; C13 ensures that each user can connect to at most one base station; and C14 and C15 represent the total inter-slice constraints on bandwidth and computing resources, respectively.

Solving the original optimization problem is challenging for the following reasons. First, the convexity of the objective function is not guaranteed. Second, the user association variable $x_{n_i,j}\left(t\right)$ is binary, which introduces integer constraints, and constraint C12 is non-convex. Therefore, the objective function forms a mixed-integer non-convex optimization problem, which is generally difficult to solve optimally. To address the computational complexity brought by the mixed-integer non-convex nature of the original problem, we relax the binary user association variables and further decompose the problem into three subproblems: user association, resource allocation, and UAV trajectory optimization, as illustrated in Figure 3.

These three subproblems are not solved by a simple one-time sequential process but are instead iteratively optimized within an alternating optimization framework, as illustrated in Figure 3. The workflow of the proposed SAC-MS algorithm is as follows: in each iteration, the UA subproblem is first solved while keeping the resource allocation and trajectory fixed. Then, using the updated association strategy and resource allocation scheme, the TO subproblem is solved. Finally, the resource allocation subproblem is addressed based on the updated association strategy and trajectory. This alternating process is repeated until convergence.

Specifically, given the bandwidth allocation ratio $b_{n,j}\left(t\right)$, the computation resource allocation ratio $f_{n,j}\left(t\right)$, and the UAV trajectory optimization strategy $l_u\left(t\right)$, the user association problem can be formulated as follows

$$sub-UA:\underset{\mathrm{x}}{\max }{\displaystyle \sum _{t\in \mathcal{T}}\left(U_{\mathrm{renu}}(t)-U_{\mathrm{cost}}(t)\right)}$$

(30)

$$\mathrm{s}.\mathrm{t}. \mathrm{C}6–\mathrm{C}7, \mathrm{C}12–\mathrm{C}13$$

(30a)

Given the bandwidth allocation $b_{n,j}\left(t\right)$, computing resource allocation $f_{n,j}\left(t\right)$, and user association strategy $x_{n_i,j}\left(t\right)$ in P1, the UAV positions can be optimized by solving the following problem.

$$sub-TO:\underset{\mathrm{l}}{\max }{\displaystyle \sum _{t\in \mathcal{T}}{\displaystyle \sum _{n\in \mathcal{N}}{\displaystyle \sum _{i\in {\mathcal{I}}_n}{\displaystyle \sum _{j\in \mathcal{J}}{\omega }_1}}}}{r}_{n_i,j}(t)$$

(31)

$$\mathrm{s}.\mathrm{t}. \mathrm{C}1–\mathrm{C}7$$

(31a)

Given the user association strategy $x_{n_i,j}\left(t\right)$ and the UAV position $l_u\left(t\right)$, the problem of bandwidth and computation resource allocation can be formulated as follows

$$sub-RA:\underset{\mathrm{b},\mathrm{f}}{\max }{\displaystyle \sum _{t\in \mathcal{T}}\left(U_{\mathrm{renu}}(t)-U_{\cos \mathrm{t}}(t)\right)}$$

(32)

$$\mathrm{s}.\mathrm{t}. \mathrm{C}6–\mathrm{C}11, \mathrm{C}14–\mathrm{C}15$$

(32a)

5. Proposed Algorithm

In this section, we propose a joint user association, UAV trajectory optimization, and resource allocation algorithm (SAC-MS) to iteratively solve the three subproblems mentioned above. First, a matching game approach is adopted for the user association problem. For the UAV trajectory optimization problem, auxiliary variables are introduced, and the SCA method is employed to solve it. Finally, the resource allocation problem is addressed using a reinforcement learning algorithm to learn the optimal strategy for bandwidth and computation resource allocation.

5.1. User Association (UA)

To solve the user association subproblem with reduced computing complexity, we apply many-to-one matching theory to obtain the optimal association strategy. In this model, each user can associate with only one base station, while each base station can serve multiple users. Therefore, as illustrated in Figure 4, the relationship between users and base stations is modeled as a many-to-one matching problem.

Formally, the user association problem can be modeled as a many-to-one matching game, represented by a tuple $\mathcal{J}, \mathcal{I}, {\succ }_{\mathcal{J}}, {\succ }_{\mathcal{I}}$, where ${\succ }_{\mathcal{J}}={\left\{{\succ }_j\right\}}_{j\in \mathcal{J}}, {\succ }_{\mathcal{I}}={\left\{{\succ }_i\right\}}_{i\in \mathcal{I}}$ denote the preference sets of base stations and users, respectively. The user association matching game is denoted by $\theta$, and the many-to-one matching between the base station set $J$ and the user set $I$ must satisfy the following conditions [38].

• $\theta \left(i\right)\subset J,\left|\theta \left(i\right)\right|⩽1,\forall i\in I_n$
• $\theta \left(j\right)\subset I_n,\left|\theta \left(j\right)\right|⩽I_j^{\max },\forall i\in I_n$
• $\theta \left(i\right)=j\iff i\in \theta \left(j\right)$

The first property restricts each user to be paired with only one base station. The second property limits the number of users that can be matched with each base station. The third property requires that a user can only be associated with a base station if the base station agrees to provide service to that user.

Let $S_j\left(i\right)$ and $S_i\left(j\right)$ denote the utility functions of base station $j$ for user $i$, and of user $i$ for base station $j$, respectively. If user $i$ prefers base station $j$ over base station $j^{\prime }$, i.e., $S_i(j)>S_i(j^{\prime })$, where $j, j^{\prime }\in \mathcal{J}$, this preference relationship is denoted as $j{\succ }_i{j}^{\prime }$. Conversely, if base station $j$ prefers user $i$ over user $i^{\prime }$, i.e., $S_j(i)>S_j(i^{\prime })$, where $i, i^{\prime }\in \mathcal{I}$, this preference relationship is denoted as $i{\succ }_j{i}^{\prime }$ [39].

Utility Function of Users: Considering resource efficiency, each user tends to select the base station that can allocate more resources. Based on this, the user’s utility function is modeled as the throughput per unit of energy consumption, i.e.,

$$S_i\left(j\right)={\displaystyle \frac{x_{n_i,j}\left(t\right)r_{n_i,j}\left(t\right)}{p_{n_i,j}\left(t\right)}}$$

(33)

Utility Function of Base Stations: To enhance overall system performance, each base station tends to serve users with the strongest received signal strength. Accordingly, the utility function of a base station is defined based on the signal quality of its associated users, aiming to maximize system utility, i.e.,

$$S_j\left(i\right)=p_{n_i,j}\left(t\right)g_{n_i,j}\left(t\right)$$

(34)

The user utility function and the base station utility function are not independent optimization objectives, but preference indicators in the many-to-one matching game. Although stable matching does not strictly maximize system utility, it eliminates all blocking pairs and achieves a Pareto-efficient equilibrium, thus effectively enhancing system performance. We provide the pseudocode for addressing the user association problem using a many-to-one bilateral stable matching game, as shown in Algorithm 1, and analyzed the computational complexity of the algorithm.

We provide the pseudocode for addressing the user association problem using a many-to-one bilateral stable matching game, as shown in Algorithm 1, and analyze the computational complexity of the algorithm.

	Algorithm 1: Association Algorithm Based on Stable Matching Game.
1:	$\mathbf{Input}:$ Preference matrices for BSs and users, Utility functions for both BSs and users to calculate preferences
2:	$\mathbf{Ouput}:$ Stable matching results between BSs and users
3:	Initialize:
4:	(a) Set all users as “free” (not yet matched);
5:	(b) Set all BSs with empty connected user lists;
6:	(c) Set BSs’ rejection list;
7:	(d) Set a list of requests for each user;
8:	Composition of preference lists:
9:	(a) BS and user exchange their listing information;
10:	(b) Each user constructs its preference list based on its own Utility Function $S_i\left(j\right)={\displaystyle \frac{x_{n_{i,}j}\left(t\right)r_{n_i,j}\left(t\right)}{p_{n_i,j}\left(t\right)}}$, and rank BSs in descending order of preference;
11:	(c) Each BS constructs its preference list based on its own Utility Function   $S_j\left(i\right)=p_{n_i,j}\left(t\right)g_{n_i,j}\left(t\right)$, and rank users in descending order of preference;
12:	Match Process:
13:	Repeat:
14:	For each user $i\in \mathcal{I}$ that is free:
15:	$i$ applies to the highest-ranked BS $j$ in its preference list;
16:	For each BS $j\in \mathcal{J}$:
17:	(a) j receives applications from users;
18:	(b) Sort users in the application list according to BS j preference list;
19:	If the number of applicants for j exceeds the quota:
20:	(a) Select the top $I_n$ users based on preference;
21:	(b) Reject the remaining users and add them to $j$ rejection list;
22:	else:
23:	(a) Accept all applicants and add them to BS $j$ waiting list;
24:	(b) Add accepted UAVs to $n$’s matched UAVs list;
25:	For each user $i\in \mathcal{I}$:
26:	If $i$ is rejected by a BS:
27:	(a) $i$ re-applies to the next most preferred BS;
28:	(b) Update the application list of $i$ and write down the BS that has been applied;
29:	end if
30:	end for
31:	For each BS $j\in \mathcal{J}$:
32:	(a) Update the waiting list based on the newly accepted users;
33:	(b) Combine the original waiting list with the newly accepted users;
34:	end for

The computational complexity of the user association matching game consists of the complexity of constructing the participants’ preference lists and the complexity of executing the game. Each user ranks the base stations in descending order according to its own preferences, thereby generating its preference list. This ranking step results in a computational complexity of $O\left(J\log \left(J\right)\right)$ for each user, and thus $O\left(JI\log \left(J\right)\right)$ for $I$ users. Similarly, the computational complexity for $J$ base stations is $O\left(JI\log \left(I\right)\right)$. Therefore, the total complexity of constructing preference lists for all users and base stations is $O\left(JI\log \left(JI\right)\right)$. During the execution of the game, each user sends an association request to a base station, and the iteration coefficient of the game determines its execution complexity. Accordingly, the computational complexity of executing the game is $O\left(I\right)$. However, the complexity of constructing the preference lists is more significant than that of executing the game. Hence, the overall computational complexity of Algorithm 1 is $O\left(JI\log \left(JI\right)\right)$.

5.2. UAV Trajectory Optimization (TO)

After obtaining the user association strategy $x_{n_i,j}\left(t\right)$, we incorporate it into the UAV trajectory optimization subproblem, which results in the following formulation.

$$\begin{array}{l}\mathbf{sub}-\mathbf{TO}:\underset{l}{\max }{\displaystyle \sum _{t\in \mathcal{T}}{\displaystyle \sum _{n\in \mathcal{N}}{\displaystyle \sum _{i\in {\mathcal{I}}_n}{\displaystyle \sum _{u\in \mathcal{U}}{r}_{n_i,u}\left(t\right)}}}}\\ ={\displaystyle \sum _{t\in \mathcal{T}}{\displaystyle \sum _{n\in \mathcal{N}}{\displaystyle \sum _{i\in {\mathcal{I}}_n}{\displaystyle \sum _{u\in \mathcal{U}}{x}_{n_i,u}}}}}(t)y_{u,n_i}(t)W_u\log \left(1+{\displaystyle \frac{p_{u,n_i}(t)g_{n_i,u}(t)}{{\displaystyle \sum _{u^{\prime }\in \mathcal{U}\setminus \left\{u\right\}}{p}_{u^{\prime },n_i}}(t)g_{n_i,u^{\prime }}(t)+{\sigma }^2}}\right)\\ ={\displaystyle \sum _{t\in \mathcal{T}}{\displaystyle \sum _{n\in \mathcal{N}}{\displaystyle \sum _{i\in {\mathcal{I}}_n}{\displaystyle \sum _{u\in \mathcal{U}}{x}_{n_i,u}}}}}(t)y_{u,n_i}(t)W_u\log \left(1+{\displaystyle \frac{p_{u,n_i}(t)h_{n_i,u}(t)}{{\displaystyle \sum _{u^{\prime }\in \mathcal{U}\setminus \left\{u\right\}}{p}_{u^{\prime },n_i}}(t)h_{n_i,u^{\prime }}(t)+{\sigma }^2{d}_{n_i,u}^{\alpha }(t)}}\right)\end{array}$$

(35)

$$\mathrm{s}.\mathrm{t}. \mathrm{C}1–\mathrm{C}7,$$

(35a)

$$C16:\log \left(1+{\displaystyle \frac{B_{u,n_i}\left(t\right)}{{\Gamma }_{n_i,u}\left(t\right)}}\right)⩾R^{th}$$

(35b)

where $B_{u,n_i}\left(t\right)=p_{u,n_i}\left(t\right)h_{n_i,u}\left(t\right)$, and ${\Gamma }_{n_i,u}\left(t\right)={\Gamma }_{n_i,u}\left(t\right)={\displaystyle \sum _{u^{\prime }\in \mathcal{U}\backslash \left\{u\right\}}{p}_{u^{\prime },n_i}\left(t\right)h_{n_i,u^{\prime }}\left(t\right)+}{\sigma }^2{d}_{n_i,u}{\left(t\right)}^{\alpha }$. Note that, due to the non-convexity of constraints C5 and C7, the UAV trajectory optimization problem is neither a concave optimization problem nor a quasi-concave maximization problem. As a result, it is generally difficult to obtain the global optimum using existing methods, and no efficient solution method currently exists. To address this issue, we adopt the SCA technique. Specifically, we first introduce an auxiliary variable set $H=\left\{h_{n_i}(t),i\in I\right\}$ to approximate and reformulate the original problem for iterative solution, i.e.,

$$C17:h_{n_i}\left(t\right)⩽{\displaystyle \frac{1}{{\left({‖l_u^{\prime }(t)-l_{n_i}^{\prime }(t)‖}^2+H^2\right)}^{\alpha /2}}}, \forall i\in I$$

(36)

Therefore, the original UAV trajectory optimization problem, denoted as $sub-TO$, can be reformulated as the following problem.

$$\begin{array}{l}sub-T{O}_1:\underset{\mathrm{l},\mathrm{h}}{\max }{\displaystyle \sum _{t\in \mathcal{T}}{\displaystyle \sum _{n\in \mathcal{N}}{\displaystyle \sum _{i\in {\mathcal{I}}_n}{\displaystyle \sum _{u\in \mathcal{U}}{r}_{n_i,u}}}}}(t)\\ ={\displaystyle \sum _{t\in \mathcal{T}}{\displaystyle \sum _{n\in \mathcal{N}}{\displaystyle \sum _{i\in {\mathcal{I}}_n}{\displaystyle \sum _{u\in \mathcal{U}}{x}_{n_i,u}}}}}(t)y_{u,n_i}(t)W_u\log \left(1+B_{u,n_i}(t)h_{u,n_i}(t)\right)\end{array}$$

(37)

Since the above optimization problem is also non-convex, we employ the SCA method to obtain a linear lower-bound approximation. Let $f\left(l_u^{\prime }(t)\right)={\displaystyle \frac{1}{{\left({‖l_u^{\prime }(t)-l_{n_i}^{\prime }(t)‖}^2+H^2\right)}^{\alpha /2}}}$, the function is neither globally convex nor concave; however, it can be approximated near a given point $l_u^{\prime (\varpi )}\left(t\right)$ using a first-order Taylor expansion, as shown below.

$$f\left(l_u^{\prime }(t)\right)⩾f\left(l_u^{\prime (\varpi )}(t)\right)+\nabla f{\left(l_u^{\prime (\varpi )}(t)\right)}^T\left(l_u^{\prime }(t)-l_u^{\prime (\varpi )}(t)\right)$$

(38)

where $f\left(l_u^{\prime (\varpi )}(t)\right)={\displaystyle \frac{1}{{\left({‖l_u^{\prime (\varpi )}(t)-l_{n_i}^{\prime }(t)‖}^2+H^2\right)}^{\alpha /2}}}$,$\nabla f=-\alpha {\displaystyle \frac{l_u^{\prime }\left(t\right)-l_{n_i}^{\prime }\left(t\right)}{{\left({‖l_u^{\prime (\varpi )}\left(t\right)-l_{n_i}^{\prime }\left(t\right)‖}^2+H^2\right)}^{\frac{\alpha }{2}+1}}}$.Therefore, the original constraint can be approximated by its lower bound, i.e.,

$$C18:h_{n_i}\left(t\right)⩽f\left(l_u^{\prime (\varpi )}(t)\right)+\nabla f{\left(l_u^{\prime (\varpi )}(t)\right)}^T\left(l_u^{\prime }(t)-l_u^{\prime (\varpi )}(t)\right), \forall i\in I$$

(39)

Similarly, constraint C5 is non-convex. To address this, we define $L^{\varpi } = \left\{l_u^{\varpi }\left(t\right)\right\}$ as the trajectory obtained at the $\varpi$th iteration. Then, by applying the first-order Taylor expansion, constraint C5 can be relaxed and approximated as follows.

$${‖l_u^{\prime }\left(t\right)-p_k‖}^2⩾2{\left(l_u^{\prime (\varpi )}\left(t\right)-p_k\right)}^T\times \left(l_u^{\prime (\varpi )}\left(t\right)-l_u^{\prime }\left(t\right)\right)+{‖p_k-l_u^{\prime (\varpi )}\left(t\right)‖}^2\stackrel{\Delta }{=}{C}_{u,k}^{\varpi }\left(t\right)$$

(40)

Therefore, constraint C5 is transformed into constraint $C19:C_{u,k}^{\varpi }\left(t\right)⩾r_{NFZ}^2$, which is convex. The equation $C_{u,k}^{\varpi }\left(t\right)⩾r_{NFZ}^2$ defines a line (a hyperplane in 2D) that is tangent to the NFZ circle at the point where the line connecting $p_k$ and $l_u^{\prime (\varpi )}(t)$ intersects the circle. The inequality $C_{u,k}^{\varpi }\left(t\right)⩾r_{NFZ}^2$ then specifies the half-space that excludes the NFZ, effectively replacing the circular keep-out zone with a linear keep-out boundary at each iteration. Since the Taylor expansion provides a lower bound for the convex quadratic function ${‖l_u^{\prime }\left(t\right)-p_k‖}^2$, this approximation ensures that the solution at iteration $\varpi +1$ will strictly satisfy the original NFZ constraint if the reference point $l_u^{\prime \left(\varpi \right)}\left(t\right)$ itself is feasible. The UAV trajectory optimization subproblem $sub-T{O}_1$ can be transformed into problem $sub-T{O}_2$.

$$\begin{array}{l}sub-T{O}_2:\underset{\mathrm{l},\mathrm{h}}{\max }{\displaystyle \sum _{t\in \mathcal{T}}{\displaystyle \sum _{n\in \mathcal{N}}{\displaystyle \sum _{i\in {\mathcal{I}}_n}{\displaystyle \sum _{u\in \mathcal{U}}{r}_{n_i,u}}}}}(t)\\ ={\displaystyle \sum _{t\in \mathcal{T}}{\displaystyle \sum _{n\in \mathcal{N}}{\displaystyle \sum _{i\in {\mathcal{I}}_n}{\displaystyle \sum _{u\in \mathcal{U}}{x}_{n_i,u}}}}}(t)y_{u,n_i}(t)W_u\log \left(1+B_{u,n_i}(t)\cdot h_{u,n_i}(t)\right)\end{array}$$

(41)

$$\mathrm{s}.\mathrm{t}. \mathrm{C}1–\mathrm{C}4, \mathrm{C}6–\mathrm{C}7, \mathrm{C}19$$

(41a)

Through the above transformations and approximations, the UAV trajectory optimization problem $sub-T{O}_2$ is converted into a convex optimization problem that can be efficiently solved using standard convex optimization solvers such as CVX [40]. In Algorithm 2, we present the pseudocode for UAV trajectory optimization based on SCA and analyze its computational complexity.

	Algorithm 2: UAV Trajectory Optimization Algorithm Based on SCA.
1:	$\mathbf{Input:}$ User Association Strategy $x_{i,j}\left(t\right)$, Slice Resource Allocation Ratio $b_n(t)$,$f_n(t)$, Initial trajectory $L^{\left(0\right)}=\left\{l_u^{\prime (0)}(t)\right\}$, NFZ’s Information $p_k$ and $r_{NFZ}$, Maximum Iterations ${\varpi }_{\max }$;
2:	$\mathbf{Output:}$ Optimized UAV trajectory $L^{*}=\left\{l_u^{\prime *}(t)\right\}$
3:	Initialize iteration index $\varpi \leftarrow 0$;
4:	Repeat:
5:	For the $\varpi$-th iteration, given reference trajectory $L^{(\varpi )}$
6:	Solve the convex optimization problem (sub-TO2):
7:	Let $L^{\left(\varpi +1\right)}=\left\{l_u^{\prime \left(\varpi +1\right)}(t)\right\}$ be the optimal solution of the above problem;
8:	Update $\varpi \leftarrow \varpi +1$;
9:	Until $\varpi >{\varpi }_{\max }$
10:	Return $L^{*}=L^{\left(\varpi \right)}$

In this study, we optimize the UAV trajectory using SCA and CVX. According to [24], the computational complexity of CVX is $O\left({\left(U\varpi \right)}^{3.5}\right)$.

5.3. Resource Allocation (RA)

We propose a reinforcement learning algorithm based on Soft Actor–Critic (SAC) to solve the resource allocation problem. SAC is a deep reinforcement learning algorithm that is well-suited for continuous action spaces. It is built upon the maximum entropy reinforcement learning framework and aims to maximize the expected cumulative reward by introducing an entropy term as a regularizer. This encourages exploration in the policy space to find the optimal policy $\mu$. In the SAC algorithm, the Markov Decision Process (MDP) is defined by the tuple $\left(\mathcal{S},\mathcal{A},\mathcal{R}\right)$ [41], where the state space $\mathcal{S}$ and the action space $\mathcal{A}$ are continuous, and $\mathcal{R}$ denotes the reward function. The three key components of the MDP are defined as follows.

(1) State Space: In this system, the allocation decisions for bandwidth and computing resources are influenced by the number and size of data packets within each slice. Therefore, the system state at each time slot is defined as the number of data packets that each slice needs to transmit, i.e., $s(t)=\left\{packe{t}_n\left(t\right),n\in N\right\}$.

(2) Action Space: The action space includes two optimization variables, defined as $a(t)=\left\{b_n\left(t\right),f_n\left(t\right),n\in N\right\}$. At time slot $t$, the agent selects an action $a(t)$ from the action space to make a decision.

(3) Reward: The reward is a function of the state and action, reflecting the quality of an action taken in a given state. To effectively solve the optimization problem, we define the reward function as follows

$$\begin{array}{l}r(t)={\xi }_1\left(U_{\mathrm{renu}}(t)-U_{\cos \mathrm{t}}(t)\right)-{\xi }_2{\displaystyle \sum _{i\in I}\max }\left\{R_e-r_{n_i,j}(t),0\right\}\\ -{\xi }_3\max \left\{T_{n_i,j}(t)-{\phi }_n^{\mathrm{comp}}(t),0\right\}\end{array}$$

(42)

The purpose of designing this reward function is to maximize system utility while ensuring a minimum rate threshold and reducing computational bottlenecks. The function consists of three components: the first term represents the difference between system revenue and cost; the second term is a penalty for insufficient data rate, ensuring that each user’s rate requirements are met; and the third term reflects the deviation between actual latency and the slice’s tolerable delay, serving as a penalty for delay violations. These three components are balanced using weighting factors ${\xi }_1,{\xi }_2$, and ${\xi }_3$ to comprehensively account for different optimization objectives.

The goal of SAC is not solely to maximize the expected return, but to maximize the expected return while also maximizing the entropy of the policy. To address the problem of resource allocation in network slicing, the resource allocation algorithm based on Soft Actor–Critic (SAC-RA) defines a maximum entropy objective formulated as follows.

$$J\left(\mu \right)={\displaystyle \sum _{t=1}^T{\mathbb{E}}_{\left(s(t),a(t)\right)\sim {\rho }_{\mu }}}\left[r\left(s(t),a(t)\right)-\tilde{\alpha }\log \left(\mu (·|s(t))\right)\right]$$

(43)

where $\tilde{\alpha }$ is the temperature parameter that balances the trade-off between reward maximization and entropy maximization, and ${\rho }_{\mu }$ represents the state-action distribution under the current policy $\mu$. In SAC-RA, the parameters ${\beta }_1$ and ${\beta }_2$ of the Q-function are updated at fixed time intervals by minimizing the soft Bellman residual [42].

$$J_Q\left({\beta }_i\right)={\mathbb{E}}_{\left(s(t),a(t)\right)~D}\left[{\displaystyle \frac{1}{2}}{\left(Q_{{\beta }_i}\left(s(t),a(t)\right)-\widehat{y}(t)\right)}^2\right]$$

(44)

where $\widehat{y}(t)=\left(r\left(t\right)+\gamma \left(\underset{\kappa =1,2}{\min }{Q}_{{\overline{\beta }}_{\kappa }}\left(s\left(t+1\right),a\left(t+1\right)\right)-\tilde{\alpha }\log {\mu }_{\phi }\left(a\left(t+1\right)|s\left(t+1\right)\right)\right)\right)$. The parameter of policy function $\phi$ is updated by

$$J_{\mu }\left(\phi \right)={\mathbb{E}}_{s(t)\sim \mathcal{D},\epsilon (t)\in \mathcal{T}}\left[\begin{array}{l}\tilde{\alpha }\log {\mu }_{\phi }\left(f_{\phi }\left(\epsilon (t);s(t)\right)|s(t)\right)\\ -\underset{\kappa =1,2}{\min }{Q}_{{\beta }_{\kappa }}\left(s(t),f_{\phi }\left(\epsilon (t);s(t)\right)\right)\end{array}\right]$$

(45)

In the above expression, $f_{\phi }\left(\epsilon (t);s(t)\right)$ is a reparameterization function [43], which maps the state $s(t)$ and noise $\epsilon (t)$ to the action $a(t)$. Specifically, given a state $s(t)$, the action is computed through $a\left(t\right)=f_{\phi }\left(\epsilon (t);s(t)\right)$. The overall framework of SAC-RA is illustrated in Figure 5, and the resource allocation algorithm SAC-RA is shown in Algorithm 3.

	Algorithm 3: SAC-RA Algorithm.
1:	$Initialize:$ policy network $\phi$, Q-functions ${\beta }_1$ and ${\beta }_2$, Parameters of target networks ${\overline{\beta }}_1\leftarrow {\beta }_1,{\overline{\beta }}_2\leftarrow {\beta }_2$, replay buffer $D$, learning rate $\lambda$, discount factor $\gamma$, entropy coefficient $\tilde{\alpha }$.
2:	for each episode do
3:	Initialize state $s(t)$;
4:	for step t = 1 to $T$do
5:	Selects action $a\left(t\right)$ according to $\mu \left(a\left(t\right)|s\left(t\right)\right)$;
6:	Execute action $a(t)$, and observe the immediate reward $r(t)$ and the     state $s\left(t+1\right)$
7:	Store $\left(s\left(t\right),a\left(t\right),r\left(t\right),s\left(t+1\right)\right)$ in the experience replay buffer $\mathcal{D}$;
8:	if replay buffer size $\ge$ batch size then
9:	Randomly sample batch $\left[s\left(t\right),a\left(t\right),r\left(t\right),s\left(t+1\right)\right]$ from buffer $\mathcal{D}$;
10:	Calculate the target Q-value, i.e., Equation (43);
11:	Update Q Network: ${\beta }_i\leftarrow {\beta }_i-\lambda {\nabla }_{{\beta }_i}{J}_Q\left({\beta }_i\right);$
12:	Update policy-function parameter: i.e., Equation (44), $\phi \leftarrow \phi -\lambda {\nabla }_{\phi }{J}_{\mu }\left(\phi \right)$;
13:	Update the target network using soft updates: ${\overline{\beta }}_i\leftarrow \tau {\beta }_i+\left(1-\tau \right){\overline{\beta }}_i$;
14:	end if
15:	end for
16:	end for

We assume that the Actor and Critic networks in SAC are $Z$-layer fully connected neural networks, with $Y_Z$ denoting the number of neurons in the $Z$-th layer. Thus, the forward/backward propagation cost of the neural networks is ${\displaystyle \sum _{\mathrm{z}=0}^{Z-1}{Y}_Z}\times Y_{Z+1}$. Let $\left|N_b\right|$ be the batch size and $N_{Episode}$ the total number of training epochs. Accordingly, the computational complexity of the SAC-based slice resource allocation algorithm is $O\left(\left({\displaystyle \sum _{\mathrm{z}=0}^{Z-1}{Y}_Z}\times Y_{Z+1}\right)\times \left|N_b\right|\times N_{Episode}\right)$. Based on the computational complexities of each algorithm discussed above, the overall computational complexity of the proposed SAC-MS algorithm is $O\left(K\left[O\left(JI\log \left(JI\right)\right)+O\left({\left(U\varpi \right)}^{3.5}\right)+O\left(\left({\displaystyle \sum _{\mathrm{z}=0}^{Z-1}{Y}_Z}\times Y_{Z+1}\right)\times \left|N_b\right|\times N_{Episode}\right)\right]\right)$.

6. Simulation Results and Analysis

In this section, we first configure the key parameters for the experiments. Then, through a series of simulation experiments, we systematically evaluate the proposed algorithms, including hyperparameter analysis, environment parameter analysis, and comparisons with benchmark algorithms.

6.1. Simulation Setup

We verify the performance of the proposed SAC-MS algorithm through simulations implemented using Python 3.11.9 and PyTorch 2.3.0. The network topology is configured over a 600 m × 600 m area, including one macro base station (MBS), three small base stations (SBSs), 100 ground users, and two no-fly zones. The shadow fading for the MBS and SBS is set to 8 dB and 10 dB, respectively [44]. The additive white Gaussian noise power ${\sigma }^2$ is set to −174 dBm, and the carrier frequency $f_c$ is 2 GHz. The UAV operates at a fixed altitude of 80 m, with a maximum speed of 15 m/s and a minimum speed of 6 m/s. For the reinforcement learning setup, the learning rates of the actor and critic networks are set to 0.001 and 0.0015, respectively. The batch size is 128, the temperature coefficient is 0.001, and the total number of training iterations is 6000. We provide the training parameters of the experiments and summarize them in

Table 4. Simulation parameters for system.

Training Parameters	Value
Number of terrestrial base stations	3
Number of UAV	1
Number of ground users	100
UAV flight altitude (m)	80
Noise Power ${\sigma }^2$(dBm)	−174
Learning rate of actor network LR_a	0.001
Learning rate of critic network LR_c	0.0015
discount factor $\gamma$	0.99
entropy coefficient $\tilde{\alpha }$	0.001
Target network soft update coefficient $\tau$	0.005
Replay Buffer Capacity $\mathcal{D}$	100,000
batch size	128
Max episode	6000
Convergence threshold	0.001
Activation function	Tanh

.

6.2. Parameter Analysis

6.2.1. Analysis of Hyperparameters

In reinforcement learning, the choice of hyperparameters significantly affects the convergence behavior, learning stability, and overall performance of the algorithm. To ensure the reliability and robustness of the proposed SAC-MS algorithm, we conduct a systematic analysis of key hyperparameters, including the learning rates of the actor and critic networks, the batch size, and the temperature coefficient.

We conduct simulations using different learning rates. As shown in Figure 6, setting the actor and critic learning rates to 0.1 and 0.15 leads to strong reward oscillations, indicating unstable updates and poor convergence. When reduced to 0.01 and 0.015, the algorithm achieves faster initial learning but converges to a lower reward. In contrast, smaller rates (0.001 and 0.0015) result in slower initial progress but lead to more stable convergence and better final performance. This highlights the trade-off between convergence speed and stability in learning rate selection.

Figure 7 compares the reward convergence under different batch sizes: 64, 128, and 256. All settings show rapid reward growth after initial exploration. The batch-size = 256 achieves faster early convergence but suffers from a noticeable performance drop around iteration 5400, indicating potential instability due to over-smoothed gradients. The batch-size = 64 yields more stable but slower learning due to high variance in updates. In contrast, batch-size = 128 provides the best trade-off, ensuring fast and stable convergence without significant fluctuations. This demonstrates that a moderate batch size (128) leads to more reliable training dynamics in SAC-MS.

As shown in Figure 8, different entropy temperature learning rates exhibit distinct convergence behaviors. A large alpha-lr (e.g., 0.1 and 0.01) results in rapid initial learning but introduces significant instability during training due to excessive entropy adjustments. Conversely, smaller values (e.g., 0.001 and 0.0001) provide smoother and more stable learning curves, ensuring consistent policy improvement. However, when the entropy temperature coefficient is too small (e.g., 0.0001), the convergence speed becomes relatively slow.

6.2.2. Analysis of Environmental Parameters

From Figure 9 and Figure 10, it can be observed that as bandwidth and computational resources increase, the system utility generally rises monotonically and gradually exhibits diminishing marginal returns. When resources are scarce (e.g., in the 70–100 range), system utility grows rapidly; as resources become sufficient (e.g., in the 115–130 range), the growth rate decreases, and system performance tends to saturate. It should be noted that in certain intervals (e.g., 100–115), the increase appears relatively larger. This is because the simulation results are based on discrete sampling points, and the additional resources in this interval happen to satisfy the service requirements of some users, resulting in a temporary improvement. Such local fluctuations do not alter the overall trend: as resources increase, the growth of system utility gradually slows and approaches saturation.

As shown in Figure 11, the system utility of all algorithms increases with the number of users. However, the superior efficiency of the proposed SAC-MS algorithm allows it to fully utilize the available resources, causing its growth to significantly slow down as it approaches the system capacity bottleneck at around 100 users. In contrast, the less efficient DDPG-MS and TD3-MS algorithms exhibit a more linear growth trend within the observed range, indicating that they have not yet fully stressed the system resources. This further demonstrates that SAC-MS can achieve higher resource utilization efficiency.

Figure 12 illustrates the optimal UAV trajectory planning results under no-fly zone constraints. As shown in the figure, the UAV not only successfully avoids the designated NFZs, ensuring flight safety and regulatory compliance, but also navigates as close as possible to user-dense areas. Such a trajectory design significantly shortens the communication distance between the UAV and ground users, thereby improving the signal-to-noise ratio (SNR), enhancing data transmission rates, and reducing communication latency. Moreover, flying near regions with high user demand facilitates more efficient task execution and real-time resource allocation.

6.2.3. Performance Comparison

Figure 13 illustrates the comparison of system utility achieved by different algorithms in the resource allocation task. As shown in the figure, the SAC algorithm converges rapidly after approximately 1000 training iterations and consistently maintains the highest utility throughout the entire training process, demonstrating its superior policy representation capability and efficient exploration performance in continuous action spaces. TD3 and DDPG follow in terms of performance, with TD3 showing better convergence stability than DDPG due to its delayed update mechanism and double Q-network structure. In contrast, DQN adopts a discrete action space, resulting in coarse-grained resource allocation and poor adaptability to dynamic environments, leading to significant fluctuations in system utility. Moreover, the hard slicing strategy does not incorporate any learning mechanism and thus cannot dynamically optimize according to environmental changes, resulting in the lowest utility level. These results further highlight the advantages of continuous action space-based deep reinforcement learning methods SAC in complex and high-dimensional resource allocation tasks.

As shown in Figure 14, the performance of TD3 under different hyperparameter settings is illustrated. Compared to TD3, SAC employs a maximum entropy framework, which enhances exploration efficiency and helps avoid local optima. In terms of system utility, SAC achieves approximately a 2.2% improvement over TD3’s best hyperparameter configuration (lr = 0.0001, batch_size = 256).

As shown in Figure 15, the performance of the DDPG-based algorithm under different hyperparameters is illustrated. Compared to DDPG, SAC demonstrates faster convergence, addresses the Q-value overestimation problem in DDPG, and is better suited for high-dimensional tasks. Relative to DDPG’s best hyperparameter combination (lr = 0.001, batch_size = 256), SAC achieves a 7.8% improvement in system utility.

Figure 16 compares DQN variants under different hyperparameter settings. DQN is suitable for discrete action spaces; however, when applied to continuous control tasks, it typically suffers from slow convergence, large fluctuations, and difficulty in achieving optimal performance. As illustrated in the figure, SAC outperforms DQN’s best hyperparameter configuration (lr = 0.001, batch_size = 128) by achieving a 31.25% improvement in system utility.

Figure 17 illustrates the trend of system utility over training iterations under different UAV flight strategies. It can be observed that when the UAV flight strategy accounts for no-fly zone constraints, the system utility is significantly higher compared to the case without such consideration. This is because no-fly zones act as hard constraints that limit the feasible flight region of the UAV. If these zones are not effectively avoided during path planning, the UAV may traverse illegal areas, resulting in communication interruptions, invalid trajectories, or resource waste. In contrast, flight strategies that consider no-fly zones can proactively avoid restricted areas and plan feasible paths, thereby ensuring communication link continuity and service quality, ultimately improving resource allocation efficiency and overall system performance.

To evaluate the impact of different user association strategies on system performance. Figure 18 compares three schemes: matching game-based association (SAC-MS), nearest-distance association (SAC-NS), and random association (SAC-RS). The figure illustrates the trend of utility values over training iterations. The results show that SAC-MS consistently achieves the highest utility, eventually stabilizing around 189, demonstrating superior system performance. This is mainly attributed to the matching game’s two-sided preference model, which dynamically matches users with base stations by considering both channel conditions and base station capacity, effectively avoiding base station overload and resource wastage, thereby enhancing overall system utility. In contrast, SAC-NS converges quickly in the early stages but considers only distance while ignoring interference and load. SAC-RS, which randomly selects base stations without considering channel state or user demand, shows poor convergence and large fluctuations, with a final utility of only around 150-indicating the weakest performance.

7. Discussion and Future Direction

The proposed SAC-MS framework provides a theoretical foundation for joint resource optimization in space–air–ground integrated networks (SAGINs) and demonstrates significant performance improvements in simulations. However, several challenges and limitations remain when moving from theoretical modeling to practical deployment, which also point to promising future research directions.

First, the limited battery capacity of UAVs constrains flight endurance and service coverage, suggesting that incorporating an energy consumption model into trajectory optimization is a valuable extension. Second, communication delays and signaling overhead in real SAGINs may weaken the timeliness of resource allocation, highlighting the need for delay-tolerant mechanisms and lightweight coordination protocols. Third, this study primarily focuses on a single-UAV scenario; although the proposed method can be extended to multi-UAV networks, the computational complexity of SCA-based trajectory optimization will significantly increase with the number of UAVs. Future work may explore distributed architectures and multi-agent reinforcement learning to enhance scalability and practicality. Fourth, the communication model relies on the assumption of perfect and instantaneous CSI, which is highly challenging to obtain in real environments with high mobility and severe fading. Developing robust optimization frameworks that can withstand CSI uncertainty and estimation errors will be an important direction. Fifth, although the proposed method has the potential to be extended to account for satellite orbital dynamics, in this study the satellite position is treated as fixed or only slowly varying over time. This limitation also provides new insights and directions for future research. Finally, the NFZs in this study are idealized as static cylindrical regions, whereas in reality they may be dynamic, irregularly shaped, or partially unknown, and may also involve multi-layer altitude restrictions. Future research could integrate real-time data from UAV traffic management (UTM) systems to enable trajectory planning under more complex and dynamic NFZ conditions.

8. Conclusions

To address challenges such as resource limitations caused by the exponential growth in user service demands, coverage blind spots in remote areas, and restrictions imposed by no-fly zones (NFZs), this paper investigated the joint optimization problem of user association, UAV trajectory planning, and network slicing resource allocation in space–air–ground integrated networks (SAGINs) under NFZ constraints. To facilitate the solution process, the original problem was decomposed into three subproblems: user association was solved using a many-to-one matching game; UAV trajectory was optimized via the sequential convex approximation (SCA) algorithm; and dynamic slice resource allocation was achieved through a deep reinforcement learning (DRL) framework (SAC). Simulation results demonstrated that, compared with benchmark algorithms, the proposed SAC-MS method more effectively satisfied users’ quality of service (QoS) requirements, avoided various no-fly zones, and achieved better resource allocation balance across network slices, thereby enhancing the overall system utility.