Joint Task Offloading and Resource Scheduling in Low Earth Orbit Satellite Edge Computing Networks ^†

Abstract

In view of the future of the Internet of Things (IoT), the number of edge devices and the amount of sensing data and communication data are expected to increase exponentially. With the emergence of new computing-intensive tasks and delay-sensitive application scenarios, terminal devices need to offload new business computing tasks to the cloud for processing. This paper proposes a joint transmission and offloading task scheduling strategy for the edge computing-enabled low Earth orbit satellite networks, aiming to minimize system costs. The proposed system model incorporates both data service transmission and computational task scheduling, which is framed as a long-term cost function minimization problem with constraints. The simulation results demonstrate that the proposed strategy can significantly reduce the average system cost, queue length, energy consumption, and task completion rate, compared to baseline strategies, thus highlighting the strategy’s effectiveness and efficiency.

1. Introduction

Traditional terrestrial networks, like 5G, rely on base stations as their fundamental units of operation. These stations enable the delivery of real-time and rapid services [1]. Nonetheless, terrestrial network deployment faces certain challenges. Primarily, the reach of terrestrial base stations is limited, making it difficult to establish them in regions such as deserts, forests, or mountains. Additionally, terrestrial base stations are susceptible to natural disasters. When events like typhoons, tsunamis, and earthquakes damage these stations, communication systems can fail, and restoring them swiftly is challenging, which leads to severe societal and economic disruptions due to interrupted communications. Consequently, satellite communication systems, especially the low Earth orbit (LEO) satellite networks, offer a flexible deployment option with global coverage, effectively complementing terrestrial networks [2,3]. In densely populated regions, satellite communication networks can provide substantial support, utilizing their resources for communication, computing, and storage to ease the burden on terrestrial systems. In locales where terrestrial networks are unable to reach, satellite networks can assume the role of transmitting information, thus complementing the terrestrial networks [4].

In the forthcoming era of the Internet of Things (IoT), achieving global connectivity and intelligence will lead to an exponential increase in the number of edge devices and the volume of sensing and communication data. Traditional centralized cloud networks will no longer be able to efficiently process the massive amounts of data generated by the IoT [5,6]. The advent of augmented reality, enhanced reality, high-definition live streaming, and other novel application scenarios further accentuates this challenge. Limitations in terminal computing power and navigation capabilities may result in suboptimal user experiences, such as the inability to interact in real time. Mobile Edge Computing (MEC), a pivotal technology in fifth-generation mobile communications, addresses these issues by decentralizing resources from central devices to network edge devices. This enables edge devices to possess certain computing capabilities for processing user requests locally, providing users with low-latency, high-bandwidth data processing services. Consequently, MEC alleviates the pressure on core network resources and effectively resolves problems such as data traffic bottlenecks in the core network [7,8,9].

In traditional satellite communication systems, satellites usually act as relays for transferring data from user devices. With the emergence of new computing-intensive tasks and delay-sensitive application scenarios, terminal devices need to offload new business computing tasks to the cloud for processing. However, the traditional mode of uploading data to the cloud for processing cannot meet the current computing-intensive business needs in terms of low latency and high bandwidth transmission. Therefore, the integration of satellite communication and edge computing is the development trend of future satellite networks. Moreover, LEO satellite communication systems inherently offer low-latency communication due to their proximity to the Earth’s surface and typically orbit at altitudes between 500 and 2000 km [10]. This close proximity significantly reduces the signal propagation delay compared to Geostationary Earth Orbit (GEO) satellites, which orbit at approximately 35,786 km. As a result, LEO satellites can achieve round-trip signal latencies as low as 20–30 ms, making them suitable for real-time and delay-sensitive applications [10,11]. The deployment of LEO satellite constellations enhances communication reliability through redundant coverage and seamless handover capabilities.

These low-latency and high-reliability characteristics of satellite communication systems, combined with edge computing and storage resources deployed on satellites, can provide users with strong computing and storage resource support [12,13,14]. This can significantly reduce data transmission between the edge and the central cloud, reduce delay, and lower energy consumption. In addition, with the increasing demand for computing and storage resources in new application scenarios, satellite edge computing can provide strong computing and storage resource support for new applications.

By integrating MEC technology with LEO satellite communication networks, satellites not only can receive and forward tasks to ground stations but also deploy MEC servers onboard, enabling LEO satellites to possess computing and storage resources. This is crucial for expanding service capabilities. Considering the characteristics of the LEO satellite communication networks and MEC technology, their integration can fully utilize the flexible satellite chain routes of LEO communication networks for efficient transmission [14]. Additionally, by deploying satellite servers, we can meet the needs of end users for faster, better, and safer applications. Therefore, building a LEO satellite edge computing network is an inevitable trend in the development of communication networks [15]. In our initial result [16], integrating edge computing technology with LEO networks can significantly reduce the service response time of LEO satellites, enhance the autonomous task processing capabilities of satellites, and improve network performance. Moreover, dynamic task offloading and transmission scheduling are critical for optimizing performance in satellite networks as they must adapt to various dynamic factors such as fluctuating user demand, the movement of satellites, changes in network topology, and environmental conditions. This adaptability ensures that tasks are efficiently offloaded to the most suitable resources in real time, thereby enhancing overall system performance and responsiveness.

The existing challenges of latency issues and bandwidth allocation must be addressed to improve system optimization. Although satellite networks are known for their low latency advantages, certain applications may still experience latency issues, especially when bandwidth is constrained during peak usage periods. Effective bandwidth allocation is crucial as it directly influences task processing speed and response times. Furthermore, optimizing bandwidth is essential in enhancing resource utilization efficiency, which ultimately contributes to better overall system performance. The LEO satellite systems integrated with edge computing need to communicate through satellites and ground stations, and there is a certain signal transmission delay between satellites and ground stations. This delay may adversely affect real-time applications and tasks that require rapid data processing; the bandwidth of satellite communications is relatively limited and insufficient for supporting large-scale data transmission and processing needs. This means that bottlenecks may be encountered when processing large-scale data sets and performing complex computing tasks; the LEO satellite systems integrated with edge computing require a large amount of energy supply to support their computing and communication needs. However, current solar battery technology and battery capacity limitations restrict the energy supply capacity of satellite systems. This may lead to the system being unable to meet the needs of high-intensity computing tasks. Additionally, satellite systems still face difficulties in maintenance and updates, as well as high costs.

The major contributions of this paper are as follows:

• This paper proposes a novel integration of Mobile Edge Computing (MEC) technology with LEO satellite communication networks while comprehensively considering constraints such as onboard computing and transmission, satellite–ground and inter-satellite links, and traffic conservation.
• This paper formulates the problem of efficient task offloading in the integrated LEO satellite edge computing network. It defines a cost function based on caching and energy consumption and models the dynamic task offloading and transmission scheduling problem as a constrained long-term cost function minimization problem.
• The original optimization problem is divided into two key components: data task transmission scheduling and computation task offloading. The data task transmission scheduling problem focuses on determining the most efficient routes for data transmission from observation satellites to ground stations, utilizing Dijkstra’s algorithm to identify the shortest paths while accounting for energy consumption. In contrast, the computation task offloading and computation scheduling problem is modeled as a Markov Decision Process (MDP), where dynamic factors such as the arrival of IoT device tasks and the status of relay satellite buffers are considered. This approach enables the application of reinforcement learning techniques, specifically the Deep Q-Network (DQN) algorithm, to develop optimal strategies for task offloading and computation scheduling, which minimize long-term system costs under varying conditions.
• Extensive simulations are performed to evaluate the proposed scheme compared with the existing peers.

The rest of the paper is organized as follows. Section 2 presents an overview of related work. Section 3 describes the system model, and Section 4 formulates the joint transmission, task offloading, and computation scheduling problem. The proposed schemes are introduced in Section 5. Simulation results are presented in Section 6. Finally, we conclude this paper in Section 7.

2. Related Works

This section outlines the current research related to task offloading and resource scheduling in LEO satellite networks.

2.1. Task Offloading in LEO Satellite Networks

In scenarios involving task offloading in LEO satellite networks, references [17,18] suggest energy-efficient algorithms for task offloading. The work in [17] examines a ground network supported by LEO satellites and offers a cloud–edge collaborative offloading strategy that accounts for service quality constraints, facilitating efficient energy offloading within a satellite network. It provides separate solutions utilizing deep reinforcement learning and game theory. Meanwhile, Ref. [18] explores a hybrid cloud–edge LEO satellite computing network with a three-layer structure, focusing on reducing total energy consumption for ground users. This research applies game theory to devise task offloading strategies for each ground user.

Studies [19,20,21] introduce algorithms that optimize delay in task offloading. Ref. [19] presents a network architecture integrating space and ground components to deliver edge computing services to LEO satellites and UAVs, focusing on reducing task execution delays through deep reinforcement learning for offloading decisions. Ref. [20] examines task offloading challenges within modern small satellite systems, proposing a strategy that optimizes delay while offloading and dismantling tasks. For satellite relay computing, Ref. [21] suggests a collaborative computing approach between satellite and ground to specifically reduce task response delays.

Other works [22,23,24,25] focus on creating energy-efficient and delay-optimized strategies for task offloading. Ref. [22] analyzes the time delays inherent in satellite–ground communications within relay computing scenarios, proposing a heuristic search algorithm to balance the computational load and reduce system energy consumption and task execution delay. Ref. [23] introduces a task offloading method employing a heuristic search algorithm specifically to lower both energy consumption and task execution delay for ground users. Ref. [24] also focuses on modern small satellite systems, suggesting a delay-optimized strategy for both offloading and dismantling tasks. Ref. [25] proposes a local search matching algorithm to determine the optimal task offloading strategy, targeting a reduction in energy usage and delay for ground users.

2.2. Resource Scheduling Algorithms for LEO Satellite Networks

The works of [26,27,28] examine the challenge of satellite transmission delays. Specifically, Ref. [26] explores the use of ground stations, high-altitude platforms, and LEO satellites to offer data offloading services to ground users. Meanwhile, Ref. [27] investigates using satellite chains for data offloading to ground stations, suggesting optimization algorithms to enhance the download throughput of data at these stations and for data transfer among cooperative satellites and from satellites to ground stations. Ref. [28] develops a transmission scheduling algorithm to maximize bandwidth usage in satellite constellations by enhancing inter-satellite link capacity. Furthermore, Ref. [29] examines the integration of LEO satellite networks with terrestrial networks, proposing a matching algorithm aimed at maximizing data transmission rates and user access, factoring in various capacity constraints.

Additionally, Refs. [30,31,32,33] delve into the scheduling of computing and storage resources in LEO satellite networks. For instance, Ref. [30] introduces a hybrid satellite–ground network utilizing software-defined networking, focusing on optimizing the long-term usage efficiency of communication, storage, and computing resources despite variable network, storage, and computing states, employing deep Q-learning algorithms for joint resource allocation optimization. Ref. [31] considers LEO satellites equipped with MEC platform computing and storage resources, proposing a service request scheduling approach to optimize resource use and service quality while reducing system costs. Ref. [32] suggests a strategy for satellite network storage resource scheduling to maintain user experience quality, and Ref. [33] explores the joint optimization of wireless resource allocation and task offloading in satellite networks to leverage the computing power of edge nodes effectively.

2.3. Task Offloading and Resource Scheduling Algorithms for LEO Networks

In addressing the challenges of task offloading and resource scheduling within LEO network environments, Ref. [34] examines the varying aspects of task segmentation and the energy priorities of edge devices. The study introduces an energy-efficient task offloading and resource scheduling algorithm to minimize both overall delay and energy usage.

Studies [35,36] put forward joint task offloading and resource scheduling algorithms leveraging deep reinforcement learning. Specifically, Ref. [35] employs a distributed algorithm grounded in convex optimization to tackle the task offloading and resource scheduling issue, recasting it as a direct optimization problem of computing resource scheduling for LEO satellites and simulating the system to maintain high energy efficiency. Meanwhile, Ref. [36] utilizes a deep reinforcement learning approach for resource allocation, addressing task offloading to the edge device and task segmentation issues, and applies a deep reinforcement learning strategy to identify the optimal offloading approach to minimize delay and energy consumption at the edge device.

Moreover, Refs. [37,38] suggest joint task offloading and resource scheduling frameworks using deep reinforcement learning. For instance, Ref. [37] proposes an integrated strategy for computing resource allocation and task offloading that adapts to dynamic changes in the network environment to enhance energy efficiency long-term, employing deep reinforcement learning to modulate the offloading strategy dynamically. Meanwhile, Ref. [38] evaluates LEO network architecture with edge computing, focusing on minimizing user task processing delay and energy use at the edge node, and proposes a comprehensive strategy for task offloading and computing resource scheduling to achieve long-term reductions in delay and energy consumption. Furthermore, Ref. [39] considers dynamic shifts in the locations of edge devices, satellites, and the network setting, using a deep reinforcement learning algorithm to optimize user association, offloading, computing, and communication resource allocation strategy for minimal long-term delay and energy use. Additionally, Ref. [40] presents a joint task offloading and resource scheduling strategy tailored for LEO edge computing aimed at minimizing system costs, with simulation outcomes indicating that the proposed strategy can notably decrease the average system cost.

Despite significant advancements in LEO satellite task offloading research, there remains a substantial gap in fully leveraging satellite collaboration for efficient resource utilization. Existing studies, summarized in the

Table 1. Comparison of related works on task offloading in LEO satellite networks.

References	Focus Area	Methodology	Key Contributions	Limitations
[17,18]	Task offloading with energy optimization	Game theory, energy-efficient algorithms	Proposes energy-efficient offloading strategies with service quality constraints	Does not address joint resource scheduling
[19,20,21]	Delay-optimized task offloading	Deep reinforcement learning	Reduces task execution delays through offloading optimization	Limited focus on computational resource scheduling
[22,23,24,25]	Energy and delay optimization	Heuristic algorithms	Balances computational load and reduces system energy consumption and delays	Lacks adaptability to dynamic network conditions
[26,27,28,29]	Satellite resource scheduling for communication	Optimization algorithms	Enhances inter-satellite bandwidth usage and throughput	Focuses only on data transmission scheduling, not computation
[29,30,31,32,33]	Resource scheduling in hybrid networks	Software-defined networking, Q-learning	Improves communication, storage, and computing efficiency in hybrid networks	Assumes static resource demands and lacks joint optimization
[34,35,36]	Joint task offloading and resource scheduling	Deep reinforcement learning, convex optimization	Proposes energy-efficient strategies for task segmentation and scheduling	Limited handling of dynamic network environments
[37,38,39,40]	Long-term optimization in LEO networks	Deep reinforcement learning, joint optimization	Provides comprehensive strategies for delay and energy optimization in dynamic scenarios	Assumes fixed task arrival rates and limited scalability
This work	Joint task offloading and dynamic resource scheduling	Deep Q-Network-based optimization	Long-term cost minimization, real-time adaptability to dynamic task arrivals and link variability	Requires high computational resources for reinforcement learning model training

, primarily focus on optimizing task execution delay and energy efficiency without accounting for the dynamic nature of LEO networks and the potential of satellite cooperation. To address these gaps, our research makes the following unique contributions. We propose a groundbreaking integration of Mobile Edge Computing (MEC) technology with LEO satellite communication networks. This integration considers unique constraints such as onboard computing capabilities, satellite–ground and inter-satellite link capacities, and traffic conservation laws, which have not been collectively addressed in prior works.

3. System Model

This section considers the system model comprising the network model, task and computation model, transmission service model, and link transmission rate model.

3.1. Network Model

The proposed LEO satellite network includes N observation satellites, K relay satellites, M IoT devices, and a ground station. Observation satellites equipped with sensors and instruments to collect data from the Earth’s surface, such as imagery or environmental measurements, focus on data acquisition and initial data processing [41,42]. Relay satellites are defined as satellites that facilitate communication by receiving data from observation satellites or IoT devices [43,44] and forwarding it to other satellites, ground stations, or processing nodes. They possess greater computational and communication capabilities to handle data relay and task offloading.

In this paper, we distinguish between two types of information handled by satellites: data and tasks. Data refer to raw information transmitted or forwarded without additional processing, such as images relayed from an observation satellite to a ground station. Tasks, on the other hand, involve computational workloads where data need to be processed to extract useful insights. For example, an IoT sensor may collect temperature readings, which require analysis to detect anomalies. In our model, if data only require transmission, we classify it as ‘data’; if computation is required, we classify it as a ‘task’.

When direct communication between an observation satellite and the ground station is not possible due to their relative positions, observation satellites gather data from the Earth’s surface and transmit it to relay satellites, which then relay the data to the ground station cache. For example, the data can be raw or processed information, such as images, sensor readings, and measurements that need to be transmitted to ground stations. Computational tasks may originate from IoT devices and require offloading to relay satellites for processing due to limited device capabilities. IoT devices create computational tasks and transmit them either to relay satellites or directly to the ground station for processing. Computational tasks encompass processing activities generated by IoT devices or satellites, requiring computational resources to execute functions like data analysis, image processing, and complex calculations. Accordingly, IoT devices establish connections with relay satellites through an association process, where each device selects an appropriate relay satellite for communication and task offloading based on factors such as signal quality, available resources, and connectivity status.

Let $O{S}_n$ denote the n-th observation satellite, $R{S}_k$ denote the k-th relay satellite with a computational capacity of ${\varphi }_k$, and $I{D}_m$ denote the m-th IoT device. Each relay satellite’s sub-channel offers a bandwidth of B, allowing multiple IoT devices to connect with relay satellites via Orthogonal Frequency Division Multiple Access (OFDMA) technology. The specific system layout is depicted in Figure 1.

In the dynamic LEO satellite scenario explored in this paper, to more clearly illustrate the system’s link dynamics, we consider that the system time T is divided into continuous intervals of duration $\tau$. Given the nature of inter-satellite links in the LEO network, each LEO satellite is equipped with four interconnections with other satellites. The links between satellites within the same orbit are comparatively stable, while those between satellites in adjacent orbits achieve relative stability through steerable beam technology. Therefore, the inter-satellite network topology is assumed to remain stable. To describe the connection status of the inter-satellite communication links, we define $x_{n,k}^o\in \{0,1\}$ as the physical link indicator between the observation satellite $O{S}_n$ and the relay satellite $R{S}_k$. If $x_{n,k}^o=1$, a physical link exists between $O{S}_n$ and $R{S}_k$; otherwise, $x_{n,k}^o=0$. Similarly, $x_{k,k^{\prime }}^r\in \{0,1\}$ represents the link indicator between relay satellites $R{S}_k$ and $R{S}_{k^{\prime }}$, where $x_{k,k^{\prime }}^r=1$ indicates that a link exists, and $x_{k,k^{\prime }}^r=0$ indicates no link. The satellite-to-ground network topology is assumed to change with each time slot, yet it remains constant within a single time slot. To describe the satellite-to-ground link status, $x_{m,k,t}^i\in \{0,1\}$ is used to indicate the existence of a physical link between the IoT device $I{D}_m$ and the relay satellite $R{S}_k$ at time slot t. Here, $x_{m,k,t}^i=1$ indicates a link exists, otherwise $x_{m,k,t}^i=0$. Likewise, $x_{k,t}^g\in \{0,1\}$ serves as the indicator for a link between the relay satellite $R{S}_k$ and the ground station at time t, where $x_{k,t}^g=1$ symbolizes an existing link, and $x_{k,t}^g=0$ symbolizes its absence.

3.2. Task and Computation Model

Consider that the computational tasks for IoT devices arrive unpredictably during each time slot and conform to a Poisson distribution. Let ${\theta }_{m,t}$ signify the computational task that arrives at IoT device $I{D}_m$ during time slot t. This task ${\theta }_{m,t}$ is characterized as ${\theta }_{m,t}=(D_{m,t},F_{m,t})$, where $D_{m,t}={\lambda }_m\tau$ denotes the size of the task ${\theta }_{m,t}$, and $F_{m,t}$ indicates the CPU cycle count necessary to process each bit of data in the task ${\theta }_{m,t}$, where ${\lambda }_m$ signifies the typical task arrival rate for device $I{D}_m$.

Considering that IoT devices often have limited computational capabilities and constrained energy resources due to their compact size and design, they are not suited for processing computationally intensive tasks locally. Performing such tasks would result in significant delays and quickly deplete the device’s battery life. Therefore, rather than executing computational tasks themselves, IoT devices offload these tasks to relay satellites. Relay satellites possess enhanced processing power and energy availability, enabling them to handle tasks efficiently either through onboard computation or by forwarding tasks to ground stations for processing. This offloading strategy not only conserves the limited resources of IoT devices but also improves overall system performance by leveraging the capabilities of more powerful network nodes. Define $z_{m,t,k,t^{\prime }}\in \{0,1\}$ as the variable indicating the transmission of task ${\theta }_{m,t}$. A value of $z_{m,t,k,t^{\prime }}=1$ means that the task ${\theta }_{m,t}$ is sent to the relay satellite $R{S}_k$ during time slot $t^{\prime }$, while $z_{m,t,k,t^{\prime }}=0$ means it is not. Furthermore, let ${\lambda }_{m,t,k,t^{\prime }}^g\in \{0,1\}$ denote the decision variable for selecting the ground station offloading mode for task ${\theta }_{m,t}$. If ${\lambda }_{m,t,k,t^{\prime }}^g=1$, it signifies that the relay satellite $R{S}_k$ chooses the ground station mode for task ${\theta }_{m,t}$ during time slot $t^{\prime }$, whereas ${\lambda }_{m,t,k,t^{\prime }}^g=0$ indicates otherwise.

3.3. Transmission Service Model and Transmission Model

In the scenario outlined in this paper, observation satellites are tasked with continuously monitoring the ground and collecting data transmission services at the outset of system operations. Denote by $D_n^o$ the data collected by the observation satellite $O{S}_n$. Assume that, following data collection, an observation satellite $O{S}_n$ is required to relay the data to the ground station through relay satellites in the following time slots. Define ${\gamma }_{n,k,t}^o\in \{0,1\}$ as the selection variable for satellite transmission relays concerning observation satellites. A value of ${\gamma }_{n,k,t}^o=1$ indicates that the observation satellite $O{S}_n$ sends data to relay satellite $R{S}_k$ in time slot t, whereas ${\gamma }_{n,k,t}^o=0$ indicates no transmission. Let ${\gamma }_{n,k,k^{\prime },t}^r\in \{0,1\}$ represent the transmission selection variable between relay satellites. If ${\gamma }_{n,k,k^{\prime },t}^r=1$, relay satellite $R{S}_k$ forwards data $D_n^o$ to relay satellite $R{S}_{k^{\prime }}$ in time slot t; otherwise, ${\gamma }_{n,k,k^{\prime },t}^r=0$. Additionally, let ${\gamma }_{n,k,t}^g\in \{0,1\}$ be the variable for selecting the ground station transmission from relay satellites. If ${\gamma }_{n,k,t}^g=1$, the relay satellite $R{S}_k$ transmits data $D_n^o$ to the ground station in the time slot t; if not, then ${\gamma }_{n,k,t}^g=0$.

3.4. Link Transmission Model

Define $R_{m,k,t}^i$ as the data rate between the IoT device $I{D}_m$ and the satellite $R{S}_k$ at time t:

$$R_{m,k,t}^i=x_{m,k,t}^{i}B{log}_2\left(1+{\displaystyle \frac{P_m^i{G}_m{G}_k^r{L}_{m,k,t}^i{L}_{m,k,t}^{pt}}{{\sigma }^2}}\right)$$

(1)

Here, $P_m^i$ is the transmission power, $G_m$ and $G_k^r$ are antenna gains, $L_{m,k,t}^i$ is path loss, $L_{m,k,t}^{pt}$ is rain attenuation, and ${\sigma }^2$ is noise power. The antenna gains and free space path loss are critical components in the link budget analysis. They determine the signal-to-noise ratio (SNR) at the receiver, which in turn affects the achievable data rate as per Shannon’s capacity formula. Higher antenna gains improve the SNR, while larger path losses deteriorate it. According to Equation $\left(23\right)$ in [45], $L_{m,k,t}^i={\left(4\pi d_{m,k,t}^{i}f/c\right)}^2$ where c is the speed of light, $d_{m,k,t}^i$ is the distance, and f is the frequency.

Let $R_{k,t}^g$ represent the link rate between the relay satellite $R{S}_k$ and the ground station in time slot t. It is given by the following:

$$R_{k,t}^g=x_{k,t}^{g}B{log}_2\left(1+{\displaystyle \frac{P_k^{rg}{G}_k^{rg}{G}_r^g{L}_{k,t}^g{L}_{k,t}^{pt}}{{\sigma }^2}}\right)$$

(2)

where $P_k^{rg}$ is the relay satellite’s transmission power to the ground station, $G_k^{rg}$ is the transmission antenna gain, $G_r^g$ is the receiving antenna’s gain, $L_{k,t}^{pt}$ denotes the rain attenuation, and $L_{k,t}^g$ indicates the free space loss, which is defined as follows: $L_{k,t}^g={\left(4\pi d_{k,t}^{g}f/c\right)}^2$ where $d_{k,t}^g$ is the distance between $R{S}_k$ and the ground station during time slot t.

Let $R_{n,k}^o$ represent the link rate between the observation satellite $O{S}_n$ and the relay satellite $R{S}_k$, which is given by Equation $\left(4\right)$ in [46]:

$$R_{n,k}^o=x_{n,k}^o{\displaystyle \frac{G_n^o{G}_k^r{P}_n{L}_{n,k}^o}{k_s{T}_s{E}_b/N_0}}$$

(3)

where $G_n^o$ is the observation satellite’s transmission gain, $G_k^r$ is the relay satellite’s receiving gain, $P_n$ is the power of $O{S}_n$, $L_{n,k}^o={\left(4\pi d_{n,k}^{o}f/c\right)}^2$ is the free space loss between $O{S}_n$ and $R{S}_k$, $k_s$ is the Boltzmann constant, $T_s$ is the system noise temperature, $E_b$ is the energy per bit, and $N_0$ is the noise power spectral density. $d_{n,k}^o$ is the distance between $O{S}_n$ and $R{S}_k$.

$R_{k,k^{\prime }}^r$ specifies the rate between the relay satellites $R{S}_k$ and $R{S}_{k^{\prime }}$, which is given by the following:

$$R_{k,k^{\prime }}^r=x_{k,k^{\prime }}^r{\displaystyle \frac{G_k^r{G}_{k^{\prime }}^r{P}_k^r{L}_{k,k^{\prime }}^r}{k_s{T}_s{E}_b/N_0}}$$

(4)

where $G_k^r$ and $G_{k^{\prime }}^r$ are the relay satellite k’s transmission gain and $k^{\prime }$’s receiving gain, $P_k^r$ is the power for inter-satellite transmission, $L_{k,k^{\prime }}^r={\left(4\pi d_{k,k^{\prime }}^{r}f/c\right)}^2$ is the free space loss between $R{S}_k$ and $R{S}_{k^{\prime }}$. $d_{k,k^{\prime }}^r$ is the distance between $R{S}_k$ and $R{S}_{k^{\prime }}$.

4. Optimization Problem Modeling

This section first models the system’s cache queue and energy consumption then defines the system cost function based on the cache queue and energy consumption. Next, it considers constraints such as association and computation limitations to model the joint task offloading and resource scheduling problem as a long-term cost function minimization problem under these constraints.

The dynamic nature of LEO networks requires a flexible approach to resource scheduling. As the number of IoT devices and their corresponding tasks vary, our system dynamically schedules the transmission and computation resources to ensure efficient processing and transmission. Continuous assessment of the operational status of satellites and ground stations allows our framework to respond promptly to changes in network conditions. The quality of communication channels is monitored to optimize data transmission paths, ensuring reliable connectivity. The relationships between satellites are considered, allowing for adaptive scheduling based on the current inter-satellite network topology.

To effectively manage these dynamic factors, our resource scheduling strategy is designed to adapt in real time. By modeling the task offloading and computation scheduling processes as a Markov Decision Process (MDP), we enable our system to make informed decisions based on the current state of the network. This approach ensures that both the data transmission and computational tasks are scheduled efficiently, minimizing latency and maximizing throughput.

4.1. System Cache Queue Modeling

As the computational tasks for IoT devices appear unpredictably and are sent to relay satellites in successive time slots, it is necessary to model the computation task cache queue at the devices. Given that relay satellites might handle data transmission services, accept and transfer computational tasks, and perform onboard tasks in each time slot, it is important to model a queue at the relay satellites to illustrate the caching scenario. Inspired by the analysis and modeling in [47,48,49], we model the queueing as follows.

4.1.1. IoT Device Queues

Let $Q_{m,t}^i$ represent the length of the task queue for computations at the IoT device $I{D}_m$ during time slot t. The formula for updating the queue $Q_{m,t}^i$ is given by the following:

$$Q_{m,t+1}^i=min\left(Q_m^{i,\max },Q_{m,t}^i-\sum _{k=1}^K\sum _{t^{\prime }=1}^t{z}_{m,t,k,t^{\prime }}{D}_{m,t^{\prime }}+D_{m,t}\right)$$

(5)

where $Q_m^{i,\max }$ denotes the maximum allowable length of the cache queue for the IoT device $I{D}_m$ and ${\sum }_{k=1}^K{\sum }_{t^{\prime }=1}^t{z}_{m,t,k,t^{\prime }}{D}_{m,t^{\prime }}+D_{m,t}$ denotes the incoming data.

4.1.2. Relay Satellite Queues

Define $Q_{k,t}$ as the queue size for the relay satellite $R{S}_k$ at time t. The update equation for the queue at the satellite $R{S}_k$ is given by the following:

$$\begin{array}{cc}\hfill Q_{k,t+1}=min& \{Q_k^{max},Q_{k,t}+\sum _{n=1}^N{\gamma }_{n,k,t}^o{D}_n^o-\sum _{k^{\prime }\ne k}^K{\gamma }_{n,k^{\prime },t}^r{D}_n^o\hfill \\ \hfill & -\sum _{n=1}^N{\gamma }_{n,k,t}^g{D}_n^o+\sum _{m=1}^M\sum _{t^{\prime }=1}^{t-1}{\lambda }_{m,t^{\prime },t}^g{D}_{m,t^{\prime },t}^g\hfill \\ \hfill & -\sum _{m=1}^M\sum _{t^{\prime }=1}^{t-1}{y}_{m,t^{\prime },k,t}min\left({\displaystyle \frac{{\phi }_k\tau }{D_{m,t^{\prime }}{F}_{m,t^{\prime }}}},D_{m,t^{\prime },k,t}\right)\}\hfill \end{array}$$

(6)

Here, $Q_k^{max}$ denotes the upper limit of the queue for $R{S}_k$; $y_{m,t^{\prime },k,t}\in \{0,1\}$ is a binary task indicator for the satellite’s task execution. A value of 1 signals that $R{S}_k$ processes the task ${\theta }_{m,t^{\prime }}$ at time t. The symbol ${\phi }_k$ stands for the computational capacity of $R{S}_k$, and $D_{m,t^{\prime },k,t}$ indicates the workload associated with task ${\theta }_{m,t^{\prime }}$ for $R{S}_k$ at t. The workload update is defined as follows:

$$D_{m,t^{\prime },k,t+1}=D_{m,t^{\prime },k,t}-min\left({\displaystyle \frac{{\phi }_k\tau }{D_{m,t}{F}_{m,t}}},D_{m,t^{\prime },k,t}\right)$$

(7)

Initially, $D_{m,t^{\prime },k,t}$ for satellite $R{S}_k$ is determined by the following:

$$D_{m,t^{\prime },k,t}=\left\{\begin{array}{cc}0,\hfill & t<t^{″},and{z}_{m,t^{\prime },k,t^{″}}=1\hfill \\ D_{m,t^{\prime }},\hfill & z_{m,t^{\prime },k,t^{″}}=1and\sum _{\overline{t}=t^{″}}^t{y}_{m,t^{\prime },k,\overline{t}},t^{″}\le t\le \overline{t}.\hfill \end{array}\right.$$

(8)

Equation (8) implies that the task is considered empty when there are no incoming data from the IoT devices, and it reflects the total task size once task execution begins.

4.2. System Energy Consumption Model

Let $E_t^c$ denote the energy consumption for executing tasks at time t, comprising energy spent transmitting tasks from IoT devices to relay satellites, relay satellites computing tasks, and transmitting tasks to ground stations. The energy consumed by ground stations is neglected due to their ample power. $E_t^c$ is given by the following:

$$E_t^c=\sum _{m=1}^M\sum _{k=1}^K{E}_{m,t,k}^t+\sum _{m=1}^M\sum _{k=1}^K{E}_{m,t,k}^c+\sum _{m=1}^M\sum _{k=1}^K{E}_{m,t,k}^g$$

(9)

In Equation (9),

• $E_{m,t,k}^t=z_{m,t,k}{D}_{m,t}^o{P}_{m,t}^i/R_{m,t,k}^i$ is the energy to transmit a task ${\theta }_{m,t}$ from IoT device $I{D}_m$ to relay $R{S}_k$ at t;
• $E_{m,t,k}^c=y_{m,t,k}{ϵ}_k{\left(D_{m,t}^o/{\phi }_k\right)}^2$ is the energy for $R{S}_k$ to compute ${\theta }_{m,t}$ at t where ${ϵ}_k$ is the energy coefficient for $R{S}_k$;
• $E_{m,t,k}^g={\lambda }_{m,t,k}{D}_{m,t}^o{P}_{m,t}^g/R_{m,t,k}^g$ is the energy to send ${\theta }_{m,t}$ from $R{S}_k$ to ground stations at t.

Define $E_t^o$ as the energy needed for data transmission at time t, incorporating data transfer from observation satellites to relay satellites, among relay satellites, and from relay satellites to ground stations. The formula for $E_t^o$ is

$$E_t^o=\sum _{n=1}^N\sum _{k=1}^K{E}_{n,k,t}^o+\sum _{k=1}^K\sum _{k^{\prime }=1,k^{\prime }\ne k}^K{E}_{n,k,k^{\prime },t}^r+\sum _{n=1}^N\sum _{k=1}^K{E}_{n,k,t}^g$$

(10)

where the following holds:

• $E_{n,k,t}^o={\gamma }_{n,k,t}^o{D}_n^o{P}_n/R_{n,k}^o$ denotes the energy for observation satellite $O{S}_n$ to send the data $D_n^o$ to the relay satellite $R{S}_k$ at t;
• $E_{n,k,k^{\prime },t}^r={\gamma }_{n,k,k^{\prime },t}^r{D}_n^o{P}^r/R_{k,k^{\prime }}^r$ quantifies the energy for relay satellite $R{S}_k$ to transmit the data $D_n^o$ to another relay satellite $R{S}_{k^{\prime }}$ at t;
• $E_{n,k,t}^g={\gamma }_{n,k,t}^g{D}_n^o{P}^{rg}/R_{k,g}^g$ defines the energy for $R{S}_k$ to transmit the data $D_n^o$ to ground stations at t.

4.3. System Cost Model

The system cost function, denoted by U, takes into account both the task queue length and energy usage and is represented as follows:

$$U=\sum _{t=1}^T{U}_t=\sum _{t=1}^T{\mu }_1{E}_t+{\mu }_2{Q}_t$$

(11)

Here, $U_t$ indicates the cost at each time t. The energy consumed, $E_t=E_t^c+E_t^o$, includes computation and data transmission at time t, while $Q_t={\sum }_{m=1}^M{Q}_{m,t}^o+{\sum }_{k=1}^K{Q}_{k,t}$ reflects the system’s queue length at the same time. The parameters ${\mu }_1$ and ${\mu }_2$ prioritize energy and queue length, respectively.

4.4. Optimization Constraints

Considering transmission links, link availability, flow conservation, and other constraints, the optimization problem needs to satisfy the following constraints.

4.4.1. Transmission Link Constraints

This study assumes that, for any time t, each IoT device can link to only one relay satellite, which is expressed as follows:

$$\sum _{k=1}^K{z}_{m,t^{\prime },k,t}^i\le 1,\forall m,t^{\prime }<t$$

(12)

Similarly, at any given t, each observation satellite is restricted to a connection with a single relay satellite, which is expressed as follows:

$$\sum _{k=1}^K{\gamma }_{n,k,t}^o\le 1,\forall n,t$$

(13)

For the relay satellites, at any moment t, data transfer is limited to transmitting from one observation satellite. This is denoted by the following:

$$\sum _{k^{\prime }=1,k^{\prime }\ne k}^K{\gamma }_{n,k,k^{\prime },t}^r\le 1,\forall n,k,t$$

(14)

Further, at any point in time t, each relay satellite can only facilitate the transmission of data from one observation satellite within a single relay transfer:

$$\sum _{n=1}^N{\gamma }_{n,k,k^{\prime },t}^r\le 1,\forall k,k^{\prime }\ne k,t$$

(15)

Finally, each relay satellite is also constrained to transmitting data from a single observation satellite to the ground station. This is defined by the following:

$$\sum _{n=1}^N{\gamma }_{n,k,t}^g\le 1,\forall k,t$$

(16)

4.4.2. Link Availability

IoT devices can transmit computational tasks to relay satellites only when connectivity is established, which is defined by

$$\sum _{t^{\prime }=1}^{t-1}{z}_{m,t^{\prime },k,t}^i·1_{\{x_{m,k}^i=0\}}=0,\forall m,k,t$$

(17)

where $1_{\left\{x\right\}}$ is an indicator function that returns 1 if x is true and 0 otherwise. Similarly, relay satellites can relay data to ground stations only with an active link:

$$\sum _{n=1}^N{\gamma }_{n,k,t}^g·1_{\{x_{k,g}^g=0\}}=0,\forall k,t$$

(18)

$$\sum _{t^{\prime }=1}^{t-1}{\lambda }_{m,t^{\prime },k,t}^g·1_{\{x_{k,g}^g=0\}}=0,\forall k,t$$

(19)

For data transfer from observation satellites to relay satellites, a valid connection is necessary:

$$\sum _{t=1}^T{\gamma }_{n,k,t}^o·1_{\{x_{n,k}^o=0\}}=0,\forall n,k$$

(20)

Finally, data can only be exchanged between relay satellites if they are interconnected:

$$\sum _{t=1}^T{\gamma }_{n,k,k^{\prime },t}^r·1_{\{x_{k,k^{\prime }}^r=0\}}=0,\forall k,k^{\prime },k\ne k^{\prime }$$

(21)

The term ‘x’ serves as an indicator of link availability between IoT devices and satellites. When $x=0$, it signifies that the link is not available; thus, the transmission variables must be set to zero, reflecting the inability to transmit data. This is supported by the indicator function, which will yield a value of 1 under these circumstances. Conversely, if $x=1$, which indicates an available link, the transmission variables can assume non-zero values, allowing for data communication. This logical structure ensures that our model accurately represents the conditions of link availability within the network.

4.4.3. Task Computation

Given that the relay satellite initiates computing tasks only when all tasks have been transmitted and that they process just one task in each time slot, we have

$$y_{m,t^{\prime },k,t}·1_{\{t<t^{″}·z_{m,t^{\prime },k,t^{″}}^i\}}=0,\forall m,t^{\prime },k,t,t^{″},t^{\prime }<t^{″}<t$$

(22)

$$\sum _{m=1}^M\sum _{t^{\prime }=1}^T{y}_{m,t^{\prime },k,t}\le 1,\forall k,t,t>t^{\prime }$$

(23)

4.4.4. Flow Conservation

In the transmission of user data via inter-satellite and satellite-to-ground communications, it is essential to adhere to flow conservation laws for both observation and relay satellites. The flow conservation constraint for an observation satellite $O{S}_n$ is defined by the following:

$$\sum _{k=1}^K\sum _{t=1}^T{\gamma }_{n,k,t}^o=1,\forall n$$

(24)

Equation (24) ensures that each observation satellite $O{S}_n$ (where n is the index of the satellite) must transmit its data to exactly one relay satellite $R{S}_k$ within the time frame $[1,T]$. The summation over k (relay satellites) and t (time slots) ensures that the total number of transmissions initiated by each observation satellite $O{S}_n$ equals 1. This reflects the principle that each observation satellite must transmit its data to exactly one relay satellite during the defined time horizon, maintaining data flow from observation satellites to relay satellites.

For a relay satellite $R{S}_k$ handling the data $D_n^o$, the constraint is formulated as follows:

$$\sum _{t=1}^T{\gamma }_{n,k,t}^o+\sum _{k^{\prime }=1,k^{\prime }\ne k}^K\sum _{t=1}^T{\gamma }_{n,k,k^{\prime },t}^r=\sum _{k^{\prime }=1,k^{\prime }\ne k}^K\sum _{t=1}^T{\gamma }_{n,k^{\prime },k,t}^r+\sum _{t=1}^T{\gamma }_{n,k,t}^g,\forall n,k$$

(25)

Equation (25) enforces the principle of flow conservation at each relay satellite $R{S}_k$ for data originating from a given observation satellite $O{S}_n$. It ensures that the total incoming data flow into the relay satellite $R{S}_k$ equals the total outgoing data flow from $R{S}_k$, preserving the continuity of the data flow within the network. The left-hand side represents the total incoming flow into the relay satellite $R{S}_k$, which includes data directly transmitted from the observation satellite and relayed from other relay satellites. The right-hand side represents the total outgoing flow from the relay satellite $R{S}_k$, which includes data relayed to the other relay satellite and transmitted to the ground station.

Equations (24) and (25) embody the principle of flow conservation, stating that for any given node in the network, the total data inflow must equal the total data outflow. This principle is vital for maintaining the integrity and efficiency of data transmission within the satellite network. By applying this conservation principle, we ensure that our model accurately tracks data movement and resource utilization across the network, thereby facilitating optimal routing and task offloading strategies.

4.5. Optimization Model

Considering constraints such as link availability and computation limitations, the optimization model for the joint dynamic task offloading and transmission scheduling problem based on the long-term cost function minimization under the constraints is the following:

$$\begin{array}{ccc}\hfill & \underset{\mathrm{X}}{min}\hfill & \hfill \underset{T\to \infty }{lim}{\displaystyle \frac{1}{T}}\mathbb{E}\left[\sum _{t=1}^T{U}_t\right]\\ \hfill & s.t.\hfill & \hfill \left(12\right)–\left(25\right)\end{array}$$

(26)

where $\mathrm{X}=\{{\gamma }_{n,k,t}^o,{\gamma }_{n,k,k^{\prime },t}^r,{\gamma }_{n,k,t}^g,z_{m,t^{\prime },k,t}^i,{\lambda }_{m,t^{\prime },k,t}^g,y_{m,t^{\prime },k,t}\}$ is the set of all variables.

5. Proposed Schemes for Joint Task Offloading and Transmission Scheduling

Solving the long-term cost minimization problem $P1$ modeled in (26) involves dynamic task offloading and computation scheduling strategies for observation satellite data tasks, IoT device computation tasks, and inter-satellite network topology stability. The original optimization problem is divided into data task transmission scheduling and computation task offloading and scheduling problems. For the data task transmission scheduling problem, a transmission scheduling strategy based on Dijkstra’s algorithm [50] is proposed. For the dynamic task offloading and scheduling problem, considering the Markov decision process, a deep learning method is adopted to propose a dynamic task offloading and scheduling strategy based on DQN.

5.1. Data Task Transmission Scheduling

This section models and addresses the data task transmission scheduling issue, aiming to devise a transmission strategy. Assuming a static inter-satellite network topology, the problem is simplified to a directed shortest path with weighted edges. Dijkstra’s algorithm is utilized to find the shortest routes from observation satellites to the ground station. To solve the capacity limitations of relay satellites, an enhanced scheduling strategy is introduced. Excluding conflicts among relay satellites, a scheduling strategy for observation satellites maps the data transmission route to ground stations. The link weights, determined by energy consumption, help model this as a directed shortest path problem. Let $W_n$ be the cumulative energy expenditure from $O{S}_n$ to the ground station, which is represented by the following:

$$W_n=\sum _{k=1}^K\sum _{t=1}^T{\gamma }_{n,k,t}^o{W}_{n,k}^o+\sum _{k=1}^K\sum _{k^{\prime }=1,k^{\prime }\ne k}^K\sum _{t=1}^T{\gamma }_{n,k,k^{\prime },t}^r{W}_{n,k,k^{\prime }}^r+\sum _{k=1}^K\sum _{t=1}^T{\gamma }_{n,k,t}^g{W}_{n,k,t}^g$$

(27)

Here, $W_{n,k}^o$ denotes the link weight between the observation satellite $O{S}_n$ and relay satellite $R{S}_k$, $W_{n,k,k^{\prime }}^r$ the link weight for the data $D_n^o$ between the relay satellites $R{S}_k$ and $R{S}_{k^{\prime }}$, and $W_{n,k,t}^g$ the link weight for the data $D_n^o$ from the relay satellite $R{S}_k$ to the ground station at time t. These weights are computed as follows.

$$W_{n,k}^o=\left\{\begin{array}{cc}{\displaystyle \frac{D_n^o{P}_n}{R_{n,k}^o}},\hfill & x_{n,k}^o=1\hfill \\ +\infty ,\hfill & x_{n,k}^o=0\hfill \end{array}\right.,W_{n,k,k^{\prime }}^r=\left\{\begin{array}{cc}{\displaystyle \frac{D_n^o{P}_k^r}{R_{k,k^{\prime }}^r}},\hfill & x_{k,k^{\prime }}^r=1\hfill \\ +\infty ,\hfill & x_{k,k^{\prime }}^r=0\hfill \end{array}\right.,W_{n,k,t}^g=\left\{\begin{array}{cc}{\displaystyle \frac{D_n^o{P}_k^g}{R_{k,g}^g}},\hfill & x_{k,g}^g=1\hfill \\ +\infty ,\hfill & x_{k,g}^g=0\hfill \end{array}\right.$$

(28)

From (28), it is observed that greater transmission energy reduces link weight. The shortest path problem between the observation satellite $O{S}_n$ and the ground station is modeled as follows:

$$\underset{{\gamma }_{n,k,t}^o,{\gamma }_{n,k,k^{\prime },t}^r,{\gamma }_{n,k,t}^g}{min}{W}_n,s.t.\left(13\right)–\left(16\right),\left(18\right),\left(20\right),\left(21\right),\left(24\right),\left(25\right)$$

(29)

The inter-satellite link’s state over time is modeled by representing the ground station as T nodes, $G{S}_t$, with $1\le t\le T$. At each time t, link weights to relay satellites depend on link conditions. Using Dijkstra’s algorithm [50] for shortest path calculations, super nodes are employed, and virtual ground station nodes have link weights set to zero. Specifically, to model the time-varying nature of the satellite–ground links, we represent the ground station as a set of virtual nodes $G{S}_t$ for each time slot t. Each virtual ground station node $G{S}_t$ corresponds to the ground station at time t, capturing the link availability and conditions at that specific time. To facilitate the application of Dijkstra’s algorithm, we introduce a ‘super node’. The super node serves as a single common destination node and is connected to all virtual ground station nodes $G{S}_t$ with edges of zero weight. This construction ensures that the shortest path algorithm can effectively consider all possible paths reaching the ground station over different time slots.

The graph $G_n=(V,E,W_n)$ is constructed, where V includes observation satellites $O{S}_n$, relay satellites, and virtual ground station nodes; E consists of satellite–ground station links, and $W_n$ contains link weights. All link weights from ground stations to super nodes are zero. By constructing the time-expanded graph $G_n=(V,E,W_n)$, we capture the dynamic topology of the inter-satellite network over time. The vertices V include the observation satellite $O{S}_n$, relay satellites $R{S}_k$ at different time slots, virtual ground station nodes $G{S}_t$ representing the ground station at time t, and the super node serving as the common destination. Edges E represent possible transmissions between nodes at consecutive time slots, with weights Wn indicating the transmission costs. We apply Dijkstra’s algorithm to this graph to find the shortest path from $O{S}_n$ to the super node. This path denotes the optimal sequence of transmissions through relay satellites and time slots, minimizing the overall transmission cost to the ground station.

Figure 2 illustrates a schematic of the link formed by one observation satellite $O{S}_n$ and six relay satellites. Applying Dijkstra’s algorithm to the augmented graph $G_n$ identifies the shortest path between $O{S}_n$ and the virtual ground station nodes. Let ${\tilde{\pi }}_{n,t}^o=\{{\tilde{\gamma }}_{n,k,t}^o,...,{\tilde{\gamma }}_{n,k,k^{\prime },t}^r,...,{\tilde{\gamma }}_{n,k,t}^g,1\le t^{\prime }\le t\}$ represent the data set $D_n^o$ from $O{S}_n$ to $G{S}_t$. Due to multiple hops and the dynamic nature of the inter-satellite link, the original transmission strategy may become infeasible, requiring adjustments. The strategy set ${\tilde{\pi }}_{n,t}^o$ is checked for conflicts. If the maximum conflict time $t_1=argmax\left\{{\tilde{\gamma }}_{n,k,k^{\prime },t}^r\right\}$ of the relay satellite transmission strategy exceeds t, the strategy is invalidated; if $t_1<t$, the strategy set ${\tilde{\pi }}_{n,t}^o$ is retained. The strategy with the minimum weight is selected as the local transmission scheduling strategy set ${\tilde{\pi }}_{n,t^{*}}^o$ for $O{S}_n$.

In the context of independently designing transmission scheduling strategies for observation satellites, instances may arise where two distinct data tasks commence transmission from a similar origin and are destined for the same endpoint concurrently. Such scenarios precipitate a transmission scheduling conflict. To address this issue, the following section introduces an algorithm capable of adjusting transmission strategies through prioritization. For illustrative purposes, consider the transmission strategies ${\tilde{\gamma }}_{n,k,k^{\prime },t}^r$ and ${\tilde{\gamma }}_{n^{\prime },k,k^{\prime },t}^{\prime r}$ associated with the conflicting data $D_n^o$ and $D_n^{\prime o}$, respectively. The algorithm’s fundamental concept is developed as follows:

• Assess the Priority of Present Data Tasks:At the current time, evaluate the transmission energy expended by both conflicting data tasks, assuming that the task with greater accumulated transmission energy is accorded higher priority. Define $E_{n,t^{\prime }-1}$ as the total transmission energy accumulated by the data task $D_n^o$ preceding time $t^{\prime }$. The expression for $E_{n,t^{\prime }-1}$ is given by the following:

  $$E_{n,t^{\prime }-1}=\sum _{k=1,t=1}^{K,t^{\prime }-1}{E}_{n,k,t}^o+\sum _{{\displaystyle \genfrac{}{}{0pt}{}{k=1,k^{\prime }=1}{k^{\prime }\ne k,t=1}}}^{K,K,t^{\prime }-1}{E}_{n,k,k^{\prime },t}^r$$

  (30)
  The accumulated energy $E_{n,t^{\prime }-1}^o$ in Equation (30) quantifies the total energy expended for transmitting data task $D_n^o$ up to time $t^{\prime }-1$, considering both observation-to-relay and relay-to-relay transmissions. Specifically, for conflicting data tasks $D_n^o$ and $D_n^{\prime o}$, if $E_{n,t^{\prime }-1}^o>E_{n^{\prime },t^{\prime }-1}$, the data task $D_n^o$ is granted higher priority (i.e., the one that has already consumed more energy up to time $t^{\prime }$). This metric is crucial for prioritizing tasks during scheduling conflicts as tasks with higher accumulated energy are considered to have greater investment and are, therefore, prioritized to minimize system cost and resource waste.
• Modify Transmission Strategy in Accordance with Priority:The aforementioned data task $D_n^o$, enjoying higher precedence, will implement the strategy encapsulated in ${\tilde{\pi }}_{n,t^{*}}^o$. Here, ${\tilde{\pi }}_{n,t}^{o,*}$ denotes the designated set of strategies for transmitting the data task $D_n^o$. Consequently, the optimal scheduling strategy set is ${\tilde{\pi }}_n^{o,*}={\tilde{\pi }}_n^o$. For the data task $D_n^{\prime }$ possessing a subordinate priority, it is imperative to revert to the observation satellite $O{S}_n$ and reformulate the transmission scheduling strategy, as detailed in the Dijkstra-based algorithm developed above. To circumvent subsequent conflicts, careful consideration must be given to arranging the set containing strategy ${\tilde{\gamma }}_{n,k,k^{\prime },t^{\prime }}^r$, ultimately achieving the refined strategy set ${\tilde{\pi }}_{n^{\prime },t^{*}}^{o,*}$.

5.2. Dynamic Task Offloading and Computation Scheduling Problem Modeling and Solving

With the transmission scheduling strategy for data tasks established, the relay satellite’s buffer queue status is known for each time slot. Due to the relay satellite’s limited buffer capacity, dynamic satellite–ground links, and the random arrival of IoT device computation tasks, the task offloading and computation scheduling problem becomes highly complex and dynamic.

This complexity makes traditional optimization methods less effective, as they may not efficiently handle the high-dimensional and stochastic nature of the problem. Therefore, we model the problem as a Markov Decision Process (MDP). By formulating the problem as an MDP, we can employ reinforcement learning techniques to find optimal policies. Specifically, we utilize the Deep Q-Network (DQN) algorithm, which integrates deep learning with Q-learning. DQN is adept at estimating the state–action value function in high-dimensional spaces, making it suitable for our model. The algorithm leverages deep neural networks to approximate the optimal Q-values and incorporates experience replay and target networks to enhance learning stability and efficiency.

The application of DQN enables the agent (relay satellite) to learn optimal task offloading and computation scheduling strategies through interactions with the environment, ultimately minimizing the long-term system cost in the face of uncertainty and dynamic conditions.

Consider the function ${\tilde{U}}_t$ as indicative of the cost associated with the relay satellite’s activities in managing the offloading of tasks and scheduling of computational procedures at time instance t. The function ${\tilde{U}}_t$ is defined mathematically as follows:

$${\tilde{U}}_t={\mu }_1{E}_t+{\mu }_2{Q}_t$$

(31)

The challenge surrounding the optimization model for dynamic task offloading and computation scheduling is formulated by the following:

$$\underset{z_{m,t^{\prime },k,t}^i,{\lambda }_{m,t^{\prime },k,t}^g,y_{m,t^{\prime },k,t}}{min}\underset{T\to \infty }{lim}{\displaystyle \frac{1}{T}}\mathbb{E}\left[\sum _{t=1}^T{\tilde{U}}_t\right],s.t.\left(12\right),\left(17\right),\left(19\right),\left(22\right),\left(23\right)$$

(32)

Through solving the above-stated Equation (32), it becomes possible to determine the strategic approach for selecting computation modes by the relay satellite, denoted as $z_{m,t^{\prime },k,t}^i$. Additionally, this leads to the identification of strategies for offloading mode selection by the ground station, represented by ${\lambda }_{m,t^{\prime },k,t}^g$, along with determining the computation scheduling strategy denoted by $y_{m,t^{\prime },k,t}$.

5.2.1. Markov Decision Process Modeling

When we closely examine Equation (32), it becomes apparent that we are dealing with a long-term optimization challenge that incorporates randomness through its variables. To effectively address this problem, it is structured as a Markov decision process, which is mathematically characterized by the tuple $(S,A,r_t)$. Here, S signifies the array of system states, $A=\left\{a_t\right\}$ signifies the suite of potential actions, and $r_t$ stands for the immediate reward function associated with the process [51]. The constituents of the state space, action space, and reward function within this context are further detailed as follows. Let $S=\left\{s_t\right\}$ denote the array of system states at a given time t, with each state $s_t$ conceptualized as follows:

$$s_t=\{Q_t,Q_t^i,L_r^{pt},x_t^i\}$$

(33)

In this expression, $Q_t=\{Q_{1,t},Q_{2,t},\dots ,Q_{k,t},\dots ,Q_{K,t}\}$ captures the buffer queue conditions for each relay satellite at time t; $Q_t^i=\{Q_{1,t}^i,Q_{2,t}^i,\dots ,Q_{m,t}^i,\dots ,Q_{M,t}^i\}$ reflects the buffer queue status for each IoT device during the same period; $L_t^{pt}=\{L_{m,k,t}^{pt},1\le m\le M,1\le k\le K\}$ indicates the rain attenuation coefficient relevant to the communication link connecting each IoT device and each relay satellite at time t; and $x_t^i=\{x_{m,k,t}^i,1\le m\le M,1\le k\le K\}$ depicts the status of logical link existence between each IoT device and each relay satellite at this time instance. At time t, IoT devices have the capability to transfer computation tasks stored within their buffer queues to relay satellites. Accordingly, let $A=\left\{a_t\right\}$ denote the comprehensive set of actions available at time t, where each action $a_t$ encapsulates the strategies for selecting task offloading modes by IoT devices at that specific moment and can be expressed as follows:

$$a_t=\{z_{m,t^{\prime },k,t},{\lambda }_{m,t^{\prime },k,t}^g,y_{m,t^{\prime },k,t},1\le m\le M,1\le t^{\prime }<t,1\le k\le K\}$$

(34)

Under the scenario wherein IoT devices occupy state $s_t$ and undertake action $a_t$, the resulting immediate reward $r_t$ is meticulously designed as follows:

$$r_t=-{\mu }_1\left(\sum _{k=1}^K{Q}_{k,t}+\sum _{m=1}^M{Q}_{m,t}^i\right)-{\mu }_2{E}_t$$

(35)

5.2.2. Determining Satellite Dynamic Link Strategy Based on DQN

Figure 3 illustrates the developed framework for task offloading and scheduling. The algorithm of DQN, introduced by the DeepMind group, is a reinforcement learning algorithm integrating deep learning principles widely used in current complex satellite networks design [19,30,52,53,54]. Unlike traditional Q-learning, DQN is adept at managing complex state spaces characterized by high dimensionality. In the realm of Q-learning, such state spaces often pose dimensionality challenges, impeding effective learning. DQN circumvents these limitations by employing deep neural networks as function approximators, which allows for the efficient handling of high-dimensional spaces by representing states as continuous variables, thus enhancing learning efficacy. Moreover, conventional Q-learning mandates a discrete action space, thereby restricting its utility in scenarios necessitating continuous action spaces. DQN addresses this limitation by leveraging deep neural networks to estimate the Q-values for each possible action, thus enabling the handling of continuous action spaces and the selection of the optimal action via optimization techniques [55]. The integration of an experience replay mechanism permits the storage of the agent’s experiences in a replay buffer, from which samples are randomly extracted for training. This approach alleviates the correlation between successive training data, thereby mitigating the sample correlation issue and bolstering training efficiency and stability. Consequently, the DQN method is proficient at maximizing interactions between intelligent agents and stochastic environments. The ‘Environment’ encapsulates the temporal dynamics that influence the state, allowing the model to maintain a distinction between the current conditions and the actions taken, until such action results in a transition to a ‘Next State’.

Figure 3 illustrates the iterative process of the Deep Q-Network (DQN) learning cycle. Although the figure does not explicitly show an entry point, the process naturally begins with the first observation of the environment, denoted as the initial state $s_0$. This initial state is used to determine the first action $a_0$, initiating the sequence of interactions between the agent and the environment. From this point forward, the system continuously updates states and actions based on learned policies. This design aligns with conventional DQN representations, where the emphasis is on the continuous reinforcement learning process rather than a discrete starting point.

During the training phase of the DQN (Deep Q-Network) algorithm, the strengths of both convolutional and recurrent neural networks are harnessed to proficiently estimate the state–action value function, known as the Q-function. The architecture of DQN comprises two distinct networks: the prediction network tasked with generating experience replay data, and the target network, which computes the state–action value, referred to as the Q-value. To expedite convergence, DQN employs an experience replay strategy, which involves the random sampling of accumulated data for training, alongside a smoothing technique for managing the data effectively. When IoT devices execute actions during the state $s_t$, the associated action value function Q is updated as follows:

$$Q_{t+1}(s,a)=Q(s,a)+\alpha \left[r\left(a_t\right|s_t)+\gamma \underset{a^{\prime }}{max}{Q}_t(s^{\prime },a^{\prime })-Q_t(s,a)\right]$$

(36)

In Equation (36), $\alpha$ denotes the learning rate, and $\gamma$ represents the discount factor. To tackle the computational intricacies of Equation (36), and prevent divergence of Q-values, DQN introduces a prediction network $Q(s,a;\theta )$ and a target network $Q(s,a;{\theta }^{\prime })$, which are responsible for predicting and assessing the Q-values, respectively. Herein, $\theta$ and ${\theta }^{\prime }$ are the parameters of the prediction network and target network. The parameters $\theta$ are updated by reducing the loss function:

$$\theta \leftarrow \theta +\nu \nabla F_t\left(Q(s,a|*)\right)$$

(37)

where $\nu$ signifies the step size, and $F_t\left(Q(s,a|*)\right)$ denotes the Q-value loss function, which is expressed as follows:

$$F_t\left(Q(s,a|*)\right))={\mathbb{E}}_{(s_t,a_t,r_t,s_{t+1})\sim U\left(D\right)}\left[{\left(y_t-Q(s,a;\theta )\right)}^2\right]$$

(38)

In Equation (38), $(s_t,a_t,r_t,s_{t+1})\sim U\left(D\right)$ symbolizes samples randomly extracted from the experience replay buffer D, and $y_t$ is the output generated by the target network at time t, which is given by the following:

$$y_t=r_t+\gamma \underset{a^{\prime }}{max}{Q}^{\prime }(s^{\prime },a^{\prime };{\theta }^{\prime })$$

(39)

The detailed procedures for the computation task offloading and scheduling algorithm, rooted in the DQN learning algorithm for interactions between satellite and IoT devices, are outlined in Algorithm 1. Begin by initializing the prediction and target networks alongside the state, action, and parameters within the experience replay buffer. Within the state $s_t$, implement the $ϵ$-greedy policy to identify the optimal action $a_t$ for the target network, facilitating the transition of the system state from $s_t$ to $s_{t+1}$. Record the experience tuple $(s_t,a_t,r_t,s_{t+1})$ in the experience replay buffer, whose maximum capacity is denoted as $D^{\max }$. Select a random batch of samples with size $D^{\mathrm{batch}}$ from this buffer and feed them into the prediction network to generate predicted Q-values. The target Q-values are computed in line with Equation (36). Then, determine the discrepancy between the predicted and target Q-values to update the parameters of the prediction network via Equation (37). The target network parameters are periodically revised utilizing the prediction network’s parameters. Iterate these steps until convergence is reached.

We define ‘resource scheduling’ in the context of our study as the strategic scheduling and management of computational and communication resources within the satellite network. This encompasses both data scheduling, which prioritizes data packet transmission, and task offloading, which involves distributing computational tasks among various network nodes. Although our primary focus is on optimizing data scheduling paths and task offloading strategies, these actions are intrinsically linked to efficient resource utilization, ensuring that the network operates optimally in conjunction with its available resources.

Algorithm 1 DQN-Based Task Offloading and Scheduling

1: Initialize prediction network $Q(s,a;\theta )$ and target network $Q^{\prime }(s,a;{\theta }^{\prime })$ 2: Initialize experience replay buffer D with capacity $D^{\max }$ 3: for each episode do 4:    Initialize state $s_t$ 5:    for each step in episode do 6:      Select action $a_t$ using $ϵ$-greedy policy 7:      Execute action $a_t$ and observe reward $r_t$ and next state $s_{t+1}$ 8:      Store $(s_t,a_t,r_t,s_{t+1})$ in D 9:      Sample random batch of size $D^{\mathrm{batch}}$ from D 10:      Compute target $y_t=r_t+\gamma {max}_{a^{\prime }}{Q}^{\prime }(s_{t+1},a^{\prime };{\theta }^{\prime })$ 11:      Compute loss $F_t=\mathbb{E}\left[{(y_t-Q(s_t,a_t;\theta ))}^2\right]$ 12:      Update prediction network parameters $\theta \leftarrow \theta +\nu {\nabla }_{\theta }{F}_t$ 13:      if step mod update_frequency == 0 then 14:         Update target network parameters ${\theta }^{\prime }\leftarrow \theta$ 15:      end if 16:      $s_t\leftarrow s_{t+1}$ 17:    end for 18: end for

6. Simulation Results and Analysis

This section uses the STK simulation software version 6.0 to build a system scenario in order to obtain real satellite and inter-satellite physical link information, apply Pytorch 2.2.2 and MATLAB 2024a to simulate the algorithms proposed in this paper and the algorithm in the literature [40], and analyze the performance differences between the two. This paper presents a novel cost model for an LEO satellite edge computing system, addressing the challenges of computation offloading. It introduces a joint computation offloading and resource allocation strategy, which decomposes the problem into two sub-problems: optimizing computation offloading using game theory and allocating communication resources via the Lagrange multiplier method. Ref. [40] is selected as a baseline due to its relevance to IoT applications and its use of energy consumption as a cost metric, which aligns with our study’s objectives. However, our approach differs significantly in methodology, as we employ a DQN-based optimization rather than the game-theoretic and Lagrange multiplier methods used in [40]. To enable a fair comparison, we replace our proposed DQN-based optimization algorithm with the JCORA method from [40] while keeping all other simulation parameters unchanged. This allows us to directly compare the performance impact of different optimization strategies under the same system conditions.

Simulation Scenario

In the simulation scenario of this paper, a LEO satellite network is built using the STK satellite tool (version 6.0), which includes 24 medium-orbit satellites distributed in three orbital planes with an altitude of 1000 km, 10 observation satellites, and one ground station. This section uses MATLAB (version 2024a) the Python programming language (version 3.8) and a Pytorch (version 2.1.2) simulation environment based on the Gym reinforcement learning environment to simulate the DQN network. During the simulation process, the number of training steps is set to 4000 steps, and the number of training episodes is set to 30 h. The parameters of the DQN network and other parameters are shown in

Table 2. Parameter settings for DQN network and simulation.

Parameter	Value
Learning Rate $\alpha$	0.001
Discount Factor $\gamma$	[0.1, 0.5, 0.9]
Experience Replay Buffer Capacity $D^{\max }$	10,000
Sample Size $D^{\mathrm{batch}}$	64
Exploration Value $ϵ$	[0.1, 1]
Link Bandwidth B	20 MHz
Arrival Rate	[0.8, 2] Mbps
${\mu }_1$ and ${\mu }_2$	[1, 3]
Rain Attenuation $L_{m,k,t}^{\mathrm{rain}}$	[10, 20] dB
Boltzmann Constant $k_s$	$1.380649\times {10}^{-23}$ J/K
Thermal Noise Temperature $T_s$	300 K
Signal-to-Noise Ratio $E_b/N_0$	17 dB
Relay Satellite Transmission Power $P^r$	40 W
Relay Satellite Transmission Antenna Gain $G_t^{\mathrm{rg}}$	27 dBi
Relay Satellite Reception Antenna Gain $G_r^{\mathrm{rg}}$	24 dBi
Observation Satellite Transmission Power $P_n$	40 W
IoT Device Transmission Power $P_m^i$	0.1 W
Time Slot Length T	0.2 s

. This section averages the simulation results of 600 independent experiments and compares the proposed algorithm with the algorithm in the literature.

Figure 4 and Figure 5 show the impact of different learning rates and discount factors on long-term rewards. As shown in the figure, a learning rate of 0.1 is very clear that the DQN network does not converge and fluctuates significantly. This is because the learning rate is too large, and the network jumps directly to the local optimal value, resulting in suboptimal results. When the learning rate is 0.0001, the network converges slowly, and even after 400 training steps, there is still a large fluctuation. This is because, during the learning process, the network converges slowly and requires more training steps to reach the final reward. When the learning rate is 0.001, the network can reach the optimal reward after 1000 steps and maintain the reward through continuous training.

Figure 5 illustrates the impact of different discount factors $\gamma$ on the long-term rewards during the training of our DQN-based task offloading and computation scheduling algorithm. The discount factor is a critical parameter in reinforcement learning algorithms, determining how future rewards are valued relative to immediate rewards. When $\gamma =0.1$, the agent exhibits slow convergence with high fluctuations in rewards. This low discount factor implies that the agent undervalues future rewards, leading to shortsighted decisions that do not contribute to long-term system optimization. For $\gamma =0.5$, the agent shows improved convergence and reduced fluctuations. The balance between immediate and future rewards allows the agent to learn better policies but still falls short of optimal performance. With $\gamma =0.9$, the agent achieves rapid convergence and higher steady-state rewards. The high emphasis on future rewards enables the agent to learn strategies that significantly minimize the long-term system cost. This result indicates that a higher discount factor is beneficial in the context of our satellite edge computing network, where long-term planning is essential due to the dynamic and continuous nature of tasks and resources.

In Figure 6, we illustrate the connection between the relay satellite’s maximum queue length and the system cost, alongside analysis of varied scenarios concerning task computation difficulty. The figure reveals that as the relay satellite’s maximum queue length grows, the system cost declines. This occurs because a longer queue allows the satellite to manage more data transmission tasks and adjust computation task scheduling, enhancing the system’s capacity to prevent data loss and stabilize cost fluctuations. Notably, the system cost achieved by our proposed method in this study is lower than that reported in prior research, and with a task computation difficulty of 1000 cycles, the cost reduction using our approach is slower compared to the method from the literature [40].

Figure 7 presents the correlation between the relay satellite’s computation capability and system cost. It demonstrates that as the satellite’s computation capability increases, the system cost decreases unevenly across various methods. Figure 7 demonstrates the relationship between the computational capability of relay satellites and the system cost in the proposed LEO satellite edge computing network. As the computational capability of relay satellites increases, the system cost decreases significantly due to improved task processing efficiency and reduced communication overhead. However, the curve exhibits a diminishing return effect, where further increases in computational capability yield only marginal reductions in system cost. This highlights the importance of balancing computational resources to achieve cost-effective performance. The figure underscores the efficiency of the proposed joint task offloading and resource scheduling strategy in optimizing system performance while minimizing costs.

Furthermore, an increase in IoT device numbers also results in uneven decreases in system costs. This is due to the reduction in task computation time as satellite computation improves, allowing the task queue to remain stable within a reasonable timeframe, thus lowering costs. Moreover, a rise in device data quantity leads to more tasks for the system to process, adding to the costs. The cost via our proposed methodology notably surpasses that of previous approaches [40], and with enhanced satellite computation, our method’s cost increase is slower than that of existing methods, underscoring the benefits of our approach.

Figure 8 evaluates the relationship between the average task arrival rate and system cost under different numbers of IoT devices for the proposed algorithm in this chapter and the algorithm from [40]. As shown in the figure, as the task arrival rate increases, the system cost decreases. This is because a higher task arrival rate leads to greater utilization of the network’s computational, communication, and caching resources, resulting in increased system costs. Moreover, the system cost achieved by the proposed algorithm in this chapter is lower than that of the algorithm in [40]. Furthermore, as the task arrival rate increases, the cost under the proposed algorithm increases at a relatively slower rate.

The system cost metric used in this study incorporates energy consumption, task processing delays, and other operational factors. By minimizing system costs, the proposed method achieves significant energy savings in addition to reducing delays and improving task completion rates. This holistic evaluation provides a comprehensive assessment of the system’s performance.

7. Conclusions

This paper has comprehensively addressed the joint dynamic task offloading and resource scheduling problem in LEO satellite edge computing networks. The proposed system model incorporates both data service transmission and computational task offloading and is framed as a long-term cost function minimization problem with constraints. Key contributions include the development of a priority-based policy adjustment algorithm for handling transmission scheduling conflicts and a DQN-based algorithm for dynamic task offloading and computation scheduling. These methods are integrated into a joint scheduling strategy that optimizes overall system performance. Simulation results demonstrate significant improvements in average system cost, queue length, energy consumption, and task completion rate, compared to baseline strategies, highlighting the strategy’s effectiveness and efficiency. Future work will extend the framework to more complex network scenarios and explore the integration of advanced machine learning techniques to further enhance security performance [56,57].