Energy-Efficient Aerial STAR-RIS-Aided Computing Offloading and Content Caching for Wireless Sensor Networks

Abstract

Unmanned aerial vehicle (UAV)-based wireless sensor networks (WSNs) hold great promise for supporting ground-based sensors due to the mobility of UAVs and the ease of establishing line-of-sight links. UAV-based WSNs equipped with mobile edge computing (MEC) servers effectively mitigate challenges associated with long-distance transmission and the limited coverage of edge base stations (BSs), emerging as a powerful paradigm for both communication and computing services. Furthermore, incorporating simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs) as passive relays significantly enhances the propagation environment and service quality of UAV-based WSNs. However, most existing studies place STAR-RISs in fixed positions, ignoring the flexibility of STAR-RISs. Some other studies equip UAVs with STAR-RISs, and UAVs act as flight carriers, ignoring the computing and caching capabilities of UAVs. To address these limitations, we propose an energy-efficient aerial STAR-RIS-aided computing offloading and content caching framework, where we formulate an energy consumption minimization problem to jointly optimize content caching decisions, computing offloading decisions, UAV hovering positions, and STAR-RIS passive beamforming. Given the non-convex nature of this problem, we decompose it into a content caching decision subproblem, a computing offloading decision subproblem, a hovering position subproblem, and a STAR-RIS resource allocation subproblem. We propose a deep reinforcement learning (DRL)–successive convex approximation (SCA) combined algorithm to iteratively achieve near-optimal solutions with low complexity. The numerical results demonstrate that the proposed framework effectively utilizes resources in UAV-based WSNs and significantly reduces overall system energy consumption.

1. Introduction

Wireless sensor networks (WSNs), comprising a large number of sensor nodes, show impressive capability in transmitting a very large number of data with high efficiency [1,2]. Their compactness, cost-effectiveness, and ease of deployment make WSNs highly effective for a wide range of real-time applications. With the rapid development of WSNs, the explosive growth of sensor devices has intensified the demand for high data rates and ultra-low-latency services [3]. Traditional cloud computing paradigms face challenges in meeting the diverse service requirements of delay-sensitive and computing-intensive tasks [4]. Recently, mobile edge computing (MEC), which provides computing and caching services for nearby sensors, has enabled local task offloading, avoiding the need to send data to distant cloud centers and thus reducing costs [5]. This has facilitated the transition of WSN devices to large-scale deployment, allowing for the real-time monitoring of environmental conditions to offer essential insights for urban planning and management. Moreover, by deploying MEC servers at access points (APs) or base stations (BSs), it is possible not only to cache popular content on the edge cloud to reduce the delay and energy consumption of sensor content requests but also to offload tasks requiring computation [6]. However, traditional MEC servers, typically deployed at fixed base stations, often suffer from limited coverage and challenges like non-line-of-sight (NLoS) transmission, which degrades signal quality and results in reduced overall service efficiency and performance [7].

In recent years, unmanned aerial vehicles (UAVs) have gained widespread application across various industries due to their mobility, low operational cost, and ability to establish easy line-of-sight (LoS) communication links [8]. These unique characteristics enable UAVs to effectively address challenges inherent to traditional communication systems, such as fixed deployment locations, high infrastructure costs, and limited adaptability to specialized scenarios [9]. Furthermore, equipped with MEC servers, UAVs can not only provide computing and caching services to sensors but also function as aerial relays to offload tasks to other nodes, thereby significantly enhancing the flexibility and efficiency of network services [10,11,12]. However, UAVs have limited computing, caching, and endurance capabilities, thus low-power solutions are crucial to improving UAV network performance [13].

Recently presented, a promising approach to reducing UAV energy consumption involves the deployment of simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs) as an alternative to UAVs for signal relaying [14]. Each element of a STAR-RIS, capable of supporting both electric and magnetic currents, can simultaneously reconfigure transmitted and reflected signals, thereby achieving full-space coverage [15,16]. However, most existing studies in this field assume that STAR-RISs are deployed in a fixed position [17], with UAVs primarily offering computing [18,19] or caching capabilities [20,21,22]. The fixed deployment of STAR-RISs limits their ability to flexibly adjust the distance between themselves and the sensors, thereby degrading task offloading transmission performance. Other studies have proposed mounting STAR-RISs on UAVs, but UAVs do not have computing and caching resources [23,24]. Consequently, UAVs serve only as flight carriers, leaving their potential computing and caching capabilities underutilized. This significantly reduces resource utilization and overall task processing efficiency in UAV-based WSNs.

While significant progress has been made in this field, several critical research gaps remain unaddressed. First, the fixed deployment of STAR-RISs restricts their adaptability to dynamic WSN environments, leading to suboptimal task offloading transmission performance. Second, while some studies equip UAVs with STAR-RISs, they overlook UAVs’ inherent computing and caching capabilities, resulting in the underutilization of WSN resources and reduced system efficiency. Third, existing research predominantly lacks a comprehensive joint optimization framework that simultaneously considers caching decisions, offloading strategies, UAV hovering positions, and STAR-RIS passive beamforming. These limitations hinder the ability to fully capitalize on the dynamic and heterogeneous resources of UAV-based WSNs. To address these limitations, we propose a novel STAR-RIS-assisted computing offloading and content caching framework to minimize system energy consumption for UAV-based WSNs. In this context, UAVs have computing and caching capabilities, which can greatly enhance task processing efficiency and response speed. Subsequently, given the limited energy capacity of UAVs, we optimize their energy utilization by separating the relay functionality. To this end, a STAR-RIS, as a passive relay, is introduced to assist the sensor nodes in forwarding tasks that can be reasonably allocated to UAV-based WSN resources. Furthermore, by installing a STAR-RIS on the UAV, the system can dynamically adjust the relative positioning between the STAR-RIS and the sensors, further enhancing efficiency in task transmission and processing. However, several key challenges must still be addressed to fully achieve this. First, traditional static caching strategies are not suitable for the dynamic characteristics of UAV-based WSNs. Therefore, it is essential to design effective caching strategies that ensure fast responses to sensor requests. Secondly, in order to ensure the efficient use of resources and avoid single-point overload, how to achieve the dynamic offloading of tasks among edge clouds, UAVs, and sensors is crucial. Third, due to the limited endurance of UAVs, STAR-RISs as relay nodes can effectively save the energy consumption of UAVs when forwarding tasks as relay nodes by optimizing signal transmission and reflection paths. Therefore, in order to further improve the endurance of the UAV system, it is crucial to design an effective transmission and reflection coefficient matrix for STAR-RISs. Finally, UAVs equipped with STAR-RISs not only cache and process sensor tasks but also serve as relay nodes, providing additional communication links between sensors and the edge cloud. Therefore, it is essential to jointly optimize caching decisions, offloading decisions, UAV hovering positions, and passive beamforming to minimize overall system energy consumption.

Tackling these challenges, the main contributions of this paper can be summarized as follows:

(1)

We propose a novel aerial STAR-RIS-aided computing offloading and content caching framework to minimize system energy consumption for WSNs. This framework leverages flexible deployment and caching, computing, and communication (3C) resources to offer adaptive computation and caching services. Additionally, a STAR-RIS is introduced as a passive relay to assist sensors in forwarding tasks, which can reasonably allocate UAV-based WSN resources. Lastly, by installing a STAR-RIS on a UAV, the system can flexibly adjust the position between the STAR-RIS and the sensors to improve task transmission performance.

(2)

Since the energy consumption minimization problem is non-convex, we decomposed the problem into four subproblems: content caching decision, computation offloading decision, UAV hovering position, and STAR-RIS resource allocation. For the subproblem of content caching decisions, the network caching decisions are optimized by utilizing a new deep reinforcement learning (DRL) algorithm. For the other subproblems, we utilize the Karush–Kuhn–Tucker (KKT) conditions and the successive convex approximation (SCA) algorithm to iteratively solve and optimize system energy consumption.

(3)

The numerical results demonstrate that the proposed STAR-RIS-aided computing offloading and content caching framework significantly reduces system energy consumption in UAV-based WSNs compared with the benchmarks, especially in scenarios with limited network resources or adverse channel conditions.

The rest of this paper is organized as follows: We first briefly review the related works of this paper in Section 2 and then give the overview and mathematical description of the system model in Section 3. In Section 4 and Section 5, the optimization algorithm and iterative solution process for the proposed model are introduced. In Section 6, we discuss the convergence and complexity of the proposed DRL-SCA algorithm. Simulation environments are presented, and the results are discussed in Section 7, followed by conclusions in Section 8.

2. Related Works

In this section, we present related works on computation offloading and content caching in three key aspects: MEC, UAVs, and STAR-RIS-aided UAVs. The three subsections are in a progressive relationship, from MEC to UAVs and then to STAR-RIS-aided UAVs, which helps to identify gaps in existing research and gradually demonstrates the superiority of the proposed framework.

2.1. Computing Offloading and Content Caching in MEC

In the context of MEC, the key element is the edge server, which provides computing resources, caching capability, and connectivity [25]. When computing tasks are offloaded to MEC nodes, content caching can effectively degrade the latency and bandwidth cost of acquiring and initializing applications [26,27]. Several studies have explored the joint optimization of content caching and computing offloading in MEC [5,19,26]. Liao et al. [5] studied a joint service caching and task offloading problem in a multi-user, multi-BS, cloud-based MEC system to optimize execution delay, energy consumption, total benefit, and task offloading rate. Yu et al. [19] proposed optimal joint service caching and task offloading strategies to minimize the overall execution delay at the mobile terminal side in a MEC scenario. Bi et al. [26] minimized user computation delay and energy consumption by jointly optimizing resource allocation, computing offloading, and content caching decisions in MEC networks.

2.2. Computing Offloading and Content Caching in UAVs

Although MEC can provide low-cost computing and caching services for nearby sensors, MEC is typically deployed at fixed BSs, which limits its adaptability to sensors’ demands, coverage, and service quality [5,7]. To overcome these limitations, UAV communication networks have become crucial due to their high mobility, strong LoS links, and fast deployment capabilities [28]. The UAVs equipped with MEC servers can plan their hovering positions or flight trajectories strategically, offering 3C resources dynamically [20]. Several studies have investigated the joint optimization of computing offloading and content caching in UAVs [21,22]. Zhao et al. [21] studied a UAV-enabled MEC network where UAVs equipped with caching and computation capabilities collaborate with ground BSs to fulfill sensor requests, enhancing communication resources for data transmission. Huang et al. [22] proposed a UAV-assisted Internet of Vehicles (IoV) framework, where both UAVs and BSs provide computing and caching services for smart vehicles, to minimize average task processing delay and maximize the UAV cache hit ratio.

2.3. Computing Offloading and Content Caching in STAR-RIS-Aided UAVs

Although UAVs can provide flexible computing and caching services, it is difficult to meet high-energy-consuming task demands due to the limited endurance capabilities of UAVs [13,17,29]. Therefore, low-power solutions are essential to improving the performance of UAV-based WSNs. Fortunately, a promising approach to reducing UAV energy consumption is the deployment of STAR-RISs as an alternative to UAVs for signal relaying [23,30]. Aung et al. [23] deployed STAR-RISs on UAVs and minimized energy consumption for IoT devices and aerial STAR-RISs by jointly optimizing task offloading, aerial STAR-RIS trajectory, amplitude, phase shift coefficients, and transmitting power in a STAR-RIS-aided MEC network. Zhang et al. [30] maximized the sum rate of all users by jointly optimizing the STAR-RISs’ beamforming vectors, the UAVs’ trajectory, and the power allocation STAR-RIS-assisted UAV communication system.

However, most studies assume that STAR-RISs are fixed and UAVs only provide computing or caching capabilities. Since STAR-RISs cannot flexibly adjust the distance between themselves and the sensor, task offloading transmission performance is reduced [30]. Some other studies assumed that STAR-RISs are mounted on UAVs but UAVs do not have computing and caching resources [23]. Since a UAV can only serve as a flight carrier, the computing and caching service resources of UAV-based WSNs cannot be utilized, which significantly reduces resource utilization and overall task processing efficiency. To overcome these limitations, we propose a novel energy-efficient aerial STAR-RIS-assisted computing offloading and content caching framework, which installs a STAR-RIS on a UAV, fully leveraging UAV-based WSNs’ computational and caching capacities for improved resource utilization and task execution efficiency. In this framework, we formulate a problem to minimize system energy consumption by jointly optimizing the content caching decisions, computing offloading decisions, UAV hovering positions, and STAR-RIS passive beamforming.

3. System Model and Problem Formulation

In this section, we provide a comprehensive description of the network overview, communication model, computation model, caching model, and energy consumption model for the proposed aerial STAR-RIS-aided WSN. We also formulate a system energy consumption minimization optimization problem. The notation is summarized in

Table 1. Summary of important notation.

Symbol	Definition
$M_c$	Total number of elements in each column of STAR-RIS
$M_r$	Total number of elements in each row of STAR-RIS
${\mathit{r}}_j$	Location of node j
${\mathit{r}}_{(m_c,m_r)}$	Location of the $(m_c,m_r)$-th STAR-RIS element
$h_{m,u}$	Channel gain from sensor m to UAV u
$h_{m,(m_c,m_r)}$	Channel gain from sensor m to the $(m_c,m_r)$-th STAR-RIS element
$g_{(m_c,m_r),c}$	Channel gain from the $(m_c,m_r)$-th STAR-RIS element to edge cloud c
P	Transmit power
B	Spectrum bandwidth
R	Transmission rate
$f_j$	Computing capability (CPU cycles per second) of node j
$\alpha$	Task offloading ratio
${\beta }_{(m_c,m_r)}^a$	Amplitude coefficients of transmitting and reflecting vectors ($a\in \left\{r,t\right\}$)
${\theta }_{(m_c,m_r)}^a$	The phase shift angle vectors of transmitting and reflecting elements ($a\in \left\{r,t\right\}$)
$k_j$	Effective switched capability of node j
$L_k$	The requested data size of task k (uplink)
${\omega }_k$	Number of CPU cycles for computing 1-bit task k

.

3.1. Network Overview

As shown in Figure 1, the considered aerial STAR-RIS-aided WSN consists of an edge cloud c, a UAV u equipped with computing and caching servers and a STAR-RIS, and many single-antenna sensors. UAV u can provide services for sensors and those sensors are denoted by $\mathcal{M}=\{1,2,\dots ,m,\dots ,M\}$. And the set of tasks is denoted by $\mathcal{K}=\{1,2,\dots ,k,\dots ,K\}$. Moreover, we assume that content popularity follows the Zipf distribution. We consider that edge cloud c and UAV u have limited caching and computing capacities and that the sensors only have limited computing capacities. The UAV has mobility and can adjust its position relative to the connected sensors to improve the quality of signal transmission. In this network, UAV u is connected to edge cloud c and to the sensors by wireless links. In particular, the STAR-RIS consists of $M_c$ × $M_r$ passive units, which are spanned as a uniform planar array (UPA). Each column and row of the UPA has $M_c$ and $M_r$ passive units, respectively [31]. The $(m_c,m_r)$-th STAR-RIS element is used to represent the element of the $m_r$-th row and $m_c$-th column of the STAR-RIS. We assume that the direct communication links between the sensors and edge cloud c are blocked and that the STAR-RIS elements work in energy-splitting (ES) mode [32]. In the proposed STAR-RIS-aided UAV system, the sensor m can process the partial tasks; then, UAV u can partially process the remaining tasks uploaded by the sensors and return the results to the sensors. At the same time, the other remaining tasks are forwarded to edge cloud c for processing through the STAR-RIS elements, and the results are returned to the sensors. In this paper, the main focus is on minimizing system energy consumption in the STAR-RIS-aided UAV system by jointly optimizing the content caching decisions, computing offloading decisions, UAV hovering positions, and STAR-RIS passive beamforming.

As illustrated in Figure 2, we make use of virtual reality (VR) as a typical application scenario. The sensors need to process a variety of virtual reality application tasks, such as object tracking, object identification, and scene rendering. Every task requires a different amount of computing capacity and data size. For instance, object-tracking tasks require a large number of data to be transmitted, while object identification tasks and scene-rendering tasks require greater computing resources. Therefore, we adopt two parameters in total for modeling heterogeneous computation tasks. For computation task k, we define $F_k=\left\{{\omega }_k{L}_k\right\}$, where ${\omega }_k$ represents the computing resources required to complete the task and $L_k$ represents the data size of the task, i.e., the size of the data that need to be transmitted to UAV u or edge cloud c.

When the computing task is not cached in the system, we consider the partial offloading scheme for delay-sensitive computation tasks in STAR-RIS-aided UAV systems. This kind of computation offloading model allows tasks to be calculated in parallel at the sensors, UAV u, and edge cloud c. The tasks processed at the sensors are referred to as local tasks, the tasks processed at UAV u are referred to as UAV-offloading tasks, and the tasks that are offloaded to edge cloud c are called edge cloud-offloading tasks. Figure 3 presents the time allocation for task processing in the STAR-RIS-aided UAV system, where the sensors utilize the same resource block with duration $T_m^{k,max}$ to transmit and compute tasks.

In the local execution phase, the sensors process their tasks by the local computing servers. In the UAV task offloading phase, some of the remaining tasks are uploaded to UAV u and are processed by the computing server of UAV u. In the edge cloud task offloading phase, some of the remaining tasks are forwarded to edge cloud c through the STAR-RIS elements for processing. When the tasks are completed, the computing results obtained at both UAV u and edge cloud c are returned to the sensors. In downlink communication, since UAV u and edge cloud c tend to have high transmit power and the computing results are usually of small size, the downloading time is comparatively negligible in the UAV task offloading phase and edge cloud task offloading phase.

3.2. Communication Model

This subsection introduces the communication model and gives the uplink data rate when the sensor offloads tasks on UAV u and edge cloud c. We assume that when the task is offloaded, the sensor does not move. A 3D Cartesian coordinate system is established to describe the locations of the sensors, UAV u, STAR-RIS, and edge cloud c. The locations of sensor m and edge cloud c are described by vector ${\mathit{r}}_m=(x_m,y_m,0)$ and vector ${\mathit{r}}_c=(x_c,y_c,0)$. We assume that UAV u is hovering at a fixed position in every slot to provide computing services for the sensors [33]. The position of UAV u is ${\mathit{r}}_u=(x_u,y_u,h_u)$, and the position of the $(m_c,m_r)$-th STAR-RIS element is ${\mathit{r}}_{(m_c,m_r)}=(x_{(m_c,m_r)},y_{(m_c,m_r)},h_{(m_c,m_r)})$.

Due to the high probability of LoS links in UAV communication, the communication channels between sensors m and UAV u, between sensor m and the $(m_c,m_r)$-th STAR-RIS element, and between the $(m_c,m_r)$-th STAR-RIS element and edge cloud c are assumed to be LoS links, all following the free-space path loss model [34]. So, the channel gain between node n and another node $n^{\prime }$ can be formulated as follows:

$$\begin{array}{cc}\hfill h_{n,n^{\prime }}=& \sqrt{g_0}{d}_{n,n^{\prime }}^{-2},\hfill \end{array}$$

(1)

where $n\ne n^{\prime }$, $n\in \left\{m,(m_c,m_r)\right\}$, and $n^{\prime }\in \left\{u,(m_c,m_r),c\right\}$. $g_0$ is the received power at a distance of 1 m for a transmission power of 1 W and $d_{n,n^{\prime }}=\sqrt{{(x_n-x_{n^{\prime }})}^2+{(y_n-y_{n^{\prime }})}^2+{(h_n-h_{n^{\prime }})}^2}$. The signal-to-interference-plus-noise ratio (SINR) of the wireless link from node n to another node $n^{\prime }$, denoted by ${\gamma }_{n,n^{\prime }}$, can be expressed as

$${\gamma }_{n,n^{\prime }}={\displaystyle \frac{P_n{\left|h_{n,n^{\prime }}\right|}^2}{{\sigma }_{n,n^{\prime }}^2}},$$

(2)

where $P_n$ is the transmit power of node n. We assume that the noise power has constant variance ${\sigma }_{n,n^{\prime }}^2$. Therefore, the transmission rate from node n to node $n^{\prime }$ can be given by

$$R_{n,n^{\prime }}=B_{n,n^{\prime }}{log}_2\left(1+{\displaystyle \frac{P_n{\left|h_{n,n^{\prime }}\right|}^2}{{\sigma }_{n,n^{\prime }}^2}}\right),$$

(3)

where $B_{n,n^{\prime }}$ is the transmission bandwidth of the wireless link from node n to node $n^{\prime }$.

3.2.1. Channel in UAV Task Offloading

UAV u equipped with the MEC server has computing resources, which allows the sensors to transfer some tasks to UAV u for processing. The signal received by UAV u can be written as

$$z_u=\sum _{m\in \mathcal{M}}{h}_{m,u}\sqrt{P_m}{s}_m+n_u,$$

(4)

where $s_m$ is the corresponding signal with $\mathbb{E}\left\{|s_m{|}^2\right\}=1$ [35]. $n_u$ is the additive white Gaussian noise (AWGN) received by UAV u. The noise power is ${\sigma }_u^2$, i.e., $n_u=n\sim \mathcal{N}(0,{\sigma }_u^2)$.

According to (2) and (3), the SNR and transmission rate of the link from sensor m to UAV u are denoted by ${\gamma }_{m,u}$ and $R_{m,u}$.

3.2.2. Channel in Edge Cloud Task Offloading

Subject to its energy limitations and task time constraints, UAV u can only perform part of its received remaining tasks. The remaining tasks are forwarded to edge cloud c for processing through the STAR-RIS. In order to better distinguish the channel gain from sensor m to the $(m_c,m_r)$-th STAR-RIS element, the channel gain from the $(m_c,m_r)$-th STAR-RIS element to edge cloud c is expressed as $g_{(m_c,m_r),c}$. Therefore, the channel gains from sensor m to the STAR-RIS and from the STAR-RIS to edge cloud c are denoted by ${\mathit{H}}_{m,s}\in {\mathbb{C}}^{M_c{M}_r\times 1}$ and ${\mathit{G}}_{s,c}^H\in {\mathbb{C}}^{1\times M_c{M}_r}$, respectively. ${\mathbf{\Theta }}_a=\mathrm{diag}[\sqrt{{\beta }_{(1,1)}^a}{e}^{j{\theta }_{(1,1)}^a},\sqrt{{\beta }_{(1,2)}^a}{e}^{j{\theta }_{(1,2)}^a},\dots ,\sqrt{{\beta }_{(m_c,m_r)}^a}{e}^{j{\theta }_{(m_c,m_r)}^a},\dots ,\sqrt{{\beta }_{(M_c,M_r)}^a}{e}^{j{\theta }_{(M_c,M_r)}^a}]$ is the transmission or reflection coefficient matrix of the STAR-RIS for the incident signal from sensor m, where $\sqrt{{\beta }_{(m_c,m_r)}^a}\in \left[0,1\right]$ and ${\theta }_{(m_c,m_r)}^a\in \left[0,2\pi \right)$ denote the amplitude and phase shift of the $(m_c,m_r)$-th STAR-RIS element for the sensor’s signal, respectively, and $a\in \left\{r,t\right\}$. Let ${\mathbf{\Theta }}_a$ be the reflection $(a=r)$ or transmission $(a=t)$ beamforming vectors. In consequence, the following constraint is required when the STAR-RIS is in ES mode:

$$\sum _{a\in \left\{r,t\right\}}{\beta }_{(m_c,m_r)}^a\le 1.$$

(5)

Hence, the signal received at edge cloud c can be obtained as

$$z_c=\sum _{m\in \mathcal{M}}{\mathit{G}}_{s,c}^H{\mathbf{\Theta }}_a{\mathit{H}}_{m,s}\sqrt{P_m}{s}_m+n_c,$$

(6)

where $n_c$ is the AWGN received at edge cloud c with variance ${\sigma }_c^2$, i.e., $n_c=n\sim \mathcal{N}(0,{\sigma }_c^2)$. If sensor m is in the transmission region, then $a=t$; otherwise, if sensor m is in the reflection region, $a=r$.

Similarly, the SNR and transmission rate of the link from sensor m to edge cloud c via the STAR-RIS are denoted by ${\gamma }_{m,c}$ and $R_{m,c}$, respectively, where the channel gain from sensor m via the STAR-RIS to edge cloud c is ${\mathit{G}}_{s,c}^H{\mathbf{\Theta }}_a{\mathit{H}}_{m,s}$.

We do not take packet loss and downlink transmission delay into account in this paper. This is due to the fact that the downlink transition rate is higher than the uplink transition rate and the size of the data after task processing is much smaller than that of the data before processing.

3.3. Computation Model

In this study, we consider a divisible computation task, allowing it to be segmented into multiple parts. Taking video analysis as an example, a large video file containing numerous frames can be divided into several video clips through segmentation. This enables a portion of the clips to be initially processed locally at the sensor, while others are handled by UAV u, and the remaining clips are offloaded to edge cloud c. Detailed explanations will be provided in subsequent sections. Additionally, we disregard the delay associated with transmitting the processed task results from the UAV and edge cloud back to the sensors, as the output data size is significantly smaller compared with that of the input data.

We define the integer caching decision variable, $X_{j^{\prime }}^k\in \{0,1\}$$(j^{\prime }\in \{u,c\})$, which indicates whether task k is cached at node $j^{\prime }$ ($X_{j^{\prime }}^k$ = 1) or not ($X_{j^{\prime }}^k$ = 0). Therefore, the task caching strategy can be represented as follows: $\mathit{X}=\left\{X_{j^{\prime }}^1,X_{j^{\prime }}^2,\dots ,X_{j^{\prime }}^k\right\}$. For the task k offloading problem of sensor m, we define decision variable ${\alpha }_{m,j}^k\in [0,1]$, $j\in \{m,u,c\}$. ${\alpha }_{m,j}^k\in [0,1]$ is the task offloading ratio of sensor m at node j. (In this paper, we assume that sensor m initiates a task request to node j. Node j senses the information of the task (task size and time constraints) and then makes a task offloading decision based on its own computing capability. We call the above process the sensing phase, and since this time is very short, we ignore this time.) Consequently, the following is a representation of the task offloading policy: $\mathit{\alpha }=\left\{{\alpha }_{m,m}^k,{\alpha }_{m,u}^k,{\alpha }_{m,c}^k\right\}$, ${\displaystyle \sum _{j\in \{m,u,c\}}}{\alpha }_{m,j}^k=1$.

Service delay is the overall service time for task k when task k is not cached, consisting of two parts: (i) uplink communication delay; (ii) computation delay. In downlink communication, since the UAV and edge cloud tend to have high transmit power and the computing results are usually of small size, the downloading time is also negligible. Next, we discuss the delay and corresponding energy consumption.

3.3.1. Energy Consumption for Uplink Communication

We denote $X_{j^{\prime }}^k=0$, as task k is not cached, and offloading decision variable ${\alpha }_{m,j^{\prime }}^k$. Therefore, the uplink transmission delay of offloading task k from sensor m to node $j^{\prime }$ can be expressed as

$$T_{m,j^{\prime }}^{tr,k}={\displaystyle \frac{{\alpha }_{m,j^{\prime }}^k{L}_k}{R_{m,j^{\prime }}}}.$$

(7)

Therefore, when task k is not cached, the total uplink communication energy consumption for sensor m offloading task k can be calculated as

$$\begin{array}{cc}\hfill E_m^{tr,k}=& \sum _{j^{\prime }\in \{u,c\}}{P}_m{T}_{m,j^{\prime }}^{tr,k}.\hfill \end{array}$$

(8)

3.3.2. Energy Consumption for Computation

We denote $X_{j^{\prime }}^k=0$, as task k is not cached, and offloading decision variable ${\alpha }_{m,j}^k$. Therefore, the computation delay for computing offloading task k at node j can be expressed as

$$\begin{array}{c}\hfill T_{m,j}^{com,k}={\displaystyle \frac{{\alpha }_{m,j}^k{\omega }_k}{f_j}},\end{array}$$

(9)

where ${\omega }_k$ is the number of required computation resources for task k, i.e., the number of CPU cycles required for computing 1-bit task data. $f_j$ is the computing capability (CPU cycles per second) of node j.

Therefore, the total computing energy consumption of sensor m for computing task request k can be expressed as

$$\begin{array}{cc}\hfill E_m^{com,k}& =\sum _{j\in \{m,u,c\}}{E}_{m,j}^{com,k}\hfill \\ \hfill & =\sum _{j\in \{m,u,c\}}{\kappa }_j{T}_{m,j}^{com,k}{f}_j^3,\hfill \end{array}$$

(10)

where ${\kappa }_j$ is the effective capacitance coefficient of node j that depends on the processor’s chip architecture.

3.4. Caching Model

In this subsection, we describe the caching model. Task caching involves storing completed tasks and associated data within UAV u or edge cloud c. Specifically, an independent resource container is maintained on UAV u or edge cloud c. The caching process operates as follows: Firstly, sensors send a computing task request. If UAV u or edge cloud c has already cached this task, the respective node notifies the sensors of its availability on the caching servers. Consequently, the sensor can avoid offloading the same task to UAV u or edge cloud c. At last, after the task is processed by UAV u or edge cloud c, the results are sent back to the sensors. This caching mechanism reduces the need for redundant task offloading, thereby lowering sensor energy consumption and minimizing offloading delays.

Despite its benefits, task caching still faces many challenges: (i) although UAV u and edge cloud c have greater caching and computational capacities compared with sensors, they still cannot cache or handle all kinds of computation tasks; (ii) unlike traditional caching strategies, task caching requires the consideration of not only the data size and computational resources necessary for each task but also task popularity. Consequently, designing an effective caching strategy presents significant challenges. We introduce an integer caching decision variable, $X_{j^{\prime }}^k\in \{0,1\}$, to indicate whether task k is cached at node $j^{\prime }$ ($X_{j^{\prime }}^k$ = 1) or not ($X_{j^{\prime }}^k$ = 0). Therefore, the computation caching strategy can be defined as $\mathit{X}=\left\{X_{j^{\prime }}^1,X_{j^{\prime }}^2,\dots ,X_{j^{\prime }}^k\right\}$. In this study, we evaluate the task duration and energy consumption on UAV u or edge cloud c in scenarios with and without task caching. For task caching ($X_{j^{\prime }}^k$ = 1), the task duration, simplified to the processing delay, is denoted by $T_{m,j^{\prime }}^{cache,k}$ and can be expressed as

$$T_{m,j^{\prime }}^{cache,k}={\displaystyle \frac{{\omega }_k}{f_{j^{\prime }}}}.$$

(11)

The primary energy consumption occurs within UAV u or edge cloud c, with sensors incurring no energy cost. Accordingly, the energy consumption associated with task caching can be formulated as

$$\begin{array}{cc}\hfill E_{m,j^{\prime }}^{cache,k}=& {\kappa }_{j^{\prime }}{T}_{m,j^{\prime }}^{cache,k}{f}_{j^{\prime }}^3,\hfill \end{array}$$

(12)

where ${\kappa }_{j^{\prime }}$ is the effective capacitance coefficient of node $j^{\prime }$ that depends on the processor’s chip architecture.

3.5. Problem Formulation

Based on the task communication, computation, and caching process mentioned above, the overall delay and energy consumption of sensor m from sending task request k to obtaining computing results are expressed as

$$\begin{array}{cc}\hfill T_m^k& =\sum _{j^{\prime }\in \{u,c\}}{X}_{j^{\prime }}^k{T}_{m,j^{\prime }}^{cache,k}+\left(1-\sum _{j^{\prime }\in \{u,c\}}{X}_{j^{\prime }}^k\right)max\left\{{\kappa }_m{T}_{m,m}^{com,k}{f}_m^3,\underset{j^{\prime }\in \{u,c\}}{max}\left\{T_{m,j^{\prime }}^{tr,k}+T_{m,j^{\prime }}^{com,k}\right\}\right\};\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill E_m^k& =\sum _{j^{\prime }\in \{u,c\}}{X}_{j^{\prime }}^k{E}_{m,j^{\prime }}^{cache,k}+\left(1-\sum _{j^{\prime }\in \{u,c\}}{X}_{j^{\prime }}^k\right)(E_m^{\mathrm{tr},\mathrm{k}}+E_m^{com,k}).\hfill \end{array}$$

(14)

In this paper, for task caching ($X_{j^{\prime }}^k$ = 1), the primary energy consumption occurs within UAV u or edge cloud c, with sensors incurring no energy cost. The primary energy consumption, simplified to the processing energy consumption, is denoted by $E_{m,j^{\prime }}^{cache,k}$. For task caching ($X_{j^{\prime }}^k$ = 0), the primary energy consumption includes the total uplink communication energy consumption for sensor m offloading task k and the total computing energy consumption of computing task request k. Therefore, we formulate a system energy consumption minimization problem by jointly optimizing caching decision $\mathit{X}=\left\{X_{j^{\prime }}^1,X_{j^{\prime }}^2,\dots ,X_{j^{\prime }}^k\right\}$ $(j^{\prime }\in \{u,c\})$, offloading decision ${\alpha }_{m,j}^k\in [0,1]$ ($j\in \{m,u,c\}$), hovering position of UAV u ${\mathit{r}}_u$, and passive beamforming $\mathbf{\Theta }$ in the STAR-RIS-aided UAV system. According to (14), the optimization problem for minimizing the total energy consumption of the STAR-RIS-aided UAV system can be formulated as

$$\begin{array}{cc}\hfill {\mathcal{P}}_1:& \underset{\begin{array}{c}\mathit{X},\mathit{\alpha },{\mathit{r}}_u,\mathbf{\Theta }\end{array}}{min}\sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}{E}_m^k\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& C_1:{\alpha }_{m,j}^k\in \left[0,1\right],\forall m,j\in \left\{m,u,c\right\},\hfill \end{array}$$

(15a)

$$\begin{array}{cc}\hfill & C_2:\sum _{j\in \left\{m,u,c\right\}}{\alpha }_{m,j}^k=1,\hfill \end{array}$$

(15b)

$$\begin{array}{cc}\hfill & C_3:\sum _{a\in \left\{r,t\right\}}{\beta }_{(m_c,m_r)}^a\le 1,{\beta }_{(m_c,m_r)}^a\in \left[0,1\right],\forall a,\forall m_c,\forall m_r,\hfill \end{array}$$

(15c)

$$\begin{array}{cc}\hfill & C_4:{\theta }_{(m_c,m_r)}^a\in \left[0,2\pi \right),\forall a,\forall m_c,\forall m_r,\hfill \end{array}$$

(15d)

$$\begin{array}{cc}\hfill & C_5:x_u\le x_u^{max},y_u\le y_u^{max},\hfill \end{array}$$

(15e)

$$\begin{array}{cc}\hfill & C_6:\sum _{k\in \mathcal{K}}{X}_{j^{\prime }}^k{S}^k\le O_{j^{\prime }},\forall k,\forall j^{\prime },\hfill \end{array}$$

(15f)

$$\begin{array}{cc}\hfill & C_7:\sum _{k\in \mathcal{K}}{\alpha }_{m,j}^k{\omega }_k\le f_j,\forall k,\forall m,\forall j,\hfill \end{array}$$

(15g)

$$\begin{array}{cc}\hfill & C_8:X_{j^{\prime }}^k\in \{0,1\},\forall k,\forall j^{\prime },\hfill \end{array}$$

(15h)

$$\begin{array}{cc}\hfill & C_9:T_m^k\le T_m^{k,max},\forall m,\forall k.\hfill \end{array}$$

(15i)

Constraint $C_1$ ensures that the task offloading ratio at each sensor must be between 0 (indicating no offloading) and 1 (indicating full offloading), ensuring that the offloading task is appropriately distributed. Constraint $C_2$ ensures that the sum of offloading ratios for task k across all offloading targets (sensors m, UAV u, and edge cloud c) is equal to 1. Constraint $C_3$ ensures that it governs the amplitude response of the $(m_c,m_r)$-th STAR-RIS element, limiting it to a range of $[0,1]$. Constraint $C_4$ defines the phase shift of the $(m_c,m_r)$-th STAR-RIS element, ensuring that it lies within the range $[0,2\pi )$. This is because the phase shift is a periodic quantity, and its effective range should be between 0 and $2\pi$. Constraint $C_5$ restricts the UAV’s coordinates, $x_u$ and $y_u$, which must not exceed the maximum allowed values $x_u^{max}$ and $y_u^{max}$, ensuring that the UAV operates within a defined area. Constraint $C_6$ ensures that the total cached content at each node does not exceed its maximum caching capacity $O_{j^{\prime }}$. Constraint $C_7$ ensures that the total computational resources allocated to handle tasks at each node do not exceed its maximum computation capacity $f_j$. Constraint $C_8$ restricts the caching binary decision variables $X_{j^{\prime }}^k$ to take the value of either 0 or 1. This is typically used to represent whether a task is cached to a specific node or not. Constraint $C_9$ ensures that the completion time of task k does not exceed the maximum tolerable time $T_m^{k,max}$ for the task. This guarantees that all tasks are completed within the allowed time window, ensuring that delay requirements are met.

Due to the different hovering positions of UAV u, the channel gain of the sensor-to-UAV channel link and the sensor-to-STAR-RIS-to-edge cloud channel link may differ, which in turn affects the transmission energy consumption. For ease of calculation, we assume that the $h_u$ of UAV u is fixed [36]. We optimize the $x_u$ and $y_u$ of UAV u to minimize the total energy consumption of the system. Meanwhile, we assume that the coordinates of STAR-RIS are the same as those of UAV u.

As shown in Figure 4, we adopt alternative optimization techniques and decompose problem (15) into four subproblems:

(1)

Content caching decision subproblem: Given $\mathit{\alpha }={\mathit{\alpha }}^0$, ${\mathit{r}}_u={\mathit{r}}_u^0$, and $\mathbf{\Theta }={\mathbf{\Theta }}^0$, i.e., when $\mathit{\alpha },{\mathit{r}}_u,$ and $\mathbf{\Theta }$ are fixed, problem (15) optimizes the caching decision vector $\mathit{X}$ to minimize the total energy consumption of the system. We adopt the DRL algorithm to optimize the content caching decision, denoted by ${\mathit{X}}^{*}$.

(2)

Computing offloading decision subproblem: Given $\mathit{X}={\mathit{X}}^{*}$, ${\mathit{r}}_u={\mathit{r}}_u^0$, and $\mathbf{\Theta }={\mathbf{\Theta }}^0$, i.e., when $\mathit{X},{\mathit{r}}_u,$ and $\mathbf{\Theta }$ are fixed, problem (15) optimizes the offloading decision vector $\mathit{\alpha }$ to minimize the total energy consumption of the system. We adopt the KKT conditions to obtain an optimal solution, denoted by ${\mathit{\alpha }}^{*}$.

(3)

UAV hovering position subproblem: Given $\mathit{X}={\mathit{X}}^{*}$, $\mathit{\alpha }={\mathit{\alpha }}^{*}$, and $\mathbf{\Theta }={\mathbf{\Theta }}^0$, i.e., when $\mathit{\alpha },\mathit{X},$ and $\mathbf{\Theta }$ are fixed, problem (15) optimizes the hovering position of UAV u vector ${\mathit{r}}_u$ to minimize the total energy consumption of the system. We adopt the SCA method to optimize the hovering position of UAV u, denoted by ${\mathit{r}}_u^{*}$.

(4)

STAR-RIS resource allocation subproblem: Given $\mathit{X}={\mathit{X}}^{*}$, $\mathit{\alpha }={\mathit{\alpha }}^{*}$, and ${\mathit{r}}_u={\mathit{r}}_u^{*}$, the transmission and reflection coefficient matrix $\mathbf{\Theta }$ is optimized to minimize the total energy consumption of the system. We adopt the SCA method to obtain an optimal solution, denoted by ${\mathbf{\Theta }}^{*}$.

Figure 4. The proposed optimization framework of the energy consumption minimization problem.

Figure 4. The proposed optimization framework of the energy consumption minimization problem.

4. Content Caching Decision Optimization

Since content caching decision optimization is only related to $\mathit{X}$ but independent of other variables in ${\mathcal{P}}_1$, caching decision optimization can be solved in advance with given offloading decisions and the hovering position of the UAV, as well as the transmission and reflection coefficient matrix. The subproblem can be written as

$$\begin{array}{cc}\hfill {\mathcal{P}}_2:& \underset{\begin{array}{c}\mathit{X}\end{array}}{min}\sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}{E}_m^k\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.\left(C_6\right)-\left(C_9\right)\hfill \end{array}$$

(16)

Due to the fact that $\mathit{X}$ is a binary vector, ${\mathcal{P}}_2$ is still subject to a mixed-integer nonlinear programming (MINLP) problem. Traditional optimization methods like SCA may not effectively handle such problems, especially in environments where the network state changes rapidly and unpredictably. The traditional binary-relax SCA approach first relaxes the binary variables $\mathit{X}$ from the discrete space $\{0,1\}$ into the continuous space [0, 1] and then forces them to round back after the SCA-based iterations. Therefore, such “relaxation” may lead the solution to converge to a local minimum in real-time dynamic systems.

To address these challenges, we utilize the proximal policy optimization (PPO) algorithm, a DRL method. As shown in Figure 5, the PPO algorithm uses neural networks to model complex relationships between system states and actions, learning directly from interactions with the environment. By leveraging the PPO algorithm, caching decisions are dynamically adjusted based on real-time network conditions, such as caching state, content popularity, historical request access frequency, and network topology. The intelligent agent iteratively updates its caching strategy to maximize a reward function reflecting caching efficiency, like the cache hit rate, enabling near-optimal caching strategies that adapt to changing conditions and enhance overall system performance.

PPO is implemented within the actor–critic framework, comprising a policy network (actor) and a value network (critic). In this setup, the actor generates actions, while the critic evaluates them. A significant limitation of the basic actor–critic framework is its low sample efficiency, which requires extensive interactions with the environment to converge. To address this, PPO introduces two major contributions: mini-batch updates to improve data efficiency and a clipped surrogate loss to constrain policy updates. The PPO algorithm allows for a small difference between the target policy ${\pi }_{\theta }\left(a_t^{ca}\right|s_t^{ca})$ and the behavior policy ${\pi }_{{\theta }_{\mathrm{old}}}\left(a_t^{ca}\right|s_t^{ca})$, where $a_t$ and $s_t$ denote the action taken and the state observed at time t. This is achieved by using a clipping function that limits the extent of policy change. If the policy update exceeds a predefined threshold, the clipping function prevents further increase.

4.1. Intelligent Caching MDP Model

In the content caching decision subproblem for aerial STAR-RIS-aided WSNs, we leverage caching state, content popularity, historical request frequency, and network topology data to realize the optimal content caching decisions. The caching update model is formulated as a Markov decision process (MDP). In each time slot $t\in \mathcal{T}$, the agent observes the current state, $s_t^{ca}\in {\mathcal{S}}^{ca}$, and selects an action $a_t^{ca}\in {\mathcal{A}}^{ca}$. Upon executing action $a_t^{ca}$, the agent receives an immediate reward $r_t^{ca}$ and the environment transitions to the next state, $s_{t+1}^{ex}$. The transition tuple $\left(s_t^{ca},a_t^{ca},r_t^{ca},s_{t+1}^{ca}\right)$ is stored in an experience replay buffer for agent training. To derive the optimal solution for problem ${\mathcal{P}}_2$ under heterogeneous scenarios, state space $s_t^{ca}$, action space $a_t^{ca}$, and reward function $r_t^{ca}$ in the proposed intelligent caching MDP model are designed as follows:

(1)

State: State $s_t^{ca}$ in slot t includes caching state information ${\mathcal{M}}_t$, content popularity ${\mathcal{P}}_t$, historical request access frequency ${\mathcal{F}}_t$, and network topology information ${\mathcal{G}}_t$. Thus, the state vector in slot t is expressed as

$$\begin{array}{c}\hfill s_t^{ca}=\left\{{\mathcal{M}}_t,{\mathcal{P}}_t,{\mathcal{F}}_t,{\mathcal{G}}_t\right\},\end{array}$$

(17)

where ${\mathcal{M}}_t$ represents the content caching status across all caching nodes in time slot t, expressed as ${\mathcal{M}}_t=\left\{{\mathcal{M}}_{u,t},M_{c,t}\right\}$. Additionally, ${\mathcal{F}}_t=\left\{{\mathcal{F}}_{1,t},\dots ,{\mathcal{F}}_{k,t},\dots ,{\mathcal{F}}_{K,t}\right\}$ denotes the historical access frequencies for all requests.

(2)

Action: During the process of caching decision, the optimal content caching decision, $a_t^{ca}$, includes the cached content across all nodes for the upcoming time slot $t+1$. The expression for $a_t^{ca}$ is given by

$$\begin{array}{cc}\hfill a_t^{ca}=& [{\mathcal{C}}_{1,c,t+1},\dots ,{\mathcal{C}}_{k,c,t+1},\dots ,{\mathcal{C}}_{K,c,t+1},{\mathcal{C}}_{1,u,t+1},\dots ,{\mathcal{C}}_{k,u,t+1},\dots ,{\mathcal{C}}_{K,u,t+1}],\hfill \end{array}$$

(18)

where ${\mathcal{C}}_{k,c,t+1}$ represents whether the content of task request k is cached in edge cloud c in time slot t + 1.

(3)

Reward: The formulation of the reward function plays a crucial role in guiding the exploration of the caching update problem and ensuring algorithm convergence. Consequently, the reward function in time slot t is defined as follows:

$$\begin{array}{c}\hfill r_t=\lambda \sum _{k=1}^{N_{st}}{R}_k+(1-\lambda )\sum _{j^{\prime }\in \{u,c\}}{\mathcal{H}}_t^{j^{\prime }},\end{array}$$

(19)

where $N_{st}$ is the maximum training steps, $\lambda$ is the weights parameter, ${\mathcal{H}}_t^{j^{\prime }}$ is the number of cache hits for node $j^{\prime }$ in time slot t, and $R_k$ represents sensor satisfaction at step k.

4.2. PPO-Based Content Caching Process

As shown in Algorithm 1, the PPO algorithm optimizes content caching decisions by iteratively adjusting the policy parameters by using a clipped objective function and advantage estimation to maximize cumulative rewards. The process starts with initializing the actor network’s policy parameters $\theta$ and ${\theta }_{\mathrm{old}}$ to ensure consistent learning. The critic network is initialized with parameters $\varphi$ to evaluate state values. Key hyperparameters, including learning rate $\alpha$, discount factor $\gamma$, and clipping parameter $ϵ$, are set to guide the training process. This setup forms the foundation for the algorithm’s ability to adapt and optimize caching strategies dynamically. Then, the environment is reset to its initial state in each episode. At each time step, the agent observes the current state ($s_t^{ca}$), selects an action $a_t^{ca}$ based on the current policy $\theta$, executes the action, and receives a reward $r_t$.

The critic network, parameterized by $\varphi$, is trained by using gradient descent to minimize the loss function

$$\begin{array}{c}\hfill L\left(\varphi \right)=\sum _{t=1}^T{\left({\delta }_t\right)}^2,\end{array}$$

(20)

where ${\delta }_t=r_t+\gamma V_{\varphi }\left(s_{t+1}^{ca}\right)-V_{\varphi }\left(s_t^{ca}\right)$. The generalized advantage estimator (GAE) is as shown in

$$\begin{array}{c}\hfill A_t=\sum _{l=0}^{\infty }{\left(\gamma \lambda \right)}^l{\delta }_{t+l}.\end{array}$$

(21)

The actor network updates its policy parameters $\theta$ by maximizing a clipped objective function, ensuring stable updates. The clipped objective function is defined as follows:

$$\begin{array}{c}\hfill J_{\mathrm{PPO}}^{\theta }\left(\theta \right)=\sum _{(s_t,a_t)}min(w{A}_t,\mathrm{clip}(w,1-ϵ,1+ϵ)A_t),\end{array}$$

(22)

where $ϵ$ is a clip fraction. The policy ratio of the target policy to the behavior policy can be expressed as

$$\begin{array}{c}\hfill w={\displaystyle \frac{{\pi }_{\theta }\left(a_t^{ca}\right|s_t^{ca})}{{\pi }_{{\theta }_{\mathrm{old}}}\left(a_t^{ca}\right|s_t^{ca})}}.\end{array}$$

(23)

Periodically, the behavior policy parameters are synchronized with the current policy to maintain consistency. By iteratively repeating these steps, the PPO algorithm effectively learns optimal caching strategies, adapting to dynamic changes in the network environment and improving overall performance.

Algorithm 1: PPO-based content caching process

5. Offloading Decision, UAV Hovering Position, and STAR-RIS Resource Allocation

Once content caching has been solved by PPO, computing offloading decision $\mathit{\alpha }$, UAV hovering position ${\mathit{r}}_u$, and STAR-RIS passive beamforming $\mathbf{\Theta }$ can be further optimized by KKT and SCA iteratively.

$$\begin{array}{cc}\hfill {\mathcal{P}}_3:& \underset{\begin{array}{c}\alpha ,{\mathit{r}}_u,\mathbf{\Theta }\end{array}}{min}\sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}((1-\sum _{j^{\prime }\in \{u,c\}}{X_{j^{\prime }}^k}^{*})(E_m^{\mathrm{tr},\mathrm{k}}+E_m^{com,k})+\sum _{j^{\prime }\in \{u,c\}}{X_{j^{\prime }}^k}^{*}{E}_{m,j^{\prime }}^{cache,k})\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& (C_1)-(C_5),(C_7),(C_9),\hfill \end{array}$$

(24a)

$$\begin{array}{cc}\hfill & \sum _{k\in \mathcal{K}}{X_{j^{\prime }}^k}^{*}{S}^k\le O_{j^{\prime }},\forall k,\forall j^{\prime },\hfill \end{array}$$

(24b)

$$\begin{array}{cc}\hfill & {X_{j^{\prime }}^k}^{*}\in \{0,1\},\forall k,\forall j^{\prime }.\hfill \end{array}$$

(24c)

5.1. Offloading Decision

Given $\mathit{X}={\mathit{X}}^{*}$, ${\mathit{r}}_u={\mathit{r}}_u^0$, and $\mathbf{\Theta }={\mathbf{\Theta }}^0$, problem (15) is rewritten as

$$\begin{array}{cc}\underset{\begin{array}{c}\alpha \end{array}}{min}=\hfill & \sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}\left(\left(1-\sum _{j^{\prime }\in \{u,c\}}{X_{j^{\prime }}^k}^{*}\right)\left(\sum _{j^{\prime }\in \{u,c\}}{P}_m{\displaystyle \frac{{\alpha }_{m,j^{\prime }}^k{L}_k}{R_{m,j^{\prime }}^0}}+\sum _{j\in \{m,u,c\}}{\kappa }_j{\alpha }_{m,j}^k{\omega }_k{f}_j^2\right)\right)\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & \left(\text{15a}\right), \left(\text{15b}\right), \left(\text{15g}\right),\hfill \end{array}$$

(25a)

$$\begin{array}{cc}\hfill & {T_m^k}^0\le T_m^{k,max},\forall m,\forall k,\hfill \end{array}$$

(25b)

where in $R_{m,j^{\prime }}^0$ and ${T_m^k}^0$, ${\mathit{r}}_u={\mathit{r}}_u^0$ and $\mathbf{\Theta }={\mathbf{\Theta }}^0$.

Consider the non-convexity of problem (25) under the given $\mathit{X}={\mathit{X}}^{*}$, ${\mathit{r}}_u^0$, and ${\mathbf{\Theta }}^0$. A proximal quadratic regularization term, i.e., ${\sum }_{m\in \mathcal{M}}{\sum }_{k\in \mathcal{K}}\xi {{\alpha }_{m,j}^k}^2$, is added to (25a) to overcome the issue, where $\xi$ is a positive scalar parameter. The regularized problem is equivalent to the original one in (25) as $\xi$→ 0. Therefore, $E_{prox}^{tot}\left({\mathit{X}}^{*},\alpha ,{\mathit{r}}_u^0,{\mathbf{\Theta }}^0\right)$ can be written as

$$\begin{array}{cc}\hfill & E_{prox}^{tot}\left({\mathit{X}}^{*},\mathit{\alpha },{\mathit{r}}_u^0,{\mathbf{\Theta }}^0\right)=\sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}\left(\left(1-\sum _{j^{\prime }\in \{u,c\}}{X_{j^{\prime }}^k}^{*}\right)\right.\hfill \\ \hfill & \left.\left(\sum _{j^{\prime }\in \{u,c\}}{P}_m{\displaystyle \frac{{\alpha }_{m,j^{\prime }}^k{L}_k}{R_{m,j^{\prime }}^0}}+\sum _{j\in \{m,u,c\}}\left({\kappa }_j{\alpha }_{m,j}^k{\omega }_k{f}_j^2+\xi {{\alpha }_{m,j}^k}^2\right)\right)\right).\hfill \end{array}$$

(26)

Hence, problem (25) is rewritten as

$$\begin{array}{cc}\hfill \underset{\begin{array}{c}\mathit{\alpha }\end{array}}{min}& E_{prox}^{tot}({\mathit{X}}^{*},\mathit{\alpha },{\mathit{r}}_u^0,{\mathbf{\Theta }}^0)\hfill \end{array}$$

(27a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& \left(\text{15a}\right), \left(\text{15b}\right), \left(\text{15g}\right), \left(\text{25b}\right).\hfill \end{array}$$

(27b)

According to (27), $E_{prox}^{tot}({\mathit{X}}^{*},\mathit{\alpha },{\mathit{r}}_u^0,{\mathbf{\Theta }}^0)$ is a function of $\mathit{\alpha }$, and its Hessian matrix is semi-positive definite. Given ${\mathit{X}}^{*}$, ${\mathit{r}}_u^0$, and ${\mathbf{\Theta }}^0$, the optimal task offloading decision vector of problem (27) is convex and can be obtained by using the KKT conditions. The Lagrange function of problem (27), denoted by $L(\mathit{\alpha },\mathit{\epsilon },\mathit{\chi },\mathit{\zeta },\mathit{\xi })$, is written as (28), where $\mathit{\epsilon }$, $\mathit{\chi }$, $\mathit{\zeta }$, and $\mathit{\xi }$ are the nonnegative Lagrangian multipliers associated with constraints (15a), (15b), (15g), and (25b), respectively.

$$\begin{array}{cc}\hfill L(\mathit{\alpha },\mathit{\epsilon },\mathit{\chi },\mathit{\zeta },\mathit{\xi })=& \sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}\left\{\left(1-\sum _{j^{\prime }\in \{u,c\}}{X_{j^{\prime }}^k}^{*}\right)\left(\sum _{j^{\prime }\in \{u,c\}}{P}_m{\displaystyle \frac{{\alpha }_{m,j^{\prime }}^k{L}_k}{R_{m,j^{\prime }}^0}}+\sum _{j\in \{m,u,c\}}\left(\xi {{\alpha }_{m,j}^k}^2\right.\right.\right.\hfill \\ \hfill & \left.\left.\left.+{\kappa }_j{\alpha }_{m,j}^k{\omega }_k{f}_j^2\right)\right)\right\}+{\epsilon }_m\left({\alpha }_{m,j}^k-1\right)+{\chi }_m\left(\sum _{j\in \{m,u,c\}}\left({\alpha }_{m,j}^k-1\right)\right)\hfill \\ \hfill & +{\zeta }_m\left(\sum _{k\in \mathcal{K}}{\alpha }_{m,j}^k{\omega }_k-f_j\right)+{\xi }_m\left({T_m^k}^0-T_m^{k,max}\right).\hfill \end{array}$$

(28)

According to the KKT conditions, the optimal task offloading decision vector ${\mathit{\alpha }}^{*}$ is given by

$${\mathit{\alpha }}^{*}=\underset{\left\{\tilde{\mathit{\alpha }},\tilde{\mathit{\epsilon }},\tilde{\mathit{\chi }},\tilde{\mathit{\zeta }},\tilde{\mathit{\xi }}\right\}}{argmin}{E}_{prox}^{tot}\left({\mathit{X}}^{*},\mathit{\alpha },{\mathit{r}}_u^0,{\mathbf{\Theta }}^0\right).$$

(29)

If $\left\{\tilde{\mathit{\alpha }},\tilde{\mathit{\epsilon }},\tilde{\mathit{\chi }},\tilde{\mathit{\zeta }},\tilde{\mathit{\xi }}\right\}$ is any point in the feasible solution set ${\mho }_{\alpha }$ that satisfies the KKT conditions (30), by solving (30), the feasible solution set ${\mho }_{\alpha }$ can be derived. The optimal task offloading decisions can be obtained, as given in (29).

$$\begin{array}{cc}\hfill & {\displaystyle \frac{\partial L}{\partial {\alpha }_{m,j}^k}}{|}_{{\alpha }_{m,j}^k={\tilde{\alpha }}_{m,j}^k,{\epsilon }_m={\tilde{\epsilon }}_m,{\chi }_m={\tilde{\chi }}_m,{\zeta }_m={\tilde{\zeta }}_i,{\xi }_m={\tilde{\xi }}_i}=0,\forall m,\forall k,\forall j,\hfill \end{array}$$

(30a)

$$\begin{array}{cc}\hfill & 0\le {\tilde{\alpha }}_{m,j}^k\le 1,\forall m,\forall k,\forall j,\hfill \end{array}$$

(30b)

$$\begin{array}{cc}\hfill & \sum _{j\in \{m,u,c\}}{\tilde{\alpha }}_{m,j}^k=1,\forall m,\forall k,\forall j,\hfill \end{array}$$

(30c)

$$\begin{array}{cc}\hfill & \sum _{k\in \mathcal{K}}{\tilde{\alpha }}_{m,j}^k{\omega }_k\le f_j,\forall m,\forall k,\forall j,\hfill \end{array}$$

(30d)

$$\begin{array}{cc}\hfill & {\widetilde{T_m^k}}^0\le T_m^{k,max},\forall m,\forall k,\forall j,\hfill \end{array}$$

(30e)

$$\begin{array}{cc}\hfill & {\tilde{\epsilon }}_m\left({\tilde{\alpha }}_{m,j}^k-1\right)=0,\forall m,\forall k,\forall j,\hfill \end{array}$$

(30f)

$$\begin{array}{cc}\hfill & {\tilde{\chi }}_m\left(\sum _{j\in \{m,u,c\}}\left({\tilde{\alpha }}_{m,j}^k-1\right)\right)=0,\forall m,\forall k,\forall j,\hfill \end{array}$$

(30g)

$$\begin{array}{cc}\hfill & {\tilde{\zeta }}_m\left(\sum _{k\in \mathcal{K}}{\tilde{\alpha }}_{m,j}^k{\omega }_k-f_j\right)=0,\forall m,\forall k,\forall j,\hfill \end{array}$$

(30h)

$$\begin{array}{cc}\hfill & {\tilde{\xi }}_m\left({\widetilde{T_m^k}}^0-T_m^{k,max}\right)=0,\forall m,\forall k,\forall j,\hfill \end{array}$$

(30i)

$$\begin{array}{cc}\hfill & {\tilde{\epsilon }}_m\ge 0,\forall m,\hfill \end{array}$$

(30j)

$$\begin{array}{cc}\hfill & {\tilde{\chi }}_m\ge 0,\forall m,\hfill \end{array}$$

(30k)

$$\begin{array}{cc}\hfill & {\tilde{\zeta }}_m\ge 0,\forall m,\hfill \end{array}$$

(30l)

$$\begin{array}{cc}\hfill & {\tilde{\xi }}_m\ge 0,\forall m.\hfill \end{array}$$

(30m)

5.2. UAV Hovering Position

Given $\mathit{X}={\mathit{X}}^{*}$, $\mathit{\alpha }={\mathit{\alpha }}^{*}$, and $\mathbf{\Theta }={\mathbf{\Theta }}^0$, problem (15) is rewritten as

$$\begin{array}{cc}\underset{\begin{array}{c}{\mathit{r}}_u\end{array}}{min}=\hfill & \sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}\left(\left(1-\sum _{j^{\prime }\in \{u,c\}}{X_{j^{\prime }}^k}^{*}\right)\sum _{j^{\prime }\in \{u,c\}}{P}_m{\displaystyle \frac{{\alpha }_{m,j^{\prime }}^k{L}_k}{R_{m,j^{\prime }}^0}}\right)\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & \left(\text{15e}\right),\hfill \end{array}$$

(31a)

$$\begin{array}{cc}\hfill & {T_m^k}^0\le T_m^{k,max},\forall m,\forall k,\hfill \end{array}$$

(31b)

where in $R_{m,j^{\prime }}^0$ and ${T_m^k}^0$, $\mathbf{\Theta }={\mathbf{\Theta }}^0$.

We adopt alternative optimization techniques and decompose the UAV hovering position problem into two steps: (1) Given $y_u=y_u^0$, problem (31) optimizes $x_u$, and we adopt SCA and the dual decomposition method to obtain an optimal solution, denoted by $x_u^{*}$. (2) Given $x_u=x_u^{*}$, problem (31) optimizes $y_u$, and we also adopt SCA and the dual decomposition method to obtain an optimal solution, denoted by $y_u^{*}$.

Given $y_u=y_u^0$, problem (31) can be rewritten as

$$\begin{array}{cc}\hfill \underset{\begin{array}{c}{x}_u\end{array}}{min}=& \sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}\left(\left(1-\sum _{j^{\prime }\in \{u,c\}}{X_{j^{\prime }}^k}^{*}\right)\sum _{j^{\prime }\in \{u,c\}}{P}_m{\displaystyle \frac{{\alpha }_{m,j^{\prime }}^k{L}_k}{R_{m,j^{\prime }}^0}}\right)\hfill \end{array}$$

(32a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& x_u\le x_u^{max},\hfill \end{array}$$

(32b)

$$\begin{array}{cc}\hfill & {T_m^k}^0\le T_m^{k,max},\forall m,\forall k,\hfill \end{array}$$

(32c)

where in $R_{m,j^{\prime }}^0$ and ${T_m^k}^0$, $y_u=y_u^0$ and $\mathbf{\Theta }={\mathbf{\Theta }}^0$.

The non-convexity of objective (32a) arises from the hovering position $x_u$ in ${\mathit{r}}_u$ in ${log}_2\left(1+{\gamma }_{m,j^{\prime }}^0\right)$. Considering that $R_{m,j^{\prime }}^0$ is replaced by their approximate variables, denoted by ${\tilde{R}}_{m,j^{\prime }}^0$, to transform (32) into a convex function, problem (32) can be reformulated as

$$\begin{array}{cc}\underset{\begin{array}{c}{x}_u\end{array}}{min}\hfill & \sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}\left(\left(1-\sum _{j^{\prime }\in \{u,c\}}{X_{j^{\prime }}^k}^{*}\right)\sum _{j^{\prime }\in \{u,c\}}{P}_m{\displaystyle \frac{{\alpha }_{m,j^{\prime }}^k{L}_k}{{\tilde{R}}_{m,j^{\prime }}^0}}\right)\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & \left(\text{32b}\right),\hfill \end{array}$$

(33a)

$$\begin{array}{cc}\hfill & {\widetilde{T_m^k}}^0\le T_m^{k,max},\forall m,\forall k,\hfill \end{array}$$

(33b)

$$\begin{array}{cc}\hfill & B_{m,j^{\prime }}{log}_2\left(1+{\gamma }_{m,j^{\prime }}^0\right)\ge {\tilde{R}}_{m,j^{\prime }}^0,\forall m,\forall j^{\prime }.\hfill \end{array}$$

(33c)

To address the non-convexity of (33c), the SCA method is used to derive a near-optimal solution. We first convexify ${log}_2\left(1+{\gamma }_{m,j^{\prime }}^0\right)$ by utilizing a logarithmic approximation as follows:

$${log}_2(1+{\gamma }_{m,j^{\prime }}^0)\ge {\displaystyle \frac{{\delta }_{m,j^{\prime }}ln{\gamma }_{m,j^{\prime }}^0+{\lambda }_{m,j^{\prime }}}{ln2}},$$

(34)

which is tight when ${\gamma }_{m,j^{\prime }}^0$ = ${\tilde{\gamma }}_{m,j^{\prime }}^0$. ${\delta }_{i,u}$ and ${\lambda }_{m,j^{\prime }}$ are two approximation constants about ${\gamma }_{m,j^{\prime }}^0$, defined as follows:

$${\delta }_{m,j^{\prime }}={\displaystyle \frac{{\tilde{\gamma }}_{m,j^{\prime }}^0}{1+{\tilde{\gamma }}_{m,j^{\prime }}^0}};$$

(35)

$${\lambda }_{m,j^{\prime }}=ln\left(1+{\tilde{\gamma }}_{m,j^{\prime }}^0\right)-{\displaystyle \frac{{\tilde{\gamma }}_{m,j^{\prime }}^0}{1+{\tilde{\gamma }}_{m,j^{\prime }}^0}}ln{\tilde{\gamma }}_{m,j^{\prime }}^0.$$

(36)

Let $x_u={\tilde{x}}_u$. Then, constraint (33c) can be approximated by its concave lower bound, as given by

$$R_{m,j^{\prime }}\ge B_{m,j^{\prime }}{\displaystyle \frac{{\delta }_{m,j^{\prime }}ln{\gamma }_{m,j^{\prime }}^0\left({\tilde{x}}_u\right)+{\lambda }_{m,j^{\prime }}}{ln2}}.$$

(37)

Therefore, problem (33) can be approximated by a convex problem, as given by

$$\begin{array}{cc}\underset{\begin{array}{c}{x}_u\end{array}}{min}\hfill & \sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}\left(\left(1-\sum _{j^{\prime }\in \{u,c\}}{X_{j^{\prime }}^k}^{*}\right)\sum _{j^{\prime }\in \{u,c\}}{P}_m{\displaystyle \frac{{\alpha }_{m,j^{\prime }}^k{L}_k}{{\tilde{R}}_{m,j^{\prime }}^0}}\right)\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & \left(\text{32b}\right), \left(\text{33b}\right),\hfill \end{array}$$

(38a)

$$\begin{array}{cc}\hfill & B_{m,j^{\prime }}{\displaystyle \frac{{\delta }_{m,j^{\prime }}ln{\gamma }_{m,j^{\prime }}^0\left({\tilde{x}}_u\right)+{\lambda }_{m,j^{\prime }}}{ln2}}\ge {\tilde{R}}_{m,j^{\prime }}^0,\forall m,\forall j^{\prime }.\hfill \end{array}$$

(38b)

The optimal hovering position $x_u^{*}$ in ${\mathit{r}}_u^{*}$ of UAV u can be readily solved by using CVX. We utilize the Lagrangian dual approach for better efficiency. The Lagrangian function $L(x_u,\mathit{\varsigma },\mathit{\iota },\mathit{\omega })$ is given in (39), where $\mathit{\varsigma }$, $\mathit{\iota }$, and $\mathit{\omega }$ are the Lagrangian multipliers associated with the constraints of problem (38).

$$\begin{array}{cc}\hfill & L(x_u,\mathit{\varsigma },\mathit{\iota },\mathit{\omega })=\sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}\left(\left(1-\sum _{j^{\prime }\in \{u,c\}}{X_{j^{\prime }}^k}^{*}\right)\sum _{j^{\prime }\in \{u,c\}}{P}_m{\displaystyle \frac{{\alpha }_{m,j^{\prime }}^k{L}_k}{{\tilde{R}}_{m,j^{\prime }}^0}}\right)+{\varsigma }_m\left(x_u-x_u^{max}\right)\hfill \\ \hfill & +{\iota }_m\left({\widetilde{T_m^k}}^0-T_m^{k,max}\right)+{\omega }_m\left({\tilde{R}}_{m,j^{\prime }}^0-B_{m,j^{\prime }}{\displaystyle \frac{{\delta }_{m,j^{\prime }}ln{\gamma }_{m,j^{\prime }}^0\left({\tilde{x}}_u\right)+{\lambda }_{m,j^{\prime }}}{ln2}}\right).\hfill \end{array}$$

(39)

The Lagrangian dual function of (39) can be given by

$$\begin{array}{c}\hfill D(\mathit{\varsigma },\mathit{\iota },\mathit{\omega })=\underset{\begin{array}{c}{x}_u\end{array}}{min}L(x_u,\mathit{\varsigma },\mathit{\iota },\mathit{\omega }).\end{array}$$

(40)

With the convexity of problem (38), the optimal solutions of both the original problem (38) and the dual problem (40) satisfy the KKT conditions. By solving ${\displaystyle \frac{\partial L\left(x_u,\mathit{\varsigma },\mathit{\iota },\mathit{\omega }\right)}{\partial x_u}}=0$, the optimal solution $x_u^{*}$ is derived. The optimal hovering position ${\mathit{r}}_u^{*}=(x_u^{*},y_u,h_u)$, where $x_u^{*}$ can be obtained as

$$\begin{array}{c}\hfill x_u^{*}=\underset{x_u}{argmin}L\left(x_u,\mathit{\varsigma },\mathit{\iota },\mathit{\omega }\right).\end{array}$$

(41)

The Lagrangian multipliers $\mathit{\varsigma }$, $\mathit{\iota }$, and $\mathit{\omega }$ are given by

$$\begin{array}{cc}\hfill {\varsigma }_m[k+1]=& {\left[{\varsigma }_m\left[k\right]+{△}_{\varsigma }\left[k\right]\left(x_u-x_u^{max}\right)\right]}^{+};\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill {\iota }_m[k+1]=& {\left[{\iota }_i\left[k\right]+{△}_{\iota }\left[k\right]\left({\widetilde{T_m^k}}^0-T_m^{k,max}\right)\right]}^{+};\hfill \end{array}$$

(43)

$$\begin{array}{cc}\hfill {\omega }_m[k+1]=& {\left[{\omega }_m\left[k\right]+{△}_{\omega }\left[k\right]\left({\tilde{R}}_{m,j^{\prime }}^0-B_{m,j^{\prime }}{\displaystyle \frac{{\delta }_{m,j^{\prime }}ln{\gamma }_{m,j^{\prime }}^0\left({\tilde{x}}_u\right)+{\lambda }_{m,j^{\prime }}}{ln2}}\right)\right]}^{+},\hfill \end{array}$$

(44)

where ${△}_1\left[k\right]=({△}_{\varsigma }\left[k\right],{△}_{\iota }\left[k\right],{△}_{\omega }\left[k\right]$ is the step size vector to update the Lagrangian multipliers $\mathit{\varsigma },\mathit{\iota }$, and $\mathit{\omega }$ in the k-th iteration, respectively. ${△}_1$ is also updated per iteration. Without loss of generality, we assume that each element in ${△}_1$ has the same step size in an iteration.

According to problem (41), the optimal solution $x_u^{*}$ is derived. Given $x_u=x_u^{*}$, problem (31) is non-convex, we also adopt SCA and the dual decomposition method to obtain an optimal solution, denoted by $y_u^{*}$. Therefore, the optimal hovering position can be given by

$$\begin{array}{c}\hfill {\mathit{r}}_u^{*}=(x_u^{*},y_u^{*},h_u).\end{array}$$

(45)

5.3. STAR-RIS Resource Allocation

Given $\mathit{X}={\mathit{X}}^{*}$, $\mathit{\alpha }={\mathit{\alpha }}^{*}$, and ${\mathit{r}}_u={\mathit{r}}_u^{*}$, problem (15) can be rewritten as

$$\begin{array}{cc}\underset{\begin{array}{c}\mathbf{\Theta }\end{array}}{min}\hfill & \sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}\left(\left(1-{X_c^k}^{*}\right)P_m{\displaystyle \frac{{\alpha }_{m,c}^k{L}_k}{{\tilde{R}}_{m,c}^{\prime }}}\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \left(\text{15c}\right), \left(\text{15d}\right),\hfill \end{array}$$

(46a)

$$\begin{array}{cc}\hfill & {T_m^k}^{\prime }\le T_m^{k,max},\forall m,\forall k,\hfill \end{array}$$

(46b)

where in $R_{m,c}^{\prime }$ and ${T_m^k}^{\prime }$, $\mathit{X}={\mathit{X}}^{*}$, $\mathit{\alpha }={\mathit{\alpha }}^{*}$, and ${\mathit{r}}_u={\mathit{r}}_u^{*}$.

The non-convexity of both objective (46a) and constraint (46b) arises from $\mathbf{\Theta }$ in ${log}_2\left(1+{\gamma }_{m,c}^{\prime }\right)$ for the given ${\mathit{X}}^{*}$, ${\mathit{\alpha }}^{*}$, and ${\mathit{r}}_u^{*}$. Similarly, an approximate variable ${\tilde{\gamma }}_{m,c}^{\prime }$ replaces ${\gamma }_{m,c}^{\prime }$ to overcome its non-convexity. Problem (46) is rewritten as

$$\begin{array}{cc}\underset{\begin{array}{c}\mathbf{\Theta }\end{array}}{min}\hfill & \sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}\left(\left(1-{X_c^k}^{*}\right)P_m{\displaystyle \frac{{\alpha }_{m,c}^k{L}_k}{{\tilde{R}}_{m,c}^{\prime }}}\right)\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & \left(\text{15c}\right), \left(\text{15d}\right),\hfill \end{array}$$

(47a)

$$\begin{array}{cc}\hfill & {\widetilde{T_m^k}}^{\prime }\le T_m^{k,max},\forall m,\forall k,\hfill \end{array}$$

(47b)

$$\begin{array}{cc}\hfill & {\gamma }_{m,c}^{\prime }\ge {\tilde{\gamma }}_{m,c}^{\prime },\forall m.\hfill \end{array}$$

(47c)

Being part of ${\gamma }_{m,c}^{\prime }$, ${\left|{\mathit{G}}_{s,c}^H{\mathbf{\Theta }}_a{\mathit{H}}_{m,s}\right|}^2$ can be rewritten as

$${\left|{\mathit{G}}_{s,c}^H{\mathbf{\Theta }}_a{\mathit{H}}_{m,s}\right|}^2={\left|{\mathit{\nu }}_a^H{\varphi }_{s,c}^H{\mathit{H}}_{m,s}\right|}^2={\mathit{\nu }}_a^H\mathbf{{\rm Y}}{\mathit{\nu }}_a,$$

(48)

where ${\mathit{\nu }}_a^H$, ${\mathit{\varphi }}_{s,c}^H$, and $\mathbf{{\rm Y}}$ can be written as

$$\begin{array}{cc}\hfill {\mathit{\nu }}_a^H=& \left[\sqrt{{\beta }_{(1,1)}^a}{e}^{j{\theta }_{(1,1)}^a},\sqrt{{\beta }_{(1,2)}^a}{e}^{j{\theta }_{(1,2)}^a},\dots ,\right.\hfill \\ \hfill & \left.\sqrt{{\beta }_{(m_c,m_r)}^a}{e}^{j{\theta }_{(m_c,m_r)}^a},\dots ,\sqrt{{\beta }_{(M_c,M_r)}^a}{e}^{j{\theta }_{(M_c,M_r)}^a}\right];\hfill \end{array}$$

(49)

$$\begin{array}{cc}\hfill {\mathit{\varphi }}_{s,c}^H=& \mathrm{diag}\left[g_{(1,1),c},g_{(1,2),c},\dots ,g_{(m_c,m_r),c},\dots ,g_{(M_c,M_r),c}\right];\hfill \end{array}$$

(50)

$$\begin{array}{cc}\hfill \mathbf{{\rm Y}}=& {\mathit{\varphi }}_{s,c}^H{\mathit{H}}_{m,s}{\mathit{H}}_{m,s}^H{\mathit{\varphi }}_{s,c}.\hfill \end{array}$$

(51)

As a result, constraints (15c) and (15d) are replaced by the following constraint:

$$\sum _{a\in \left\{r,t\right\}}{\mathit{\nu }}_a^2\left[(m_c,m_r)\right]\le 1,$$

(52)

where ${\mathit{\nu }}_a\left[(m_c,m_r)\right]$ is the $(m_c,m_r)$-th element of ${\mathit{\nu }}_a^H$. Clearly, (52) is convex.

To tackle non-convex constraint (47c), the SCA method is adopted. Specifically, since ${\mathit{\nu }}_a^H\mathbf{{\rm Y}}{\mathit{\nu }}_a$ is convex about ${\mathit{\nu }}_a$, its lower bound can be derived as follows:

$${\mathit{\nu }}_a^H\mathbf{{\rm Y}}{\mathit{\nu }}_a\ge -{\mathit{\nu }}_a^{\left(n\right)H}\mathbf{{\rm Y}}{\mathit{\nu }}_a^{\left(n\right)}+2\mathrm{Re}\left[{\mathit{\nu }}_a^H\mathbf{{\rm Y}}{\mathit{\nu }}_a^{\left(n\right)}\right],$$

(53)

where ${\mathit{\nu }}_a^{\left(n\right)}$ is obtained in the previous iteration. Then, constraint (47c) can be replaced by the following constraint:

$$-{\mathit{\nu }}_a^{\left(n\right)H}\mathbf{{\rm Y}}{\mathit{\nu }}_a^{\left(n\right)}+2\mathrm{Re}\left[{\mathit{\nu }}_a^H\mathbf{{\rm Y}}{\mathit{\nu }}_a^{\left(n\right)}\right]\ge {\displaystyle \frac{{\tilde{\gamma }}_{m,c}^{\prime }{\sigma }_c^2}{P_m}},$$

(54)

which is linear. As a result, problem (47) is rewritten as

$$\begin{array}{cc}\underset{\begin{array}{c}{\mathit{\nu }}_a\end{array}}{min}\hfill & \sum _{m\in \mathcal{M}}\sum _{k\in \mathcal{K}}\left(\left(1-{X_c^k}^{*}\right)P_m{\displaystyle \frac{{\alpha }_{m,c}^k{L}_k}{{\tilde{R}}_{m,c}^{\prime }}}\right)\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & \left(\text{47b}\right), \left(52\right), \left(54\right).\hfill \end{array}$$

(55)

Although convex problem (55) can be solved, e.g., by using the CVX toolbox, we utilize the Lagrangian dual approach to improve the computational efficiency. The Lagrangian function $L(\mathit{\nu },\mathit{\varrho },\mathit{\varpi },\mathit{\rho })$ is given in (56), where $\mathit{\varrho }$, $\mathit{\varpi }$, and $\mathit{\rho }$ are the Lagrangian multipliers associated with the three constraints of problem (55).

$$\begin{array}{cc}\hfill & L(\mathit{\nu },\mathit{\varrho },\mathit{\varpi },\mathit{\rho })=\sum _{k\in \mathcal{K}}\left(\left(1-{X_c^k}^{*}\right)P_m{\displaystyle \frac{{\alpha }_{m,c}^k{L}_k}{{\tilde{R}}_{m,c}^{\prime }}}\right)+{\varrho }_m\left[{\widetilde{T_m^k}}^{\prime }-T_m^{k,max}\right]\hfill \\ \hfill & +{\varpi }_m\left[\sum _{a\in \left\{r,t\right\}}{\mathit{\nu }}_a^2\left[(m_c,m_r)\right]-1\right]+{\rho }_m\left[{\displaystyle \frac{{\tilde{\gamma }}_{m,c}^{\prime }{\sigma }_c^2}{P_m}}+{\mathit{\nu }}_a^{\left(n\right)H}\mathbf{{\rm Y}}{\mathit{\nu }}_a^{\left(n\right)}-2\mathrm{Re}\left[{\mathit{\nu }}_a^H\mathbf{{\rm Y}}{\mathit{\nu }}_a^{\left(n\right)}\right]\right].\hfill \end{array}$$

(56)

The Lagrangian dual function is written as

$$D(\mathit{\varrho },\mathit{\varpi },\mathit{\rho })=\underset{\begin{array}{c}\nu \end{array}}{min}L\left(\mathit{\nu },\mathit{\varrho },\mathit{\varpi },\mathit{\rho }\right).$$

(57)

With the convexity of problem (55), the optimal solutions of both the original problem (55) and problem (57) satisfy the KKT conditions. By solving ${\displaystyle \frac{\partial L\left(\mathit{\nu },\mathit{\varrho },\mathit{\varpi },\mathit{\rho }\right)}{\partial {\nu }_a}}=0$, the optimal solution ${\mathit{\nu }}_a^{*}$ is derived. The optimal ${\mathit{\nu }}_a^{*}$ is obtained as

$${\mathit{\nu }}_a^{*}=\underset{{\mathit{\nu }}_a}{argmin}L\left(\mathit{\nu },\mathit{\varrho },\mathit{\varpi },\mathit{\rho }\right).$$

(58)

The Lagrangian multipliers $\mathit{\varrho },\mathit{\varpi }$, and $\mathit{\rho }$ are given by

$$\begin{array}{cc}\hfill {\varrho }_m\left[k+1\right]=& {\left\{{\varrho }_m\left[k\right]+{△}_{\varrho }\left[k\right]\left[{\widetilde{T_m^k}}^{\prime }-T_m^{k,max}\right]\right\}}^{+};\hfill \end{array}$$

(59)

$$\begin{array}{cc}\hfill {\varpi }_m\left[k+1\right]=& {\left\{{\varpi }_m\left[k\right]+{△}_{\varpi }\left[k\right]\left[\sum _{a\in \left\{r,t\right\}}{\mathit{\nu }}_a^2\left[(m_c,m_r)\right]-1\right]\right\}}^{+};\hfill \end{array}$$

(60)

$$\begin{array}{cc}\hfill {\rho }_m\left[k+1\right]=& {\left\{{\rho }_m\left[k\right]+{△}_{\rho }\left[k\right]\left({\displaystyle \frac{{\tilde{\gamma }}_{m,c}^{\prime }{\sigma }_c^2}{P_m}}+{\mathit{\nu }}_a^{\left(n\right)H}\mathbf{{\rm Y}}{\mathit{\nu }}_a^{\left(n\right)}-2\mathrm{Re}\left[{\mathit{\nu }}_a^H\mathbf{{\rm Y}}{\mathit{\nu }}_a^{\left(n\right)}\right]\right)\right\}}^{+},\hfill \end{array}$$

(61)

where ${△}_2\left[k\right]={({△}_{\varrho }\left[k\right],{△}_{\varpi }\left[k\right],{△}_{\rho }\left[k\right])}^T$ is the step size vector to update the Lagrangian multipliers $\mathit{\varrho }$, $\mathit{\varpi }$, and $\mathit{\rho }$ in the k-th iteration, respectively. ${△}_2$ is updated in each iteration.

According to (49) and (58), the optimal transmission and reflection matrix ${\mathbf{\Theta }}_a^{*}$ is obtained as

$${\mathbf{\Theta }}_a^{*}=\mathrm{diag}{\left({\mathit{\nu }}_a^{*}\right)}^H.$$

(62)

6. Convergence and Complexity Analysis

Algorithm 2 provides details of the DRL-SCA algorithm in its entirety. As a whole loop, the generated action ${\mathit{X}}^{*}$ by the PPO algorithm will be taken into (29), (38), and (55) as a given parameter for computing offloading decision, UAV hovering position, and STAR-RIS passive beamforming optimization. Then, utilizing block coordinate descent (BCD) to iterative optimize computing offloading decision ${\mathit{\alpha }}^{*}$, UAV hovering position ${\mathit{r}}_u=(x_u^{*},y_u^{*},h_u)$, and STAR-RIS passive beamforming $\mathbf{\Theta }$. Each iteration’s solution serves as the input of feasible points for the subsequent one. Therefore, ${\mathit{\alpha }}^{*}$, ${\mathit{r}}_u^{*}$, and ${\mathbf{\Theta }}^{*}$ will become a new state for the PPO agent in the next time slot.

Algorithm 2: DRL-SCA-based caching decision, offloading decision, hovering position of the UAV, and passive beamforming of STAR-RIS for UAV-based WSNs

Therefore, the change of action will eventually lead to the transition of state, which drives the agent to learn to improve the operation objective-related reward. Even if the system finds it hard to obtain an analytical prediction due to the environment complexity and embedded SCA, deep reinforcement learning can still pave the way to figure out a feasible solution in such a complicated dynamic environment.

Lemma 1. 

At least in finite iterations, Algorithm 2 can converge to a local suboptimal solution.

Proof. 

The initial problem (15) is decomposed into four subproblems and addressed iteratively by using the BCD method. Specifically, subproblems (16), (29), (38), and (55) are optimized in an alternating sequence to acquire a suboptimal solution. The solution obtained in each iteration is subsequently used as the feasible input for the following iteration.

Let $\eta ({\mathit{X}}^l,{\mathit{\alpha }}^l,{\mathit{r}}_u^l,{\mathbf{\Theta }}^l)$ represent the value of the original objective function (15) obtained during the l-th iteration.

The DRL algorithm generates an improved solution ${\mathit{X}}^{l+1}$ in Step 3 that meets the condition

$$\begin{array}{c}\eta ({\mathit{X}}^l,{\mathit{\alpha }}^l,{\mathit{r}}_u^l,{\mathbf{\Theta }}^l)\stackrel{\left(a\right)}{\ge }\eta ({\mathit{X}}^{l+1},{\mathit{\alpha }}^l,{\mathit{r}}_u^l,{\mathbf{\Theta }}^l),\hfill \end{array}$$

(63)

where (a) follows the inherent nature of learning approaches that always tend to seek a better reward defined in (19).

In Step 4, the suboptimal solution for offloading decision ${\mathit{\alpha }}^{l+1}$ can be obtained by solving (29), where (b) follows its convexity. In Step 5, the suboptimal solution for UAV hovering position ${\mathit{r}}_u^{l+1}$ can be obtained by solving (38), where (c) follows its convexity at the given feasible point due to constraints (32b), (33b), and (38b), and it can be optimally solved due to its convexity. In Step 6, the suboptimal solution for STAR-RIS passive beamforming ${\mathbf{\Theta }}^{l+1}$ can be obtained by solving (55), where (d) follows its convexity at the given feasible point due to constraints (47b), (52), and (54), and it can be optimally solved due to its convexity.

$$\begin{array}{cc}\eta ({\mathit{X}}^{l+1},{\mathit{\alpha }}^l,{\mathit{r}}_u^l,{\mathbf{\Theta }}^l)\hfill & \stackrel{\left(\mathrm{b}\right)}{\ge }\eta ({\mathit{X}}^{l+1},{\mathit{\alpha }}^{l+1},{\mathit{r}}_u^l,{\mathbf{\Theta }}^l)\hfill \\ \hfill & \stackrel{\left(\mathrm{c}\right)}{\ge }\eta ({\mathit{X}}^{l+1},{\mathit{\alpha }}^{l+1},{\mathit{r}}_u^{l+1},{\mathbf{\Theta }}^l)\hfill \\ \hfill & \stackrel{\left(\mathrm{d}\right)}{\ge }\eta ({\mathit{X}}^{l+1},{\mathit{\alpha }}^{l+1},{\mathit{r}}_u^{l+1},{\mathbf{\Theta }}^{l+1}).\hfill \end{array}$$

(64)

The inequality of Equation (64) induces that subproblems (29), (38), and (55) with regard to energy consumption are always nonincreasing after each iteration. It is observed that the objective function does not increase with each iteration. Due to the constraints, the minimum achievable energy consumption is bounded below by a finite value. Consequently, Algorithm 2 is assured to converge to at least a locally suboptimal solution within a finite number of iterations. □

7. Performance Evaluation and Discussion

In this section, we present numerical results to evaluate the effectiveness of the proposed energy-efficient aerial STAR-RIS-aided computing offloading and content caching framework for WSNs. The simulation scenario and parameter settings are initially outlined, followed by an in-depth discussion of the simulation results.

7.1. Simulation Scenario and Parameter Settings

In this section, we evaluate the performance of the proposed energy-efficient aerial STAR-RIS-aided computing offloading and content caching framework through numerical experiments. The simulation setup is detailed below.

We assume that the reference locations of the ground edge cloud (c), the UAV (u), and the STAR-RIS are positioned at $(0,20,0)$ meters, $(x_u^{*},y_u^{*},20)$ meters, and $(x_u^{*},y_u^{*},20)$ meters, respectively. Additionally, the four ground sensors are fixed at $(-40,40,0)$ meters, $(-40,-40,0)$ meters, $(40,40,0)$ meters, and $(40,-40,0)$ meters, representing a typical distributed sensor network configuration [37]. The detailed simulation parameters are summarized in

Table 2. Simulation parameters.

Notation	Simulation Value	Notation	Simulation Value
${\kappa }_j$	$1\times {10}^{-28}$	$P_m$	30 dBm
$f_c$	$1\times {10}^{10}$ r/s	$B_{m,j^{\prime }}$	3.2 MHz
$f_u$	$40\times {10}^7$ r/s	$L_k$	$6\times {10}^8$ bit
$f_m$	$2\times {10}^8$ r/s	${\omega }_k$	600 r/bit
$T_m^{k,max}$	0.1 s	$M_c$×$M_r$	9
$g_0$	$-30$ dB	$x_u^{max}$	40 m
${\sigma }_{n,n^{\prime }}^2$	$-174$ dBm	$y_u^{max}$	40 m

[38]. For example, the data size to be transmitted is set to $L_k=6\times {10}^8$ bits, representing a typical high-volume data transmission scenario in UAV-based WSNs. Moreover, we modeled data transmission as periodic, assuming a consistent flow of information typical of high-demand applications [39]. The maximum hovering range of the UAV is constrained within 40 m ($x_u^{max},y_u^{max}=40$ m), while the communication bandwidth $B_{m,j^{\prime }}$ is set to 3.2 MHz, ensuring sufficient transmission capacity for the tasks. $w_k$ represents the 600 r/bit computing resources required to complete task k. $M_c\times M_r$ indicates that the STAR-RIS consists of nine passive units.

For the caching DRL algorithm, detailed parameter settings are provided to ensure stability and convergence of the learning process: Different learning rates $\alpha$ are evaluated (e.g., 0.0001, 0.0003, and 0.0005) to assess their impact on convergence speed and stability. A learning rate of 0.0003 is found to provide the best balance between performance and robustness. The clipping parameter $ϵ$ is set to 0.2. This parameter constrains policy updates, ensuring stable and gradual improvements during training. The discount factor $\gamma$ is fixed at 0.99, which balances immediate rewards and long-term benefits, crucial to achieving consistent performance across episodes. These parameter settings were chosen to optimize the performance of the PPO algorithm within the DRL-SCA framework and ensure the reliable operation of the proposed model under dynamic conditions [40,41].

Moreover, to demonstrate the advantages of the proposed framework, we evaluate its performance in multiple existing comparative scenarios, including (1) full offloading, (2) fixed positions, (3) no STAR-RIS, and (4) no caching [30,42]. These existing comparative setups provide insights into the effectiveness of the proposed joint optimization framework in improving energy efficiency and overall system performance. All simulation scenarios are detailed below:

(1)

Proposed policy: We minimize the total energy consumption by jointly optimizing the content caching decision, offloading decision, the hovering position of the UAV, and the STAR-RIS transmission and reflection coefficient matrix in the STAR-RIS-aided UAV system.

(2)

Fixed-position policy: The fixed-position policy is to set the UAV position at a fixed position without dynamic adjustment. The reason for setting this comparison strategy is to analyze the effectiveness and advantages of optimizing the UAV hovering position.

(3)

Full offloading policy: The full offloading policy mainly offloads all request tasks sent by users to the UAV or cloud for independent processing. The reason for setting this comparison strategy is to analyze the effectiveness and advantages of the partial offloading strategy.

(4)

No-STAR-RIS policy: The no-STAR-RIS policy is to directly transmit the task request sent by the user to the edge cloud without using the STAR-RIS transmission and reflection coefficient matrix for transmission. The reason for setting this comparison strategy is to analyze the effectiveness and advantages of the STAR-RIS transmission and reflection coefficient matrix.

(5)

No-caching policy: The no–caching strategy consists of not to using the cache service capability on the UAV or edge cloud and not caching the requested content sent by the user in advance. Therefore, under this strategy, the user directly uses the partial offloading strategy to process the task locally or in the UAV or edge cloud. The reason for setting this comparison strategy is to analyze the effectiveness and advantages of the content caching strategy.

7.2. The Discussion of the Simulation Results

This section evaluates and discusses the performance of the proposed DRL-SCA algorithm in the energy-efficient aerial STAR-RIS-aided computing offloading and content caching framework by comparing it with various benchmark methods across diverse scenarios.

Figure 6 plots the total energy consumption of the system versus the number of iterations. We see that the proposed method can ensure that the total energy consumption of the system converges to the optimal value after only serval iterations, confirming that the optimal caching strategy, partial offloading strategy, the hovering position of the UAV, and the STAR-RIS transmission and reflection matrix are always available. Moreover, the convergence speed of the four benchmark schemes is slightly slower than that of the proposed scheme. Finally, this figure can verify that the energy consumption of the proposed STAR-RIS-aided computing offloading and content caching framework for UAV systems is less than that of other benchmark schemes.

Figure 7 shows the total energy consumption of the system versus the network bandwidth. The results show that the total energy consumption of the five schemes decreases with the increase in network bandwidth. The reason is that the increase in network bandwidth for offloading improves the transmission rate between sensors and the UAV, as well as the transmission rate among the sensors, the STAR-RIS, and the edge cloud, reducing the transmission delay and energy consumption. At a lower network bandwidth, the proposed caching strategy optimization, UAV hovering position optimization, partial offloading optimization, and STAR-RIS transmission reflection coefficient matrix significantly reduce energy consumption compared with other comparison strategies. However, as the network bandwidth increases, it helps to increase the task transmission rate, making the impact of network bandwidth in system energy consumption dominant. The system energy consumption of other benchmark solutions gradually approaches that of the proposed solution, but through the proposed joint optimization policy, our system performance is always optimal.

Figure 8 shows that as the CPU cycles required for computing 1 bit of task data increase, the system consumes more computing resources when calculating tasks of the same size, thereby increasing computing energy consumption. However, when the computing power of drones and local sensors is limited, more tasks are offloaded to the edge cloud for processing, which increases transmission energy consumption. At the same time, as the number of offloaded tasks increases, network resources become increasingly limited. Under the condition of limited network resources, the cache strategy optimization, drone hovering position optimization, and STAR-RIS transmission reflection coefficient matrix optimization strategies we proposed can significantly reduce system transmission energy consumption compared with other comparison strategies. Therefore, the system energy consumption gap between the proposed solution and other benchmark solutions will gradually widen. However, as network resources become increasingly limited, the network energy consumption under full offloading is only affected by computing resources, resulting in an increase in computing energy consumption, and the transmission energy consumption does not change much. Therefore, the system energy consumption of full offloading solutions gradually approaches that of the proposed partial offloading solution.

Figure 9 evaluates the total energy consumption of the proposed scheme and other benchmark schemes for various computation task sizes. First, the increase in computation task sizes leads to an increase in the total energy consumption of the system, because as the computation task sizes increase, transmission energy consumption gradually increases, and the computing energy consumption for processing unit tasks also increases. When local sensors and drones are unable to handle tasks due to their computation capability limitations, more computation tasks have to be offloaded to the distant edge cloud for processing. This not only increases computing energy consumption but also, due to the limited resources of UAVs and local sensors, offloading tasks to a distant cloud for processing, greatly increasing transmission energy consumption. Furthermore, the advantage of the proposed scheme over other benchmark schemes in the total energy consumption is marginal when we set the computation task size to small, while the advantage becomes increasingly substantial upon increasing the task size.

Figure 10 shows that the system energy consumption of all STAR-RIS-related schemes decreases significantly as the number of STAR-RIS elements increases. This is because STAR-RIS elements can provide more channel gain and effectively reduce transmission energy consumption. However, the no-STAR-RIS policy directly transmits some task requests sent by the sensor to the edge cloud without using the STAR-RIS transmission reflection coefficient matrix for transmission. Therefore, as the number of STAR-RIS elements increases, the system energy consumption under this strategy remains unchanged. At a smaller number of STAR-RIS elements, the energy consumption of the proposed cache strategy optimization, drone hovering position optimization, partial offloading optimization, and STAR-RIS transmission reflection coefficient matrix are significantly reduced compared with other comparison strategies. However, as the number of STAR-RIS elements increases, this provides more channel gain, which greatly reduces the transmission energy consumption of the system offloading tasks to a more distant cloud for processing. Therefore, the gap between the system energy consumption of other benchmark solutions and the energy consumption of the proposed solution gradually narrows, but through the proposed joint optimization strategy, our system performance is always in the optimal state.

Figure 11 illustrates that the system energy consumption of all schemes increases significantly with the increase in the sensors’ transmit power. This is because as the sensor’s transmission power increases, the transmission delay gradually decreases, but the transmission energy consumption is proportional to the sensor’s power. Therefore, when the sensor’s offloading ratio remains unchanged, the overall energy consumption of the system gradually increases. At smaller sensor’s transmit power, the energy consumption of the proposed cache strategy optimization, drone hovering position optimization, partial offloading optimization, and STAR-RIS transmission reflection coefficient matrix is significantly reduced compared with other comparison strategies. However, as the sensor’s transmission power increases, it greatly increases the transmission energy consumption of the system offloading tasks to the edge cloud or UAV for processing. Therefore, the gap between the system energy consumption of other benchmark solutions and the energy consumption of the proposed solution gradually narrows, but through the proposed joint optimization strategy, our system performance is always in the optimal state.

We define the SINR as ${\gamma }_{n,n^{\prime }}={\displaystyle \frac{P_n{\left|h_{n,n^{\prime }}\right|}^2}{{\sigma }_{n,n^{\prime }}^2}}$. It can be seen from Figure 12 that at greater SINR, the signal strength is relative to the noise. That is to say, the signal transmission quality will be better, thus improving transmission efficiency, which helps to reduce energy consumption during the transmission process. At a smaller SINR, the energy consumption of the proposed caching strategy optimization, UAV hovering position optimization, partial offloading optimization, and STAR-RIS transmission reflection coefficient matrix is significantly reduced compared with other comparison strategies. However, as the SINR increases, this provides a better channel state, which greatly reduces the transmission energy consumption of the system offloading tasks to a more distant cloud for processing. Therefore, the gap between the system energy consumption of other benchmark solutions and the energy consumption of the proposed solution gradually narrows, but through the proposed joint optimization strategy, our system performance is always in the optimal state.

Figure 13 illustrates the convergence of the average weighted reward per episode for the DRL caching agent at varying learning rates. As seen in the figure, it is evident that the DRL caching agent converges rapidly at all learning rates and achieves optimal performance at a learning rate of 0.0003. A higher learning rate results in the current Q-value having more influence than the prior Q-value, leading to faster updates. However, excessively high learning rates may destabilize the learning process and hinder convergence by over-adjusting to recent rewards. Therefore, selecting an appropriate learning rate is crucial to balancing the learning speed and stability for optimal performance.

The above simulation results demonstrate the effectiveness of the proposed DRL-SCA algorithm in optimizing energy consumption in the aerial STAR-RIS-aided computing offloading and content caching framework. Across various scenarios, our joint optimization strategy consistently outperforms benchmark methods. Specifically, the results reveal that the proposed framework achieves faster convergence in energy optimization, as shown by the rapid decrease in energy consumption over iterations. The analysis highlights that key factors such as network bandwidth, computation task size, CPU cycles per bit, STAR-RIS element count, and sensors’ transmission power significantly impact the system’s energy performance. For instance, the proposed strategy exhibits substantial energy savings, particularly under conditions of limited network resources or high computational demand, owing to the efficient coordination of offloading and caching decisions, UAV hovering positions, and STAR-RIS passive beamforming. Additionally, the results demonstrate that higher STAR-RIS element counts and better SINR conditions further enhance energy efficiency by improving channel gain and transmission quality. The sensitivity analysis of DRL caching learning rates confirms the stability and rapid convergence of the DRL component, with optimal performance achieved at a learning rate of 0.0003. Overall, the results validate that the proposed framework achieves superior energy efficiency and scalability while maintaining robustness under dynamic conditions, showcasing its applicability in real-world UAV-based wireless sensor networks.

8. Conclusions

In this paper, the energy-efficient STAR-RIS-aided computing offloading and content caching framework is proposed in order to meet the service requirements of delay-sensitive tasks for UAV-based WSNs. Firstly, we formulated the system energy consumption minimization problem, aiming to jointly optimize content caching decisions, computing offloading decisions, UAV hovering positions, and STAR-RIS passive beamforming. Subsequently, to tackle the non-convex problem of system energy consumption minimization, we decomposed it into four subproblems and proposed a DRL-SCA algorithm for iterative optimization, achieving near-optimal solutions with low complexity. According to numerical results, the suggested framework significantly reduces the network energy consumption of the overall system in aerial STAR-RIS-aided WSNs, exhibiting a fast convergence rate.

In our future research, we plan to investigate the impact of various design parameters, including the power-to-weight ratio, to further optimize the overall system performance. We will explore the application of multi-agent proximal policy optimization (MAPPO) to enhance the content caching decisions, computing offloading decisions, UAV hovering positions, and STAR-RIS passive beamforming decision process by enabling collaborative optimization among multiple agents, further improving adaptability and the overall performance of the framework. To enhance real-world applicability, future work will focus on developing lightweight optimization models for key parameters and testing the framework under dynamic conditions, such as varying data loads, network congestion, and hardware constraints. Additionally, we will incorporate more complex real-world factors, including UAV mobility, multi-task coordination, and heterogeneous sensor networks, to further expand the framework’s applicability. Finally, we plan to explore advanced feature extraction and training techniques to improve the scalability of MAPPO algorithms for large-scale and complex network environments.