Joint Beamforming and Trajectory Design for Aerial Intelligent Reflecting Surface-Aided Secure Transmission

Abstract

This paper studies the secure transmission challenge confronted with future communication systems. In our considered model, the confidential communication between legitimate users is strengthened by an aerial intelligent reflecting surface (AIRS) deployed on aerial platforms such as an unmanned aerial vehicle (UAV). The average secrecy rate for all time slots is first investigated to improve information security during AIRS flights. Then, the transmit beamforming, phase-shifting matrix, and trajectory of AIRS are jointly designed, aiming to maximize the average secrecy rate performance between legitimate users. On account of the non-convexity of the formulated objective function and the coupled three key variables, we resort to an alternative strategy that converts the original objective into three sub-problems and solves them recursively. In particular, the transmit beamforming is designed on the basis of the generalized eigenvalue optimization method, and the closed-form solution is derived. For the AIRS-related optimization, a Minorization-Maximization (MM)-based algorithm and a deep deterministic policy gradient (DDPG)-based method are proposed to derive the solutions of phase shift matrix and trajectory, respectively. Simulation results show that AIRS assistance can obtain nearly twice the secrecy performance of terrestrial intelligent reflecting surface (TIRS) systems.

1. Introduction

Wireless communication revolutionizes human life in various aspects, gradually leading us to a more convenient and intelligent society. Despite the advantages of wireless communication, the broadcast nature of the wireless medium makes it vulnerable to interception attacks by malicious users. This poses a challenge to secure wireless communication. To conquer this challenge, physical layer security (PLS) emerged and attracted tremendous attention from both academia and industry [1,2,3]. As a valuable method to ensure the security of wireless communication, PLS utilizes the randomness and uniqueness of wireless channels to protect the confidential signals from being intercepted by adversaries and achieve secure and reliable communication [4].

As a promising technology of PLS, cooperative relaying technology can help expand the communication range and maintain the wireless network’s security [5,6]. However, the performance gain of traditional active wireless relays require an additional power consumption, and relays usually work in a half-duplex model with lower spectral efficiency. Even though the full-duplex mode can enhance the spectral efficiency, additional interference is introduced in confidential signal transmission [7]. Recently, the emerging intelligent reflective surface (IRS) technology provides a novel solution to the above challenges in PLS systems [8,9,10,11]. The IRS is a uniform planar array composed of amount of composite material elements, and the phase shift of each composite element can be independently regulated to reflect incident electromagnetic waves [12,13]. Thereby, the propagation environment and performance can be improved with the assistance of IRS. Moreover, compared with cooperative relays, IRS also has the advantage of realizing full-duplex communication with lower energy consumption because the passive reflecting elements do not decode or amplify and forward information signals [14]. In [15,16], the programmable IRS was introduced to assist confidential information transmission, and the active beamformer at transmitter and passive beamformer at IRS were jointly optimized for secrecy performance maximization. To further enhance information security, IRS-empowered secure transmission scenarios are introduced into multiple-input multiple-output (MIMO) systems [17,18] as well as millimeter-wave (mmWave) and terahertz (THz) systems [19,20]. It is proved that, in comparison to baseline schemes without IRS, the help of the IRS can significantly suppress information leakage at malicious users, hence boosting the secrecy performance.

The IRS deployments involved in the above work mainly focus on the terrestrial IRS (TIRS). That is, the IRS is installed on the walls or building surfaces in terrestrial networks. Such a deployment restricts TIRS only to achieve half-angle reflection (i.e., 180 degree panoramic angle reflection) and serve users in half of the space [21]. In complicated environments such as urban areas, the half-angle reflection of TIRS makes it necessary for multiple IRSs to cover all users in the wireless network, which results in both additional communication overhead and signal attenuation. Recently, the aerial IRS (AIRS) that can establish full-angle reflection has attracted much attention [22]. In the AIRS-aided communications, AIRS is mounted on aerial platforms such as UAVs and satellites, enjoying more flexible deployment. Thus, AIRS can cater to the real-time communication environment and easily construct hardened line-of-sight (LoS) links for both ground and aerial users. It is shown in [21] that AIRS promotes a higher probability of LoS links between communication users, and its achievable data rate is more than twice that of TIRS. However, the benefits of AIRS in secure communications are rarely explored. Recent work in [23,24,25,26,27] mainly focused on AIRS-authorized communications without regard to information security or the TIRS-aided secure air-ground communications. Specifically, Refs. [23,24] jointly designed AIRS’s placement and three dimensions ($3D$) passive beamforming in one time slot to maximize the worst-case signal-to-noise ratio (SNR) in the target communication area. Refs. [25,26,27] proposed joint optimization schemes of TIRS phase shift and UAV trajectory based on successive convex approximation (SCA) and a deep deterministic policy gradient (DDPG) framework to information security. The above IRS-aided secure transmission schemes in [23,24,25,26,27] cannot be directly applied in AIRS-empowered secure communication systems because of different system models. In summary, all the above work is listed in

Table 1. Related work on IRS and UAV assisted communication.

Reference	Scenario	Description	Design Objective
[15,16]	Secure communication in MISO system	TIRS-assisted secure communication between multi-antenna access point and a ground user	Maximize the secrecy rate of the legitimate communication link
[17,18]	Secure transmission in MIMO system	TIRS-aided secure transmission from BS to multi-users	Maximize the secrecy rate
[19,20]	Secure transmission in mmWave/THz systems	TIRS-aided secure transmission of terrestrial network in mmWave/THz bands	Maximize the system secrecy rate
[22]	AIRS-aided cellular network	AIRS-assisted communication between BS and aerial users	Maximize mean signal-to-interference-noise ratio (SINR)
[23,24]	AIRS-assisted communication system	AIRS-enhanced terrestrial communication within a given area	Maximize the worst-case signal-to-noise ratio
[25]	Secure UAV communication system	TIRS-aided secure transmission from UAV to a terrestrial user	Maximize the average secrecy rate
[26]	Robust secure UAV communication system	TRIS-aided secure transmission in mmWave UAV system with imperfect CSI	Maximize the sum secrecy rate of all users
[27]	TIRS-aided UAV communication system	Multi-TIRSs aided communication from UAV to ground user	Maximize the received power at the ground user

. It can be seen that the listed IRS-aided secure UAV communication schemes focus on terrestrial IRS-aided propagation between UAV and ground users rather than the secure transmission problem between ground users. Moreover, the AIRS mounted on the aerial platform enjoys the flexibility and mobility of aerial platforms and the passive reflection characteristics of IRS, but results in coupled phase shift and trajectory design, leading to an intractable secure transmission problem.

Inspired by the aforementioned challenges, in this paper, we investigate the AIRS-assisted secure communication system, where the transmitter with multi-antennas tries to emit confidential signals to its desired destination, while one passive eavesdropper remains silent and tries to decode the confidential signals. Since the direct link is blocked by the infrastructure, AIRS mounted on an aerial platform is employed to facilitate LoS links and help secure transmission. The main objective is to maximize the average secrecy rate during flight period of AIRS, and investigate the effects of the phase-shift and trajectory of AIRS on secure transmission. We summarize our contributions as follows.

• A novel AIRS-assisted secure transmission scenario is investigated. With massive reflecting elements mounted on an aerial platform, AIRS facilitates the free-space LoS link transmission of confidential signals.
• Aiming to guarantee the information security, the average secrecy rate during the flight time is introduced as the secrecy performance metric. Then, we formulate the average secrecy rate maximization problem by jointly designing transmit beamforming, phase-shift matrix, and trajectory of AIRS.
• To cope with the non-convex problem that is formulated, we propose an iterative algorithm. The solutions are derived alternately by dividing the original problem into three sub-problems.
• For the AIRS, the Minorization-Maximization (MM) algorithm-based method is proposed to derive the closed-form solution of phase shifts. Additionally, the DDPG-based algorithm is proposed to design AIRS’s trajectory.

2. System Model

In this paper, an AIRS-empowered secure communication system is considered, in which a source node Alice with $N_t$ antennas emits confidential signals to its desired user Bob. At the same time, Eve attempts to intercept the emitted information as a passive eavesdropper. As shown in Figure 1, the direct connection between Alice and Bob is blocked by some infrastructure in a complex urban environment, and an AIRS is exploited to help the confidential signal transmission from Alice to Bob. We assume that the AIRS consists of M reflecting elements and an aerial platform (e.g., UAV [28]). Owing to massive reflector elements installed on the aerial platform, AIRS can make the transmit beam diffract obstacles and establish an LoS link between ground users.

In the AIRS-assisted secure communication system, the height of Alice is assumed as $h_A$, and then its $3D$ Cartesian coordinate location can be denoted as $[{\mathbf{l}}_A,h_A]=[x_A,y_A,h_A]$. The locations of the legitimate user Bob and the eavesdropper Eve are assumed to be $[{\mathbf{l}}_D,0]=[x_D,y_D,0]$ and $[{\mathbf{l}}_E,0]=[x_E,y_E,0]$, respectively. For the AIRS, we assume the flight period T of the aerial platform is divided into N equally spaced time slots, and each time slot duration is ${\delta }_t$, i.e., $T=N{\delta }_t$. In the n-th time slot, the location of the AIRS is assumed to be $\left[\mathbf{q}\right[n],h]$, where h is the constant altitude during AIRS flying to avoid collision with buildings, and $\mathbf{q}\left[n\right]={[x\left[n\right],y\left[n\right]]}^T$ is the coordinate in the $xy$-plane. In particular, due to mission and network deployment requirements, the starting and target positions of AIRS are defined as $[{\mathbf{q}}_0,h]=[x_0,y_0,h]$ and $[{\mathbf{q}}_T,h]=[x_T,y_T,h]$, respectively. The maximum speed of the AIRS during flight is assumed to be $V_{max}$, thereby the maximum distance that the AIRS can fly between two adjacent slots is $V_{max}{\delta }_t$, ${\delta }_t$ denoting the time duration of each time slot.

Note that, as mentioned in [29,30], the mobility of aerial platforms may lead to challenges in obtaining instantaneous CSI of aerial platform-ground user links. Here, we assume that the ground control station connected with AIRS can make full use of all available resources to estimate the CSI and angle of arrival between users and the aerial platform coated with AIRS [31]. Moreover, we suppose that the eavesdropper is an idle legal user in the communication network but unauthorized for the private signals. Thus, both legitimate and wiretap CSI can be perfectly known to the system.

2.1. Channel Model

Since the AIRS is deployed on an air platform and flies in the air, the Alice-IRS link is dominated by the LoS path. Thus, the free-space path loss model is employed, then the Alice-IRS channel of the nth time slot is assumed as

$${\mathbf{H}}_{SI}^H\left[n\right]=\sqrt{{\rho }_0{d}_{AI}^{-2}\left[n\right]}{\mathbf{a}}_r({\vartheta }_a^I,{\vartheta }_e^I){\mathbf{a}}_t{\left({\vartheta }_a^A\right)}^H,$$

(1)

wherein the distance from Alice to AIRS is assumed as $d_{AI}\left[n\right]=\sqrt{\parallel \mathbf{q}\left[n\right]-{\mathbf{l}}_A{\parallel }^2+{\parallel h-h_A\parallel }^2}$ and the channel gain at the reference distance $d_0=1$ m is denoted as ${\rho }_0$. ${\mathbf{a}}_r({\vartheta }_a^I,{\vartheta }_e^I)$ is the receive array response vector at the AIRS, and ${\vartheta }_a^I$ and ${\vartheta }_e^I$ are the azimuth and the elevation angles of arrival (AoA). In addition, ${\mathbf{a}}_t\left({\vartheta }_a^A\right)$ represents the transmit array response vector at Alice and ${\vartheta }_a^A$ is the azimuth angles of departure (AoD). With respect to the uniform square planner array (USPA) assumption at AIRS (The USPA with $\sqrt{M}\times \sqrt{M}$-element on the $xz$-plane is adopted.), the receive array response vector for AIRS ${\mathbf{a}}_r({\vartheta }_a^I,{\vartheta }_e^I)$ is written as

$$\begin{array}{cc}\hfill {\mathbf{a}}_r({\vartheta }_a^I,{\vartheta }_e^I)=& \hfill \frac{1}{\sqrt{M}}\left[1,\dots ,e^{j\frac{2\pi }{\lambda }d\left(pcos\left({\vartheta }_a^I\right)sin\left({\vartheta }_e^I\right)+qcos\left({\vartheta }_e^I\right)\right)},\right.\hfill \\ \hfill & \hfill {\left.\dots ,e^{j\frac{2\pi }{\lambda }d\left((\sqrt{M}-1)cos\left({\vartheta }_a^I\right)sin\left({\vartheta }_e^I\right)+(\sqrt{M}-1)cos\left({\vartheta }_e^I\right)\right)}\right]}^T,\hfill \end{array}$$

(2)

where $d=\lambda /2$ is the distance between two adjacent antennas at AIRS, $\lambda =c/f_c$ is the length of wave. The parameters $0\le p\le \sqrt{M}$ and $0\le q\le \sqrt{M}$ are the antenna indices in the $2D$ plane.

A uniform linear array (ULA) is utilized at transmitter Alice, thereby the transmit array response vector ${\mathbf{a}}_t\left({\vartheta }_a^A\right)$ can be expressed as

$${\mathbf{a}}_t\left({\vartheta }_a^A\right)=[1,e^{j\frac{2\pi }{\lambda }dsin\left({\vartheta }_a^A\right)},\dots ,e^{j(N_t-1)\frac{2\pi }{\lambda }dsin\left({\vartheta }_a^A\right)}].$$

(3)

Then, the channel between AIRS and Bob or Eve is expressed as

$${\mathbf{g}}_k\left[n\right]=\sqrt{{\rho }_0{d}_k^{-\alpha }\left[n\right]}{\mathbf{a}}_t({\vartheta }_{a,k}^I,{\vartheta }_{e,k}^I),$$

(4)

in which $k=\{ID,IE\}$ is an index term to indicate the link between AIRS and Bob or Eve. $d_{ID}=\sqrt{\parallel \mathbf{q}\left[n\right]-{\mathbf{l}}_B{\parallel }^2+h^2}$ is the distance from AIRS to Bob, and $d_{IE}=\sqrt{\parallel \mathbf{q}\left[n\right]-{\mathbf{l}}_E{\parallel }^2+h^2}$ denotes the distance from AIRS to Eve. $\alpha$ is the path loss exponent, and ${\mathbf{a}}_t({\vartheta }_{a,k}^I,{\vartheta }_{e,k}^I)$ is the transmit array response vector at AIRS and is obtained according to (2).

2.2. Signal Model

In our considered AIRS-assisted secure communication system, the source node Alice tries to emit confidential signal $x_s$ to AIRS using a transmit beamformer $\mathbf{w}\left[n\right]$, then AIRS adjusts the phase shift of its elements to reflect signals to Bob, and Eve can also intercept the reflected signal. At the n-th time slot, the signal received by Bob can be obtained as

$$y_D={\mathbf{g}}_{ID}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}{\left[n\right]}^H\mathbf{w}\left[n\right]x_s+n_D,$$

(5)

where $\mathbb{E}\left\{\right|x_s{|}^2\}=P_s$, the transmit beamformer at Alice is $\mathbf{w}\left[n\right]\in {\mathbb{C}}^{N_t\times 1}$ with ${\parallel \mathbf{w}\left[n\right]\parallel }^2\le 1$. ${\mathbf{g}}_{ID}\left[n\right]\in {\mathbb{C}}^{M\times 1}$ is the AIRS-Bob channel and $n_D\sim \mathcal{CN}(0,N_0)$ is the additive white Gaussian noise (AWGN) received at the Bob. We assume $\mathsf{\Theta }\left[n\right]=\mathrm{diag}{\{e^{j{\theta }_1\left[n\right]},\dots ,e^{j{\theta }_M\left[n\right]}\}}^T$ denote the reflecting matrix at AIRS and ${\theta }_m\left[n\right]\in [0,2\pi ),m=1,\dots ,M$ is the phase shift of reflecting element at the n-th time slot.

Then, the signal intercepted by Eve is represented as

$$y_E={\mathbf{g}}_{IE}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}^H\mathbf{w}\left[n\right]x_s+n_E.$$

(6)

where ${\mathbf{g}}_{IE}\left[n\right]\in {\mathbb{C}}^{M\times 1}$ is the AIRS-Eve channel, which can be obtained based on (4). $n_E\sim \mathcal{CN}(0,N_0)$ is received noise.

In terms of the received signals in the above equations, the signal-to-noise ratio (SNR) at Bob and Eve are represented as

$${\gamma }_D=\frac{P_s}{N_0}{|{\mathbf{g}}_{ID}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}^H\left[n\right]\mathbf{w}\left[n\right]|}^2.$$

(7)

$${\gamma }_E=\frac{P_s}{N_0}{|{\mathbf{g}}_{IE}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}^H\left[n\right]\mathbf{w}\left[n\right]|}^2.$$

(8)

Then, the received information rates at Bob and Eve are obtained as

$$R_D\left[n\right]={log}_2(1+\frac{P_s}{N_0}|{\mathbf{g}}_{ID}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}^H\left[n\right]\mathbf{w}\left[n\right]{|}^2),$$

(9)

$$R_E\left[n\right]={log}_2(1+\frac{P_s}{N_0}|{\mathbf{g}}_{IE}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}^H\left[n\right]\mathbf{w}\left[n\right]{|}^2).$$

(10)

According to the definition of secrecy rate in [32], the secrecy rate performance for an average of N time slots, i.e., average secrecy rate, is written as

$$\begin{array}{ll}{R}_s^{aver}& =\frac{1}{N}\sum _{n=1}^N{R}_s\left[n\right]\\ & =\frac{1}{N}\sum _{n=1}^N{[R_D\left[n\right]-R_E\left[n\right]]}^{+}\\ & =\frac{1}{N}\sum _{n=1}^N{log}_2\left(\frac{1+\frac{P_S}{N_0}{|{\mathbf{g}}_{ID}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}^H\left[n\right]\mathbf{w}\left[n\right]|}^2}{1+\frac{P_S}{N_0}{|{\mathbf{g}}_{IE}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}^H\left[n\right]\mathbf{w}\left[n\right]|}^2}\right)\end{array}$$

(11)

wherein ${[·]}^{+}$ means that the secrecy rate of the communication is non-negative, e.g., ${\left[a\right]}^{+}=max(a,0)$.

2.3. Problem Formulation

With the objective to maximize the achievable average secrecy rate in N time slots, a joint optimization problem of transmit beamforming, phase-shift, and trajectory of the AIRS is formulated, i.e.,

$$\underset{\mathbf{w}\left[n\right],\mathsf{\Theta }\left[\mathbf{n}\right],\mathbf{Q}}{max}\frac{1}{N}\sum _{n=1}^N{R}_s\left[n\right]$$

$$\mathrm{s}.\mathrm{t}.\left\{\begin{array}{ll}C1:& {\parallel \mathbf{w}\left[n\right]\parallel }^2\le 1,\\ C2:& {\theta }_m\in [0,2\pi ),m=1,\dots ,M,\\ C3:& \mathbf{q}\left[0\right]={\mathbf{q}}_0,\mathbf{q}\left[N\right]={\mathbf{q}}_T,\\ C4:& \parallel \mathbf{q}[n+1]-\mathbf{q}\left[n\right]\parallel \le V_{max}{\delta }_t,n=1,\dots ,N.\end{array}\right.$$

(12)

where $\mathbf{Q}=\left[\mathbf{q}\left[0\right],\dots ,\mathbf{q}\left[n\right],\dots ,\mathbf{q}\left[N\right]\right]$ denotes AIRS trajectory. The first constraint $C1$ is the transmit power constraint of transmit beamformer $\mathbf{w}$, and the constraint $C2$ is the adjustable range of AIRS’s phase-shift. Constraints $C3$ and $C4$ are related to AIRS’s position and trajectory. In particular, $C3$ constrains the starting position before the first slot and the target position at the end of N slots, while $C4$ represents the maximum distance constraint between two adjacent slots due to the limitation of maximum flight speed.

It is intuitive that $C2$ is the non-convex constraint since it establishes the unit modulus constraints of the elements of $\mathsf{\Theta }$. Moreover, in the objective function three variables $\left(\mathbf{w}\right[n],\mathsf{\Theta }[n],\mathbf{Q})$ are coupled with each other for each time slot. The above conditions result in the non-convexity of the formulated problem in (12), which is difficult to solve. As an alternative, the following iterative algorithm is proposed, and the solutions of transmit beamforming, phase-shift matrix, and trajectory of AIRS will be derived alternately.

3. Average Secrecy Rate Maximization

Instead of seeking the solutions of the formulated average secrecy rate maximization problem in (12) directly, we try to divide the original problem into several sub-problems of $\mathbf{w}\left[n\right]$, $\mathsf{\Theta }\left[n\right]$, and $\mathbf{Q}$, respectively, then solve them recursively. The idea is based on the fact that, for the objective function, the transmit beamforming $\mathbf{w}\left[n\right]$ design is only related to the phase shift $\mathsf{\Theta }\left[n\right]$ as well as the position $\mathbf{q}\left[n\right]$ at the same time slot, while it is independent of the variables of other time slots. Thus, we propose an iterative algorithm to derive $\mathbf{w}\left[n\right]$ and $\mathsf{\Theta }\left[n\right]$ alternately. With the obtained $\mathbf{w}\left[n\right]$ and $\mathsf{\Theta }\left[n\right]$, the trajectory of AIRS $\mathbf{q}\left[n\right]$ at the n-th time slot is designed using the DDPG algorithm, which is a deep reinforcement learning algorithm for learning continuous actions.

3.1. Transmit Beamforming Design

Given the phase-shift matrix $\mathsf{\Theta }\left[n\right]$ and the trajectory of AIRS, the sub-problem of the transmit beamforming vector $\mathbf{w}\left[n\right]$ at Alice can be written as

$$\underset{\mathbf{w}\left[n\right]}{max}\frac{1}{N}\sum _{n=1}^N{log}_2\left(\frac{{\mathbf{w}}^H\left[n\right]\left[\frac{1}{\xi }{\mathbf{I}}_{N_t}+\frac{P_s}{N_0}{\mathbf{H}}_{SI}\left[n\right]\mathsf{\Theta }{\left[n\right]}^H{\mathbf{g}}_{ID}\left[n\right]{\mathbf{g}}_{ID}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}^H\left[n\right]\right]\mathbf{w}\left[n\right]}{{\mathbf{w}}^H\left[n\right]\left[\frac{1}{\xi }{\mathbf{I}}_{N_t}+\frac{P_s}{N_0}{\mathbf{H}}_{SI}\left[n\right]\mathsf{\Theta }{\left[n\right]}^H{\mathbf{g}}_{IE}\left[n\right]{\mathbf{g}}_{IE}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}^H\left[n\right]\right]\mathbf{w}\left[n\right]}\right).$$

$${\mathrm{s}.\mathrm{t}.\parallel \mathbf{w}\left[n\right]\parallel }^2=\xi ,0\le \xi \le 1.$$

(13)

In the above problem, we definite a parameter $\xi \in [0,1]$ and introduce it to convert the inequality power constraint into equality. It can be observed that the sub-problem with respect to transmit beamformer $\mathbf{w}\left[n\right]$ is a generalized eigenvector problem. Hence, we proposed the following Proposition 1 to obtain the solution of (13).

Proposition1. 

The closed-form solution of $\mathbf{w}\left[n\right]$ that maximizes the average secrecy rate can be derived as

$${\mathbf{w}}^{opt}\left[n\right]={\mathbf{e}}^{unit},$$

(14)

where ${\mathbf{e}}^{unit}$ is the unit-norm eigenvector of matrix ${\mathbf{H}}_E^{-1}{\mathbf{H}}_D$ based on its maximum eigenvalue. The matrix ${\mathbf{H}}_E$ is defined as ${\mathbf{H}}_E={\mathbf{I}}_{N_t}+\frac{P_s}{N_0}{\mathbf{H}}_{SI}\left[n\right]\mathsf{\Theta }{\left[n\right]}^H{\mathbf{g}}_{IE}\left[n\right]{\mathbf{g}}_{IE}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}^H\left[n\right]$, while matrix ${\mathbf{H}}_D={\mathbf{I}}_{N_t}+\frac{P_s}{N_0}{\mathbf{H}}_{SI}\left[n\right]\mathsf{\Theta }{\left[n\right]}^H{\mathbf{g}}_{ID}\left[n\right]{\mathbf{g}}_{ID}^H\left[n\right]\mathsf{\Theta }\left[n\right]{\mathbf{H}}_{SI}^H\left[n\right]$.

Proof. 

Please see Appendix A.    □

3.2. Phase-Shift Design

With the given AIRS trajectory and the obtained transmit beamformer $\mathbf{w}\left[n\right]$, the average secrecy rate maximization problem of phase-shift $\mathsf{\Theta }\left[n\right]$ can be written as

$$\underset{\widehat{\mathbf{\theta }}\left[n\right]}{max}\frac{1}{N}\sum _{n=1}^N{log}_2\left(\frac{{\widehat{\mathbf{\theta }}}^T\left[n\right][\frac{1}{M}{\mathbf{I}}_M+{\overline{\mathbf{R}}}_D]{\widehat{\mathbf{\theta }}}^{*}\left[n\right]}{{\widehat{\mathbf{\theta }}}^T\left[n\right][\frac{1}{M}{\mathbf{I}}_M+{\overline{\mathbf{R}}}_E]{\widehat{\mathbf{\theta }}}^{*}\left[n\right]}\right).$$

$$\mathrm{s}.\mathrm{t}.\left\{\begin{array}{ll}\widehat{\mathbf{\theta }}\left[n\right]={\left[e^{j{\theta }_1\left[n\right]},\dots ,e^{j{\theta }_m\left[n\right]},\dots ,e^{j{\theta }_M\left[n\right]}\right]}^T,& \\ {\theta }_m\left[n\right]\in [0,2\pi ),m=1,\dots ,M.\end{array}\right.$$

(15)

In the above problem, we define the vector $\widehat{\mathbf{\theta }}\left[n\right]$, then the phase-shift matrix $\mathsf{\Theta }\left[n\right]$ in (12) is expressed as $\mathsf{\Theta }\left[n\right]=\mathrm{diag}\left\{\widehat{\mathbf{\theta }}\left[n\right]\right\}$. The matrices ${\overline{\mathbf{R}}}_D$ and ${\overline{\mathbf{R}}}_E$ are expressed as ${\overline{\mathbf{R}}}_D=\frac{P_s}{N_0}\mathrm{diag}\left\{{\mathbf{g}}_{ID}^H\left[n\right]\right\}{\mathbf{H}}_{SI}^H\left[n\right]\mathbf{w}\left[n\right]{\mathbf{w}}^H\left[n\right]{\mathbf{H}}_{SI}\left[n\right]\mathrm{diag}\left\{{\mathbf{g}}_{ID}\left[n\right]\right\}$ and ${\overline{\mathbf{R}}}_E=\frac{P_s}{N_0}\mathrm{diag}\left\{{\mathbf{g}}_{IE}^H\left[n\right]\right\}$${\mathbf{H}}_{SI}^H\left[n\right]\mathbf{w}\left[n\right]·{\mathbf{w}}^H\left[n\right]{\mathbf{H}}_{SI}\left[n\right]\mathrm{diag}\left\{{\mathbf{g}}_{IE}\left[n\right]\right\}$, respectively.

It is proved that the problem in (15) is also in the form of generalized eigenvalue problems, but unit modulus constraints ${\theta }_m\left[n\right]\in [0,2\pi )$ make it difficult to solve using the traditional generalized eigenvector solver. To cope with this difficulty, here we propose to utilize the MM method [33,34] to compose a surrogate objective function and derive the closed-form solution. The design of the MM algorithm is more challenging when malicious eavesdroppers are introduced into the communication system. We first find the surrogate function to approximate the average secrecy rate, then solve the optimal phase shifts based on the reformulated problem. Here, we define the vector $\mathbf{\phi }={\widehat{\mathbf{\theta }}}^{*}\left[n\right]$, and the matrices ${\widehat{\mathbf{R}}}_D=\frac{1}{M}{\mathbf{I}}_M+{\overline{\mathbf{R}}}_D$ and ${\widehat{\mathbf{R}}}_E=\frac{1}{M}{\mathbf{I}}_M+{\overline{\mathbf{R}}}_E$. Then, the objective function in the above problem (15) can be rewritten as $f(\mathbf{\phi })={\mathbf{\phi }}^H{\widehat{\mathbf{R}}}_D\mathbf{\phi }/{\mathbf{\phi }}^H{\widehat{\mathbf{R}}}_E\mathbf{\phi }$. It is assumed ${\mathbf{\phi }}_{iter}^i$ denotes the value of $\mathbf{\phi }$ in the i-th iteration. Then, a lower bound $g(\mathbf{\phi }|{\mathbf{\phi }}_{iter}^i)$ that touches $f(\mathbf{\phi })$ at point ${\mathbf{\phi }}_{iter}^i$ can be constructed.

Proposition2. 

Objective function $f(\mathbf{\varphi })$ is lower bounded by

$$\begin{array}{c}\hfill f(\mathbf{\phi })=\frac{{\mathbf{\phi }}^H{\widehat{\mathbf{R}}}_D\mathbf{\phi }}{{\mathbf{\phi }}^H{\widehat{\mathbf{R}}}_E\mathbf{\phi }}\ge g(\mathbf{\phi }|{\mathbf{\phi }}_{iter})+[f\left({\mathbf{\phi }}_{iter}\right)-g\left({\mathbf{\phi }}_{iter}\right|{\mathbf{\phi }}_{iter})],\end{array}$$

(16)

where $f\left({\mathbf{\phi }}_{iter}\right)-g\left({\mathbf{\phi }}_{iter}\right|{\mathbf{\phi }}_{iter})$ is a constant term.

Proof. 

Please see Appendix B.    □

According to Proposition 2, it is clear that, for the $(i+1)$-th iteration operation of the MM method, both the second and third terms on the right-hand side of (16) are constants. Then, we have the conclusion that the optimal reflecting matrix is determined by the first part $g(\mathbf{\phi }|{\mathbf{\phi }}_{iter})$. In other words, the optimization problem of phase shifts for $(i+1)$-th iteration is equivalent to

$$\begin{array}{c}\hfill {\mathbf{\phi }}_{iter}^{i+1}=\underset{\mathbf{\phi }}{argmax}\mathfrak{R}[{\left({\mathit{u}}^i\right)}^H\mathbf{\phi }],s.t.{\theta }_m\left[n\right]\in [0,2\pi ),\end{array}$$

(17)

where the operation $\mathfrak{R}[·]$ means taking the real part of $[·]$. The expression for the vector ${\mathbf{u}}^i$ can be derived from Proposition 2 and written as

$$\begin{array}{c}\hfill {\mathbf{u}}^i=\frac{{\widehat{\mathbf{R}}}_D{\mathbf{\phi }}_{iter}^i}{{\left({\mathbf{\phi }}_{iter}^n\right)}^H{\widehat{\mathbf{R}}}_E{\mathbf{\phi }}_{iter}^i}-\frac{{\left({\mathbf{\phi }}_{iter}^i\right)}^H{\widehat{\mathbf{R}}}_D{\mathbf{\phi }}_{iter}^i}{{\left[{\left({\mathbf{\phi }}_{iter}^i\right)}^H{\widehat{\mathbf{R}}}_E{\mathbf{\phi }}_{iter}^i\right]}^2}\times [{\widehat{\mathbf{R}}}_E-{\lambda }_{max}\left({\widehat{\mathbf{R}}}_E\right){\mathbf{I}}_M]{\mathbf{\phi }}_{iter}^i.\end{array}$$

(18)

On the basis of basic algebraic manipulations [35], the solution of the above problem in (17) can be achieved as

$$\angle {\mathbf{\phi }}_{iter}^{i+1}=\angle {\mathbf{u}}^i.$$

(19)

Consequently, in the $(i+1)$-th iteration, the solution of $\widehat{\mathbf{\theta }}$ can be derived as $\angle {\widehat{\mathbf{\theta }}}^{i+1}=-\angle {\mathit{u}}^i$. When the iterative operation of the MM method converges, the optimal ${\theta }_m^{opt}\left[n\right]$ can be obtained.

3.3. AIRS Trajectory Design

With the obtained transmit beamformer $\mathbf{w}$ and phase-shift vector $\widehat{\mathbf{\theta }}$ at AIRS, the AIRS trajectory is designed for average secrecy rate maximization, i.e.,

$$\begin{array}{ll}\hfill & \underset{\mathbf{Q}}{max}\frac{1}{N}\sum _{n=1}^N{R}_s\left[n\right]\\ & \hfill \mathrm{s}.\mathrm{t}.\left\{\begin{array}{l}\mathbf{q}\left[0\right]={\mathbf{q}}_0,\mathbf{q}\left[N\right]={\mathbf{q}}_T,\\ \left|\right|\mathbf{q}[n+1]-\mathbf{q}\left[n\right]\left|\right|\le V_{max}{\delta }_t.\end{array}\right.\hfill \end{array}$$

(20)

For the definition of channel model in Section 2.1, the location of AIRS affects the path loss coefficient of all communication links. Thus, the trajectory $\mathbf{Q}=\left[\mathbf{q}\left[0\right],\dots ,\mathbf{q}\left[n\right],\dots ,\mathbf{q}\left[N\right]\right]$ is coupled with all channel coefficients, which establishes the non-convexity of the problem in (20). To cope with the non-convex problem, we resort to the DDPG algorithm by designing novel deep neural networks, i.e., actor networks and critic networks. The actor network embraces the state of the communication system as an input feature and generates the action. The critic network inputs the state and the action then yields the reward. We detail the design of the neural networks and their corresponding input and output in the next subsections.

3.3.1. State, Action, and Reward

To maximize the average secrecy rate of the AIRS-empowered secure communication, all system-related conditions are included in the state space $\mathcal{S}$, which is shown in

Table 2. Description of states in the DDPG algorithm.

No.	State	Numeric Types	Dimension
1	transmit beamformer	complex vector	$N_t$
2	phase-shift vector of AIRS	complex vector	M
3	AIRS’s current speed	real scalar	1
4	location of Alice	real vector	3
5	location of Bob	real vector	3
6	location of Eve	real vector	3
7	starting position of AIRS	real vector	3
8	target position of AIRS	real vector	3
9	current location of AIRS	real vector	3

. More specifically, the locations of all users, phase-shift and trajectory of AIRS, and transmit beamformer are considered states in our learning algorithm. As for the choice of actions, we select the moving direction and velocity v of the AIRS in each time slot as the action in the action space $\mathcal{A}$ of the neural network.

Due to the fact that our considered AIRS is mounted on an UAV, we also consider several practical constraints for the flight of the UAV, that is, the maximum flight speed $V_{max}$, the flight’s starting position ${\mathbf{q}}_0$ and target position ${\mathbf{q}}_T$. Thus, the reward function of the neural network includes the following terms: (1) The rate at Bob is greater than the minimum rate requirement; (2) the speed does not exceed the maximum speed of the AIRS; and (3) AIRS has to fly to the finish position ${\mathbf{q}}_T$ within N time slots.

3.3.2. Loss Function

In the DDPG algorithm, there are four networks, namely the evaluate actor network, evaluate critic network, target actor network, and target critic network. In the n-th time slot, the actor network $\pi (s;{\mu }^{\pi })$ is responsible for iteratively updating the policy network parameters ${\mu }^{\pi }$ and selecting the current action $a\left(n\right)$ according to the current state $s\left(n\right)$. The target actor network ${\pi }^{{}^{\prime }}(s;{{\mu }^{\pi }}^{{}^{\prime }})$ is to select the optimal next action $a(n+1)$ according to the next state $s(n+1)$ in the experience pool and realize the update of the network parameters ${{\mu }^{\pi }}^{{}^{\prime }}={\mu }^{\pi }$. The role of the critic network $Q(s,a;{\mu }^Q)$ is an iterative update of the value network parameters ${\mu }^Q$ and the calculation of current Q. The role of target critic network $Q^{\prime }(s,a;{{\mu }^Q}^{{}^{\prime }})$ is to calculate the target $Q^{{}^{\prime }}$ and periodically update network parameters ${\mu }^{Q^{{}^{\prime }}}$.

We can utilize the loss function to measure the goodness of the algorithm by describing the difference between the target network and the virtual network. Then, we optimize the loss function to make the predicted value closer to the actual value. Therefore, the loss functions of the actor network and the critic network are analyzed as

$$L\left({\mu }^{\pi }\right)=\frac{1}{N_b}\sum _{n=1}^N-Q(s\left(n\right),\pi \left(s\left(n\right)\right)),$$

(21)

$$L\left({\mu }^Q\right)=\frac{1}{N_b}\sum _{n=1}^N{[r\left(n\right)+\gamma Q^{{}^{\prime }}(s(n+1),{\pi }^{{}^{\prime }}\left(s(n+1)\right))-Q(s\left(n\right),a\left(n\right))]}^2,$$

(22)

where the parameter $N_b$ is defined as the size of the mini-batch, and $\gamma$ is the discount factor, $r\left(n\right)$ is the reward function received by agent at the n-th time slot, and the reward function is as

$$r\left(n\right)={\eta }_{rate}{R}_s\left[n\right]+{\eta }_{dist}{d}_{AT}\left[n\right],$$

(23)

where ${\eta }_{rate}$ is a binary penalty indicator on secrecy rate $R_s\left[n\right]$ with ${\eta }_{rate}=1$ if the constraints on the AIRS’s fight are guaranteed and ${\eta }_{rate}=0$ otherwise. In addition, ${\eta }_{dist}$ denotes a constant penalty term on the reduced distance $d_{AT}\left[n\right]$ between current position and the position of the last time slot, in which $d_{AT}\left[n\right]=\sqrt{\parallel \mathbf{q}\left[n\right]-{\mathbf{q}}_T{\parallel }^2}$.

3.3.3. Neural Network

In the n-th time slot, the actor network selects the current state $s\left(n\right)$ from $\mathcal{S}$ as the input to the network and then generates the action $a\left(n\right)$ on the basis of the policy function. The inputs of actor network are nine state parameters listed in Table 2, and two action parameters, the magnitude and direction of the AIRS velocity, will be output from the output layer using the activation function tanh. The angle range of AIRS velocity direction is $(-\pi ,\pi )$. The neural network diagram of actor is shown in the Figure 2. The critic network inputs the current $s\left(n\right)$ and $a\left(n\right)$ then generates $Q\left(s\right(n),a(n\left)\right)$ for evaluating the actor network based on our designed reward function. The neural network diagram of critic is shown in Figure 3 and our whole DDPG algorithm is summarized in the following Algorithm 1. The parameters involved in the neural network are listed in

Table 3. Notations for parameters used in the neural network.

Parameter	Description	Parameter	Description
$a\left(n\right)$	action at n-th time slot	$s\left(n\right)$	state at n-th time slot
$\pi (s;{\mu }^{\pi })$	current actor network	${\pi }^{{}^{\prime }}(s;{\mu }^{{\pi }^{{}^{\prime }}})$	target actor network
$Q(s,a;{\mu }^Q)$	current critic network	$Q^{{}^{\prime }}(s,a;{\mu }^{Q^{{}^{\prime }}})$	target critic network
$d_{AT}\left[n\right]$	distance from $\mathbf{q}\left[n\right]$ to target ${\mathbf{q}}_T$	$r\left(n\right)$	reward function
${\eta }_{rate}$	binary penalty indicator on $R_s\left[n\right]$	$\gamma$	discount factor
${\eta }_{dist}$	binary penalty indicator on $d_{AT}\left[n\right]$	$N_b$	size of mini-batch

.

Algorithm 1 DDPG Algorithm

1: Input transmit beamformer $\mathbf{w}$, phase shift matrix $\mathsf{\Theta }$, locations of all users, maximum velocity, starting and target position of AIRS, actor and critic network structures. 2: Output actor network with ${\mu }^{\pi }$, critic network with ${\mu }^Q$, and the trajectory $\mathbf{Q}=\left[\mathbf{q}\right[0],\dots ,\mathbf{q}[n],\dots ,\mathbf{q}[N\left]\right]$ in state space. 3: Initialize evaluate actor network $\pi (s;{\mu }^{\pi })$ and evaluate critic network $Q(s,a;{\mu }^Q)$ with weight ${\mu }^{\pi }$ and ${\mu }^Q$. 4: Initialize the target actor network ${\pi }^{{}^{\prime }}(s;{{\mu }^{\pi }}^{{}^{\prime }})$ and the critic network $Q^{{}^{\prime }}(s,a;{{\mu }^Q}^{{}^{\prime }})$, ${{\mu }^{\pi }}^{{}^{\prime }}\leftarrow {\mu }^{\pi }$ and ${{\mu }^Q}^{{}^{\prime }}\leftarrow {\mu }^Q$. 5: Initialize the experience replay buffer $\mathcal{M}$. 6: for each episode do 7:      Initialize a random exploration noise $\mathcal{N}$, and include AIRS’s starting position, velocity, phase-shift matrix and transmit beamformer into state space. 8:      for each time slot $n\in N$ do 9:         Get the initial observation state $s\left(n\right)$, and obtain $a\left(n\right)=\pi (s\left(n\right);{\mu }^{\pi })+\mathcal{N}$. 10:       The central controller takes the action $a\left(n\right)$ and observes reward $r\left(n\right)$, then the model transfers to a new state $s(n+1)$. 11:       Store $\mathcal{T}\left(n\right)=\left(s\right(n),a(n),r(n),s(n+1\left)\right)$ from the experience reply buffer $\mathcal{M}$. 12:    end for 13:    Sample randomly a minibatch of $N_b$ transitions $\mathcal{T}\left(n\right)$ from $\mathcal{M}$ and calculate the $y\left(n\right)=r\left(n\right)+\gamma Q^{{}^{\prime }}(s(n+1),{\pi }^{{}^{\prime }}\left(s(n+1)\right))$. 14:    Minimize the the loss function $L\left({\mu }^Q\right)$ to update parameter ${\mu }^Q$. 15:    Minimize the loss function $L\left({\mu }^{\pi }\right)$ to update parameter ${\mu }^{\pi }$. 16:    Soft updates for the target network. 17:    ${{\mu }^Q}^{{}^{\prime }}=\tau {\mu }^Q+(1-\tau ){{\mu }^Q}^{{}^{\prime }}$. 18:    ${{\mu }^{\pi }}^{{}^{\prime }}=\tau {\mu }^{\pi }+(1-\tau ){{\mu }^{\pi }}^{{}^{\prime }}$. 19: end for

3.4. Iterative Algorithm Description

On the basis of Propositions 1 and 2 and Algorithm 1, the transmit beamformer $\mathbf{w}$ and phase-shift vector $\widehat{\mathbf{\theta }}$ and trajectory $\mathbf{Q}$ of AIRS can be updated alternatively, and the solution of average secrecy rate maximization problem is obtained. We summarize the whole process of iterative algorithm as Algorithm 2. In addition, since solutions of each sub-problems are derived based on the performance maximization, the average secrecy rate after each iteration is non-decreasing. Consequently, the proposed iterative algorithm can be guaranteed to be convergent in the case of transmit power limitation.

The optimization problem investigated in this paper is non-convex, so we divide the objective problem into three sub-problems and iteratively solve the sub-problems separately. Each sub-problem is the solution obtained by maximizing the secrecy rate of the system. Thus, our proposed algorithm ensures that the objective function of this paper is non-decreasing after each iteration. Moreover, because of the transmit power limitation, the achievable secrecy rate is bounded by a finite value. Therefore, the convergence of our proposed iterative algorithm is guaranteed.

Algorithm 2 Algorithm for Average Secrecy Rate Maximization

1: Initialize ${\mathbf{w}}^0\left[n\right]$, ${\widehat{\mathbf{\theta }}}^0\left[n\right]$, ${\mathbf{Q}}^0=[\mathbf{q}\left[0\right],\dots ,\mathbf{q}\left[n\right],\dots ,\mathbf{q}\left[N\right]$, and set locations of all users, the maximum velocity of AIRS, and $\epsilon$. 2: Calculate the average secrecy rate $R_s^{aver}(\mathbf{w}{\left[n\right]}^0,\widehat{\theta }{\left[n\right]}^0,{\mathbf{Q}}^0)$. 3: Set an index term of iteration $i=0$. 4: repeat 5:    $i=i+1$ 6:    Given ${\widehat{\mathbf{\theta }}}^{i-1}\left[n\right]$ and ${\mathbf{Q}}^{i-1}$, derive the transmit beamformer ${\mathbf{w}}^i\left[n\right]$ using Proposition 1. 7:    With the obtained ${\mathbf{w}}^i\left[n\right]$ and given ${\mathbf{Q}}^{i-1}$, find ${\widehat{\mathbf{\theta }}}^i\left[n\right]$ according to Proposition 2. 8:    With the obtained ${\mathbf{w}}^i\left[n\right]$ and ${\widehat{\mathbf{\theta }}}^i\left[n\right]$, initialize the state space $\mathcal{S}$, and derive the trajectory of AIRS ${\mathbf{Q}}^i$ based on DDPG method in Algorithm 1. 9:    Calculate average secrecy rate ${\left(R_s^{aver}\right)}^i=R_s^{aver}({\mathbf{w}}^i\left[n\right],{\widehat{\mathbf{\theta }}}^i\left[n\right],{\mathbf{Q}}^i)$. 10: until$|{\left(R_s^{aver}\right)}^i-{\left(R_s^{aver}\right)}^{i-1}|\le \epsilon$. 11: return${\mathbf{w}}^{opt}\left[n\right]={\mathbf{w}}^i\left[n\right]$, ${\widehat{\mathbf{\theta }}}^{opt}\left[n\right]={\widehat{\mathbf{\theta }}}^i\left[n\right]$,$n=1,\dots ,N$, and ${\mathbf{Q}}^{opt}={\mathbf{Q}}^i$.

3.5. Complexity Analysis

We assume that the proposed iterative algorithm converges after $N_{iter}$ iterations. The complexity of our proposed iterative algorithm can be calculated relying on the transmit beamforming design, AIRS phase shift design and AIRS trajectory design, and $N_{iter}$. For the design of transmit beamforming, on the basis of Proposition 1, the complexity is obtained as $\mathcal{O}(N_t^3+N_t{M}^2)$. For the design of AIRS phase shifts, the complexity relies mainly on the number of iterations of MM algorithm $I_{iter}$ and the calculation of Equation (18). Therefore, the complexity of obtaining phase shifts is $\mathcal{O}\left((M^3+M^2{N}_t)I_{iter}\right)$. For the design of AIRS trajectory, we assume L layers used in the deep neural network and $n_k$ neurons in the k-th layer. With $N_b$ mini batches and N time slots, the complexity of DDPG algorithm is obtained as $\mathcal{O}(N_{b}N({\sum }_{k=1}^{L-1}{n}_k{n}_{k+1}))$. Consequently, the complexity of our proposed iterative algorithm is about $\mathcal{O}(N_{iter}(N_t^3+I_{iter}(M^3+M^2{N}_t)+N_{b}N({\sum }_{k=1}^{L-1}{n}_k{n}_{k+1})))$.

4. Simulation Results and Analysis

In this section, simulation results are presented to evaluate the average secrecy rate performance of our proposed AIRS-aided secure transmission scheme. Unless otherwise stated, it is assumed that Alice is located at ${\mathbf{l}}_A=[50,30]$ m with the height $h_A=10$ m, while the desired destination and passive eavesdropper are located at $[100,150,0]$ and $[150,180,0]$, respectively. Moreover, we assume that AIRS needs to fly from the starting position ${\mathbf{q}}_0=[0,0]$ m to the target position ${\mathbf{q}}_T=[200,200]$ m within $N=20$ time slots. The constant flight altitude of AIRS is $h=50$ m and the maximum flight speed is limited to $V_{max}=20$ m/s. In addition, the number of antennas equipped with Alice is $N_t=20$, and the number of reflecting elements at AIRS is $M=64$. The element spacing at Alice and AIRS are both set to a half wavelength. All channels are modeled based on the rank-one channel model mentioned in Section 2.1. Without loss of generality, we set the path loss exponent to $\alpha =2.5$ and the channel gain at the reference distance $d_0=1$ m to ${\rho }_0=-10$ dBm. The transmit power is $P_s=20$ dBm.

In addition, the neural network used in the DDPG algorithm involves two networks, the actor network and the critic network. We set the number of neurons in four hidden layers of the actor network to $(128,128,64,64)$. In the critic network, there are also four hidden layers and the number of neurons in each layer is set to $(128,128,64,64)$. The learning rates of actor network and critic network are set to $3\times {10}^{-4}$ and $1\times {10}^{-3}$, and the constant penalty term is set to ${\eta }_{dist}=0.1$. The discount factor is set to $\gamma =0.99$, and the deviation of exploration noise is ${\sigma }_{\mathcal{N}}=0.1$. The experience reply buffer is $1\times {10}^4$ and mini-batches is 4. The soft update coefficient is assumed as $\tau =0.005$.

In the AIRS-assisted secure communication system, we propose an iterative algorithm for secrecy rate maximization. The effectiveness of our proposed algorithm depends on whether it converges and how fast it converges. Thereby, we select two time slots during the AIRS flight as samples to study the convergence property. As shown in Figure 4, the iterative algorithm converges monotonically and has a fast convergence speed for both sampling time slots. Specifically, for time slot 1, the secrecy rate performance reaches a smooth state after 10 iterations, while the secrecy rate performance in time slot 2 reaches a plateau after nine iterations. That is, after 10 iterations, both time slots converge, and the secrecy rate performance remains unchanged in the remaining iterations. After the iterative algorithm converges, the solution of joint design of transmit beamforming, phase shift and trajectory for each slot can be obtained, and the secrecy performance can be maximized. Results reveal that the theoretical analysis in Section 3 can be valid from the simulation perspective.

The transmit power is a key factor that affects the security performance of physical layer security systems. An increase in transmit power can enhance the signal strength received by all receivers. Therefore, it can increase the information rate of legitimate user Bob, and it can also increase the information leakage of illegal user Eve. Figure 5 displays the variation of the average secrecy rate with different transmit power at transmitter Alice. It can be observed that increasing power at Alice to a certain extent can enhance the secrecy rate performance. The reason is that increasing the transmit power to a certain extent can bring more information rate received at Bob, thereby increasing the secrecy rate of each time slot. However, the excessive transmit power can also result in more information being eavesdropped on. Therefore, setting the appropriate power is crucial for the secure transmission of information. Moreover, the performance comparison of $M=81$ and $M=64$ reveal that the more reflecting elements the AIRS is equipped with, the better the average secrecy rate performance can be obtained.

To verify the effectiveness and performance of the developed AIRS-assisted secure transmission scenario, the secrecy performance of terrestrial IRS (TIRS)-based scenarios is also included as a baseline in Figure 5. Different from the AIRS mounted on aerial platforms, numerous reflecting elements of TIRS are installed on the surface of buildings or other infrastructure on the ground. Thus, TIRS in terrestrial networks can only achieve a coverage of ${180}^{\circ }$, while AIRS can achieve a full coverage of ${360}^{\circ }$ due to its flexibility and mobility in the air. It can be observed from Figure 5 that, in comparison with TIRS-assisted secure transmission, our proposed AIRS-assisted scenario can obtain more secrecy rate gain. When the transmit power is $P_s=10$ dBm and the number of reflecting elements is $M=64$, AIRS assistance can obtain a secrecy rate gain of $34.5\%$ more than TIRS-assisted systems. Moreover, when the number of reflecting elements of AIRS is increased to $M=81$, the secrecy rate of our proposed AIRS-assisted scheme is almost twice that of TIRS systems. The above superior performance relies on the full-angle coverage of AIRS, which enables AIRS to facilitate enhanced LoS connectivity for terrestrial users. Thus, in contrast to the terrestrial LoS connection constructed by TIRS, which may suffer from terrestrial obstacles and strong fading, the air LoS can achieve better transmission performance, which in turn leads to the average secrecy rate improvement.

In wireless communication systems, the configuration of multiple antennas can significantly improve the information transmission rate and security because more antennas lead to sharper information beams, thereby reducing information leakage to eavesdroppers. Figure 6 analyzes the impact of the number of transmit antennas on the average secrecy rate. In Figure 6, our proposed trajectory design method is compared with two baselines, straight trajectory and polyline trajectory, and all trajectories are shown in Figure 7. Specifically, for a straight trajectory, we assume that the AIRS flies straight from its starting position of the horizontal coordinate ${\mathbf{q}}_0=[0,0]$ m to the target position ${\mathbf{q}}_T=[200,200]$ m with a constant velocity $v=14.14$ m/s. For the polyline trajectory, assuming that AIRS also starts from the starting position, it first flies straight to Alice’s position, then to Bob’s position, and finally to the target position. In contrast to the preset fixed flight trajectory, our proposed trajectory can use the DDPG algorithm to design an optimal trajectory that maximizes the system secrecy rate based on real-time channel state.

In Figure 6, it can be derived that, as the number of transmit antennas increases, the average secrecy rate is improved greatly. Intuitively, this is because it is equipped with more transmit antennas at Alice enabling more spatial degrees of freedom for confidential information transmission and results in improvement of secrecy rate. In addition, results also show that our proposed AIRS trajectory method designed based on the DDPG algorithm can achieve the best average secrecy rate performance, while the performance of the polyline trajectory is better than that of the straight trajectory. Compared with linear and polyline trajectories, since the dynamic communication environment and distance-based path loss are taken into account in the trajectory design of our paper, the state and actions of the neural network are updated with the dynamic channel conditions. Then, the optimal real-time position of AIRS can be output by the designed neural network. Therefore, the proposed DDPG-based trajectory design can obtain the best secrecy rate performance compared to the preset trajectories. Particularly, when the transmit power is $P_s=100$ mW and the number of transmit antennas is $N_t=20$, the proposed trajectory design can achieve a secrecy rate gain of $15\%$ more than polyline trajectory.

In addition, as analyzed by Figure 5, the system with transmit power $P_s=100$ mW can obtain more secrecy rate gain than the system with power $P_s=50$ mW.

5. Conclusions

In this paper, we investigated an AIRS-empowered secure communication system. In the system, an AIRS mounted on an aerial platform is introduced to help confidential information transmission. With the objective to guarantee information security during flight period, the average secrecy rate was investigated, and the transmit beamforming at transmitter and phase-shift matrix as well as trajectory of AIRS are jointly designed. To cope with the formulated non-convex problem, we first divided the secrecy rate maximization problem into three sub-problems and proposed an iterative algorithm. Specifically, we optimized the IRS phase shift matrix by the MM algorithm and designed the trajectory of AIRS by the DDPG algorithm. Simulation results have demonstrated that our proposed algorithm can obviously enhance the secrecy performance.