Full-Duplex Relaying Systems with Massive MIMO: Equal Gain Approach

Abstract

In this paper, the uplink spectral efficiency performance of a massive MIMO system based on full-duplex relay communication is investigated in Rician fading channels. The relay station is equipped with a large number of antennas, while multiple source and destination nodes are located at both ends of the transceiver. Each source and destination node is equipped with a single antenna. The relay station adopts Maximum Ratio Combining/Maximum Ratio Transmission (MRC/MRT) and Equal Gain Combining/Equal Gain Transmission (EGC/EGT) schemes to perform linear preprocessing on the received signals. Approximate expressions for uplink spectral efficiency under both MRC/MRT and EGC/EGT schemes are derived, and the effects of antenna number, signal-to-noise ratio (SNR), and loop interference on spectral efficiency are analyzed. In addition, the impact of full-duplex and half-duplex modes on system performance is compared, and a hybrid relay scheme is proposed to maximize the total spectral efficiency by dynamically switching between full-duplex and half-duplex modes based on varying levels of loop interference. Finally, a novel power allocation scheme is proposed to maximize energy efficiency under given total spectral efficiency and peak power constraints at both the relay and source nodes. The results show that the impact of loop interference can be eliminated by using a massive receive antenna array, leading to the disappearance of inter-pair interference and noise. Under these conditions, the spectral efficiency of the system can be improved up to $2N$ times, while the transmission power of the user and relay nodes can be reduced to $1/N_{\mathrm{rx}}$ and $1/N_{\mathrm{tx}}$, respectively.

1. Introduction

Since the 1960s, relay communication has remained a vital research direction in the field of wireless communications [1,2]. As a core technology of modern wireless communication systems, large-scale multiple-input multiple-output (MIMO) significantly enhances system capacity and spectral efficiency by deploying a large number of transceiver antennas and fully exploiting spatial diversity and multiplexing gains. However, the practical implementation of this technology poses several challenges. First, the deployment of large-scale antenna arrays at the base station demands sophisticated signal processing algorithms, which not only increase computational complexity but also place stringent requirements on hardware processing capabilities. Second, when large-scale MIMO is applied to full-duplex relay systems, signal processing becomes even more intricate [3,4,5,6,7]. In theory, full-duplex relay technology can potentially double the system capacity by enabling simultaneous transmission and reception on the same frequency band [8,9]. In practice, however, it encounters several critical obstacles, such as loop interference suppression, hardware impairment compensation, mitigation of nonlinear effects, and strict synchronization constraints [10,11,12]. These issues significantly hinder the practical deployment of full-duplex communication and represent key challenges in the current research on full-duplex relay-based large-scale MIMO systems.

In response to the aforementioned challenges, extensive research has been conducted from multiple perspectives, yielding significant advancements. In the area of interference suppression and system performance optimization, Guo et al. [13] proposed an innovative zero-forcing (ZF) precoding technique and systematically analyzed the coupling relationships among loop interference, antenna array size, and transmission power, as well as their collective impact on system throughput. Their findings confirmed the remarkable advantage of large-scale antenna arrays in mitigating interference. Regarding multi-user communication scenarios, Gupta et al. [14] developed a full-duplex large-scale MIMO relay system model involving multiple user pairs. Based on the MRC/MRT scheme, they derived exact closed-form expressions for spectral efficiency and, for the first time, quantitatively evaluated the joint influence of phase-shift errors and residual interference on system performance. To address the issue of imperfect channel state information (CSI), Shokoohifard et al. [15] investigated a two-hop full-duplex multi-antenna relay network employing a decode-and-forward strategy under non-ideal CSI conditions. Their study revealed the roles of critical parameters—including the number of antennas, relay location, and self-interference cancellation capability—in determining the achievable spectral efficiency. In the domain of channel adaptability, Peng et al. [16] made a groundbreaking contribution by establishing a universal power scaling law for Ricean fading channels, offering theoretical guidance for a variety of service environments. Similarly, Dong et al. [17], starting from the characteristics of Rayleigh fading, pioneered the joint optimization of power control strategies and ADC quantization resolution, elucidating their combined effect on the achievable data rate. Notably, Ma et al. [18] introduced a hybrid transmission mode selection mechanism that integrates full-duplex, half-duplex, and direct transmission, and applied an advanced fractional programming technique to enhance system energy efficiency. However, this study still exhibits noticeable limitations in terms of channel modeling completeness, dynamic multi-user scheduling, and the practical constraints of large-scale MIMO deployment. As such, its proposed scheme remains difficult to directly apply to more complex real-world communication scenarios.

Although significant theoretical progress has been made in existing research, several critical challenges remain unresolved in practical engineering deployments. Specifically, current large-scale MIMO relay systems predominantly rely on high-performance signal processing schemes such as Maximum Ratio Combining/Maximum Ratio Transmission (MRC/MRT) and Zero Forcing (ZF). While these techniques exhibit excellent spectral efficiency, their implementation requires complex precoding computations and high-precision RF components, which substantially increase the hardware complexity and deployment costs. These factors severely limit their applicability in power- or cost-sensitive scenarios. Notably, Equal Gain Combining/Equal Gain Transmission (EGC/EGT), as a low-complexity linear processing scheme, offers significant advantages such as simplified hardware implementation and reduced computational burden. However, systematic investigations of EGC/EGT in large-scale MIMO systems remain insufficient, particularly under Ricean fading channels—a typical non-uniform propagation environment. There is a clear lack of rigorous theoretical analysis and modeling frameworks for evaluating its end-to-end performance under such conditions. Therefore, this paper focuses on full-duplex relay-based large-scale MIMO systems and conducts a comparative analysis of uplink spectral efficiency between MRC/MRT and EGC/EGT schemes under Ricean fading conditions.

In this paper, the uplink spectral efficiency of a massive MIMO-based full-duplex relay system in a Rician channel is mainly studied. Specific contributions are as follows:

• The conclusion shows that using a massive transceiver antenna array in the relay station can effectively reduce the loop interference caused by full duplex. At this time, the inter-pair interference and noise disappear, and the transmitting power of each source and relay decreases proportionally when the number of transmitting and receiving antennas of the relay station tends to infinity.
• In terms of linear reception, the more economical EGC/EGT approach is used to linearly receive the signal, and the results are compared with those obtained by the MRC/MRT approach. Finally, the simulation results and theoretical analysis are obtained. In addition, the performance of full duplex mode and half duplex mode is compared. When the level of loop interference is low, the system performance of full duplex mode is significantly higher than that of half duplex mode. Based on this, a hybrid relay mode is proposed, which can quickly switch between full duplex mode and half duplex mode under different loop interference levels to maximize the total spectral efficiency.
• In this paper, a new power allocation scheme is proposed, which can achieve maximum energy efficiency under the given total spectral efficiency and the limit of peak power of the relay node and the source node. Geometric programming (GPs) is used to solve the problem, and the results show that compared with the average power distribution algorithm, the proposed power distribution scheme can significantly improve the system performance.

Notations: Use uppercase boldface letters to denote matrices and lowercase boldface letters to denote vectors, respectively. $\mathbf{E}\left\{·\right\}$ represents expectation, ${\mathbf{X}}^{*}$, ${\mathbf{X}}^T$ and ${\mathbf{X}}^H$ represent conjugation, transpose, and conjugate transpose, respectively, $∥\mathbf{X}∥$ represents the Euclidean norm, and $tr\left(\mathbf{X}\right)$ represents the trace of the matrix.

2. System Model

The full-duplex relay communication system has multiple pairs of DF, which have N source nodes and destination nodes, and a common relay station R to share the same time-frequency resources [19]. The relay station works in full-duplex mode, and its system model is shown in Figure 1. Assuming that the relay station of the system has M antennas and $M\gg N$, all source and destination nodes use a single antenna, the relay station adopts $N_{\mathrm{rx}}$ root receiving antenna and $N_{\mathrm{tx}}$ root transmitting antenna, the total number of antennas at the relay station is $N=N_{\mathrm{rx}}+N_{\mathrm{tx}}$. In addition, due to effects such as large-scale fading and shadow fading, there is no direct link between $S_{\mathrm{n}}$ and $D_{\mathrm{n}}$.

At time i, N source nodes $S_{\mathrm{n}}$, $n=1,\dots ,N$, simultaneously send their respective signals $\sqrt{p_s}{x}_n\left[i\right]$ to the repeater station, and the repeater station sends signals $\sqrt{p_R}s\left[i\right]\in {\mathbb{C}}^{N_{tx}\times 1}$ to N destination nodes. To ensure the average transmit power of the source node and the relay node, assume $E\left\{{\left|x_n\left[i\right]\right|}^2\right\}=1$ and $E\left\{{∥s\left[i\right]∥}^2\right\}=1$. The signal sent by the user is $x\left[i\right]=\left[x_1\left[i\right]\right.x_2\left[i\right]\dots {\left.x_N\left[i\right]\right]}^T$, then the received signal of the relay station and the destination node is

$${\mathbf{y}}_{\mathrm{R}}\left[i\right]=\sqrt{p_{\mathrm{S}}}{\mathbf{G}}_{\mathrm{SR}}x\left[i\right]+\sqrt{p_{\mathrm{R}}}{\mathbf{G}}_{\mathrm{RR}}s\left[i\right]+{\mathbf{n}}_{\mathrm{R}}\left[i\right]$$

(1)

$${\mathbf{y}}_{\mathrm{D}}\left[i\right]=\sqrt{p_{\mathrm{R}}}{\mathbf{G}}_{\mathrm{RD}}^{T}s\left[i\right]+{\mathbf{n}}_{\mathrm{D}}\left[i\right]$$

(2)

Among them ${\mathbf{G}}_{\mathrm{SR}}\in {\mathbb{C}}^{N_{rx}\times N}$ and ${\mathbf{G}}_{\mathrm{RD}}^T\in {\mathbb{C}}^{N\times N_{tx}}$, respectively, the N source node to the relay station Repeaters transmitting terminal and the receiving end to the channel matrix N destination node, ${\mathbf{G}}_{\mathrm{RR}}\in {\mathbb{C}}^{N_{rx}\times N_{tx}}$ is the channel matrix between the receiving end and the transmitting end of the relay station, that is, the loop interference, ${\mathbf{G}}_{\mathrm{RR}}\sim CN\left(0,{\sigma }_{\mathrm{LI}}^2\right)$, ${\sigma }_{\mathrm{LI}}^2$ represents the loop interference intensity. It is assumed that the distance between the transmitting antenna and the receiving antenna is much greater than the distance between the components, so the channel between the transmitting antenna and the receiving antenna is independently distributed. ${\mathbf{n}}_{\mathrm{R}}\left[i\right]$ and ${\mathbf{n}}_{\mathrm{D}}\left[i\right]$ are Gaussian white noise at the relay end and the destination node, respectively, and each element is subject to independent and identically distributed $CN\left(0,1\right)$.

2.1. Channel Estimation

In Rician channels, the relay cannot directly obtain CSI [20], so it is necessary to estimate ${\mathbf{G}}_{\mathrm{SR}}$ and ${\mathbf{G}}_{\mathrm{RD}}$, and the estimated channel is the real channel. The channel between the base station and the user is

$$\mathbf{G}=\mathbf{H}{\mathbf{D}}^{1/2}$$

(3)

Here, $\mathbf{H}\in {\mathbb{C}}^{M\times N}$ is the small-scale fading between the user and the base station, where a deterministic LoS component $\overline{\mathbf{H}}$ and the random component $\tilde{\mathbf{H}}$ of the corresponding scattered signal and ${\left[\mathbf{H}\right]}_{mn}=h_{mn}$; $\mathbf{D}\in {\mathbb{C}}^{N\times N}$ is a large-scale fading diagonal matrix, and ${\left[\mathbf{D}\right]}_{nn}={\beta }_n$.

The Rician K-factor, defined as the power ratio between the LoS and NLoS components, is given by the following:

$$K={\displaystyle \frac{P_{\mathrm{LoS}}}{P_{\mathrm{NLoS}}}}$$

(4)

A larger K value indicates a stronger LoS component in the channel. Based on this, the fast fading channels are modeled as follows:

$${\mathbf{H}}_{\mathrm{SR}}=\sqrt{{\displaystyle \frac{K_{SR}}{K_{SR}+1}}}{\overline{\mathbf{H}}}_{\mathrm{SR}}+\sqrt{{\displaystyle \frac{1}{K_{SR}+1}}}{\tilde{\mathbf{H}}}_{\mathrm{SR}}$$

(5)

$${\mathbf{H}}_{\mathrm{RD}}=\sqrt{{\displaystyle \frac{K_{RD}}{K_{RD}+1}}}{\overline{\mathbf{H}}}_{\mathrm{RD}}+\sqrt{{\displaystyle \frac{1}{K_{RD}+1}}}{\tilde{\mathbf{H}}}_{\mathrm{RD}}$$

(6)

Among them, said $\tilde{\mathbf{H}}$ random component, and the average random component of the elements is zero, real part and imaginary part are independent, each has a variance with distribution [21]; $\overline{\mathbf{H}}$ represents a deterministic component whose arbitrary rank is

$${\left[{\overline{\mathbf{H}}}_{\mathrm{SR}}\right]}_{mn}={\left[{\overline{\mathbf{H}}}_{\mathrm{RD}}\right]}_{mn}={\mathrm{e}}^{-j\left(m-1\right)\left(2\pi d_1/\lambda \right)sin\left({\theta }_n\right)}$$

(7)

where $d_1$ is the antenna spacing, $\lambda$ is the wavelength, and ${\theta }_n$ is the n user arrival Angle. In order to obtain CSI, the uplink pilot sequence $\tau$ needs to be sent for channel estimation. The pilot matrix received by the transmitter and receiver of the relay is

$${\mathbf{Y}}_{\mathrm{rp}}=\sqrt{\tau p_{\mathrm{p}}}{\mathbf{G}}_{\mathrm{SR}}{\Phi }_{\mathrm{S}}+\sqrt{\tau p_{\mathrm{p}}}{\overline{\mathbf{G}}}_{\mathrm{RD}}{\Phi }_{\mathrm{D}}+{\mathbf{N}}_{\mathrm{rp}}$$

(8)

$${\mathbf{Y}}_{\mathrm{tp}}=\sqrt{\tau p_{\mathrm{p}}}{\overline{\mathbf{G}}}_{\mathrm{SR}}{\Phi }_{\mathrm{S}}+\sqrt{\tau p_{\mathrm{p}}}{\mathbf{G}}_{\mathrm{RD}}{\Phi }_{\mathrm{D}}+{\mathbf{N}}_{\mathrm{tp}}$$

(9)

Among them, ${\overline{\mathbf{G}}}_{\mathrm{SR}}\in {\mathbb{C}}^{N_{\mathrm{tx}}\times N}$ and ${\overline{\mathbf{G}}}_{\mathrm{RD}}\in {\mathbb{C}}^{N_{\mathrm{rx}}\times N}$, respectively, the N source node to relay the sender and the N destination node to the relay channel matrix at the receiving end; $p_{\mathrm{p}}$ is the energy of the pilot symbol [22]; ${\mathbf{N}}_{\mathrm{rp}}$ and ${\mathbf{N}}_{\mathrm{tp}}$ are Gaussian white noise subject to independent distribution; ${\Phi }_{\mathrm{S}}\in {\mathbb{C}}^{N\times \tau }$ and ${\Phi }_{\mathrm{D}}\in {\mathbb{C}}^{N\times \tau }$ are pilot sequences of ${\mathrm{S}}_n$ and ${\mathrm{D}}_n$, assuming that all pilot sequences are paired orthogonally. Therefore, ${\Phi }_{\mathrm{S}}{\Phi }_{\mathrm{S}}^H={\mathbf{I}}_N$, ${\Phi }_{\mathrm{D}}{\Phi }_{\mathrm{D}}^H={\mathbf{I}}_N$, ${\Phi }_{\mathrm{S}}{\Phi }_{\mathrm{D}}^H=0_N$, and demand $\tau \ge 2N$. In order to acquire the CSI, uplink pilot signals must be transmitted for channel estimation. In the uplink system model, users transmit pilot sequences of length $\tau$, and the relay then estimates the corresponding channels based on the received pilots. The channel estimates can be obtained using the minimum mean square error (MMSE) estimation method.

$${\widehat{\mathbf{G}}}_{\mathrm{SR}}={\displaystyle \frac{1}{\sqrt{\tau p_{\mathrm{p}}}}}{\mathbf{Y}}_{\mathrm{rp}}{\Phi }_{\mathrm{S}}^H{\tilde{\mathbf{D}}}_{\mathrm{SR}}={\mathbf{G}}_{\mathrm{SR}}{\tilde{\mathbf{D}}}_{\mathrm{SR}}+{\displaystyle \frac{1}{\sqrt{\tau p_{\mathrm{p}}}}}{\mathbf{N}}_{\mathrm{S}}{\tilde{\mathbf{D}}}_{\mathrm{SR}}$$

(10)

$${\widehat{\mathbf{G}}}_{\mathrm{RD}}={\displaystyle \frac{1}{\sqrt{\tau p_{\mathrm{p}}}}}{\mathbf{Y}}_{\mathrm{tp}}{\Phi }_{\mathrm{D}}^H{\tilde{\mathbf{D}}}_{\mathrm{RD}}={\mathbf{G}}_{\mathrm{RD}}{\tilde{\mathbf{D}}}_{\mathrm{RD}}+{\displaystyle \frac{1}{\sqrt{\tau p_{\mathrm{p}}}}}{\mathbf{N}}_{\mathrm{D}}{\tilde{\mathbf{D}}}_{\mathrm{RD}}$$

(11)

Among them, ${\tilde{\mathbf{D}}}_{\mathrm{SR}}={\left({\mathbf{D}}_{\mathrm{SR}}^{-1}/\left(\tau p_{\mathrm{p}}\right)+{\mathbf{I}}_{\mathrm{N}}\right)}^{-1}$, ${\tilde{\mathbf{D}}}_{\mathrm{RD}}={\left({\mathbf{D}}_{\mathrm{RD}}^{-1}/\left(\tau p_{\mathrm{p}}\right)+{\mathbf{I}}_{\mathrm{N}}\right)}^{-1}$, ${\mathbf{N}}_{\mathrm{S}}={\mathbf{N}}_{\mathrm{rp}}{\Phi }_{\mathrm{S}}^H$, ${\mathbf{N}}_{\mathrm{D}}={\mathbf{N}}_{\mathrm{tp}}{\Phi }_{\mathrm{D}}^H$. The estimated matrix errors of ${\mathbf{G}}_{\mathrm{SR}}$ and ${\mathbf{G}}_{\mathrm{RD}}$ are represented by ${\epsilon }_{\mathrm{SR}}$ and ${\epsilon }_{\mathrm{RD}}$, respectively, which can be written as

$${\mathbf{G}}_{\mathrm{SR}}={\widehat{\mathbf{G}}}_{\mathrm{SR}}+{\epsilon }_{\mathrm{SR}}$$

(12)

$${\mathbf{G}}_{\mathrm{RD}}={\widehat{\mathbf{G}}}_{\mathrm{RD}}+{\epsilon }_{\mathrm{RD}}$$

(13)

where ${\epsilon }_{\mathrm{SR}}$, ${\epsilon }_{\mathrm{RD}}$, ${\widehat{\mathbf{G}}}_{\mathrm{SR}}$, and ${\widehat{\mathbf{G}}}_{\mathrm{RD}}$ are independent of each other, and ${\epsilon }_{\mathrm{SR}}$, ${\epsilon }_{\mathrm{RD}}$, ${\widehat{\mathbf{G}}}_{\mathrm{SR}}$ and ${\widehat{\mathbf{G}}}_{\mathrm{RD}}$ are subject to $CN\left(0,{\mathbf{D}}_{\mathrm{SR}}-{\widehat{\mathbf{D}}}_{\mathrm{SR}}\right)$, $CN\left(0,{\mathbf{D}}_{\mathrm{RD}}-{\widehat{\mathbf{D}}}_{\mathrm{RD}}\right)$, $CN\left(0,{\widehat{\mathbf{D}}}_{\mathrm{SR}}\right)$ and $CN\left(0,{\widehat{\mathbf{D}}}_{\mathrm{RD}}\right)$. The nth diagonal element of ${\widehat{\mathbf{D}}}_{\mathrm{SR}}$ and ${\widehat{\mathbf{D}}}_{\mathrm{RD}}$ is ${\sigma }_{\mathrm{X},n}^2=\tau p_P{\beta }_{\mathrm{X},n}^2/\left(\tau p_P{\beta }_{\mathrm{X},n}^2+1\right)$, $\mathrm{X}=\mathrm{SR},\mathrm{RD}$.

2.2. Data Processing

Perform linear reception processing on the signal received by the trunk [23], that is, multiply it by a linear reception matrix ${\mathbf{W}}^T$, and divide the received signal into N data streams, that is

$$\mathbf{r}\left[i\right]={\mathbf{W}}^T{\mathbf{y}}_R\left[i\right]=\sqrt{p_{\mathrm{S}}}{\mathbf{W}}^T{\mathbf{G}}_{\mathrm{SR}}\mathbf{x}\left[i\right]+\sqrt{p_{\mathrm{R}}}{\mathbf{W}}^T{\mathbf{G}}_{\mathrm{RR}}\mathbf{s}\left[i\right]+{\mathbf{W}}^T{\mathbf{n}}_{\mathrm{R}}\left[i\right]$$

(14)

Then the nth dataflow decodes ${\mathrm{S}}_n$ and the nth element of $\mathbf{r}\left[i\right]$ is denoted by

$$r_n\left[i\right]=\sqrt{p_{\mathrm{S}}}{\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},n}{x}_n\left[i\right]+\sqrt{p_{\mathrm{S}}}\sum _{j\ne n}^N{\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},j}{x}_j\left[i\right]+\sqrt{p_{\mathrm{R}}}{\mathbf{w}}_n^T{\mathbf{G}}_{\mathrm{RR}}\mathbf{s}\left[i\right]+{\mathbf{w}}_n^T{\mathbf{n}}_{\mathrm{R}}\left[i\right]$$

(15)

Here, ${\mathbf{g}}_{\mathrm{SR},n}$ and ${\mathrm{w}}_n$ are the nth columns of ${\mathbf{G}}_{\mathrm{SR}}$ and $\mathbf{W}$, respectively, and $x_n\left[i\right]$ represents the nth element of $\mathbf{x}\left[i\right]$.

Due to the delay of relay processing, the relay sends signal $\mathbf{s}\left[i\right]$ as

$$\mathbf{s}\left[i\right]=\mathbf{Ax}\left[i-d\right]$$

(16)

Here, $\mathbf{A}\in {\mathbb{C}}^{N_{\mathrm{tx}}\times N}$ is the sexual preconditioning matrix and d is the delay processing. Therefore, the received signal at ${\mathrm{D}}_n$ is

$$y_{D,n}\left[i\right]=\sqrt{p_{\mathrm{R}}}{\mathbf{g}}_{\mathrm{RD},n}^T{\mathbf{a}}_n{x}_n\left[i-d\right]+\sqrt{p_{\mathrm{R}}}\sum _{j\ne n}^N{\mathbf{g}}_{\mathrm{RD},n}^T{\mathbf{a}}_j{x}_j\left[i-d\right]+{\mathrm{n}}_{\mathrm{D},n}\left[i\right]$$

(17)

Here, ${\mathbf{g}}_{\mathrm{RD},n}$ and ${\mathbf{a}}_n$ are the nth columns of ${\mathbf{G}}_{\mathrm{RD}}$ and $\mathbf{A}$, respectively, and ${\mathrm{n}}_{\mathrm{D},n}\left[i\right]$ is the nth element of ${\mathbf{n}}_{\mathrm{D}}\left[i\right]$.

2.3. Linear Treatment Approach

2.3.1. MRC/MRT Processing

The MRC detection matrix and the MRT precoding matrix can be written as [24]

$${\mathbf{W}}^T={\mathbf{W}}_{\mathrm{MRC}}^T={\widehat{\mathbf{G}}}_{\mathrm{SR}}^H$$

(18)

$$\mathbf{A}={\mathbf{A}}_{\mathrm{MRT}}={\alpha }_{\mathrm{MRT}}{\widehat{\mathbf{G}}}_{\mathrm{RD}}^{*}$$

(19)

where ${\alpha }_{\mathrm{MRT}}$ is a normalized constant and satisfies the conditions of $E\left\{{∥s\left[i\right]∥}^2\right\}=1$, so

$${\alpha }_{\mathrm{MRT}}=\sqrt{{\displaystyle \frac{1}{N_{tx}{\displaystyle \sum _{n=1}^N}{\sigma }_{\mathrm{RD},n}^2}}}$$

(20)

2.3.2. EGC/EGT Processing

When the signal is processed by the equal-gain technique, the detection matrix and the pre-coded matrix can be written as

$${\mathbf{W}}^T={\mathbf{W}}_{\mathrm{EGC}}^T={\displaystyle \frac{{\widehat{\mathbf{G}}}_{\mathrm{SR}}^H}{\left|{\widehat{\mathbf{G}}}_{\mathrm{SR}}\right|}}$$

(21)

$$\mathbf{A}={\mathbf{A}}_{\mathrm{EGT}}={\alpha }_{\mathrm{EGT}}{\displaystyle \frac{{\widehat{\mathbf{G}}}_{\mathrm{RD}}^{*}}{\left|{\widehat{\mathbf{G}}}_{\mathrm{RD}}^{*}\right|}}$$

(22)

where ${\alpha }_{\mathrm{EGT}}$ is a normalized constant and satisfies the conditions of $E\left\{{∥s\left[i\right]∥}^2\right\}=1$, so

$${\alpha }_{\mathrm{EGT}}=\left|{\widehat{\mathbf{G}}}_{\mathrm{RD}}^{*}\right|\sqrt{{\displaystyle \frac{1}{N_{tx}{\displaystyle \sum _{n=1}^N}{\sigma }_{\mathrm{RD},n}^2}}}$$

(23)

3. Large Antenna Array Loop Interference Elimination

3.1. Using Large-Scale Antenna Arrays

When receiving antennas tend to infinity, the subvector Spaces of useful signal and loop interference are almost orthogonal, so the effect of loop interference can be eliminated by increasing the number of receiving antennas [25]. Assuming that for any $N_{\mathrm{tx}}$, $N_{\mathrm{rx}}/N_{\mathrm{tx}}$ is fixed and $N_{\mathrm{rx}}\to \infty$, the received signal at the trunk can be written as

$${\displaystyle \frac{r_n\left[i\right]}{N_{rx}{\sigma }_{\mathrm{SR},n}^2}}\stackrel{a.s.}{\to }\sqrt{p_{\mathrm{S}}}{x}_n\left[i\right],\mathrm{for}\mathrm{MRC}/\mathrm{MRT}$$

(24)

$${\displaystyle \frac{r_n\left[i\right]}{N_{rx}{\sigma }_{\mathrm{SR},n}^2}}\stackrel{a.s.}{\to }{\displaystyle \frac{\sqrt{p_{\mathrm{S}}}{x}_n\left[i\right]}{\left|{\widehat{\mathbf{g}}}_{\mathrm{SR},n}^H\right|}},\mathrm{for}\mathrm{EGC}/\mathrm{EGT}$$

(25)

As can be seen from the above formula, when the receiving antenna tends to infinity, the loop interference disappears, and the interpair interference and noise also disappear. After the signal is processed by MRC/MRT and EGC/EGT, the original noise interference is eliminated and only the useful signal exists. At this time, the channel capacity between ${\mathrm{S}}_n\to \mathrm{R}$ will increase to infinity, and the system performance is only affected by the quality of the channel between $\mathrm{R}\to {\mathrm{D}}_n$, independent of loop interference.

3.2. Adopt Large Transmitting Antenna and Low Transmitting Power

The transmitting power $p_{\mathrm{R}}$ of the relay station determines the size of the loop interference [26]. Therefore, reducing the transmitting power can reduce the influence of the loop interference, but it will also affect the channel quality between $\mathrm{R}\to {\mathrm{D}}_n$, and the overall performance of the system will be worse. When the number of transmitting antennas is large, the relay station can concentrate the transmitted energy in the direction of the destination node and reduce the energy in other directions [27]. Therefore, when large-scale antennas are used in the repeater station, the relay transmission power can be reduced while keeping the communication link of $\mathrm{R}\to {\mathrm{D}}_n$ unchanged.

Assuming that N and $E_{\mathrm{R}}$ are fixed, the transmitting energy of the trunk is $p_{\mathrm{R}}=E_{\mathrm{R}}/N_{\mathrm{tx}}$, and for any finite $N_{\mathrm{rx}}$, when $N_{\mathrm{tx}}\to \infty$, the signal received by the trunk after MRC/MRT processing and EGC/EGT processing is

$$r_n\left[i\right]\stackrel{a.s.}{\to }\sqrt{p_{\mathrm{S}}}{\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},n}{x}_n\left[i\right]+\sqrt{p_{\mathrm{S}}}\sum _{j\ne n}^N{\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},j}{x}_j\left[i\right]+{\mathbf{w}}_n^T{\mathbf{n}}_{\mathrm{R}}\left[i\right]$$

(26)

The signal received by the destination node after MRC/MRT processing and EGC/EGT processing is

$$y_{D,n}\left[i\right]\stackrel{a.s.}{\to }\left\{\begin{array}{c}\sqrt{{\displaystyle \frac{{\sigma }_{\mathrm{RD},n}^4{E}_{\mathrm{R}}}{{\sum }_{j=1}^N{\sigma }_{\mathrm{RD},j}^2}}}{x}_n\left[i-d\right]+{\mathrm{n}}_{\mathrm{D},n}\left[i\right],\mathrm{for}\mathrm{MRC}/\mathrm{MRT}\\ \sqrt{{\displaystyle \frac{{\beta }_{\mathrm{RD},n}^2{\sigma }_{\mathrm{RD},n}^4{E}_{\mathrm{R}}}{{\sum }_{j=1}^N{\sigma }_{\mathrm{RD},j}^2}}}{x}_n\left[i-d\right]+{\mathrm{n}}_{\mathrm{D},n}\left[i\right],\mathrm{for}\mathrm{EGC}/\mathrm{EGT}\end{array}\right.$$

(27)

As can be seen from the above formula, by scaling the transmitting antenna to $1/N_{\mathrm{tx}}$, the loop interference at the trunk end disappears. Although the transmit power is reduced, the array gain increases with the increase in $N_{\mathrm{tx}}$, so the useful signal strength received by the destination node is sufficient. At the same time, due to the orthogonality of the channel vector, the interference between user pairs disappears, and the channel quality from the relay to the destination node is good.

4. System Performance Analysis

4.1. Reachable Rate Analysis

In this section, an approximate expression for the uplink spectral efficiency of the massive MIMO-based full-duplex relay system is derived, which is the achievable uplink rate of ${\mathrm{S}}_n\to \mathrm{R}$ and $\mathrm{R}\to {\mathrm{D}}_n$ after MRC/MRT and EGC/EGT processing. Because the reachable rate is affected by the weakest link or bottleneck link, the final value is the minimum reachable rate from the source node to the trunk and from the trunk to the destination node. The reachable rate is calculated using an alternative method where the received signal is considered to be a known average gain multiplied by the useful signal, plus uncorrelated noise. This approach is often applied to massive MIMO systems, and its advantage is that it does not need to provide destination instantaneous CSI, in which case the reachable rate of ${\mathrm{S}}_n\to \mathrm{R}\to {\mathrm{D}}_n$ is

$$R_n=min\left\{R_{\mathrm{SR},n},R_{\mathrm{RD},n}\right\}$$

(28)

Here, $R_{\mathrm{SR},n}$ and $R_{\mathrm{RD},n}$ are the achievable rates of the transmission links of ${\mathrm{S}}_n\to \mathrm{R}$ and $\mathrm{R}\to {\mathrm{D}}_n$, respectively, (15) can be rewritten as

$$r_n\left[i\right]=\sqrt{p_{\mathrm{S}}}\mathbb{E}\left\{{\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},n}\right\}{x}_n\left[i\right]+{\tilde{n}}_{\mathrm{R},n}\left[i\right]$$

(29)

Here, ${\tilde{n}}_{\mathrm{R},n}\left[i\right]$ is the effective noise, which can be written as

$$\begin{array}{cc}\hfill {\tilde{n}}_{\mathrm{R},n}\left[i\right]=& \sqrt{p_{\mathrm{S}}}\left({\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},n}-\mathbb{E}\left\{{\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},n}\right\}\right)x_n\left[i\right]\hfill \\ \hfill & +\sqrt{p_{\mathrm{S}}}\sum _{j\ne n}^N{\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},j}{x}_j\left[i\right]+\sqrt{p_{\mathrm{R}}}{\mathbf{w}}_n^T{\mathbf{G}}_{\mathrm{RR}}\mathbf{s}\left[i\right]+{\mathbf{w}}_n^T{\mathbf{n}}_{\mathrm{R}}\left[i\right]\hfill \end{array}$$

(30)

In this case, the useful signal and the effective noise in (29) are not correlated. Since the uncorrelated additive noise in the worst case has the characteristic of independent Gaussian white noise with the same variance, the reachable rate expression from the source node to the relay end can be written as

$$R_{\mathrm{SR},n}={log}_2\left(1+{\displaystyle \frac{p_{\mathrm{S}}{\left|\mathrm{E}\left\{{\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},n}\right\}\right|}^2}{p_{\mathrm{S}}\mathrm{Var}\left({\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},n}\right)+\mathrm{M}{\mathrm{P}}_n+\mathrm{L}{\mathrm{I}}_n+\mathrm{A}{\mathrm{N}}_n}}\right)$$

(31)

Among them, $\mathrm{M}{\mathrm{P}}_n$, $\mathrm{L}{\mathrm{I}}_n$ and $\mathrm{A}{\mathrm{N}}_n$ are multi-pair interference, loop interference and additive noise interference, respectively.

$$\mathrm{M}{\mathrm{P}}_n=p_{\mathrm{S}}\sum _{j\ne n}^N\mathrm{E}\left\{{\left|{\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},j}\right|}^2\right\}$$

(32)

$$\mathrm{L}{\mathrm{I}}_n=p_{\mathrm{R}}\mathrm{E}\left\{{∥{\mathbf{w}}_n^T{\mathbf{G}}_{\mathrm{RR}}\mathbf{A}∥}^2\right\}$$

(33)

$$\mathrm{A}{\mathrm{N}}_n=\mathrm{E}\left\{{∥{\mathbf{w}}_n∥}^2\right\}$$

(34)

Similarly, the expression of the reachable rate from the trunk to the destination node is

$$R_{\mathrm{RD},n}={log}_2\left(1+{\displaystyle \frac{p_{\mathrm{R}}{\left|\mathrm{E}\left\{{\mathbf{g}}_{\mathrm{RD},n}^T{\mathbf{a}}_n\right\}\right|}^2}{p_{\mathrm{R}}\mathrm{Var}\left({\mathbf{g}}_{\mathrm{RD},n}^T{\mathbf{a}}_n\right)+p_{\mathrm{R}}{\displaystyle \sum _{j\ne n}^N}\mathrm{E}\left\{{\left|{\mathbf{g}}_{\mathrm{RD},n}^T{\mathbf{a}}_j\right|}^2\right\}+1}}\right)$$

(35)

In massive MIMO, the central limit theorem guarantees the reliability of approximating the effective noise as additive Gaussian noise, where the effective noise is the sum of several parts. Therefore, the reachable speed limits in Equations (31) and (35) are accurate; (35) assumes that the destination node ${\mathrm{D}}_n$ applies the statistics of the channel gain (that is, $\mathrm{E}\left\{{\mathbf{g}}_{\mathrm{RD},n}^T{\mathbf{a}}_n\right\}$) to decode the transmitted signal, where it is not necessary to know the CSI. To prove that the derived reachability rate expression can accurately predict system performance, (35) can be compared with the traversal reachability rate received using Sprite, where the relays ${\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},j}$, ${\mathbf{G}}_{\mathrm{RR}}$, and ${\mathbf{g}}_{\mathrm{RD},n}^T{\mathbf{a}}_j$, $j=1,\dots ,N$, are known. In this case, the reachable rate of ${\mathrm{S}}_n\to \mathrm{R}\to {\mathrm{D}}_n$ is

$${\tilde{R}}_n=min\left\{{\tilde{R}}_{\mathrm{SR},n},{\tilde{R}}_{\mathrm{RD},n}\right\}$$

(36)

Here, the expressions for ${\tilde{R}}_{\mathrm{SR},n}$ and ${\tilde{R}}_{\mathrm{RD},n}$ are given by

$${\tilde{R}}_{\mathrm{SR},n}=\mathrm{E}\left\{{log}_2\left(1+{\displaystyle \frac{p_{\mathrm{S}}{\left|{\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},n}\right|}^2}{p_{\mathrm{S}}{\displaystyle \sum _{j\ne n}^N}{\left|{\mathbf{w}}_n^T{\mathbf{g}}_{\mathrm{SR},j}\right|}^2+p_{\mathrm{R}}{∥{\mathbf{w}}_n^T{\mathbf{G}}_{\mathrm{RR}}\mathbf{A}∥}^2+{∥{\mathbf{w}}_n∥}^2}}\right)\right\}$$

(37)

$${\tilde{R}}_{\mathrm{RD},n}=\mathrm{E}\left\{{log}_2\left(1+{\displaystyle \frac{p_{\mathrm{R}}{\left|{\mathbf{g}}_{\mathrm{RD},n}^T{\mathbf{a}}_n\right|}^2}{p_{\mathrm{R}}{\displaystyle \sum _{j\ne n}^N}{\left|{\mathbf{g}}_{\mathrm{RD},n}^T{\mathbf{a}}_j\right|}^2+1}}\right)\right\}$$

(38)

It can be seen from the subsequent simulation that the reachable rate expressions of Equations (28) and (36) are relatively consistent, especially when the transmitting antenna and receiving antenna are large.

4.2. MRC/MRT Approach

Theorem 1.

After MRC/MRT processing, the exact expression of the reachable rate of $S_n\to R\to D_n$ can be obtained by means of Equation (28)

$$R_n=R_n^{\mathit{MR}}={log}_2\left(1+min\left(R_{\mathit{SR},n}^{\mathit{MR}},R_{\mathit{RD},n}^{\mathit{MR}}\right)\right)$$

(39)

Here, the expressions for $R_{\mathit{SR},n}^{\mathit{MR}}$ and $R_{\mathit{RD},n}^{\mathit{MR}}$ are given by

$$R_{\mathit{SR},n}^{\mathit{MR}}={log}_2\left(1+{\displaystyle \frac{p_S\frac{{\sigma }_{\mathit{SR},n}^2{N}_{rx}}{K_{\mathit{SR},n}+1}\left(K_{\mathit{SR},n}+{\eta }_{\mathit{SR},n}\right)}{\frac{p_S{\beta }_{\mathit{SR},n}}{K_{\mathit{SR},n}+1}{\eta }_{\mathit{SR},n}+\frac{p_S}{N_{rx}\left(K_{\mathit{SR},n}+{\eta }_{\mathit{SR},n}\right)}{\displaystyle \sum _{j\ne n}^N}{\Delta }_1+p_R{\sigma }_{LI}^2+1}}\right)$$

(40)

$$R_{\mathit{RD},n}^{\mathit{MR}}={log}_2\left(1+{\displaystyle \frac{p_R{\sigma }_{\mathit{RD},n}^4{N}_{tx}{\left(K_{\mathit{RD},n}+{\eta }_{\mathit{RD},n}\right)}^2}{p_R{\beta }_{\mathit{RD},n}{\sigma }_{\mathit{RD},n}^2{\eta }_{\mathit{RD},n}\left(K_{\mathit{RD},n}+1\right)+\frac{1}{N_{tx}}{p}_R{\sigma }_{\mathit{RD},n}^2\left(K_{\mathit{RD},n}+1\right){\displaystyle \sum _{j\ne n}^N}{\Delta }_2+{\left(K_{\mathit{RD},n}+1\right)}^2{\displaystyle \sum _{j=1}^N}{\sigma }_{\mathit{RD},j}^2}}\right)$$

(41)

$${\Delta }_1={\displaystyle \frac{{\sigma }_{\mathit{SR},j}^2}{K_{\mathit{SR},j}+1}}\left(K_{\mathit{SR},n}{K}_{\mathit{SR},j}{\varphi }_{nj}^2+N_{rx}{K}_{\mathit{SR},j}{\eta }_{\mathit{SR},n}+N_{rx}{K}_{\mathit{SR},n}{\eta }_{\mathit{SR},j}+N_{rx}{\eta }_{\mathit{SR},n}{\eta }_{\mathit{SR},j}\right)$$

(42)

$${\Delta }_2={\displaystyle \frac{{\sigma }_{\mathit{RD},j}^2}{K_{\mathit{RD},j}+1}}\left(K_{\mathit{RD},n}{K}_{\mathit{RD},j}{\varphi }_{nj}^2+N_{tx}{K}_{\mathit{RD},j}{\eta }_{\mathit{RD},n}+N_{tx}{K}_{\mathit{RD},n}{\eta }_{\mathit{SR},j}+N_{tx}{\eta }_{\mathit{RD},n}{\eta }_{\mathit{RD},j}\right)$$

(43)

4.3. EGC/EGT Approach

Theorem 2.

After EGC/EGT processing, the exact expression of the reachable rate of $S_n\to R\to D_n$ can be obtained by the method of Equation (28)

$$R_n=R_n^{\mathit{EG}}={log}_2\left(1+min\left(R_{\mathit{SR},n}^{\mathit{EG}},R_{\mathit{RD},n}^{\mathit{EG}}\right)\right)$$

(44)

Here, the expressions for $R_{\mathit{SR},n}^{\mathit{EG}}$ and $R_{\mathit{EG},n}^{\mathit{MR}}$ are given by

$$R_{\mathit{SR},n}^{\mathit{EG}}={log}_2\left(1+{\displaystyle \frac{p_S\frac{{\sigma }_{\mathit{SR},n}^2{N}_{rx}}{K_{\mathit{SR},n}+1}\left(K_{\mathit{SR},n}+{\eta }_{\mathit{SR},n}\right)}{\frac{p_S{\beta }_{\mathit{SR},n}}{K_{\mathit{SR},n}+1}-p_S\frac{{\sigma }_{\mathit{SR},n}^2}{K_{\mathit{SR},n}+1}\left(N_{rx}{K}_{\mathit{SR},n}{\eta }_{\mathit{SR},n}+K_{\mathit{SR},n}+{\eta }_{\mathit{SR},n}\right)+\frac{p_S}{N_{rx}\left(K_{\mathit{SR},n}+{\eta }_{\mathit{SR},n}\right)}{\displaystyle \sum _{j\ne n}^N}{\Delta }_1+p_R{\sigma }_{LI}^2+1}}\right)$$

(45)

$$R_{\mathit{RD},n}^{\mathit{EG}}={log}_2\left(1+{\displaystyle \frac{p_R{\sigma }_{\mathit{RD},n}^4{N}_{tx}{\left(K_{\mathit{RD},n}+{\eta }_{\mathit{RD},n}\right)}^2}{\begin{array}{cc}\hfill & p_R{\beta }_{\mathit{RD},n}\left(K_{\mathit{RD},n}+1\right)\left(K_{\mathit{RD},n}+{\eta }_{\mathit{RD},n}\right)+{\left(K_{\mathit{RD},n}+1\right)}^2\sum _{j=1}^N{\sigma }_{\mathit{RD},j}^2\hfill \\ \hfill & -p_R{\sigma }_{\mathit{RD},n}^4\left(K_{\mathit{RD},n}+1\right)\left(N_{tx}{K}_{\mathit{RD},n}{\eta }_{\mathit{RD},n}+K_{\mathit{RD},n}+{\eta }_{\mathit{RD},n}\right)+\frac{1}{N_{tx}}{p}_R{\sigma }_{\mathit{RD},n}^2\left(K_{\mathit{RD},n}+1\right)\sum _{j\ne n}^N{\Delta }_2\hfill \end{array}}}\right)$$

(46)

5. Performance Evaluation

5.1. Spectral Eefficiency

Next, the system performance is analyzed in terms of spectral efficiency, and let T denote the coherence time, during which the user sends the $\tau$ pilot sequence of length A for training and the rest time for data transmission; therefore, the uplink spectral efficiency is

$$S_{\mathrm{FD}}^{\mathrm{A}}={\displaystyle \frac{T-\tau }{T}}\sum _{n=1}^N{R}_n^{\mathrm{A}}$$

(47)

Here, $\mathrm{A}\in \left\{\mathrm{MR},\mathrm{EG}\right\}$ is the MRC/MRT treatment and the EGC/EGT treatment, respectively. According to Theorems 1 and 2 and Equation (47), the expression of full-duplex spectral efficiency after MRC/MRT and EGC/EGT processing is

$$S_{\mathrm{FD}}^{\mathrm{MR}}={\displaystyle \frac{T-\tau }{T}}\sum _{n=1}^N{log}_2\left(1+min\left(R_{\mathrm{SR},n}^{\mathrm{MR}},R_{\mathrm{RD},n}^{\mathrm{MR}}\right)\right)$$

(48)

$$S_{\mathrm{FD}}^{\mathrm{EG}}={\displaystyle \frac{T-\tau }{T}}\sum _{n=1}^N{log}_2\left(1+min\left(R_{\mathrm{SR},n}^{\mathrm{EG}},R_{\mathrm{RD},n}^{\mathrm{EG}}\right)\right)$$

(49)

5.2. Energy Efficiency

In this section, the advantages of having a large number of antennas in a relay station are studied.

Case 1: Assume $p_{\mathrm{P}}$, $E_{\mathrm{S}}$, and $E_{\mathrm{R}}$ are fixed, where $p_{\mathrm{S}}=E_{\mathrm{S}}/N_{\mathrm{rx}}$, $p_{\mathrm{R}}=E_{\mathrm{R}}/N_{\mathrm{tx}}$. When $N_{\mathrm{rx}},N_{\mathrm{tx}}\to \infty$, the approximate expression of spectral efficiency after MRC/MRT treatment and EGC/EGT treatment is

$$S_{\mathrm{FD}}^{\mathrm{MR}}\to {\displaystyle \frac{T-\tau }{T}}{\displaystyle \sum _{n=1}^N}{log}_2\left(1+min\left(E_{\mathrm{S}}{\sigma }_{\mathrm{SR},n}^2{\displaystyle \frac{K_{\mathrm{SR},n}+{\eta }_{\mathrm{SR},n}}{K_{\mathrm{SR},n}+1}},{\displaystyle \frac{E_{\mathrm{R}}{\sigma }_{\mathrm{RD},n}^4{\left(K_{\mathrm{RD},n}+{\eta }_{\mathrm{RD},n}\right)}^2}{{\left(K_{\mathrm{RD},n}+1\right)}^2{\displaystyle \sum _{j=1}^N}{\sigma }_{\mathrm{RD},j}^2}}\right)\right)$$

(50)

$$S_{\mathrm{FD}}^{\mathrm{EG}}\to {\displaystyle \frac{T-\tau }{T}}\sum _{n=1}^N{log}_2\left(1+min\left(E_{\mathrm{S}}{\sigma }_{\mathrm{SR},n}^2{\displaystyle \frac{K_{\mathrm{SR},n}+{\eta }_{\mathrm{SR},n}}{K_{\mathrm{SR},n}+1}},A\right)\right)$$

(51)

Among them,

$$A={\displaystyle \frac{E_{\mathrm{R}}{\sigma }_{\mathrm{RD},n}^4{\left(K_{\mathrm{RD},n}+{\eta }_{\mathrm{RD},n}\right)}^2}{{\left(K_{\mathrm{RD},n}+1\right)}^2{\displaystyle \sum _{j=1}^N}{\sigma }_{\mathrm{RD},j}^2-E_{\mathrm{R}}{\sigma }_{\mathrm{RD},n}^4\left(K_{\mathrm{RD},n}+1\right)K_{\mathrm{RD},n}{\eta }_{\mathrm{RD},n}}}$$

(52)

When the above formula is reduced to $1/N_{\mathrm{rx}}$ and $1/N_{\mathrm{tx}}$ in proportion, the overall performance of the system will remain unchanged, assuming that for any n, there is ${\beta }_{\mathrm{SR},n}={\beta }_{\mathrm{RD},n}=1$, then Equations (50) and (51) can be written as

$$S_{\mathrm{FD}}^{\mathrm{MR}}\to {\displaystyle \frac{T-\tau }{T}}N{log}_2\left(1+min\left(E_{\mathrm{S}}{\sigma }_1^2{\displaystyle \frac{K_{\mathrm{SR},n}+{\eta }_{\mathrm{SR},n}}{K_{\mathrm{SR},n}+1}},{\displaystyle \frac{E_{\mathrm{R}}{\sigma }_1^2}{N}}{\left({\displaystyle \frac{K_{\mathrm{RD},n}+{\eta }_{\mathrm{RD},n}}{K_{\mathrm{RD},n}+1}}\right)}^2\right)\right)$$

(53)

$$S_{\mathrm{FD}}^{\mathrm{EG}}\to {\displaystyle \frac{T-\tau }{T}}N{log}_2\left(1+min\left(E_{\mathrm{S}}{\sigma }_1^2{\displaystyle \frac{K_{\mathrm{SR},n}+{\eta }_{\mathrm{SR},n}}{K_{\mathrm{SR},n}+1}},B\right)\right)$$

(54)

$$B={\displaystyle \frac{E_{\mathrm{R}}{\sigma }_1^2}{N}}{\left({\displaystyle \frac{K_{\mathrm{RD},n}+{\eta }_{\mathrm{RD},n}}{K_{\mathrm{RD},n}+1}}\right)}^2-{\displaystyle \frac{{\left(K_{\mathrm{RD},n}+{\eta }_{\mathrm{RD},n}\right)}^2}{\left(K_{\mathrm{RD},n}+1\right)K_{\mathrm{RD},n}{\eta }_{\mathrm{RD},n}}}$$

(55)

Among them, ${\sigma }_1^2=\tau p_{\mathrm{P}}/\left(\tau p_{\mathrm{P}}+1\right)$. As can be seen from the above formula, the use of large-scale antennas can reduce the user’s transmission power, and the total spectral efficiency is N times higher than before.

Case 2: Assuming $E_{\mathrm{S}}$ and $E_{\mathrm{R}}$ are fixed, with $p_{\mathrm{P}}=p_{\mathrm{S}}=E_{\mathrm{S}}/\sqrt{N_{\mathrm{rx}}}$ and $p_{\mathrm{R}}=E_{\mathrm{R}}/\sqrt{N_{\mathrm{tx}}}$, when $N_{\mathrm{rx}}\to \infty$ and $N_{\mathrm{tx}}=k{N}_{\mathrm{rx}}$, $k>0$, the approximate expression of spectral efficiency is

$$S_{\mathrm{FD}}^{\mathrm{MR}}\to {\displaystyle \frac{T-\tau }{T}}\sum _{n=1}^N{log}_2\left(1+min\left(\tau E_{\mathrm{S}}^2{\beta }_{\mathrm{SR},n}^2{N}_{rx}{\displaystyle \frac{K_{\mathrm{SR},n}}{K_{\mathrm{SR},n}+1}},{\displaystyle \frac{\sqrt{k}\tau E_{\mathrm{S}}{E}_{\mathrm{R}}{\beta }_{\mathrm{RD},n}^4{K}_{\mathrm{RD},n}^2}{{\left(K_{\mathrm{RD},n}+1\right)}^2{\displaystyle \sum _{j=1}^N}{\beta }_{\mathrm{RD},j}^2}}\right)\right)$$

(56)

$$S_{\mathrm{FD}}^{\mathrm{EG}}\to {\displaystyle \frac{T-\tau }{T}}\sum _{n=1}^N{log}_2\left(1+min\left(\tau E_{\mathrm{S}}^2{\beta }_{\mathrm{SR},n}^2{N}_{rx}{\displaystyle \frac{K_{\mathrm{SR},n}}{K_{\mathrm{SR},n}+1}},C\right)\right)$$

(57)

$$C={\displaystyle \frac{\sqrt{k}{\tau }^2{E}_{\mathrm{S}}{E}_{\mathrm{R}}{\beta }_{\mathrm{RD},n}^4{K}_{\mathrm{RD},n}^2}{E_{\mathrm{R}}{\beta }_{\mathrm{RD},n}\left(K_{\mathrm{RD},n}+1\right)+\tau E_{\mathrm{S}}{\left(K_{\mathrm{RD},n}+1\right)}^2{\displaystyle \sum _{j=1}^N}{\beta }_{\mathrm{RD},j}^2}}$$

(58)

According to the above formula, when the uplink frequency transmission energy and transmission power are the same, the spectral efficiency can be reduced to $1/\sqrt{N_{\mathrm{rx}}}$ and $1/\sqrt{N_{\mathrm{tx}}}$.

5.3. Compare Full-Duplex and Half-Duplex

In this section, system performance in full duplex and half duplex modes is compared. Compared to full duplex, half duplex mode does not cause loop interference, but it also results in a reduction in the transmission rate, and the transmission time is only half that of full duplex mode. Therefore, we assume that the user sends the same power in the coherence interval, that is, the coherence time used in half duplex mode is twice that in full duplex mode. At this time, the approximate expression of spectral efficiency after MRC/MRT processing and EGC/EGT processing is

$$S_{\mathrm{HD}}^{\mathrm{MR}}={\displaystyle \frac{T-\tau }{2T}}\sum _{n=1}^N{log}_2\left(1+min\left(R_{\mathrm{SR},n}^{\mathrm{MR}},R_{\mathrm{RD},n}^{\mathrm{MR}}\right)\right)$$

(59)

$$S_{\mathrm{HD}}^{\mathrm{EG}}={\displaystyle \frac{T-\tau }{2T}}\sum _{n=1}^N{log}_2\left(1+min\left(R_{\mathrm{SR},n}^{\mathrm{EG}},R_{\mathrm{RD},n}^{\mathrm{EG}}\right)\right)$$

(60)

In this paper, a hybrid relay mode is proposed. When ${\sigma }_{\mathrm{LI}}^2\le {\sigma }_{\mathrm{LI},0}^2$, the spectral efficiency of full-duplex mode is higher than that of half-duplex mode, and ${\sigma }_{\mathrm{LI},0}^2$ is the balance of the spectral efficiency of the full-duplex and half-duplex modes.

$$\mathrm{Hybrid}\mathrm{Relay}\mathrm{Mode}=\left\{\begin{array}{c}\mathrm{Full-}\mathrm{duplex},S_{\mathrm{FD}}^{\mathrm{A}}\ge S_{\mathrm{HD}}^{\mathrm{A}}\\ \mathrm{Half-}\mathrm{duplex},otherwise\end{array}\right.$$

(61)

5.4. Power Distribution

In the previous section, we assumed that the transmission power is the same for all users. However, different power distribution schemes can lead to varying system performance. Therefore, in this section, we assume that the transmission powers are different for different users, and the training time $\tau$ and pilot power $p_{\mathrm{P}}$ are given. Based on Ngo’s design approach [28], we propose a power allocation algorithm to maximize the energy utilization of each node. Assuming that the transmitted power of the nth source is $p_{\mathrm{S},\mathrm{n}}$, the energy efficiency in full-duplex mode is

$$\mathrm{E}{\mathrm{E}}^{\mathrm{A}}={\displaystyle \frac{S_{\mathrm{FD}}^{\mathrm{A}}}{\frac{T-\tau }{T}\left({\displaystyle \sum _{n=1}^N}{p}_{\mathrm{S},\mathrm{n}}+p_R\right)}}$$

(62)

To translate this into mathematics, it can be written as

$$\begin{array}{cc}\hfill \mathrm{maximize}& \mathrm{E}{\mathrm{E}}^{\mathrm{A}}\hfill \\ \hfill \mathrm{subject}\mathrm{to}& S_{\mathrm{FD}}^{\mathrm{A}}=S_0^{\mathrm{A}}\hfill \\ \hfill & 0\le p_{\mathrm{S},\mathrm{n}}\le p_0,n=1,\dots ,N\hfill \\ \hfill & 0\le p_R\le p_1\hfill \end{array}$$

(63)

Here, $S_0^{\mathrm{A}}$ is the given spectral efficiency, and $p_0$ and $p_1$ are the peaks of $p_{\mathrm{S},\mathrm{n}}$ and $p_R$.

From (48), (49) and (62), it can be seen that the energy optimization problem in (62) can be rewritten as

$$\begin{array}{cc}\hfill \mathrm{minimize}& {\displaystyle \sum _{n=1}^N}{p}_{\mathrm{S},\mathrm{n}}+p_R\hfill \\ \hfill \mathrm{subject}\mathrm{to}& {\displaystyle \frac{T-\tau }{T}}{\displaystyle \sum _{n=1}^N}{log}_2\left(1+min\left(R_{\mathrm{SR},n}^{\mathrm{A}},R_{\mathrm{RD},n}^{\mathrm{A}}\right)\right)=S_0^{\mathrm{A}}\hfill \\ \hfill & 0\le p_{\mathrm{S},\mathrm{n}}\le p_0,n=1,\dots ,N\hfill \\ \hfill & 0\le p_R\le p_1\hfill \end{array}$$

(64)

Among them, $R_{\mathrm{SR},n}^{\mathrm{A}}={\displaystyle \frac{a_n{p}_{\mathrm{S},\mathrm{n}}}{b_n{p}_{\mathrm{S},\mathrm{n}}-c_n{p}_{\mathrm{S},\mathrm{n}}+{\displaystyle \sum _{j\ne n}^N}{d}_j{p}_{\mathrm{S},\mathrm{j}}+e_n{p}_R+1}}$, $R_{\mathrm{RD},n}^{\mathrm{A}}={\displaystyle \frac{f_n{p}_R}{g_n{p}_R-h_n{p}_R+l_n}}$.

When the system uses MRC/MRT processing, $a_n={\displaystyle \frac{{\sigma }_{\mathrm{SR},n}^2{N}_{rx}}{K_{\mathrm{SR},n}+1}}\left(K_{\mathrm{SR},n}+{\eta }_{\mathrm{SR},n}\right)$, $b_n={\displaystyle \frac{{\beta }_{\mathrm{SR},n}{\eta }_{\mathrm{SR},n}}{K_{\mathrm{SR},n}+1}}$, $c_n=0$, $d_n={\Delta }_1/N_{rx}\left(K_{\mathrm{SR},n}+{\eta }_{\mathrm{SR},n}\right)$, $e_n={\sigma }_{LI}^2$, $f_n={\sigma }_{\mathrm{RD},n}^4{N}_{tx}{\left(K_{\mathrm{RD},n}+{\eta }_{\mathrm{RD},n}\right)}^2$, $g_n={\beta }_{\mathrm{RD},n}{\sigma }_{\mathrm{RD},n}^2{\eta }_{\mathrm{RD},n}\left(K_{\mathrm{RD},n}+1\right)+{\sigma }_{\mathrm{RD},n}^2\left(K_{\mathrm{RD},n}+1\right){\displaystyle \sum _{j\ne n}^N}{\Delta }_2/N_{tx}$, $h_n=0$, $l_n={\left(K_{\mathrm{RD},n}+1\right)}^2{\displaystyle \sum _{j=1}^N}{\sigma }_{\mathrm{RD},j}^2$.

When the system uses EGC/EGT processing, $a_n={\displaystyle \frac{{\sigma }_{\mathrm{SR},n}^2{N}_{rx}}{K_{\mathrm{SR},n}+1}}\left(K_{\mathrm{SR},n}+{\eta }_{\mathrm{SR},n}\right)$, $b_n={\displaystyle \frac{{\beta }_{\mathrm{SR},n}}{K_{\mathrm{SR},n}+1}}$, $c_n={\sigma }_{\mathrm{SR},n}^2\left(N_{rx}{K}_{\mathrm{SR},n}{\eta }_{\mathrm{SR},n}+K_{\mathrm{SR},n}+{\eta }_{\mathrm{SR},n}\right)/K_{\mathrm{SR},n}+1$, $d_n={\Delta }_3/N_{rx}\left(K_{\mathrm{SR},n}+{\eta }_{\mathrm{SR},n}\right)$, $e_n={\sigma }_{LI}^2$, $f_n={\sigma }_{\mathrm{RD},n}^4{N}_{tx}{\left(K_{\mathrm{RD},n}+{\eta }_{\mathrm{RD},n}\right)}^2$, $g_n={\beta }_{\mathrm{RD},n}\left(K_{\mathrm{RD},n}+1\right)\left(K_{\mathrm{RD},n}+{\eta }_{\mathrm{RD},n}\right)+{\sigma }_{\mathrm{RD},n}^2\left(K_{\mathrm{RD},n}+1\right){\displaystyle \sum _{j\ne n}^N}{\Delta }_4/N_{tx}$, $h_n={\sigma }_{\mathrm{RD},n}^4\left(K_{\mathrm{RD},n}+1\right)\left(N_{tx}{K}_{\mathrm{RD},n}{\eta }_{\mathrm{RD},n}+K_{\mathrm{RD},n}+{\eta }_{\mathrm{RD},n}\right)$, $l_n={\left(K_{\mathrm{RD},n}+1\right)}^2{\displaystyle \sum _{j=1}^N}{\sigma }_{\mathrm{RD},j}^2$.

Therefore, (64) is equivalent to

$$\begin{array}{cc}\hfill \mathrm{minimize}& {\displaystyle \sum _{n=1}^N}{p}_{\mathrm{S},\mathrm{n}}+p_R\hfill \\ \hfill \mathrm{subject}\mathrm{to}& {\displaystyle \frac{T-\tau }{T}}{\displaystyle \sum _{n=1}^N}{log}_2\left(1+{\gamma }_n\right)=S_0^{\mathrm{A}}\hfill \\ \hfill & {\gamma }_n\le {\displaystyle \frac{a_n{p}_{\mathrm{S},\mathrm{n}}}{b_n{p}_{\mathrm{S},\mathrm{n}}-c_n{p}_{\mathrm{S},\mathrm{n}}+{\displaystyle \sum _{j\ne n}^N}{d}_j{p}_{\mathrm{S},\mathrm{j}}+e_n{p}_R+1}},n=1,\dots ,N\hfill \\ \hfill & {\gamma }_n\le {\displaystyle \frac{f_n{p}_R}{g_n{p}_R-h_n{p}_R+l_n}},n=1,\dots ,N\hfill \\ \hfill & 0\le p_{\mathrm{S},\mathrm{n}}\le p_0,n=1,\dots ,N\hfill \\ \hfill & 0\le p_R\le p_1\hfill \end{array}$$

(65)

Since $a_n$, $b_n$, $c_n$, $d_n$ and $e_n$ is all greater than or equal to 0, (64) is rewritten as

$$\begin{array}{cc}\hfill \mathrm{minimize}& {\displaystyle \sum _{n=1}^N}{p}_{\mathrm{S},\mathrm{n}}+p_R\hfill \\ \hfill \mathrm{subject}\mathrm{to}& \prod \left(1+{\gamma }_n\right)=2^{{\displaystyle \frac{T{S}_0^{\mathrm{A}}}{T-\tau }}}\hfill \\ \hfill & {\displaystyle \frac{b_n}{a_n}}{\gamma }_n-{\displaystyle \frac{c_n}{a_n}}{\gamma }_n+{\displaystyle \sum _{j\ne n}^N}{\displaystyle \frac{d_j}{a_n}}{p}_{\mathrm{S},\mathrm{j}}{\gamma }_n{p}_{\mathrm{S},\mathrm{n}}^{-1}+{\displaystyle \frac{e_n}{a_n}}{p}_R{\gamma }_n{p}_{\mathrm{S},\mathrm{n}}^{-1}+{\displaystyle \frac{1}{a_n}}{\gamma }_n{p}_{\mathrm{S},\mathrm{n}}^{-1}\le 1,\forall n\hfill \\ \hfill & {\displaystyle \frac{g_n}{f_n}}{\gamma }_n-{\displaystyle \frac{h_n}{f_n}}{\gamma }_n+{\displaystyle \frac{l_n}{f_n}}{\gamma }_n{p}_R^{-1}\le 1,n=1,\dots ,N\hfill \\ \hfill & 0\le p_{\mathrm{S},\mathrm{n}}\le p_0,n=1,\dots ,N\hfill \\ \hfill & 0\le p_R\le p_1\hfill \end{array}$$

(66)

We find that the above objective functions and inequality constraints are positive functions, and can effectively find approximate solutions by solving GP sequences. More precisely, when ${\eta }_n={\widehat{\gamma }}_n{\left(1+{\widehat{\gamma }}_n\right)}^{-1}$ and ${\kappa }_n={\widehat{\gamma }}_n^{-{\eta }_n}\left(1+{\widehat{\gamma }}_n\right)$ are used, ${\kappa }_n{\widehat{\gamma }}_n^{{\eta }_n}$ can be used to approximate $1+{\gamma }_n$. Therefore, near point ${\widehat{\gamma }}_n$, the left side of the equation constraint can be approximated as

$$\prod \left(1+{\gamma }_n\right)\approx \Pi {\kappa }_n{\gamma }_n^{{\eta }_n}$$

(67)

Therefore, the optimization problem of (66) can be transformed into a geometric programming problem, and the specific algorithm is as follows (Algorithm 1).

Algorithm 1 Successive approximation algorithm

1: Initialization: Set $i=1$, select ${\gamma }_n$ as the initial value of ${\gamma }_{n,1}\left(n=1,\dots ,N\right)$, assume error $\epsilon$, maximum number of iterations L and parameter $\alpha$. 2: Iteration i: Calculate ${\eta }_{n,i}={\gamma }_{n,i}{\left(1+{\gamma }_{n,i}\right)}^{-1}$ and ${\kappa }_{n,i}={\gamma }_{n,i}^{-{\eta }_{n,i}}\left(1+{\gamma }_{n,i}\right)$, and then solve the geometric programming problem. $$\begin{array}{cc}\hfill \mathrm{minimize}& \sum _{n=1}^N{p}_{\mathrm{S},\mathrm{n}}+p_R\hfill \\ \hfill \mathrm{subject}\mathrm{to}& \prod \left(1+{\gamma }_n\right)=2^{{\displaystyle \frac{T{S}_0^{\mathrm{A}}}{T-\tau }}}\hfill \\ \hfill & {\displaystyle \frac{b_n}{a_n}}{\gamma }_n-{\displaystyle \frac{c_n}{a_n}}{\gamma }_n+\sum _{j\ne n}^N{\displaystyle \frac{d_j}{a_n}}{p}_{\mathrm{S},\mathrm{j}}{\gamma }_n{p}_{\mathrm{S},\mathrm{n}}^{-1}+{\displaystyle \frac{e_n}{a_n}}{p}_R{\gamma }_n{p}_{\mathrm{S},\mathrm{n}}^{-1}+{\displaystyle \frac{1}{a_n}}{\gamma }_n{p}_{\mathrm{S},\mathrm{n}}^{-1}\le 1,\forall n\hfill \\ \hfill & {\displaystyle \frac{g_n}{f_n}}{\gamma }_n-{\displaystyle \frac{h_n}{f_n}}{\gamma }_n+{\displaystyle \frac{l_n}{f_n}}{\gamma }_n{p}_R^{-1}\le 1,n=1,\dots ,N\hfill \\ \hfill & 0\le p_{\mathrm{S},\mathrm{n}}\le p_0,n=1,\dots ,N\hfill \\ \hfill & 0\le p_R\le p_1\hfill \\ \hfill & {\alpha }^{-1}{\gamma }_{n,i}\le {\gamma }_n\le \alpha {\gamma }_{n,i}\hfill \end{array}$$ Let us call ${\gamma }_n^{*},n=1,\dots ,N$ the solution. 3: If ${max}_n\left|{\gamma }_{n,i}-{\gamma }_n^{*}\right|<\epsilon$ or $i=L$, the algorithm ends; otherwise, turn to step 4. 4: Set $i=i+1$ and ${\gamma }_{n,i}={\gamma }_n^{*}$, and return to step 2.

6. Numerical Results

In this paper, the full-duplex relay scenario under massive MIMO is studied. The simulation environment used is as follows: Users are evenly distributed in hexagonal cells with a radius of 1000 m; the minimum distance between the user and the base station is 100 m; the number of users per cell $N=5$; the noise power is 1; pilot sequence length $\tau =10$; the symbol length of each frame $T=196$; number of antennas $N_{\mathrm{rx}}=N_{\mathrm{tx}}$.

Figure 2 shows the results of the total spectral efficiency as a function of SNR for the MRC/MRT and EGC/EGT treatments for different antenna numbers. At this time, the signal-to-noise ratio $\mathrm{SNR}=p_{\mathrm{S}}$; loop interference ${\sigma }_{\mathrm{LI}}^2=2\mathrm{dB}$. The spectral efficiency of different linear processing methods is given by Equations (39) and (44), and the Monte Carlo simulation is given by Equations (28), (31) and (35). It can be seen from the figure that the simulation value of the spectral efficiency is not much different from the theoretical value after linear processing. At 15 dB, the spectral efficiency difference between the two schemes is only 0.53 bits/s/Hz. Its overall trend is close to the simulation value. The system spectral efficiency increases with the increase in the signal-to-noise ratio and eventually tends to a constant value. When the signal-to-noise ratio is fixed, the larger the number of antennas, the greater the spectral efficiency. Moreover, the simulation curve of EGC/EGT is always slightly lower than that of MRC/MRT. Compared with the MRC/MRT method, the EGC/EGT method has the advantages of a simple hardware structure and low cost. Moreover, it only relies on the phase of the signal and does not require amplitude information. Although it is slightly inferior to the maximum ratio merging/transmission scheme in terms of SNR, its advantage in the trade-off between engineering implementation and complexity is obvious, and it is suitable for some occasions with high requirements for signal quality. Therefore, later on, it can be considered to use the simpler EGC/EGT scheme instead of the MRC/MRT scheme for linear processing of the signal.

Figure 3 compares the calculation methods of two spectral efficiency simulations, namely the spectral efficiency simulation values of Equations (28) and (36). Among them, the method adopted by Formula (28) does not require the destination to know the instantaneous CSI, while Formula (36) uses the wizard reception to give a definite instantaneous CSI for later calculation. As can be seen from the figure, the performance gap between the two calculation methods is very small, and the curve trend increases with the increase in SNR, which proves that it is feasible to use channel statistics to detect sent signals. Therefore, the spectrum efficiency expression given by Equation (28) can accurately predict system performance.

Figure 4 shows the change trend of the user’s required transmission energy demand per 1 bit/s/Hz. At this time, assume that $p_R=N{p}_S$, ${\beta }_{SR,n}={\beta }_{RD,n}=1,n=1,2,\dots ,N$, studied the relationship between the number of base station antennas and the energy required by the user to send signals in $p_P=15\mathrm{dB}$ and $p_P=p_S$, and verified that the use of a large-scale antenna array can improve the system performance. According to the loopback interference intensity and the user’s transmission, power is divided into four cases: (1) ${\sigma }_{\mathrm{LI}}^2=3p_P=15\mathrm{dB}$; (2) ${\sigma }_{\mathrm{LI}}^2=10p_P=15\mathrm{dB}$; (3) ${\sigma }_{\mathrm{LI}}^2=3p_P=p_S$; (4) ${\sigma }_{\mathrm{LI}}^2=10p_P=p_S$. It can be seen from the figure that the energy required for users to send signals gradually decreases as the number of base station antennas increases. When the loopback interference is strong and the number of antennas is constant, the energy efficiency can be improved by increasing the number of base station antennas. In addition, the intensity of the loopback interference affects the spectral efficiency. The required transmission energy increases with the increase in the loopback interference intensity. However, for cases where the loopback interference intensity is large and the number of base station antennas is relatively small, even if the transmission energy is infinite, a higher spectral efficiency cannot be achieved. Therefore, large-scale antenna arrays need to be used to reduce the impact of the loopback interference and thereby improve the spectral efficiency. When the number of base station antennas is fixed, the transmission energy required by the MRC/MRT scheme is always slightly lower than that of the EGC/EGT scheme, but generally not much different. Considering the characteristics of low complexity and low latency of the EGC/EGT scheme, the simpler EGC/EGT scheme can be used to replace the MRC/MRT scheme in linear processing later.

Figure 5 shows the spectral efficiency performance of the system in full-duplex and half-duplex modes under different loop interference intensities. Assume that $p_{\mathrm{R}}=p_{\mathrm{P}}=p_{\mathrm{S}}=15\mathrm{dB}$, ${\beta }_{\mathrm{SR},n}={\beta }_{\mathrm{RD},n}=1$ and $N_{\mathrm{rx}}=N_{\mathrm{tx}}=80$. As can be seen from the figure, in full-duplex mode, MRC/MRT processing is superior to EGC/EGT processing in terms of performance, and its spectral efficiency gradually decreases with the increase in loop interference. When the loop interference is very large, the spectral efficiency of the two linear processing modes is very close. In half duplex mode, the spectral efficiency is not affected by loop interference and, therefore, remains unchanged. Notably, when $N_{\mathrm{rx}}=N_{\mathrm{tx}}=80$, the full-duplex mode can suppress the loop interference power by up to 18 dB, thereby maintaining its performance advantage. When the loop interference intensity is low, the spectral efficiency of full-duplex mode is much higher than that of half-duplex mode, because half-duplex mode needs to be multiplied by 1/2 preprocessing coefficient. When the intensity of loop interference is strong, the existence of loop interference affects the full-duplex processing, and the spectral efficiency of the half-duplex mode is higher than that of the full-duplex mode. Therefore, the above observations also prove that in order to improve the spectral efficiency, different processing modes should be selected in different cases.

Figure 6 shows the trend of the spectral efficiency of half-duplex and full-duplex relays in relation to the number of base station antennas. It can be seen from the figure that when the MRC/MRT and EGC/EGT schemes are adopted, the system spectral efficiency obtained in the full-duplex mode is significantly higher than that in the half-duplex mode. This is because the half-duplex mode can only perform receiving or transmitting at the same time, and the improvement in spectral efficiency performance is relatively slow. Moreover, the EGC/EGT scheme is only slightly lower than the MRC/MRT scheme. It indicates that the adoption of the EGC/EGT scheme in linear processing is feasible. Furthermore, the spectral efficiency of both increases with the increase in the number of base station antennas. Thus, it can be seen that configuring a large number of antennas at the relay end can effectively reduce the impact of loopback interference and improve the system performance in full-duplex mode.

Figure 7 shows the relationship between the total spectral efficiency of the system and the cumulative distribution function in the half-duplex, full-duplex and hybrid relay modes. Assume that $p_{\mathrm{R}}=p_{\mathrm{P}}=p_{\mathrm{S}}=15\mathrm{dB}$, ${\sigma }_{\mathrm{LI}}^2=8.5\mathrm{dB}$ and $N_{\mathrm{rx}}=N_{\mathrm{tx}}=150$. It can be seen from the figure that the performance of the MRC/MRT scheme is superior to that of the EGC/EGT scheme, but the overall trend is not much different. When considering the MRC/MRT scheme or the EGC/EGT scheme separately, under the same cumulative distribution function, the full-duplex mode can provide higher spectral efficiency for the system, and the performance fluctuation of the full-duplex mode is smaller in different situations, showing good stability. Furthermore, the hybrid relay mode established in this chapter can flexibly select the optimal scheme of spectral efficiency for full-duplex and half-duplex modes in different scenarios, thereby providing the best system spectral efficiency performance.

Figure 8 shows the relationship between the bit error rate (BER) of the system and the signal-to-noise ratio after MRC/MRT and EGC/EGT processing in full duplex mode. At this point, the number of characters sent per frame is $L=60$, ${\sigma }_{\mathrm{LI}}^2=8.5\mathrm{dB}$, $p_{\mathrm{P}}=10\mathrm{dB}$, $p_{\mathrm{R}}=p_{\mathrm{S}}=\mathrm{SNR}$ and $N_{\mathrm{rx}}=N_{\mathrm{tx}}=50$, and the relay station adopts 2PSK mode to modulate and demodulate the signal. It can be seen from the figure that with the increase in the signal-to-noise ratio, the bit error rate performance of both the MRC/MRT scheme and the EGC/EGT scheme has improved, and the bit error rate performance of the MRC/MRT scheme is superior to that of the EGC/EGT scheme. The gap in bit error rate between the two gradually widens. Although the bit error rate performance of the EGC/EGT scheme is slightly lower than that of the MRC/MRT scheme, it still provides better spectral efficiency performance within a certain range. When $\mathrm{SNR}=-20\mathrm{dB}$, the bit error rates of the two linear processing methods differ by only about 0.02 dB, which proves that the adoption of the EGC/EGT scheme in large-scale MIMO is feasible.

Figure 9 shows the relationship between the number of antennas of the base station and the bit error rate after MRC/MRT and EGC/EGT processing of the signal sent by the user. At this time, three situations are considered: (1) $p_S=E_S{p}_R=E_R/N_{tx}$; (2) $p_S=E_S/N_{rx}{p}_R=E_R$; (3) $p_S=E_S/N_{rx}{p}_R=E_R/N_{tx}$, in which $E_S$ and $E_R$, respectively, send the energy source node and relay node, assume that $E_S=E_R=13\mathrm{dB}$, $p_P=13\mathrm{dB}$. As can be seen from the figure, the bit error rate performance of MRC/MRT and EGC/EGT processing decreases with the increase in the number of base station antennas, and eventually tends to a fixed value, and the bit error rate performance of case 2 is the worst among the three, because the loop interference at this time has a great impact on the signal receiving end. The bit error rate performance of case 1 and case 3 has little difference when the number of antennas is small. Then, with the increase in the number of antennas, the bit error rate performance of case 3 is gradually better than that of case 1. This is because the use of a large-scale antenna array reduces the influence of loop interference on receiving nodes, thus improving the spectral efficiency performance of the system. Finally, for MRC/MRT and EGC/EGT schemes, the bit error rate performance of MRC/MRT in case 2 is better than that of EGC/EGT schemes, but in case 1 and case 3, the bit error rate performance of the two schemes is basically the same. Therefore, EGC/EGT schemes can be considered to replace MRC/MRT schemes in future work.

7. Conclusions

This paper investigates the performance of large-scale MIMO systems based on full-duplex relay communication in a Rician fading channel environment. The relay station employs the MMSE method to obtain an imperfect CSI channel estimation matrix. In terms of linear processing, the EGC/EGT scheme is applied for the first time and compared with the conventional MRC/MRT scheme. Both exact and approximate closed-form expressions for system spectral efficiency are derived. The simulation results demonstrate that employing a large antenna array can effectively eliminate the impact of loop interference, thereby suppressing inter-pair interference and noise. When the number of relay antennas approaches infinity, the transmission powers of the users and the relay can be reduced to $1/N_{\mathrm{rx}}$ and $1/N_{\mathrm{tx}}$, respectively. Moreover, the spectral efficiency performance of the EGC/EGT scheme is only slightly inferior to that of the traditional MRC/MRT scheme, with a negligible difference overall. For instance, when $\mathrm{SNR}=-20\mathrm{dB}$, the bit error rates of the two linear processing methods differ by only about 0.02 dB. The spectral efficiencies of full-duplex and half-duplex systems are also compared, showing that the full-duplex mode can improve spectral efficiency by a factor of $2N$. Furthermore, a hybrid relay scheme is proposed, which allows for the optimal selection between full-duplex and half-duplex modes under given conditions, thereby significantly enhancing system spectral efficiency. Finally, a power allocation strategy is introduced to further optimize spectral efficiency. Under constraints on total spectral efficiency and peak power, the transmission powers of both users and the relay are optimally allocated, resulting in significant improvements in energy and spectral efficiency. Considering the low complexity, low latency, and simple hardware structure of the EGC/EGT scheme, it is of great practical significance to adopt this scheme as a substitute for the MRC/MRT scheme in large-scale MIMO systems based on full-duplex relay communication.