Enhance the Performance of Expectation Propagation Detection in Spatially Correlated Massive MIMO Channels via DFT Precoding

Abstract

Expectation Propagation (EP) has emerged as a promising detection algorithm for large-scale multiple-input multiple-output (MIMO) systems owing to its excellent performance and practical complexity. However, transmit antenna correlation significantly degrades the performance of EP detection, especially when the number of transmit and receive antennas is equal and high-order modulation is adopted. Based on the fact that the eigenvector matrix of the channel transmit correlation matrix approaches asymptotically to a discrete Fourier transform (DFT) matrix, a DFT precoder is proposed to effectively eliminate transmit antenna correlation. Simulation results demonstrate that for high-order, high-dimensional massive MIMO systems with strong transmit antenna correlation, employing the proposed DFT precoding can significantly accelerate the convergence of the EP algorithm and reduce the detection error rate.

1. Introduction

With the rapid evolution of 5G and the advent of 6G, demands for spectral and energy efficiency have spurred interest in massive multiple-input multiple-output (mMIMO) technology [1,2,3]. In a rich scattering wireless propagation environment, equipping transmitters and receivers with a larger number of antennas will increase the degree of freedom (DoF) of the channel and hence the multiplexing gain. Consequently, the system spectral efficiency can be considerably increased. On the other hand, the signals received by the receiver through multiple antennas are a linear mixture of a large number of transmitted symbols with noise superimposed. This poses significant challenges to signal detection. While optimal maximum-likelihood (ML) detection yields the best performance, its complexity increases exponentially with the number of transmit antennas, making it completely impractical for mMIMO systems. Linear detectors such as Zero Forcing (ZF) and Minimum Mean Square Error (MMSE) offer lower complexity and are easy to implement in practical systems; however, these algorithms exhibit unsatisfactory performance [4]. Nonlinear detection methods, such as Sphere Decoding (SD), Tree Search (TS), and MMSE with Successive Interference Cancellation (MMSE-SIC), can provide better detection accuracy, but their computational complexity grows rapidly with the system dimension, making them impractical for massive MIMO systems.

Recently, artificial intelligence (AI)-based approaches have also been explored for MIMO detection [5,6]. Expectation Propagation (EP) is one such Bayesian-inspired method that has emerged as a promising approximate Bayesian approach [7], which strikes a balance between detection performance and computational complexity in large-scale MIMO detection [8,9]. Regarding the EP algorithm, most existing studies focus on enhancing it by reducing algorithm complexity or improving detection performance [10,11,12,13,14]. Meanwhile, some research efforts have focused on integrating EP with deep learning or neural networks to attain more robust and superior performance [15,16,17,18,19,20].

Although EP has demonstrated excellent performance under ideal independent and identically distributed (i.i.d.) Rayleigh fading channels, massive real-world MIMO systems often experience spatial channel correlation [21,22,23], which poses a significant challenge to EP detection accuracy. There are numerous application scenarios where both the transmitter and receiver are equipped with large-scale antennas with correlation between antennas. Examples include backhaul links [24], millimeter-wave communications [25], V2X communications [26], and communications between UAVs and the ground [27]. With the increase in frequency bands and technological development, future mobile terminals and IoT terminals may also be equipped with large-scale antennas. In addition, for reconfigurable intelligent surface (RIS)-aided communication scenarios, the total equivalent channel from the transmitter to the receiver via the RIS may also exhibit spatial correlation [28]. At the same time, high-dimensional signal detection requires EP or similar Bayesian algorithms, which are quite sensitive to antenna correlation [29].

To address this problem, some existing works have investigated the performance of EP in correlated channels. For instance, [19] and [20] leverage deep learning to optimize EP parameters, while [13] proposes a preconditioned conjugate gradient (pCG) method to replace matrix inversion, thereby improving numerical stability and reducing complexity. Although these approaches demonstrate enhanced robustness under spatial correlation, they generally treat correlation as a secondary evaluation scenario rather than the main design objective.

This limitation becomes even more critical in the context of millimeter-wave (mmWave) massive MIMO—a key enabling technology for 5G and 6G networks. In practical deployments, antennas are packed in close proximity and thus inherently correlated [30], which significantly undermines detection performance. While having full channel state information (CSI) at the transmitter could, in principle, allow techniques such as beamforming to mitigate correlation effects [31], this is difficult to realize in practice: feedback overhead scales prohibitively with the array size, and mmWave channels are highly time-varying, especially in high-mobility scenarios such as communications between cars and base stations.

Motivated by these challenges, in this work, we focus on the effect of transmit antenna correlation and propose a simple precoding scheme to alleviate such correlation and improve EP detection performance in mMIMO systems. Since the eigenvector matrix of the channel transmit correlation matrix asymptotically approaches a discrete Fourier transform (DFT) matrix [22,32], a DFT precoder is effective in eliminating transmit antenna correlation for mMIMO systems. Simulation results show that when the number of antennas at the transmitter and receiver is equal and the transmitted symbols adopt high-order modulation, the proposed DFT precoder can significantly speed up EP convergence and reduce the detection error rate. One particular advantage of the proposed scheme is that it does not require any CSI at the transmitter—neither the instantaneous CSI nor the statistical CSI [33]. To the best of our knowledge, there is no similar method in the existing literature. The proposed precoding can be implemented in the digital domain, as well as in the analog domain [34] or the beam domain [35], since spatial DFT transforms the signal into the beam domain.

Notations: Boldface lowercase letters $\mathbf{x}$ and uppercase letters $\mathbf{X}$ denote column vectors and matrices, respectively. For a matrix X, ${\left[\mathbf{X}\right]}_{i,j}$ denotes the element in the i-th row and j-th column. $\parallel \mathbf{X}\parallel$ and ${\parallel \mathbf{X}\parallel }_{\mathrm{F}}$ denotes the 2-norm and the Frobenius norm, respectively. ${\mathbf{X}}^{\mathrm{T}}$ and ${\mathbf{X}}^{\mathrm{H}}$ denote, respectively, the transpose and the conjugate transpose of $\mathbf{X}$. $\mathrm{diag}\left(\mathbf{X}\right)$ and $\mathrm{diag}\left(\mathbf{x}\right)$, respectively, denote the diagonal extraction operator and the expansion from vector to diagonal matrix, respectively. $\mathrm{sort}\left(\mathbf{x}\right)$ denotes sorting the elements of $\mathbf{x}$ in descending order. In addition, $\mathbb{E}[·]$ denotes the expectation, and $\mathcal{CN}(\mathbf{0},\mathbf{A})$ denotes the circularly symmetric complex Gaussian random vector with covariance matrix $\mathbf{A}$. $\mathcal{CN}\left(\mathbf{x}:\mathit{\mu },\mathbf{A}\right)$ represents the joint probability density function of $\mathbf{x}$, which is circularly symmetric about $\mathit{\mu }$ and follows a complex Gaussian distribution with covariance matrix $\mathbf{A}$. Furthermore, the upright j denotes the square root of minus one ($\mathrm{j}=\sqrt{-1}$), while the italic j denotes an integer. $A\propto B$ stands for A is proportional to B.

2. System Model

The system we are considering is a point-to-point narrowband mMIMO system as shown in Figure 1, where the transmitter and the receiver are equipped with $N_t$ and $N_r$ antennas, respectively. The data symbol vector $\mathbf{x}={(x_1,x_2,\cdots ,x_{N_t})}^{\mathrm{T}}$ is transmitted over $N_t$ transmit antennas. The elements of $\mathbf{x}$ are independently and equally likely drawn from the M-ary quadrature amplitude modulation (M-QAM) constellation $\mathrm{\Theta }$, which is normalized such that $\mathbb{E}\left[\right|x_i{|}^2]=1$ for $i=1,2,\cdots ,N_t$.

We assume that the time duration of $\mathbf{x}$ (i.e., the symbol interval) is much shorter than the channel coherence time, such that the channel does not change during the transmission of $\mathbf{x}$. Then, the received signal can be expressed as

$$\mathbf{y}=\mathbf{H}\mathbf{x}+\mathbf{n}$$

(1)

where $\mathbf{y}={(y_1,y_2,\cdots ,y_{N_r})}^{\mathrm{T}}\in {\mathbb{C}}^{N_r\times 1}$, $\mathbf{H}\in {\mathbb{C}}^{N_r\times N_t}$ is the mMIMO channel coefficient matrix, and $\mathbf{n}\sim \mathcal{CN}(0,{\sigma }_n^2\mathbf{I})$ is the additive white Gaussian noise (AWGN). The signal-to-noise ratio (SNR) of the system can be defined as $\mathrm{SNR}=N_t/{\sigma }_n^2$, which is the ratio of the total transmit power ${\mathbb{E}[\parallel \mathbf{x}\parallel }^2]=N_t$ to the noise power at each antenna.

Similar to most research (e.g., [8]), we also assume that perfect CSI, i.e.,$\mathbf{H}$, is available at the receiver but unknown to the transmitter.

If we ignore the receiver side correlation, the spatial correlation among transmit antennas can be captured as $\mathbf{H}=\mathbf{W}{\mathbf{R}}^{\mathrm{H}/2}$, where $\mathbf{W}$ has i.i.d. entries drawn from $\mathcal{CN}(0,1)$ [36]. The spatial correlation matrix at the transmitter is given by

$$\mathbf{R}={\displaystyle \frac{1}{N_r}}\mathbb{E}\left[{\mathbf{H}}^{\mathrm{H}}\mathbf{H}\right]={\mathbf{R}}^{1/2}{\mathbf{R}}^{\mathrm{H}/2}.$$

(2)

Since the matrix $\mathbf{R}$ defined above is Hermitian, it can be decomposed into

$$\mathbf{R}=\mathbf{UE}{\mathbf{U}}^{\mathrm{H}}$$

(3)

where $\mathbf{U}$ is a unitary matrix and $\mathbf{E}$ is a diagonal matrix whose diagonal elements are the eigenvalues of $\mathbf{R}$.

There exist many models for the correlation matrix $\mathbf{R}$. For example, the geometrical one-ring scattering model [37] assumes no line-of-sight path between the transmitter and receiver, with the departure angle (AoD) of all scatterers relative to the transmitter uniformly distributed around the center AoD $\theta$ and an angular spread of $\Delta$. If the transmit antennas are linearly and uniformly distributed at intervals of half a wavelength, then the $(i,j)$-th elements of $\mathbf{R}$ can be computed as

$${\left[\mathbf{R}\right]}_{i,j}={\displaystyle \frac{1}{2\Delta }}{\int }_{-\Delta }^{\Delta }{\mathrm{e}}^{-\mathrm{j}\pi \left(\alpha +\theta \right)\left(i-j\right)sin\alpha }\mathrm{d}\alpha$$

(4)

The most popular model for $\mathbf{R}$ in the literature is the exponential model [38], where $\mathbf{R}$ is a Toeplitz matrix and its $(i,j)$-th element is given by

$${\left[\mathbf{R}\right]}_{i,j}={\rho }^{|i-j|}$$

(5)

where $\rho \in [0,1)$ denotes the correlation coefficient between adjacent antennas. In the following, we will mainly consider Equation (5) as the transmit antenna correlation model.

3. Expectation Propagation

3.1. Expectation Propagation Algorithm

Expectation Propagation [7] is an approximate Bayesian inference method that seeks a tractable distribution $q\left(\mathbf{x}\right)$ to approximate the posterior probability distribution $p\left(\mathbf{x}\right|\mathbf{y})\propto p(\mathbf{y}\left|\mathbf{x}\right)p\left(\mathbf{x}\right)$. In the context of MIMO signal detection problem, $p\left(\mathbf{y}|\mathbf{x}\right)=\mathcal{CN}\left(\mathbf{y}:\mathbf{Hx},{\sigma }_n^2\mathbf{I}\right)$ is the channel likelihood function, and $q\left(\mathbf{x}\right)$ is chosen as a complex Gaussian distribution $\mathcal{CN}(\mathbf{x}:\mathit{\mu },\mathbf{\Sigma })$ with mean vector $\mathit{\mu }$ and covariance matrix $\mathbf{\Sigma }$. The prior distribution of $\mathbf{x}$, $p\left(\mathbf{x}\right)$, is approximated as a complex Gaussian distribution $\mathcal{CN}\left(\mathbf{x}:\mathrm{diag}{\left(\mathbf{\Lambda }\right)}^{-1}\mathit{\gamma },\mathrm{diag}{\left(\mathbf{\Lambda }\right)}^{-1}\right)$ where $\mathit{\gamma }={({\gamma }_1,{\gamma }_2,\cdots ,{\gamma }_{N_t})}^{\mathrm{T}}\in {\mathbb{C}}^{N_t\times 1}$ and $\mathbf{\Lambda }={({\lambda }_1,{\lambda }_2,\cdots ,{\lambda }_{N_t})}^{\mathrm{T}}\in {\mathbb{C}}^{N_t\times 1}$ are variational parameters updated iteratively.

Let L be the maximum number of iterations. The EP algorithm proceeds as follows:

(1)

Initialization: Set ${\lambda }_i^{\left(1\right)}=1$ and ${\gamma }_i^{\left(1\right)}=0$ for all $i=1,2,\dots ,N_t$. Form vectors ${\mathbf{\Lambda }}^{\left(1\right)}={[{\lambda }_1^{\left(1\right)},{\lambda }_2^{\left(1\right)},\dots ,{\lambda }_{N_t}^{\left(1\right)}]}^T$ and ${\mathit{\gamma }}^{\left(1\right)}={[{\gamma }_1^{\left(1\right)},{\gamma }_2^{\left(1\right)},\dots ,{\gamma }_{N_t}^{\left(1\right)}]}^T$.

(2)

Posterior update: At iteration l, compute

$$\begin{array}{cc}\hfill {\mathbf{\Sigma }}^{\left(l\right)}& ={\left({\sigma }_n^{-2}{\mathbf{H}}^{\mathrm{H}}\mathbf{H}+\mathrm{diag}\left({\mathbf{\Lambda }}^{\left(l\right)}\right)\right)}^{-1},\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\mathit{\mu }}^{\left(l\right)}& ={\mathbf{\Sigma }}^{\left(l\right)}\left({\sigma }_n^{-2}{\mathbf{H}}^{\mathrm{H}}\mathbf{y}+{\mathit{\gamma }}^{\left(l\right)}\right).\hfill \end{array}$$

(7)

(3)

Cavity distribution: For each symbol $x_i$, compute the parameters of cavity distribution $q_{\setminus i}\left(x_i\right)=\mathcal{CN}(x_i:t_i^{\left(l\right)},h_i^{2\left(l\right)})$ as

$$\begin{array}{cc}\hfill h_i^{2\left(l\right)}& ={\displaystyle \frac{{\mathrm{\Sigma }}_{ii}^{\left(l\right)}}{1-{\mathrm{\Sigma }}_{ii}^{\left(l\right)}{\lambda }_i^{\left(l\right)}}},\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill t_i^{\left(l\right)}& =h_i^{2\left(l\right)}\left({\displaystyle \frac{{\mu }_i^{\left(l\right)}}{{\mathrm{\Sigma }}_{ii}^{\left(l\right)}}}-{\gamma }_i^{\left(l\right)}\right).\hfill \end{array}$$

(9)

(4)

Moment matching and parameters update: Define the distribution $\tilde{p}\left(x_i\right)\propto q_{\setminus i}\left(x_i\right)p\left(x_i\right)$, compute its mean and variance:

$$\begin{array}{cc}\hfill m_i^{\left(l\right)}& ={\displaystyle \frac{{\sum }_{x_i\in \mathrm{\Theta }}{x}_{i}exp\left(-\frac{|x_i-t_i^{\left(l\right)}{|}^2}{h_i^{2\left(l\right)}}\right)}{{\sum }_{x_i\in \mathrm{\Theta }}exp\left(-\frac{|x_i-t_i^{\left(l\right)}{|}^2}{h_i^{2\left(l\right)}}\right)}},\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill V_i^{\left(l\right)}& ={\displaystyle \frac{{\sum }_{x_i\in \mathrm{\Theta }}{|x_i-m_i^{\left(l\right)}|}^{2}exp\left(-\frac{|x_i-t_i^{\left(l\right)}{|}^2}{h_i^{2\left(l\right)}}\right)}{{\sum }_{x_i\in \mathrm{\Theta }}exp\left(-\frac{|x_i-t_i^{\left(l\right)}{|}^2}{h_i^{2\left(l\right)}}\right)}},\hfill \end{array}$$

(11)

where $\mathrm{\Theta }$ is the modulation constellation set. Next, update the variational parameters as

$$\begin{array}{cc}\hfill {\lambda }_i^{(l+1)}& ={\displaystyle \frac{1}{V_i^{\left(l\right)}}}-{\displaystyle \frac{1}{h_i^{2\left(l\right)}}},\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\gamma }_i^{(l+1)}& ={\displaystyle \frac{m_i^{\left(l\right)}}{V_i^{\left(l\right)}}}-{\displaystyle \frac{t_i^{\left(l\right)}}{h_i^{2\left(l\right)}}}.\hfill \end{array}$$

(13)

To stabilize convergence, apply damping with factor $\beta \in (0,1]$:

$$\begin{array}{cc}\hfill {\lambda }_i^{(l+1)}& \leftarrow (1-\beta ){\lambda }_i^{\left(l\right)}+\beta {\lambda }_i^{(l+1)},\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill {\gamma }_i^{(l+1)}& \leftarrow (1-\beta ){\gamma }_i^{\left(l\right)}+\beta {\gamma }_i^{(l+1)}.\hfill \end{array}$$

(15)

(5)

Iteration: Repeat steps 2–4 until convergence or the maximum number of iterations L is reached.

(6)

Decision: The final detection output is taken as

$$\widehat{\mathbf{x}}=\mathrm{Hard}\left({\mathit{\mu }}^{\left(L\right)}\right),$$

(16)

where the hard-decision function $\mathrm{Hard}(·)$ maps the elements of ${\mathit{\mu }}^{\left(L\right)}$ to a point in $\mathrm{\Theta }$.

In summary, the EP algorithm constructs a Gaussian approximation to the posterior distribution of the transmitted signal. During the iterative process, the discrepancy between the first- and second-order moments of the symbol-wise posterior distribution and its Gaussian approximation is progressively reduced. For high-dimensional signal detection with high-order modulation, the EP algorithm converges rapidly and achieves significant performance gains over conventional methods [8].

3.2. Transmit Antenna Correlation Degrades the Performance of EP

The performance of the EP algorithm is closely related to the characteristics of the channel matrix $\mathbf{H}$. One of the unfavorable factors in the channel structure is the correlation of transmit antennas [29,39]. We have conducted extensive simulations, and the results show that the correlation between transmit antennas degrades the performance of the EP algorithm. The higher the correlation coefficient, the closer the number of transmit and receive antennas, and the higher the modulation order, the more severe the performance loss. Figure 2 shows an example of the simulation results. Figure 2a shows the symbol error rate (SER) performance of a MIMO system with different numbers of antennas under different channel correlations, where the number of transmit and receive antennas is configured to be the same. As the antenna correlation increases (i.e., larger $\rho$), SER performance degrades significantly. Compared with the uncorrelated i.i.d. Rayleigh channel ($\rho =0$), the performance loss caused by high correlation is quite significant. When $\rho =0.8$, the performance loss at SER = $1\times {10}^{-3}$ is about 10.1 dB, 12.1 dB and 13.1 dB for $N_t=32,64$, and 128, respectively. Figure 2b shows the SER performance of a MIMO system with different numbers of receive antennas, while the number of transmit antennas is fixed at $N_t=64$. As the number of receive antennas increases, the SER performance improves significantly, and the impact of transmit antenna correlation diminishes accordingly. Therefore, the scenario where the impact of transmit antenna correlation most needs to be addressed is when the number of transmit antennas is equal to the number of receive antennas.

The impact of transmit antenna correlation on the performance of EP detection is understandable. Consider a MIMO system where only two symbols $x_1,x_2$ are transmitted over antenna 1 and antenna 2, respectively. The received signal is given by

$$\mathbf{y}={\mathbf{h}}_1{x}_1+{\mathbf{h}}_2{x}_2+\mathbf{n},$$

(17)

where ${\mathbf{h}}_1$ and ${\mathbf{h}}_2$ are the first two columns of $\mathbf{H}$. Given the received signal $\mathbf{y}$, the posterior distribution of $\mathbf{x}={\left[x_1{x}_2\right]}^{\mathrm{T}}$ is

$$p(\mathbf{x}\mid \mathbf{y})\propto exp\left(-{\displaystyle \frac{1}{{\sigma }_n^2}}\parallel \mathbf{y}-{\mathbf{h}}_1{x}_1-{\mathbf{h}}_2{x}_2{\parallel }^2\right).$$

(18)

Using a complete orthonormal basis ${\mathbf{v}}_1,{\mathbf{v}}_2$, ${\mathbf{h}}_1$ and ${\mathbf{h}}_2$ can be decomposed as ${\mathbf{h}}_1=\parallel {\mathbf{h}}_1\parallel {\mathbf{v}}_1$ and ${\mathbf{h}}_2={\alpha }_1{\mathbf{v}}_1+{\alpha }_2{\mathbf{v}}_2$, where ${\mathbf{v}}_2\perp {\mathbf{v}}_1$, $\parallel {\mathbf{v}}_1\parallel =\parallel {\mathbf{v}}_2\parallel =1$. The projection of $\mathbf{y}$ onto the space spanned by $\{{\mathbf{v}}_1,{\mathbf{v}}_2\}$ is $\tilde{\mathbf{y}}={\beta }_1{\mathbf{v}}_1+{\beta }_2{\mathbf{v}}_2$. Since $\tilde{\mathbf{y}}$ is the sufficient statistics for the signal detection, by replacing $\mathbf{y}$ with $\mathbf{y}$ and substituting the orthogonal expansion of ${\mathbf{h}}_{\mathbf{1}}$ and ${\mathbf{x}}_2$ into (18), we have

$$p(x_1,x_2\mid \mathbf{y})\propto exp\left(-\frac{1}{{\sigma }_n^2}|{\beta }_1-\parallel {\mathbf{h}}_1\parallel x_1-{\alpha }_1{x}_2{|}^2\right)exp\left(-\frac{1}{{\sigma }_n^2}|{\beta }_2-{\alpha }_2{x}_2{|}^2\right).$$

(19)

This indicates that $x_1$ and $x_2$ are posteriorly correlated. The EP algorithm tries to solve the posterior mean by iteratively approximating $\mathbf{x}$ as independent Gaussian variables. The posterior correlation shown in (19) will affect the Gaussian approximation in the EP algorithm, thereby influencing the convergence speed and convergence accuracy of the EP algorithm. Specifically, from the principle and procedure of the EP algorithm, it can be seen that the posterior distribution is approximated by a joint Gaussian distribution with mean $\mathit{\mu }$ and covariance $\mathbf{\Sigma }$. However, after updating $\mathit{\mu }$ and $\mathbf{\Sigma }$, the cavity distribution is computed using only the diagonal elements of $\mathbf{\Sigma }$. Subsequently, moment matching and parameter updates are carried out independently for each symbol. As a result, spatial channel correlation induces posterior dependence among symbols, which conflicts with the independence assumption inherent in EP, thereby leading to performance degradation.

Note that the correlation coefficient of $x_1$ and $x_2$ conditioned on a given signal $\mathbf{y}$ is dictated by the magnitude of ${\alpha }_1={\mathbf{v}}_1^{\mathrm{H}}{\mathbf{h}}_2={\parallel {\mathbf{h}}_{\mathbf{1}}\parallel }^{-\mathbf{1}}{\mathbf{h}}_{\mathbf{1}}^{\mathrm{H}}{\mathbf{h}}_{\mathbf{2}}$. With two highly correlated transmit antennas, we have a larger $\rho ={\displaystyle \frac{1}{N_r}}\mathbb{E}\left[{\mathbf{h}}_1^{\mathrm{H}}{\mathbf{h}}_2\right]$, which implies a lower probability of ${\alpha }_1$ being small, a higher chance of $x_1$ and $x_2$ being highly correlated, and hence a higher approximation error in EP.

The impact of antenna correlation can also be observed through the cavity variance, which is defined in (8). A larger cavity variance $h_i^2$ indicates lower symbol reliability, and consequently, a higher detection error probability [8,39].

At the initialization stage, following [10], we set $\mathit{\gamma }=\mathbf{0}$ and ${\mathrm{\Lambda }}_i=\lambda =1$. By decomposing $\mathbf{H}=\mathbf{U}\mathbf{S}{\mathbf{V}}^H$ and combining Equations (6)–(8), the initial cavity variance for symbol $x_i$ is given by

$$h_i^2={\displaystyle \frac{{\sum }_{j=1}^{N_t}(\frac{{\sigma }_n^2}{S_j^2+\lambda {\sigma }_n^2}|v_{ij}{|}^2)}{{\sum }_{j=1}^{N_t}(\frac{S_j^2}{S_j^2+\lambda {\sigma }_n^2}|v_{ij}{|}^2)}},$$

(20)

where $v_{ij}$ is the $(i,j)$-th element of the right singular matrix $\mathbf{V}$, and $S_j$ is the j-th singular value. Equation (20) indicates that the singular values $\mathbf{S}$ and the energy distribution in $\mathbf{V}$ have a great influence on cavity variance $h^2$. Figure 3 shows some statistical results of the cavity variance obtained via Monte Carlo simulations. We can see that the initial $h^2$ for correlated channels is uniformly larger than that in the i.i.d. case. This is especially evident when the transmitter and receiver have the same number of antennas ($N_t=N_r$). If the receiver is equipped with more antennas than the transmitter ($N_r>N_t$), $h^2$ is significantly smaller. This enables us to predict that when the number of receive antennas is greater than that of transmit antennas, the performance of EP improves rapidly, and the impact of antenna correlation on it weakens accordingly, which is consistent with the observations in Figure 2.

4. Eliminating Antenna Correlation Using DFT Precoding

In MIMO systems, precoding/beamforming generally stands for a linear mapping that transforms the original data symbols into a vector to be transmitted over multiple antennas [31,40,41]. Theoretically, the optimal mapping should be channel-specific—the precoder should be optimized with specific $\mathbf{H}$.

In large-scale antenna systems, it is no easy task for the transmitter to obtain sufficiently accurate CSI, which often results in a certain overhead burden. As wavelengths become increasingly shorter, even statistical CSI is not easy to obtain accurately. For instance, in millimeter-wave communications, the sub-centimeter-level movements of surrounding pedestrians and vehicles can alter the propagation environment and affect the channel’s correlation matrix. In this paper, we focus on precoding schemes that do not require instantaneous CSI, nor do they depend on the specific channel correlation matrix. This is possible because in mMIMO systems, the correlation matrices share some common structural features. For example, the correlation coefficient decreases rapidly as the antenna spacing increases, meaning the entire correlation matrix is mainly determined by the correlation between adjacent antennas.

With a precoding matrix (precoder) $\mathbf{P}\in {\mathbb{C}}^{N_t\times N_t}$ applied in the system, the original signal vector $\mathbf{x}$ is first transformed into $\mathbf{Px}$, and then transmitted through $N_t$ transmit antennas. We assume that precoding does not change the average transmit power of the signal, i.e., ${\mathbb{E}[\parallel \mathbf{Px}\parallel }^2]=\mathbb{E}\left[\parallel \mathbf{x}{\parallel }^2\right]$. The received signal becomes

$$\mathbf{y}=\mathbf{H}\mathbf{P}\mathbf{x}+\mathbf{n}=\tilde{\mathbf{H}}\mathbf{x}+\mathbf{n},$$

(21)

where $\tilde{\mathbf{H}}=\mathbf{H}\mathbf{P}$ is the equivalent channel matrix taking into account the effect of precoding.

From the perspective of the EP algorithm, $\tilde{\mathbf{H}}$ is the channel matrix input for executing the algorithm. Similar to Equation (2), the equivalent transmitter-side correlation matrix for this equivalent channel matrix is given by

$${\mathbf{R}}_{\mathrm{eq}}={\displaystyle \frac{1}{N_r}}\mathbb{E}\left[{\mathbf{P}}^{\mathrm{H}}{\mathbf{H}}^{\mathrm{H}}\mathbf{HP}\right]={\mathbf{P}}^{\mathrm{H}}\mathbf{R}\mathbf{P}={\mathbf{P}}^{\mathrm{H}}{\mathbf{UEU}}^{\mathrm{H}}\mathbf{P}.$$

(22)

Now, it is obvious that by choosing the precoding matrix as $\mathbf{P}=\mathbf{U}$, the equivalent correlation matrix becomes ${\mathbf{R}}_{\mathrm{eq}}=\mathbf{E}$, where all the off-diagonal elements are zero, implying that the transmit antenna correlation has been completely eliminated.

For uniform linear arrays, the correlation matrix has a Toeplitz form [22]. By using Szegö’s theorem [32], $\mathbf{R}$ will approach a circulant matrix for sufficiently large $N_t$. It is well known that the eigenvector matrix of a circulant matrix is the unitary DFT matrix. So we have

$$\mathbf{R}\approx \mathbf{F}\tilde{\mathbf{E}}{\mathbf{F}}^{\mathrm{H}},$$

(23)

where $\mathbf{F}$ is the normalized DFT matrix, $\tilde{\mathbf{E}}$ is a diagonal matrix, which is actually $\mathbf{E}$ with its diagonal elements permuted.

Equation (23) is equivalent to ${\mathbf{F}}^{\mathrm{H}}\mathbf{RF}\approx \tilde{\mathbf{E}}$. The approximation error can be measured by comparing the diagonal elements of ${\mathbf{F}}^{\mathrm{H}}\mathbf{RF}$ with those of $\tilde{\mathbf{E}}$. Define

$$e={\displaystyle \frac{{\parallel \mathrm{sort}\left(\mathrm{diag}\left({\mathbf{F}}^{\mathrm{H}}\mathbf{RF}\right)\right)-\mathrm{sort}\left(\mathrm{diag}\left(\mathbf{E}\right)\right)\parallel }^2}{{\parallel \mathbf{E}\parallel }_{\mathrm{F}}^2}}.$$

(24)

It is obvious that the smaller the value of e, the closer ${\mathbf{F}}^{\mathrm{H}}\mathbf{RF}$ is to a diagonal matrix, and the closer $\mathbf{F}$ approaches $\mathbf{U}$ (up to column permutation). Figure 4a is the simulation results of e under different correlation levels. As $N_t$ increases, e decreases, confirming that in the large-scale regime, the DFT matrix is very close to the eigenvector matrix of $\mathbf{R}$.

Another method to evaluate the approximation error is to observe the diagonal property of ${\mathbf{R}}_{\mathrm{eq}}={\mathbf{P}}^{\mathrm{H}}\mathbf{RP}$. If $\mathbf{F}=\mathbf{U}$, then ${\mathbf{R}}_{\mathrm{eq}}=\mathbf{E}$ must be a diagonal matrix. The deviation between $\mathbf{F}$ and $\mathbf{U}$ will result in non-zero off-diagonal elements of ${\mathbf{R}}_{\mathrm{eq}}$. Define diagonal dominance ratio of ${\mathbf{R}}_{\mathrm{eq}}$ as

$$D={\displaystyle \frac{\parallel \mathrm{diag}\left({\mathbf{R}}_{\mathrm{eq}}\right){\parallel }^2}{\parallel {\mathbf{R}}_{\mathrm{eq}}{\parallel }_{\mathrm{F}}^2}}.$$

(25)

The degree to which D approaches 1 represents the degree to which $\mathbf{F}$ approaches $\mathbf{U}$. We can clearly see from Figure 4b that as the number of antennas increases, D will become very close to 1.

A conclusion we can immediately draw from the above analysis is that for massive MIMO systems, if we choose the precoding matrix $\mathbf{P}=\mathbf{F}$, then we will have ${\mathbf{R}}_{\mathrm{eq}}\approx \tilde{\mathbf{E}}$; that is, the adoption of DFT precoding can effectively eliminate the correlation among transmit antennas.

5. Simulation Results

In this section, we use simulations to verify our proposed precoding scheme. We select no precoding and eigenvalue decomposition (EVD)-based precoding (i.e., $\mathbf{P}=\mathbf{U}$) as the baselines. In all simulations, we adopt Equation (5) as the model for the correlation matrix, set $L=5$ as the number of iterations for the EP algorithm, and use $\beta =0.5$ as the damping factor in Equations (14) and (15).

Simulation results for a $64\times 64$ MIMO system with 16-QAM under transmit-side correlation are summarized in Figure 5a. It can be observed that the performance of the proposed DFT precoding matches that of the EVD-based precoding scheme very well. At an SER of $1\times {10}^{-3}$, the performance gains are about 4 dB, 5.2 dB, and 8 dB for transmit correlation coefficients $\rho =0.5$, $\rho =0.7$, and $\rho =0.8$, respectively. Figure 5b examines the proposed precoding scheme under varying antenna configurations. At $\mathrm{SER}={10}^{-3}$, the gains achieved via the proposed DFT precoding for the $32\times 32$, $64\times 64$, and $128\times 128$ antenna configurations are approximately 6.2 dB, 5 dB, and 4 dB, respectively. Figure 5c presents the simulation results for different modulation orders, including quadrature phase shift keying (QPSK), 16-QAM, and 64-QAM. At $\mathrm{SER}={10}^{-3}$, the SNR gains relative to the unprecoded scheme are approximately 5 dB for 16-QAM and 9 dB for 64-QAM, while a slight performance reduction is observed for QPSK. This is because QPSK is less affected by spatial correlation. This result indicates that higher-order modulations derive greater benefits from precoding under correlated channel conditions.

The proposed DFT precoding scheme only modifies the transmitter without altering the EP algorithm at the receiver; thus, it does not affect the receiver’s algorithm complexity or hardware cost. Introducing precoding at the transmitter does introduce a certain degree of computational or hardware complexity if DFT is implemented in the analog domain or via lens [35]. However, due to the availability of fast DFT algorithms, the increase in complexity is limited. If this scheme is applied to a wideband MIMO-OFDM system, the complexity of precoding will be comparable to that of OFDM modulation.

6. Conclusions

As the number of antennas continues to grow and transmit antennas are deployed with increasing density, coupling and correlation between antennas become increasingly unavoidable. On the other hand, the signal dimensionality brought by large-scale antennas makes it difficult for traditional signal detection methods to achieve good performance, leaving Bayesian methods such as EP as the only viable option. However, antenna correlation has a significant impact on the performance of the EP algorithm. This paper proposes a simple CSI-free scheme that can effectively eliminate the correlation between antennas and significantly improve system performance by implementing the DFT transform at the transmitter.

Although DFT precoding can mitigate performance losses caused by antenna correlation, and some qualitative discussions have been provided in this paper, how the structural characteristics of the channel matrix affect the EP algorithm remains an important issue to be addressed.