Capacity Analysis of Lattice Reduction Aided Equalizers for Massive MIMO Systems

Abstract

Massive multi-input-multi-output (MIMO) systems are the future of the communication system. The proper design of the MIMO system needs an appropriate choice of detection algorithms. At the same time, Lattice reduction (LR)-aided equalizers have been well investigated for MIMO systems. Many studies have been carried out over the Korkine–Zolotareff (KZ) and Lenstra–Lenstra–Lovász (LLL) algorithms. This paper presents an analysis of the channel capacity of the massive MIMO system. The mathematical calculations included in this paper correspond to the channel correlation effect on the channel capacity. Besides, the achievable gain over the linear receiver is also highlighted. In this study, all the calculations were further verified through the simulated results. The simulated results show the performance comparison between zero forcing (ZF), minimum mean squared error (MMSE), integer forcing (IF) receivers with log-likelihood ratio (LLR)-ZF, LLR-MMSE, KZ-ZF, and KZ-MMSE. The main objective of this work is to show that, when a lattice reduction algorithm is combined with the convention linear MIMO receiver, it improves the capacity tremendously. The same is proven here, as the KZ-MMSE receiver outperforms its counterparts in a significant margin.

1. Introduction

The continuous demand for high speed and reliable communication systems puts the MIMO systems the most demanding research topic [1]. As a further enhancement of MIMO technology, massive MIMO can increase the capacity in many folded and also enhance the system energy efficiency [2,3]. However, all the benefits come with the burden of challenges. In the MIMO system, the design of low complex signal processing techniques is the main challenge of concern. Therefore, it is critical to address the challenge of developing low complex signal processing techniques for increasing the capacity of the massive MIMO system. In this paper, the authors discuss this critical problem and examine the low complex LR-aided receivers for the channel capacity enhancement.

Several studies state that maximum-likelihood (ML) is the optimal receiver for a MIMO system. Nevertheless, it is practically unimplementable as its complexity increases exponentially with the number of antennas used in a MIMO system [4,5,6]. The linear receivers such as matched filter (MF), zero-forcing (ZF), and minimum mean squared error (MMSE) equalizers gain lots of attention because of their low signal processing complexity [7]. However, these receivers have limitations such as the cost and poor bit error rate (BER) performance [1,8]. This results in the development of nonlinear receivers, such as successive interference cancellation (SIC) [9], decision feedback equalizer (DFE) [10,11,12,13], probabilistic data association (PDA)-based MIMO receiver, and semidefinite programming relaxation (SDPR)-based MIMO detector [14,15,16,17]. These receiver systems can minimize the performance gap in comparison with the ML receiver but with a significant increase in the system complexity. The sphere detector (SD) receiver [18] is introduced, and it can provide near-optimal performance to enhance the performance of the MIMO system. The basic approach of this SD algorithm is concentrated on a tree searching algorithm. More advanced receiver systems such as fixed-complexity SD (FSD) [19] and the K-best detector [20] have been developed, but the system complexity remains very high considering the massive MIMO system [21].

Lattice reduction (LR) [21] is a powerful and efficient algorithm to improve system performance with reduced complexity [22]. The basic approach of the LR-aided receiver system is to convert the lattice basis formed by the channel matrix into a near orthogonal lattice basis. The existence of this short and orthogonal basis leads to a tremendous improvement in the MIMO system performance [1,23]. Extensive research work is ongoing, and different variants of LR algorithms have been proposed, such as Lenstra–Lenstra–Lovász algorithm (LLL) algorithm, Seysen’s algorithm, and Brun’s algorithm [1,24,25]. In the context of the MIMO detection, LLL algorithm is widely used because of its polynomial time complexity [21]. Furthermore, numerous research works are motivated to develop low complexity LR algorithms [20,22,26,27,28]. In [8,22], the complex-valued LLL algorithm is proposed to reduce the size of the channel matrix by half in comparison to that of a real-valued channel matrix. The authors of [8] investigated the improvement of the capacity due to LR-aided receivers. They utilized LLL algorithm-based lattice reduction-aided ZF receiver for their analysis.

Apart from the LLL algorithm, researchers are continuously working on the KZ algorithm and Minkowski reduction algorithm [21,29,30]. The KZ algorithm is more efficient in terms of generating more orthogonal basis vector than the LLL algorithm. Therefore, KZ reduced basis is considered to be the best reduced basis for the decoding purpose in a MIMO system. Consequently, it results in significant improvement in the MIMO system. As in [31,32], the KZ algorithm, and particularly complex KZ algorithm, is able to provide full diversity with reduced complexity.

This document provides an assessment of the effectiveness of the lattice reduction algorithms for enhancing the capacity of the massive MIMO system. The main objective of this paper is to present the capabilities of the KZ aided linear receiver to improve the capacity of a massive MIMO system. In particular, the main contributions of this paper are the following:

• This study provides a mathematical calculation to find the capacity gain of the MMSE receiver over the ZF receiver.
• It highlights the impact of the channel correlation on the capacity.
• An expression to established the capacity gain due to lattice reduction-aided receivers over the linear counterpart is derived.
• The lattice reduction-aided receivers are shown to perform much better than the conventional linear receiver in terms of capacity gain in a massive MIMO system. It shows that the lattice reduction-aided receivers perform much better than LLL-enabled IF receiver.
• The achievable capacity corresponding to LLL- and KZ-based lattice reduction-aided receivers are compared, establishing that KZ-aided receivers perform much better.

The rest of this paper is organized as follows. Section 2 provides a literature survey about related work. Brief descriptions about some receiver systems such as MF-, ZF-, MMSE-, IF-, and LLR-aided receivers are presented in Section 3. After that, we provide an overview of the Nakagami-m channel with its probability distribution function and statistical behavior in Section 4. Section 5 represents the mathematical calculation to show the impact of the channel correlation over the capacity of a MIMO system. In addition, this section presents the capacity improvement due to the use of LLR-aided receivers and provides a comparative study between LLL and KZ lattice reduction. All the analytical results are included in Section 6. Finally, we summarize the paper in Section 7.

2. Related Work

In [1], the authors investigated the performance of Seysen’s Algorithmic (SA)-based LR-aided MMSE receiver. The performance is analyzed in terms of BER corresponding to a large scale MIMO system. The prime focus of this paper is to implement the SA algorithm using Very-large-scale integration (VLASI).

In [21], the authors proposed element based lattice reduction (ELR) combined with K-best detector for the MIMO system. They demonstrated that the proposed system provides better BER performance than the conventional LLL methods with a reduction in complexity. Moreover, it also indicates the hardware architecture of the proposed algorithm.

In [23], the authors proposed a MIMO detection technique with LR aided fixed-complexity tree search. As demonstrated, the intended receiver can achieve near ML performance for a massive MIMO system with higher-order modulation. This paper provides a detailed analysis of the complexity of the system and also BER comparison with 256 QAM modulation.

The authors of [8] provided a detailed mathematical model for calculating the capacity of a MIMO system with complex LLL (CLLL) based LR-aided receivers. Furthermore, the performance is compared, keeping ZF as the basic building block for receiver systems. This paper also demonstrates the performance comparison of CLLL, dual CLLL, and dual ELR receiver systems.

The authors of [32] studied the utility of complex KZ reduction (CKZ) algorithm for the MIMO system and figured out its advantage of reduced complexity with full diversity. The BER performance of the proposed CKZ algorithm along with ZF, MMSE, and SIC receivers are compared with the linear receiver in this paper. The most important outcome of this paper is that the CKZ-aided receiver outperforms the LLL-aided receiver and achieves full diversity.

In [33], the author provided a detailed quantitative analysis of lattice decoding in terms of the proximity factor. They provided a comparative analysis of the primal and dual reduction corresponding to both LLL and KZ aided ZF and SIC receivers. This paper also includes mathematical expression and analytical results corresponding to the upper bounds of the proximity factor.

In [34], the authors pointed out the shortcomings of the KZ and LLL algorithms and proposed boosted KZ and LLL algorithms. They also provided some quantitative analysis of ergodic rate and orthogonal deficiency of conventional LR algorithms along with the boosted ones. The complexity of those algorithms is analyzed as a function of the dimension of the massive MIMO system.

The authors of [35] proposed a hybrid LLL algorithm for the MIMO system and showed that it has fast convergence about the conventional LLL algorithm. However, the proposed algorithm degrades system performance.

In [36], the authors analyzed LR-aided receiver for the realistic MIMO channel and also emphasized the performance complexity tradeoff. They proposed an LR-aided sphere decoding MIMO detector for complexity reduction. Besides, the MIMO system performance is investigated based on BER performance concerning variation antenna configuration, channel correlation coefficient, and channel estimation error. The proposed LR-SD system can provide near-optimal performance. Furthermore, this study provides a detailed analysis of the system complexity.

In [37], the authors proposed an LR-SIC receiver for the massive MIMO system with low-precision ADCs. The prime focus of this paper to select the proper receiver system to make the system more energy efficient.

In [38], the authors investigated the performance of integer forcing (IF) receivers for the MIMO system. Moreover, to design IF the receiver, they used the LLL algorithm and proved that the IF receiver outperforms the MMSE and ZF receivers. Nevertheless, in our paper, we combine LLL and KZ reduction techniques with MMSE receivers and show that LLR-MMSE and KZ-MMSE receivers provide much better performance than the IF receivers. Similarly, in [34], the authors used different LR algorithms to design the IF receiver and studied its performance in terms of ergodic capacities and orthogonal deficiencies. The main focus of this paper is to develop the lattice reduction enabled IF receivers. In this study, we demonstrated the effectiveness of ZF and MMSE algorithms when they were combined with the LLR and KZ algorithms and proved they are better than IF, as presented in [34]. In [39], the authors put emphasis more on the LLL algorithm-based receiver for large scale MIMO systems and proposed sophisticated QR decomposition-based LLL algorithms. In this paper, the authors demonstrated the performance of the proposed LLL algorithms concerning BER performances. Nevertheless, its effect on the channel capacity is not analyzed. The main objective of this paper is to show the benefits of KZ decomposition-based receivers in terms of the capacity improvements along with its comparison with the LLR-aided receivers.

Overall, the works mentioned above deal with the LR-based MIMO detection techniques and their performance analysis is in regard to the system BER performance and the complexity of the system. Therefore, this paper focuses on the impact of the channel correlation over the capacity of a massive MIMO system. The proposed method aims to improve the capacity by using an LR-aided receiver for the massive MIMO system. Moreover, this paper also highlights the utility of KZ-aided receivers over LLL-aided receivers in terms of the improvement in the capacity of the MIMO system.

3. Receivers

This study considers $N_t\times N_r$ MIMO system having $N_t$ transmitting antennas and $N_r$ receiving antennas. The MIMO system equation can be written as

$$Y=Hx+n$$

(1)

where the transmit signal, x, is involved with time varying flat fading wireless channels, H. Here, n is the $N_r\times 1$ additive white Gaussian noise (AWGN) vector and $E\left[x{x}^H\right]=\frac{\rho }{N_t}I$.

3.1. MF

The matched filter (MF) [7] improves the SNR level of the signal but ignores the effect of the interference. The ZF receiver performs better when columns in H have orthogonal elements. The output signal for in case of a MF can be written as,

$${\overline{y}}_{mf}=H^{T}x$$

(2)

3.2. ZF

The complete suppression of the interference has been made possible by ZF receiver [7]. In addition, it is done by taking the pseudo inverse of the channel matrix.

$$H^{†}={\left(H^{T}H\right)}^{-1}{H}^T$$

(3)

The output signal from the ZF receiver is given by,

$${\overline{y}}_{zf}=H^{†}x$$

(4)

The output SNR for of the nth branch for the ZF receiver is given by,

$${\rho }_{zf,n}=\frac{\rho }{{[{\left(H^{H}H\right)}^{-1}]}_{nn}},1\le n\le N_t$$

(5)

The problem in the ZF receiver appears when there are ill-conditioned H matrices and this leads the increase in noise variance after the matrices inversion. Without noise, ZF acts as an optimal receiver similar to ML. However, it produces a noise amplification.

3.3. MMSE

To eliminate the effect of noise amplification due to the ZF receiver, a more robust equalizer, MMSE receiver, has been introduced [7,40,41]. The receive signal from the MMSE receiver is given by:

$${\overline{y}}_{mmse}={(H^{T}H+\frac{N_t}{\rho }I)}^{-1}{H}^{T}x$$

(6)

The output SNR of the nth branch for the MMSE receiver can be represented as

$${\rho }_{mmse,n}=\frac{\rho }{{[{(H^{H}H+\frac{1}{\rho }I)}^{-1}]}_{nn}}-1,1\le n\le N_t$$

(7)

The MMSE receiver takes care of both noise and interference. Furthermore, the MMSE receiver provides an improved solution for the ill-conditioned H matrices, hence provides improved performance in comparison to the ZF receiver. In terms of complexity, the MF receiver is much less complex than both ZF and MMSE detectors.

3.4. IF

As in [38], the received signal from the IF receiver can be presented as

$$\tilde{y}=Ax+\left(BH-A\right)H+Bz$$

(8)

where $Ax$ is the intended signal component and effective noise component is $\left(BH-A\right)H+Bz$. B is the equalization matrix related to the IF receiver. The non-singular integer matrix A is obtained from the LLL decomposition of channel matrix H.

3.5. LR-Aided Receiver

Several LR algorithms have been proposed over the years; of those, the LLL algorithm is the most popular one [20,21]. The main motivation for using LR-aided receiver is to nullify the channel effect by taking advantage of orthogonalization. LR algorithm produces the reduced channel matrix $\tilde{H}=HT$, where T is a unimodular matrix. Therefore, the MIMO system equation can be rewritten as,

$$Y=HT\left(T^{-1}x\right)+n=\tilde{H}z+n$$

(9)

where $z=T^{-1}x$. The reduced channel matrix $\tilde{H}$ is in better condition, and that helps to improve the system performance. As in MIMO $QR$ processing is very commonly used, the LLL algorithm is modified for the Q and R matrices. The LR algorithm is applied in the $QR$ decomposed H to find the $\tilde{Q}$ and $\tilde{R}$. Furthermore, those can be used to obtain the reduced channel matrix, $\tilde{H}=\tilde{Q}\tilde{R}$.

By combining the linear receiver with the LLR algorithm, the system performance can be further improved. Moreover, this approach can reduce the performance gap between the conventional linear receiver and the optimal ML receiver.

4. Nakagami-m Channel

The authors considered a correlated Nakagami-m channel. This channel model was selected because of its inherent features of producing different fading environments. The Rayleigh fading model provides a diversity order of one, whereas the Nakagami-m fading model supports a diversity order of m. In particular, it can describe the LOS and NLOS conditions. The channel characteristic can be represented by its probability distribution function (PDF) [42,43,44,45], which, for the Nakagami-m channel, it is defined as

$$P_{\gamma }\left(\gamma \right)=\frac{2}{\mathsf{\Gamma }\left(m\right)}{(\frac{m}{\overline{\gamma }})}^m{\gamma }^{m-1}exp(\frac{-m\gamma }{\Omega });\gamma \ge 0$$

(10)

where $\gamma$ represent the instantaneous SNR.

Figure 1 and Figure 2 represent the PDF and the cumulative distribution function (CDF) corresponding to the Nakagami-m channel.

5. Mathematical Analysis

5.1. Impact of Channel Correlation on the Capacity

As in [45,46], with ZF, the sub-channel capacity of a MIMO system can be represented as

$$C_{zf}=lo{g}_2[1+\frac{SNR}{{\left(H^{H}H\right)}^{-1}}]$$

(11)

Now, from [45] we have

$$\begin{array}{c}\hfill H^{H}H={(R_{RX}^{1/2}G{R}_{TX}^{H/2})}^H(R_{RX}^{1/2}G{R}_{TX}^{H/2})\\ \hfill =R_{RX}^{H/2}{G}^H{R}_{TX}^{1/2}{R}_{RX}^{1/2}G{R}_{TX}^{H/2}\\ \hfill =R_{RX}G{G}^H{R}_{TX}\end{array}$$

(12)

where the correlation matrices at the Tx and Rx side are presented here by $R_{TX}$ and $R_{RX}$. Here, G is a i.i.d.complex Gaussian vector with zero mean and unit variance.

Therefore $C_{zf}$ in correlated environment can be written as

$$C_{z{f}_{corr}}=lo{g}_2[1+\frac{SNR}{{\left(R_{RX}G{G}^H{R}_{TX}\right)}^{-1}}]$$

(13)

$$\begin{array}{c}\hfill =lo{g}_2[1+SNR\left(R_{RX}G{G}^H{R}_{TX}\right)]\end{array}$$

(14)

Similarly, with MMSE, the sub-channel capacity [45] can be defined as

$$C_{mmse}=lo{g}_2[1+\frac{1}{{(I_{N_r+SNR(H^{H}H)})}^{-1}}]$$

(15)

In high SNR condition,

$${(I_{N_r+SNR(H^{H}H)})}^{-1}={\left(SNR\left(H^{H}H\right)\right)}^{-1}[1-I_{N_r}{\left(SNR\left(H^{H}H\right)\right)}^{-1}+\mathcal{O}\left(SN{R}^{-2}\right)]$$

(16)

Therefore,

$$C_{mmse}=lo{g}_2[\frac{SNR\left(H^{H}H\right)}{1-{(I_{N_r+SNR(H^{H}H)})}^{-1}+\mathcal{O}\left(SN{R}^{-2}\right)}]$$

(17)

Under the correlated channel condition, the channel capacity with MMSE receiver can be represented as,

$$C_{mms{e}_{corr}}=lo{g}_2[\frac{SNR\left(R_{RX}G{G}^H{R}_{TX}\right)}{1-{(I_{N_r+SNR(R_{RX}G{G}^H{R}_{TX})})}^{-1}+\mathcal{O}\left(SN{R}^{-2}\right)}]$$

(18)

Therefore, one can compare the achievable capacity with MMSE and ZF receiver and the capacity difference [45] can be written as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill △C_{corr}=lo{g}_2\left(SNR\right)-lo{g}_2[1-I_{N_r(SNR.R_{RX}G{G}^H{R}_{TX}}{)}^{-1}+\mathcal{O}\left(SN{R}^{-2}\right)]\\ \hfill -lo{g}_2[SNR+{\left(R_{RX}G{G}^H{R}_{TX}\right)}^{-1}]\end{array}\end{array}$$

(19)

5.2. Improvement in Channel Capacity: LR-Aided Receiver

In this section, we present the achievable channel capacity gain due to LR aided MMSE receiver in comparison to the conventional MMSE receiver. The capacity for a MIMO system with MMSE [45,46,47] can be expressed as,

$$C_{mmse}=lo{g}_2[\frac{1}{{(I_{N_r}+SNR{H}^{H}H)}^{-1}}]$$

(20)

Similarly, for a LR-aided MMSE receiver, the achievable capacity can be represented as

$$C_{MMSE}^{LR}=lo{g}_2[\frac{1}{{(I_{N_r}+SNR{\tilde{H}}^H\tilde{H})}^{-1}}]$$

(21)

At high SNR condition, the factor ${(I_{N_r}+SNR{H}^{H}H)}^{-1}$ can be simplified as

$$\begin{array}{c}\hfill {(I_{N_r}+SNR{H}^{H}H)}^{-1}={\left(SNR{H}^{H}H\right)}^{-1}-I_{N_r}{\left(SNR{H}^{H}H\right)}^{-2}+o\left(SN{R}^{-3}\right)\end{array}$$

(22)

$$\begin{array}{c}\hfill ={\left(SNR{H}^{H}H\right)}^{-1}[1-I_{N_r}{\left(SNR{H}^{H}H\right)}^{-1}+o\left(SN{R}^{-2}\right)]\end{array}$$

(23)

Therefore, one can calculate the capacity difference between the LR-MMSE and MMSE as,

$$\begin{array}{c}{C}_{MMSE}^{LR}-C_{mmse}=lo{g}_2(SNR.H^{H}H)-lo{g}_2[1-I_{N_r}{\left(SNR{H}^{H}H\right)}^{-1}+o\left(SN{R}^{-2}\right)]\hfill \\ \hfill -lo{g}_2(SNR.{\tilde{H}}^H\tilde{H})+lo{g}_2[1-I_{N_r}{(SNR{\tilde{H}}^H\tilde{H})}^{-1}+o\left(SN{R}^{-2}\right)]\end{array}$$

(24)

$$\begin{array}{c}\hfill C_{MMSE}^{LR}-C_{mmse}=lo{g}_2(\frac{{\tilde{H}}^H\tilde{H}}{H^{H}H})+lo{g}_2(\frac{1-I_{N_r}{\left(SNR{H}^{H}H\right)}^{-1}+o\left(SN{R}^{-2}\right)}{1-I_{N_r}{(SNR{\tilde{H}}^H\tilde{H})}^{-1}+o\left(SN{R}^{-2}\right)})\end{array}$$

(25)

Equation (25) represents the capacity improvement due to LR-MMSE system.

5.3. Comparison between LLL and KZ Lattice Reduction

Lattice reduction based decoding is a reliable technique and utmost necessary for a large scale MIMO system. It utilizes a lattice structure (signal constellation) for reducing the receiver complexity. The main idea of this decoding is to solve the closest vector problem (CVP) [30]. The standard approach to solve this CVP is to have first lattice reduction and then local search. In addition, for large-scale MIMO systems, it is advantageous to have the basis vectors as short as possible, which is known as the shortest vector problem (SVP) [30].

As in [33,48], a lattice can be defined as

$$L\left(B\right)=\left\{\sum _{i=1}^n{x}_i{b}_i\right\};x\in {\mathbb{Z}}^n$$

(26)

where $B=\left[b_1\dots \dots \dots b_n\right]$ is called the basis of lattice L. Both KZ and LLL algorithms utilize the Gram–Schmidt orthogonalization and one can compute the same as follows

$${\widehat{b}}_i=b_i-\sum _{j=1}^{i=1}{\mu }_{i,j}{\widehat{b}}_j, for i=1,\dots ,n$$

(27)

where ${\mu }_{i,j}=〈b_i,\widehat{b_j}〉/{∥\begin{array}{c}\widehat{b_j}\end{array}∥}^2$. Using matrix notation, Gram–Schmidt orthogonalization can be written as $B=\widehat{B}{\mu }^T$, where $\widehat{B}=\left[{\widehat{b}}_1\dots \dots \dots {\widehat{b}}_n\right]$ and $\mu$ is a lower-triangular matrix with unit diagonal elements.

Now, in the case of KZ reduction, a basis B is considered to be reduced if it maintain the following conditions:

(a)

$b_1$ represents the shortest nonzero vector of lattice L.

(b)

$|{\mu }_{i,j}|\le 1/2$, for $1\le j<i\le n$.

(c)

If $L^{n-1}$ represents the orthogonal projection of L on the orthogonal complement spanned by $b_1$, then the projections $b_i={\mu }_{i,1}{b}_1$ of $b_2,\dots ,b_n$ yield a KZ basis of $L^{n-1}$.

Similarly, a basis is LLL reduced if it satisfies the following conditions;

(a)

$|{\mu }_{i,j}|\le 1/2$, for $1\le j<i\le n$.

(b)

${∥\begin{array}{c}\widehat{b_i}\end{array}∥}^2+{\mu }_{i,i-1}^2{∥\begin{array}{c}\widehat{b_{i-1}}\end{array}∥}^2\ge \delta {∥\begin{array}{c}\widehat{b_{i-1}}\end{array}∥}^2$, for $1<i\le n$.

One can decompose the LLL algorithm as mentioned above; Clause (a) is the size reduction and Clause (b) is the Lovász condition. The value parameter $\delta$ is chosen to have a tradeoff between quality and complexity. Its value can be anything between 1/4 and 1, but the optimal is 3/4.

Particularly, in this section, we compare the most commonly used lattice reduction techniques, namely Lenstra–Lenstra–Lovász (LLL) and Korkine–Zolotarev (KZ) algorithms. From the computational complexity point of view, KZ is much more expensive compared to the LLL algorithm but is much more efficient.

6. Results

This section represents the performance analysis of the massive MIMO system under realistic channel environment. To evaluate the performance, the authors used the MATLAB platform. The correlated Nakagami-m channel was taken to have a realistic environment. In addition, the authors also assumed equal power distribution for the transmitter section and full knowledge of channel state information (CSI) at the receiver side only.

Figure 3 shows the performance comparison between the different receiver systems. The LLL algorithm was used to perform the lattice reduction. The performance was compared considering the MF receiver. As in Figure 3, with the increase in SNR, MMSE receiver approaches the ZF receiver. The same is true for the LLR-aided receiver system. In Figure 3, at an SNR 2dB, the capacities corresponding to MF, MMSE, and LLR-MMSE receivers are 1.994, 3.559, and 4.472 bits/S/Hz, respectively. Now, at relatively high SNR (at 28 dB), the channel capacities for the same receivers are 2.492, 6.52, and 12.17 bits/S/Hz, respectively. Therefore, it is clear that there is a considerable capacity gain due to the lattice reduction-aided receiver system. This is because lattice reduction-aided receivers convert the ill-conditioned channel to a much better channel.

Figure 4 represents the effect of the number of antennas over the channel capacities. The objective of this analysis is to figure out the capacity improvement due to LR-aided receivers.

At a particular SNR level of 24 dB (as in Figure 4), the channel capacities for ZF receivers with $6\times 6$, $12\times 12$, and $30\times 30$ antenna configurations are 8.093, 9.801, and 11.88 bits/S/Hz, respectively. Furthermore, with the same antenna configurations, for LLR-ZF receiver, the channel capacities are 16.19, 19.6, and 23.75 bits/S/Hz, respectively. Through this quantitative analysis, one can figure out that LLR-aided receivers improve the channel capacity tremendously. Similarly, if we compare the MMSE and LLR-MMSE receivers for these antenna configurations at same SNR level, we can visualize that LLR-MMSE is providing capacity increments of 7.608, 10.43, and 19.49 bits/S/Hz, respectively, concerning the MMSE receiver.

Therefore, the lattice reduction-aided receivers effectively enhance the performance of the MIMO system, and, out of all the presented receiver systems, the LLLR-MMSE receiver provides the best solution for capacity enhancement.

The impact of the correlated channel over the capacity is presented in Figure 5. For this particular simulated result, we considered a $30\times 30$ MIMO system with one receiver side correlated Nakagami-m channel (m = 1). Considering a SNR = 24 dB and correlation coefficient $\left({\rho }_{rx}\right)$ = 0.2, the channel capacities for ZF, MMSE, LLR-ZF, and LLR-MMSE are 4.326, 6.7263, 8.652, and 17.83 bits/S/Hz, respectively. Moreover, it establishes the utility of a lattice reduction-aided receiver system.

Figure 6 shows the statistical analysis of the channel capacity for a MIMO system with LLR-MMSE receiver.

Figure 7 presents the statistical analysis of the channel capacity for a MIMO system with the LLR-MMSE receiver with the variation m. There is a significant effect of m over the channel capacity and the higher is the value of m, the higher is the capacity of the MIMO system.

The variation of OD with respect to the dimension of the MIMO system is presented in Figure 8. It can be seen that the lattice reduction algorithms significantly improve the channel condition.

Figure 9 describes the performance of the ZK and LLL reduction algorithm with the change in ${\rho }_{rx}$. The KZ reduction algorithm has a better ability to reduce OD in comparison to the LLL algorithm. In addition, in identifies the effectiveness of the KZ algorithm when the channel correlation is high (${\rho }_{rx}$ = 0.5).

Figure 10 represents the effect of the channel correlation over the capacity. The MMSE outperformed the ZF receiver. Therefore, the authors considered only the MMSE receiver. Particularly, this result deals with the comparison between the conventional MF and MMSE receivers with KZ/LLL lattice reduction-aided receivers for a $10\times 10$ MIMO system. Now, at 24 dB SNR and ${\rho }_{rx}$ = 0, capacities corresponding to MF, MMSE, LLR-MMSE, and KZ-MMSE are 2.219, 10.4, 19.11, and 21.66 bits/S/Hz, respectively. It reveals the impact of the lattice reduction-aided receivers over the channel capacity of a MIMO system. Furthermore, at the same SNR level but with ${\rho }_{rx}$ = 0.2, it is seen in the figure that the KZ receiver performs better than the other receivers.

In the comparison for KZ-MMSE and LLR-MMSE under SNR = 24 dB and ${\rho }_{rx}$ = 0, we observed that KZ-MMSE receiver outperforms the LLR-MMSE receiver by 2.55 bits/S/Hz. However, with the increased channel correlation (${\rho }_{rx}$ = 0.2), the capacity difference between the two receiver system is 9.1840 bits/S/Hz. This proves the effectiveness of the KZ-MMSE receivers over the correlated channel condition.

Figure 11 describes the impact of the dimension of the MIMO system over capacity for different receiver algorithms, and for this particular result the authors consider 10 dB SNR and uncorrelated channel condition. The most critical point that one can observe is that, as the number of antennas increases, the performance of LLR-MMSE and KZ-MMSE tends to converge. However, the lattice reduction-aided receivers present a better performance in comparison to the conventional receiver system.

In this study, the authors compared the running times of ZF, MMSE, LLR-MMSE, and KZ-MMSE, as a part of complexity analysis. To obtain the above result, we took the median of the average runtime for 1000 random matrices. As we considered some random matrices, we observed specific long run times; thus, by taking the median, we smoothened the average time. Figure 12 shows the variation of average run time as a function of the numbers of antennas. The run time is dependent on the system configuration of the simulation platform. We used Lenevo laptop with a 2.4-GHz Intel (R)Core (TM) i3-311M CPU processor, with MATLAB running under Windows XP. As in the figure, with the lattice reduction-aided receiver, the complexity of the system is increased in comparison to the conventional receivers. Nevertheless, MIMO detector with LLR-aided technique is able to produce full diversity with reasonable complexity and that is much lower than the ML receiver.

Figure 13 shows the comparison between the IF receiver [34,38] with the LLR-MMSE and KZ-MMSE receivers. The IF receiver outperforms the ZF and MMSE receivers. However, the LLR-MMSE and KZ-MMSE receivers show better performance. Moreover, KZ-MMSE receiver outperforms the other receiver systems.

The LLR-aided receivers improves the massive MIMO system performance significantly. In this study, we analyzed the performance of the same receiver system with multiple users in a single cell. The authors observed that, under spatially correlated channel condition, LLR-aided receiver systems have significantly improved performance when compared with than the conventional linear receiver system; it is most suitable for the usage in a multi-user environment. Figure 14 shows the impact of the number of users (3/6) and the number of antennas at the receiver side (2/4), over the channel capacity, with 12 antennas at the transmitter side.

As depicted in Figure 3, Figure 4 and Figure 7, the ZF receiver outperforms the MMSE receiver. Then, to summarize all the relevant results, the authors have evaluated the MMSE and LR aided MMSE receivers, as presented in

Table 1. Achievable capacity comparison between different Receivers.

Receiver	Capacity (Bits/S/Hz)
	${\mathit{\rho }}_{\mathit{rx}}$ = 0		${\mathit{\rho }}_{\mathit{rx}}$ = 0.2
System	SNR = 3 dB	SNR = 30 dB	SNR = 3 dB	SNR = 30 dB
MF	2.17	2.22	1.531	1.533
MMSE	3.709	11.66	2.453	7.051
LLR-MMSE	5.519	21.73	2.455	9.425
KZ-MMSE	12.57	23.23	5.264	19.26

and

Table 2. Achievable capacity comparison with the variation in number of antennas.

	Capacity (Bits/S/Hz)
Antenna Configuration	$\mathbf{4}\times \mathbf{4}$	$\mathbf{10}\times \mathbf{10}$	$\mathbf{20}\times \mathbf{20}$
ME	2.189	2.273	2.293
MMSE	5.897	6.779	10.25
LLR-MMSE	10.16	12.45	18.52
KZ-MMSE	15.14	17.42	20.29

.

Numerous research studies that analyze the capacity improvement due to LLR-aided receivers are presented in the literature. However, there is a lack of interest in the analysis of KZ-aided receivers. Therefore, the main contribution of this paper is to present a comprehensive evaluation and analysis of the effectiveness of KZ-aided receiver for massive MIMO systems. As shown in Table 1 and Table 2, KZ aided receiver performs much better than other receiver systems under uncorrelated and correlated channel conditions. Consequently, the results prove the fact that KZ-aided receiver can be considered as a reliable solution for the massive MIMO systems.

7. Conclusions

The growing demand for high data rate systems motivates researchers to explore the sophisticated signal processing tools for massive MIMO systems. However, system complexity is the main limitation. This paper discusses the effectiveness of the lattice reduction-aided receivers for massive MIMO systems. In this paper, we analyze the KZ- and LLR-based lattice reduction algorithms. The performance of the massive MIMO system is analyzed in terms of the improvement in achievable capacity using the lattice reduction-aided receivers. Through the mathematical calculation and simulated results, this study showed that the lattice reduction-aided receivers outperform the conventional linear receivers, to enhance the channel capacity under the correlated channel condition. Furthermore, the KZ reduction-aided receivers is much more efficient in reducing the orthogonal defect under the correlated channel condition in comparison to LLL algorithms. This efficiency results in a robust conditioned channel and significantly impacts the achievable capacity, and, as a consequence, KZ reduction-aided receivers outperforms the LLL reduction-aided receivers. Under correlated channel condition (${\rho }_{rx}$ = 0.2) and high SNR condition (SNR = 30 dB), the KZ-MMSE receiver provides an increment in the capacity of 17.73, 12.209, and 9.835 Bits/S/Hz for MF, MMSE, and LLR-MMSE receivers, respectively. Therefore, this presents KZ reduction-aided receivers as a default choice for a massive MIMO system.

As future work, the authors will analyze the achievable capacity for millimeter-wave massive MIMO systems under the realistic highly dynamic system. This study also has limitations since the authors did not explore the impact of the Doppler shift, and this analysis will be highly beneficial for a high-speed vehicular communication system. As an extension of the presented study, the authors plan to focus on the performance analysis of the lattice reduction-aided receivers under the influence of multi-user interference (MUI), particularly at a cell edge environment.