Sparse-Based Millimeter Wave Channel Estimation With Mutual Coupling Effect

Abstract

The imperfection of antenna array degrades the communication performance in the millimeter wave (mmWave) communication system. In this paper, the problem of channel estimation for the mmWave communication system is investigated, and the unknown mutual coupling (MC) effect between antennas is considered. By exploiting the channel sparsity in the spatial domain with mmWave frequency bands, the problem of channel estimation is converted into that of sparse reconstruction. The MC effect is described by a symmetric Toeplitz matrix, and the sparse-based mmWave system model with MC coefficients is formulated. Then, a two-stage method is proposed by estimating the sparse signals and MC coefficients iteratively. Simulation results show that the proposed method can significantly improve the channel estimation performance in the scenario with unknown MC effect and the estimation performance for both direction of arrival (DOA) and direction of departure (DoD) can be improved by about 8 dB by reducing the MC effect about 4 dB.

1. Introduction

Millimeter wave (mmWave) communication with the frequency bands of 30–300 GHz will be a promising technology in the 5G cellular networks [1,2,3]. The critical challenge is the significant path loss in the mmWave frequency bands, and that large antenna arrays are adopted to provide the beamforming gain and compensate for the path loss [4]. Additionally, to improve the performance of mmWave communication, the channel estimation methods are essential to obtain the associated channel parameters including the direction of arrival (DoA) and the direction of departure (DoD) [5,6,7,8,9]. In [10], a joint DoA/DoD estimation method for Impulse Radio-Ultra Wide Band (IR-UWB) peer-to-peer communications is proposed, where the multi-path scenario is considered.

To exploit the sparse scattering nature of mmWave channels, the sparse-based methods have been proposed to convert the channel estimation problems into the problems of sparse reconstruction. For example, in [11], a joint sparse and low-rank structure is exploited, and a two-stage compressed sensing (CS) method has been proposed for the mmWave channel estimation; the approximate message passing (AMP) method has been extended by the nearest neighbor pattern learning algorithm to improve the attainable channel estimation performance in [12]; a channel estimation algorithm based on the alternating direction method of multipliers has been given in [7]. However, in the practical mmWave communication systems, the imperfections of antenna arrays degrade the performance of channel estimation [13,14,15]. The DoA estimation methods with the unknown mutual coupling (MC) effect have been proposed in [16,17,18]. However, in the present papers, the sparsity of mmWave channel and the MC effect between antennas have not been considered simultaneously.

In this paper, the estimation problem for the mmWave channel is addressed. By exploiting the channel sparsity in the spatial domain, a CS-based method is proposed to convert the problem of channel estimation into that of sparse reconstruction. Additionally, the MC effect between antennas is described by a symmetric Toeplitz matrix, and the sparse-based system model with MC is formulated. Then, a novel two-stage channel estimation method is proposed by estimating the sparse signals and the MC coefficients iteratively. The remainder of this paper is organized as follows. The system model of mmWave communication is elaborated in Section 2. The proposed channel estimation method with unknown MC is presented in Section 3. Section 4 gives the simulation results. Finally, Section 5 concludes the paper.

Notations: Matrices are denoted by capital letters in boldface (e.g., $\mathit{A}$), and vectors are denoted by lowercase letters in boldface (e.g., $\mathit{a}$). ${\mathit{I}}_N$ denotes an $N\times N$ identity matrix. ${\mathit{I}}_N$ denotes an $N\times N$ identity matrix. $\mathcal{E}\left\{·\right\}$ denotes the expectation operation. $\mathcal{CN}\left(\mathit{a},\mathit{B}\right)$ denotes the complex Gaussian distribution with the mean being $\mathit{a}$ and the variance matrix being $\mathit{B}$. ${\parallel ·\parallel }_2$, ⊗, $Tr\left\{·\right\}$, $vec\left\{·\right\}$, ${(·)}^{\ast }$, ${(·)}^{\mathrm{T}}$ and ${(·)}^{\mathrm{H}}$ denote the ${\ell }_2$ norm, the Kronecker product, the trace of a matrix, the vectorization of a matrix, the conjugate, the matrix transpose and the Hermitian transpose, respectively.

2. System Model of mmWave Communcation

2.1. System Model with MC Effect

As shown in Figure 1, the mmWave MIMO communication system has M antennas in the base station (BS) and N antennas in the mobile station (MS). The transmitting and receiving antennas have the same polarization (horizontal polarization or vertical polarization). Additionally, we solve the problem of 1-D problem and not 2-D problem in both TX and RX sides. In this paper, we only use the analog beamforming in BS and MS, and can extend to the hybrid beamforming structure easily.

The beamfroming vector used in the transmitter (BS) is $\mathit{a}\left(t\right)\in {\mathbb{C}}^{M\times 1}$, and the beamforming vector used in the received (MS) is $\mathit{b}\left(t\right)\in {\mathbb{C}}^{N\times 1}$, so in the time instant t, the received signal can be expressed as

$$\begin{array}{c}\hfill r\left(t\right)={\left[{\mathit{C}}_{\mathrm{R}}\mathit{b}\left(t\right)\right]}^{\mathrm{H}}\mathit{H}\left[{\mathit{C}}_{\mathrm{T}}\mathit{a}\left(t\right)\right]s\left(t\right)+n\left(t\right),\end{array}$$

(1)

where $s\left(t\right)$ denotes the tramitted symbol, $\mathit{H}\in {\mathbb{C}}^{N\times M}$ denotes the mmWave channel matrix, and $n\left(t\right)$ denotes the additive white Gaussian noise (AWGN). The MC matrices in the transmitter and received are denoted as ${\mathit{C}}_{\mathrm{T}}$ and ${\mathit{C}}_{\mathrm{R}}$, respectively.

Usually, the MC matrices in the transmitter and receiver are, respectively, described by [19]

$$\begin{array}{c}{\mathit{C}}_{\mathrm{T}}\triangleq \left(Z_{\mathrm{TA}}+Z_{\mathrm{TL}}\right){\left({\mathit{Z}}_{\mathrm{T}}+Z_{\mathrm{TL}}\mathit{I}\right)}^{-1},\hfill \end{array}$$

(2)

$$\begin{array}{c}{\mathit{C}}_{\mathrm{R}}\triangleq \left(Z_{\mathrm{RA}}+Z_{\mathrm{RL}}\right){\left({\mathit{Z}}_{\mathrm{R}}+Z_{\mathrm{RL}}\mathit{I}\right)}^{-1},\hfill \end{array}$$

(3)

where $Z_{\mathrm{TL}}$ and $Z_{\mathrm{TA}}$ denote the terminating load and the antenna impedance in transmitter, and $Z_{\mathrm{RL}}$ and $Z_{\mathrm{RA}}$ denote the terminating load and the antenna impedance in receiver. ${\mathit{Z}}_{\mathrm{R}}$ and ${\mathit{Z}}_{\mathrm{T}}$ denote the mutual impedance matrix in receiver and transmitter, respectively.

The $m_1$-th row and $m_2$-th column of mutual impedance matrix ${\mathit{Z}}_{\mathrm{T}}$ can be expressed as [20,21,22]

$$\begin{array}{c}\hfill Z_{\mathrm{T},m_1,m_2}=\left\{\begin{array}{cc}30(0.5772+ln\left(2\gamma L\right)-g_{\mathrm{C}}\left(2\gamma L\right)+j{g}_{\mathrm{S}}\left(2\gamma L\right)),\hfill & m_1=m_2\hfill \\ 30(g_{\mathrm{R}}(m_1,m_2)+j{g}_{\mathrm{X}}(m_1,m_2)),\hfill & m_1\ne m_2\hfill \end{array}\right.\end{array}$$

(4)

where $\gamma \triangleq 2\pi /\lambda$, and L denotes the length of dipole antennas. $g_{\mathrm{R}}(m_1,m_2)$ and $g_{\mathrm{X}}(m_1,m_2)$ are defined respectively as

$$\begin{array}{cc}{g}_{\mathrm{R}}(m_1,m_2)\triangleq \hfill & sin\left(\gamma L\right)[g_{\mathrm{S}}\left({\nu }_0\right)-g_{\mathrm{S}}\left({\mu }_0\right)+2g_{\mathrm{S}}\left({\mu }_1\right)\hfill \\ & -2g_{\mathrm{S}}\left({\nu }_1\right)]+cos\left(\gamma L\right)[g_{\mathrm{C}}\left({\mu }_0\right)+g_{\mathrm{C}}\left({\nu }_0\right)-2g_{\mathrm{C}}\left({\mu }_1\right)\hfill \\ & -2g_{\mathrm{C}}\left({\nu }_1\right)+2g_{\mathrm{C}}\left(\gamma d(m_1,m_2)\right)]-[2g_{\mathrm{C}}\left({\mu }_1\right)+2g_{\mathrm{C}}\left({\nu }_1\right)\hfill \\ & -4g_{\mathrm{C}}\left(\gamma d(m_1,m_2)\right)],\hfill \end{array}$$

(5)

$$\begin{array}{cc}{g}_{\mathrm{X}}(m_1,m_2)\triangleq \hfill & sin\left(\gamma L\right)[g_{\mathrm{C}}\left({\nu }_0\right)-g_{\mathrm{C}}\left({\mu }_0\right)+2g_{\mathrm{C}}\left({\mu }_1\right)\hfill \\ & -2g_{\mathrm{C}}\left({\nu }_1\right)]+cos\left(\gamma L\right)[-g_{\mathrm{S}}\left({\mu }_0\right)-g_{\mathrm{S}}\left({\nu }_0\right)+2g_{\mathrm{S}}\left({\mu }_1\right)\hfill \\ & +2g_{\mathrm{S}}\left({\nu }_1\right)-2g_{\mathrm{S}}\left(\gamma d(m_1,m_2)\right)]+[2g_S\left({\mu }_1\right)+2g_{\mathrm{S}}\left({\nu }_1\right)\hfill \\ & -4g_{\mathrm{S}}\left(\gamma d(m_1,m_2)\right)],\hfill \end{array}$$

(6)

where $d(m_1,m_2)$ denotes the distance between the $m_1$-th antenna and the $m_2$-th antenna. ${\mu }_0$, ${\nu }_0$, ${\mu }_1$ and ${\nu }_1$ are defined, respectively, as

$$\begin{array}{c}{\mu }_0=\gamma \left(\sqrt{d^2(m_1,m_2)+L^2}-L\right),\hfill \end{array}$$

(7)

$$\begin{array}{c}{\nu }_0=\gamma \left(\sqrt{d^2(m_1,m_2)+L^2}+L\right),\hfill \end{array}$$

(8)

$$\begin{array}{c}{\mu }_1=\gamma \left(\sqrt{d^2(m_1,m_2)+0.25L^2}-0.5L\right),\hfill \end{array}$$

(9)

$$\begin{array}{c}{\nu }_1=\gamma \left(\sqrt{d^2(m_1,m_2)+0.25L^2}+0.5L\right).\hfill \end{array}$$

(10)

$g_{\mathrm{C}}\left(x\right)$ and $g_{\mathrm{S}}\left(x\right)$ are defined respectively as

$$\begin{array}{c}{g}_{\mathrm{C}}\left(x\right)\triangleq {\int }_{-\infty }^x\frac{cos\left(t\right)}{t}dt,\hfill \end{array}$$

(11)

$$\begin{array}{c}{g}_{\mathrm{S}}\left(x\right)\triangleq {\int }_0^x\frac{sin\left(t\right)}{t}dt.\hfill \end{array}$$

(12)

Similarly, the mutual impedance matrix ${\mathit{Z}}_{\mathrm{R}}$ can be also obtained from the expression of ${\mathit{Z}}_{\mathrm{T}}$.

However, the expresses for ${\mathit{Z}}_{\mathrm{T}}$ and ${\mathit{Z}}_{\mathrm{R}}$ in (4) are too complex to analysis. Since ${\mathit{Z}}_{\mathrm{T}}$ and ${\mathit{Z}}_{\mathrm{R}}$ depend on the length of dipole antennas and the distances between antennas, the MC matrices ${\mathit{C}}_{\mathrm{T}}$ and ${\mathit{C}}_{\mathrm{R}}$ can be approximated, respectively, by two symmetric Toeplitz matrices [23,24,25].

$$\begin{array}{cc}\hfill {\mathit{C}}_{\mathrm{T}}& \approx T\left({\mathit{c}}_{\mathrm{T}}\right),\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill {\mathit{C}}_{\mathrm{R}}& \approx T\left({\mathit{c}}_{\mathrm{R}}\right),\hfill \end{array}$$

(14)

where $T\left({\mathit{c}}_{\mathrm{T}}\right)\in {\mathbb{C}}^{M\times M}$ is defined as

$$\begin{array}{c}\hfill T\left({\mathit{c}}_{\mathrm{T}}\right)\triangleq \left[\begin{array}{ccccc}{c}_{\mathrm{T},0}& c_{\mathrm{T},1}& c_{\mathrm{T},2}& \dots & c_{\mathrm{T},M-1}\\ c_{\mathrm{T},1}& c_{\mathrm{T},0}& c_{\mathrm{T},1}& \dots & c_{\mathrm{T},M-2}\\ c_{\mathrm{T},2}& c_{\mathrm{T},1}& c_{\mathrm{T},0}& \dots & c_{\mathrm{T},M-3}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ c_{\mathrm{T},M-1}& c_{\mathrm{T},M-2}& c_{\mathrm{T},M-3}& \dots & c_{\mathrm{T},0}\end{array}\right],\end{array}$$

(15)

and $T\left({\mathit{c}}_{\mathrm{R}}\right)\in {\mathbb{C}}^{N\times N}$ is defined similarly. Additionally, for the MC matrices, we also have

$$\begin{array}{c}1=|c_{\mathrm{T},0}|\ge |c_{\mathrm{T},1}|\ge \dots \ge |c_{\mathrm{T},M-1}|,\hfill \end{array}$$

(16)

$$\begin{array}{c}1=|c_{\mathrm{R},0}|\ge |c_{\mathrm{R},1}|\ge \dots \ge |c_{\mathrm{R},N-1}|.\hfill \end{array}$$

(17)

Therefore, in the scenario with MC between antennas, the received signal in (1) can be rewritten as

$$\begin{array}{c}\hfill r\left(t\right)={\mathit{b}}^{\mathrm{H}}{T}^{\mathrm{H}}\left({\mathit{c}}_{\mathrm{R}}\right)\mathit{H}T\left({\mathit{c}}_{\mathrm{T}}\right)\mathit{a}s\left(t\right)+n\left(t\right).\end{array}$$

(18)

Usually, the mmWave channel can be described by a geometric channel model

$$\begin{array}{c}\hfill \mathit{H}={\displaystyle \sum _{k=0}^{K-1}}{a}_k\mathit{c}\left({\varphi }_k\right){\mathit{d}}^{\mathrm{H}}\left({\psi }_k\right),\end{array}$$

(19)

where K denotes the number of paths, $a_k$ denotes the complex gain of the k-th path, ${\varphi }_k$ and ${\psi }_k$ are the DoD and DoA, respectively. We define the following vectors to collect DoD/DoA

$$\begin{array}{cc}\hfill \mathit{\varphi }& \triangleq {\left[\begin{array}{c}{\varphi }_0,{\varphi }_1,\dots ,{\varphi }_{K-1}\end{array}\right]}^{\mathrm{T}},\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill \mathit{\psi }& \triangleq {\left[\begin{array}{c}{\psi }_0,{\psi }_1,\dots ,{\psi }_{K-1}\end{array}\right]}^{\mathrm{T}}.\hfill \end{array}$$

(21)

$\mathit{c}\left({\varphi }_k\right)$ and ${\mathit{d}}^{\mathrm{H}}\left({\psi }_k\right)$ are the steering vectors of receiver and transmitter, and can be expressed as

$$\begin{array}{cc}\hfill \mathit{c}\left({\varphi }_k\right)& =\frac{1}{\sqrt{N}}{\left[\begin{array}{c}1,e^{j2\pi \frac{d}{\lambda }sin\left({\varphi }_k\right)},\dots ,e^{j2\pi \frac{(N-1)d}{\lambda }sin\left({\varphi }_k\right)}\end{array}\right]}^{\mathrm{T}}\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill \mathit{d}\left({\psi }_k\right)& =\frac{1}{\sqrt{M}}{\left[\begin{array}{c}1,e^{j2\pi \frac{d}{\lambda }sin\left({\psi }_k\right)},\dots ,e^{j2\pi \frac{(M-1)d}{\lambda }sin\left({\psi }_k\right)}\end{array}\right]}^{\mathrm{T}},\hfill \end{array}$$

(23)

where d denotes the distance between adjacent antennas, and $\lambda$ denotes the wavelength.

We define the following matrices

$$\begin{array}{cc}\hfill \mathit{C}& \triangleq \left[\begin{array}{c}\mathit{c}\left({\varphi }_0\right),\mathit{c}\left({\varphi }_1\right),\dots ,\mathit{c}\left({\varphi }_{K-1}\right)\end{array}\right]\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill \mathit{D}& \triangleq \left[\begin{array}{c}\mathit{d}\left({\psi }_0\right),\mathit{d}\left({\psi }_1\right),\dots ,\mathit{d}\left({\psi }_{K-1}\right)\end{array}\right],\hfill \end{array}$$

(25)

and the channel model can be rewritten as

$$\begin{array}{c}\hfill \mathit{H}=\mathit{C}\mathit{G}{\mathit{D}}^{\mathrm{H}},\end{array}$$

(26)

where $\mathit{G}\in {\mathbb{C}}^{K\times K}$ is a diagonal matrix and $\mathit{G}\triangleq diag\{a_0,a_1,\dots ,a_{K-1}\}$. Substituting (26) into (18), we can obtain

$$\begin{array}{cc}\hfill r\left(t\right)& ={\mathit{b}}^{\mathrm{H}}\left(t\right)T^{\mathrm{H}}\left({\mathit{c}}_{\mathrm{R}}\right)\mathit{C}\mathit{G}{\mathit{D}}^{\mathrm{H}}T\left({\mathit{c}}_{\mathrm{T}}\right)\mathit{a}\left(t\right)s\left(t\right)+n\left(t\right)\hfill \\ & =\left[{\left({\mathit{D}}^{\mathrm{H}}T\left({\mathit{c}}_{\mathrm{T}}\right)\mathit{a}\left(t\right)\right)}^{\mathrm{T}}\otimes \left({\mathit{b}}^{\mathrm{H}}\left(t\right)T^{\mathrm{H}}\left({\mathit{c}}_{\mathrm{R}}\right)\mathit{C}\right)\right]\mathit{g}s\left(t\right)+n\left(t\right)\hfill \\ & =\left[{\mathit{a}}^{\mathrm{T}}\left(t\right)\otimes {\mathit{b}}^{\mathrm{H}}\left(t\right)\right]\left[T^{\mathrm{T}}\left({\mathit{c}}_{\mathrm{T}}\right){\mathit{D}}^{\ast }\otimes T^{\mathrm{H}}\left({\mathit{c}}_{\mathrm{R}}\right)\mathit{C}\right]\mathit{g}s\left(t\right)+n\left(t\right),\hfill \end{array}$$

(27)

where the vector $\mathit{g}\triangleq vec\left\{\mathit{G}\right\}$.

With the sampling interval $T_{\mathrm{s}}$, we can collect the P sampled signals into a vector

$$\begin{array}{c}\hfill \mathit{r}\triangleq {\left[\begin{array}{c}r\left(0\right),r\left(T_{\mathrm{s}}\right),\dots ,r\left((P-1)T_{\mathrm{s}}\right)\end{array}\right]}^{\mathrm{T}}.\end{array}$$

(28)

Then, with $s\left(t\right)=1$, we can obtain

$$\begin{array}{cc}\hfill \mathit{r}& =\underset{\mathbf{\Psi }}{\underbrace{\left[\begin{array}{c}{\mathit{a}}^{\mathrm{T}}\left(0\right)\otimes {\mathit{b}}^{\mathrm{H}}\left(0\right)\\ {\mathit{a}}^{\mathrm{T}}\left(T_{\mathrm{s}}\right)\otimes {\mathit{b}}^{\mathrm{H}}\left(T_{\mathrm{s}}\right)\\ ⋮\\ {\mathit{a}}^{\mathrm{T}}\left((P-1)T_{\mathrm{s}}\right)\otimes {\mathit{b}}^{\mathrm{H}}\left((P-1)T_{\mathrm{s}}\right)\end{array}\right]}}\left[T^{\mathrm{T}}\left({\mathit{c}}_{\mathrm{T}}\right){\mathit{D}}^{\ast }\otimes T^{\mathrm{H}}\left({\mathit{c}}_{\mathrm{R}}\right)\mathit{C}\right]\mathit{g}+\mathit{n}\hfill \\ & =\mathbf{\Psi }\left[T^{\mathrm{T}}\left({\mathit{c}}_{\mathrm{T}}\right)\otimes T^{\mathrm{H}}\left({\mathit{c}}_{\mathrm{R}}\right)\right]({\mathit{D}}^{\ast }\otimes \mathit{C})\mathit{g}+\mathit{n}\hfill \end{array}$$

(29)

where $\mathit{n}\triangleq {\left[\begin{array}{c}n\left(0\right),n\left(T_{\mathrm{s}}\right),\dots ,n\left((P-1)T_{\mathrm{s}}\right)\end{array}\right]}^{\mathrm{T}}$.

2.2. Sparse-Based mmWave Channel Model

To estimate the mmWave channel $\mathit{H}$, we can discretize the DoD and DoA into grids, and the channel model in (26) can be rewritten as

$$\begin{array}{c}\hfill \mathit{H}={\mathit{EUF}}^{\mathrm{H}},\end{array}$$

(30)

where we have

$$\begin{array}{cc}\hfill \mathit{E}& \triangleq \left[\begin{array}{c}\mathit{c}\left({\zeta }_0\right),\mathit{c}\left({\zeta }_1\right),\dots ,\mathit{c}\left({\zeta }_{N_{\mathrm{r}}-1}\right)\end{array}\right]\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill \mathit{F}& \triangleq \left[\begin{array}{c}\mathit{d}\left({\xi }_0\right),\mathit{d}\left({\xi }_1\right),\dots ,\mathit{d}\left({\xi }_{N_{\mathrm{t}}-1}\right)\end{array}\right].\hfill \end{array}$$

(32)

$N_{\mathrm{r}}$ and $N_{\mathrm{t}}$ are the numbers of DoD and DoA grids, respectively. ${\zeta }_n$ and ${\xi }_n$ are the n-th discretized grids of DoD and DoA, respectively. $\mathit{U}$ is a sparse matrix, and entry at the $n_1$-th row and $n_2$-th column of $\mathit{U}$ is

$$\begin{array}{c}\hfill U_{n_1,n_2}=\left\{\begin{array}{cc}{a}_k,\hfill & {\zeta }_{n_1}={\varphi }_k\mathrm{and}{\xi }_{n_2}={\psi }_k\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right..\end{array}$$

(33)

Therefore, the received signal in (29) can be rewritten in a sparse model as

$$\begin{array}{c}\hfill \mathit{r}=\mathbf{\Psi }\left[T^{\mathrm{T}}\left({\mathit{c}}_{\mathrm{T}}\right)\otimes T^{\mathrm{H}}\left({\mathit{c}}_{\mathrm{R}}\right)\right]({\mathit{F}}^{\ast }\otimes \mathit{E})\mathit{u}+\mathit{n},\end{array}$$

(34)

where $\mathit{u}\triangleq vec\left\{\mathit{U}\right\}$ is a sparse vector. As shown in (34), the sparse model is different from the conventional compressed sensing model, where the additional matrix $\left[T^{\mathrm{T}}\left({\mathit{c}}_{\mathrm{T}}\right)\otimes T^{\mathrm{H}}\left({\mathit{c}}_{\mathrm{R}}\right)\right]$ is introduced to describe the unknown MC effect between antennas.

3. Sparse-Based Channel Estimation With Unknown MC Effect

With the sparse model (34), we propose a two-stage method to estimate the mmWave Channel with the unknown MC between antennas. In the two-stage method, the sparse vector can be estimated firstly, and then the MC matrix $\left[T^{\mathrm{T}}\left({\mathit{c}}_{\mathrm{T}}\right)\otimes T^{\mathrm{H}}\left({\mathit{c}}_{\mathrm{R}}\right)\right]$ is estimated with the estimated $\widehat{\mathit{u}}$. In the sparse reconstruction processes, the orthogonal matching pursuit (OMP) method [26,27] can be adopted.

To estimate the MC vectors ${\mathit{c}}_{\mathrm{T}}$ and ${\mathit{c}}_{\mathrm{R}}$, we can rewrite the system model in (34) as

$$\begin{array}{cc}\hfill \mathit{r}& =\mathbf{\Psi }\left({\mathit{Q}}_{\mathrm{F}}^{\ast }\otimes {\mathit{Q}}_{\mathrm{E}}\right)\underset{\mathit{\vartheta }}{\underbrace{\left({\mathit{I}}_{N_{\mathrm{t}}}\otimes {\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\otimes {\mathit{c}}_{\mathrm{R}}^{\ast }\right)\mathit{u}}}+\mathit{n}\hfill \end{array}$$

(35)

where we define

$$\begin{array}{cc}\hfill {\mathit{Q}}_{\mathrm{F}}& \triangleq \left[\begin{array}{c}{\mathit{Q}}_{\mathrm{F}}\left({\xi }_0\right),{\mathit{Q}}_{\mathrm{F}}\left({\xi }_1\right),\dots ,{\mathit{Q}}_{\mathrm{F}}\left({\xi }_{N_{\mathrm{t}}-1}\right)\end{array}\right],\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill {\mathit{Q}}_{\mathrm{E}}& \triangleq \left[\begin{array}{c}{\mathit{Q}}_{\mathrm{E}}\left({\zeta }_0\right),{\mathit{Q}}_{\mathrm{E}}\left({\zeta }_1\right),\dots ,{\mathit{Q}}_{\mathrm{E}}\left({\zeta }_{N_{\mathrm{r}}-1}\right)\end{array}\right].\hfill \end{array}$$

(37)

${\mathit{Q}}_{\mathrm{F}}\left({\xi }_n\right)$ is a matrix and the entries are from the vector $\mathit{d}\left({\xi }_n\right)$, and ${\mathit{Q}}_{\mathrm{E}}\left({\zeta }_n\right)$ is a matrix and the entries are from the vector $\mathit{c}\left({\zeta }_n\right)$. Both ${\mathit{Q}}_{\mathrm{F}}\left({\xi }_n\right)$ and ${\mathit{Q}}_{\mathrm{E}}\left({\zeta }_n\right)$ can be obtained from the following lemma.

Lemma 1.

For complex symmetric Toeplitz matrix$\mathit{A}=Toeplitz\left\{\mathit{a}\right\}\in {\mathbb{C}}^{M\times M}$and complex vector$\mathit{c}\in {\mathbb{C}}^{M\times 1}$, we have [28,29]

$$\mathit{Ac}=\mathit{Qa},$$

(38)

where$\mathit{a}$is a vector formed by the first row of$\mathit{A}$, and$\mathit{Q}={\mathit{Q}}_1+{\mathit{Q}}_{\mathit{2}}$with the p-th ($p=0,1,\dots ,M-1$) row and q-th ($q=0,1,\dots ,M-1$) column entries being

$$\begin{array}{cc}\hfill {\left[{\mathit{Q}}_1\right]}_{p,q}& =\left\{\begin{array}{cc}{c}_{p+q},\hfill & p+q\le M-1\hfill \\ 0,\hfill & \mathit{otherwise}\hfill \end{array}\right.,\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill [{\mathit{Q}}_2{]}_{p,q}& =\left\{\begin{array}{cc}{c}_{p-q},\hfill & p\ge q\ge 1\hfill \\ 0,\hfill & \mathit{otherwise}\hfill \end{array}\right..\hfill \end{array}$$

(40)

We can obtain the following equations

$$\begin{array}{cc}\hfill \mathit{\vartheta }& =\left({\mathit{I}}_{N_{\mathrm{t}}}\otimes {\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\otimes {\mathit{c}}_{\mathrm{R}}^{\ast }\right)\mathit{u}\hfill \\ & =vec\left\{{\mathit{I}}_N{\mathit{c}}_{\mathrm{R}}^{\ast }{\mathit{u}}^{\mathrm{T}}\left({\mathit{I}}_{N_{\mathrm{t}}}\otimes {\mathit{c}}_{\mathrm{T}}^{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right)\right\}\hfill \\ & =\left[\left({\mathit{I}}_{N_{\mathrm{t}}}\otimes {\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right)\mathit{u}\otimes {\mathit{I}}_N\right]{\mathit{c}}_{\mathrm{R}}^{\ast }\hfill \\ & =vec\left\{{\mathit{c}}_{\mathrm{R}}^{\ast }{\mathit{u}}^{\mathrm{T}}\left({\mathit{I}}_{N_{\mathrm{t}}}\otimes {\mathit{c}}_{\mathrm{T}}^{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right){\mathit{I}}_{M{N}_{\mathrm{t}}{N}_{\mathrm{r}}}\right\}\hfill \\ & =({\mathit{I}}_{M{N}_{\mathrm{t}}{N}_{\mathrm{r}}}\otimes {\mathit{c}}_{\mathrm{R}}^{\ast })\left({\mathit{I}}_{N_{\mathrm{t}}}\otimes {\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right)\mathit{u}\hfill \\ & =({\mathit{I}}_{M{N}_{\mathrm{t}}{N}_{\mathrm{r}}}\otimes {\mathit{c}}_{\mathrm{R}}^{\ast })vec\left\{{\mathit{I}}_{M{N}_{\mathrm{r}}}({\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}})\mathit{U}\right\}\hfill \\ & =({\mathit{I}}_{M{N}_{\mathrm{t}}{N}_{\mathrm{r}}}\otimes {\mathit{c}}_{\mathrm{R}}^{\ast })({\mathit{U}}^{\mathrm{T}}\otimes {\mathit{I}}_{M{N}_{\mathrm{r}}})vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}\hfill \\ & =({\mathit{I}}_{M{N}_{\mathrm{t}}{N}_{\mathrm{r}}}\otimes {\mathit{c}}_{\mathrm{R}}^{\ast })vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes \mathit{U}\right\}\hfill \\ & =vec\left\{{\mathit{c}}_{\mathrm{R}}^{\ast }{vec}^{\mathrm{T}}\left\{{\mathit{c}}_{\mathrm{T}}\otimes \mathit{U}\right\}\right\}\hfill \end{array}$$

(41)

$$\begin{array}{c}=\left(vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes \mathit{U}\right\}\begin{array}{c}\hfill \otimes {\mathit{I}}_N{\mathit{c}}_{\mathrm{R}}^{\ast }.\end{array}\right)\hfill \end{array}$$

(42)

Therefore, with (41), the system model in (35) can be rewritten as

$$\begin{array}{cc}\hfill \mathit{r}=& \underset{{\mathbf{\Xi }}_{\mathrm{T}}}{\underbrace{\mathbf{\Psi }\left({\mathit{Q}}_{\mathrm{F}}^{\ast }\otimes {\mathit{Q}}_{\mathrm{E}}\right)({\mathit{I}}_{M{N}_{\mathrm{t}}{N}_{\mathrm{r}}}\otimes {\mathit{c}}_{\mathrm{R}}^{\ast })({\mathit{U}}^{\mathrm{T}}\otimes {\mathit{I}}_{M{N}_{\mathrm{r}}})}}vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right)+\mathit{n},\hfill \end{array}$$

(43)

and with (42), the system model in (35) can be rewritten as

$$\begin{array}{cc}\hfill \mathit{r}=& \underset{{\mathbf{\Xi }}_{\mathrm{R}}}{\underbrace{\mathbf{\Psi }\left({\mathit{Q}}_{\mathrm{F}}^{\ast }\otimes {\mathit{Q}}_{\mathrm{E}}\right)(vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes \mathit{U}\right.\otimes {\mathit{I}}_{\mathrm{N}})}}{\mathit{c}}_{\mathrm{R}}^{\ast }+\mathit{n},\hfill \end{array}$$

(44)

We will use (43) and (44) to estimate the MC vectors c_T and c_R, respectively.

The steepest descent-based method is proposed to estimate the MC vectors. For c_T, we define the following objective function from (43)

$$\begin{array}{c}\hfill f_{\mathrm{T}}\left({\mathit{c}}_{\mathrm{T}}\right)\triangleq {∥\mathit{r}-{\mathbf{\Xi }}_{\mathrm{T}}vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}∥}_2^2.\end{array}$$

(45)

Then, we can obtain

$$\begin{array}{cc}\hfill \frac{\partial f_{\mathrm{T}}\left({\mathit{c}}_{\mathrm{T}}\right)}{\partial {\mathit{c}}_{\mathrm{T}}^{\ast }}=& -\frac{\partial {\mathit{r}}^{\mathrm{H}}{\mathbf{\Xi }}_{\mathrm{T}}vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}}{\partial {\mathit{c}}_{\mathrm{T}}^{\ast }}-\frac{\partial {vec}^{\mathrm{H}}\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}{\mathbf{\Xi }}_{\mathrm{T}}^{\mathrm{H}}\mathit{r}}{\partial {\mathit{c}}_{\mathrm{T}}^{\ast }}\hfill \\ & +\frac{\partial {vec}^{\mathrm{H}}\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}{\mathbf{\Xi }}_{\mathrm{T}}^{\mathrm{H}}{\mathbf{\Xi }}_{\mathrm{T}}vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}}{\partial {\mathit{c}}_{\mathrm{T}}^{\ast }}\hfill \\ \hfill =& -{\mathit{r}}^{\mathrm{H}}{\mathbf{\Xi }}_{\mathrm{T}}\frac{\partial vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}}{\partial {\mathit{c}}_{\mathrm{T}}^{\ast }}-{\left({\mathbf{\Xi }}_{\mathrm{T}}^{\mathrm{H}}\mathit{r}\right)}^{\mathrm{T}}\frac{\partial vec\left\{{\mathit{c}}_{\mathrm{T}}^{\ast }\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}}{\partial {\mathit{c}}_{\mathrm{T}}^{\ast }}\hfill \\ & +{({\mathbf{\Xi }}_{\mathrm{T}}^{\mathrm{H}}{\mathbf{\Xi }}_{\mathrm{T}}vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\})}^{\mathrm{T}}\frac{\partial vec\left\{{\mathit{c}}_{\mathrm{T}}^{\ast }\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}}{\partial {\mathit{c}}_{\mathrm{T}}^{\ast }}\hfill \\ & +{vec}^{\mathrm{H}}\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}{\mathbf{\Xi }}_{\mathrm{T}}^{\mathrm{H}}{\mathbf{\Xi }}_{\mathrm{T}}\frac{\partial vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}}{\partial {\mathit{c}}_{\mathrm{T}}^{\ast }}\hfill \\ \hfill =& {({\mathbf{\Xi }}_{\mathrm{T}}^{\mathrm{H}}{\mathbf{\Xi }}_{\mathrm{T}}vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}-{\mathbf{\Xi }}_{\mathrm{T}}^{\mathrm{H}}\mathit{r})}^{\mathrm{T}}\frac{\partial vec\left\{{\mathit{c}}_{\mathrm{T}}^{\ast }\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}}{\partial {\mathit{c}}_{\mathrm{T}}^{\ast }}\hfill \\ \hfill =& ({\mathbf{\Xi }}_{\mathrm{T}}vec\left\{{\mathit{c}}_{\mathrm{T}}\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\}{-\mathit{r})}^{\mathrm{T}}{\mathbf{\Xi }}_{\mathrm{T}}^{\ast }{\Omega }_{\mathrm{T}},\hfill \end{array}$$

(46)

where ${\Omega }_{\mathrm{T}}\triangleq \left[\begin{array}{c}{\omega }_{\mathrm{T},0},{\omega }_{\mathrm{T},1},\dots ,{\omega }_{\mathrm{T},M-1}\end{array}\right]$, and the m-th column of Ω_T is defined as

$$\begin{array}{c}\hfill {\mathit{\omega }}_{\mathrm{T},m}\triangleq vec\left\{{\mathit{e}}_m^M\otimes {\mathit{I}}_{N_{\mathrm{r}}}\right\},\end{array}$$

(47)

and ${\mathit{e}}_m^M$ is a M × 1 vector with the m-th entry being 1 and other entries being 0.

Similarly, we defined the following objective function to estimate ${\mathit{c}}_{\mathrm{R}}$ from (44)

$$\begin{array}{c}\hfill f_{\mathrm{R}}\left({\mathit{c}}_{\mathrm{R}}\right)\triangleq {∥\mathit{r}-{\mathbf{\Xi }}_{\mathrm{R}}{\mathit{c}}_{\mathrm{R}}^{\ast }∥}_2^2.\end{array}$$

(48)

Then, we can obtain

$$\begin{array}{cc}\hfill \frac{\partial f_{\mathrm{R}}\left({\mathit{c}}_{\mathrm{R}}\right)}{\partial {\mathit{c}}_{\mathrm{R}}^{\ast }}& =-\frac{\partial {\mathit{r}}^{\mathrm{H}}{\mathbf{\Xi }}_{\mathrm{R}}{\mathit{c}}_{\mathrm{R}}^{\ast }}{\partial {\mathit{c}}_{\mathrm{R}}^{\ast }}-\frac{\partial {\mathit{c}}_{\mathrm{R}}^{\mathrm{T}}{\mathbf{\Xi }}_{\mathrm{R}}^{\mathrm{H}}\mathit{r}}{\partial {\mathit{c}}_{\mathrm{R}}^{\ast }}+\frac{\partial {\mathit{c}}_{\mathrm{R}}^{\mathrm{T}}{\mathbf{\Xi }}_{\mathrm{R}}^{\mathrm{H}}{\mathbf{\Xi }}_{\mathrm{R}}{\mathit{c}}_{\mathrm{R}}^{\ast }}{\partial {\mathit{c}}_{\mathrm{R}}^v}\hfill \\ \hfill & =\left({\mathit{c}}_{\mathrm{R}}^{\mathrm{T}}{\mathbf{\Xi }}_{\mathrm{R}}^{\mathrm{H}}-{\mathit{r}}^{\mathrm{H}}\right){\mathbf{\Xi }}_{\mathrm{R}}.\hfill \end{array}$$

(49)

In Algorithm 1, the details about the proposed method to estimate the mmWave channel is given with the unknown MC effect. In Algorithm 1, the computational complexity of the sparse reconstruction can be obtained as $\mathcal{O}\left(K{N}_{\mathrm{t}}{N}_{\mathrm{r}}P\right)+\mathcal{O}\left(K^4\right)+\mathcal{O}\left(K^{3}P\right)$. The computational complexity of steepest descent is $\mathcal{O}\left(PM{N}_{\mathrm{r}}\right)+\mathcal{O}\left(PN{N}_{\mathrm{t}}\right)+\mathcal{O}\left(P^2{N}_{\mathrm{t}}\right)$. Therefore, with $K\ll N_{\mathrm{t}}$, the computational complexity can be finally obtained as $\mathcal{O}\left(K{N}_{\mathrm{t}}{N}_{\mathrm{r}}P\right)$.

Algorithm 1 Channel estimation with MC effect

1: Input: received signal $\mathit{r}$, the matrices $\mathbf{\Psi }$, ${\mathit{Q}}_F$ and $\mathit{E}$, the maximum of iteration $N_{\mathrm{iter}}$, the step $\delta$. 2: Initialization:$t=0$, ${\widehat{\mathit{c}}}_{\mathrm{T}}^{\mathrm{T}}={[1,{\mathbf{0}}_{1\times (M-1)}]}^{\mathrm{T}}$, ${\widehat{\mathit{c}}}_{\mathrm{R}}^{\mathrm{T}}={[1,{\mathbf{0}}_{1\times (N-1)}]}^{\mathrm{T}}$, $\widehat{\mathit{u}}={\mathbf{0}}_{N_{\mathrm{T}}{N}_{\mathrm{R}}\times 1}$. 3: while$t\le N_{\mathrm{iter}}$do 4:  $t\leftarrow t+1$. 5:  Obtain MC matrices $T\left({\widehat{\mathit{c}}}_{\mathrm{T}}\right)$ and $T\left({\widehat{\mathit{c}}}_{\mathrm{R}}\right)$. 6:  Obtain ${\mathbf{\Lambda }}^{\prime }\triangleq \mathbf{\Psi }\left[T^{\mathrm{T}}\left({\mathit{c}}_{\mathrm{T}}\right)\otimes T^{\mathrm{H}}\left({\mathit{c}}_{\mathrm{R}}\right)\right]({\mathit{F}}^{\ast }\otimes \mathit{E})$, and $\mathbf{\Lambda }\triangleq \left[\begin{array}{c}{\mathit{\lambda }}_0,{\mathit{\lambda }}_1,\dots ,{\mathit{\lambda }}_{N_{\mathrm{T}}{N}_{\mathrm{R}}-1}\end{array}\right]$ is the column-normalized matrix of ${\mathbf{\Lambda }}^{\prime }$. 7:  $k=1$, $\mathit{z}=\mathit{r}$, $\rho =\varnothing$. 8:  while $k\le K$ do 9:   $n_{\max }=arg{max}_n\left|{\mathit{z}}^{\prime }{\lambda }_n\right|$. 10:   $\rho \leftarrow \rho \cup n_{\max }$. 11:   $\mathit{z}=\mathit{r}-{\mathbf{\Lambda }}_{\rho }{\mathbf{\Lambda }}_{\rho }^{†}\mathit{r}$, where ${\mathbf{\Lambda }}_{\rho }$ is a matrix with the $\left|\rho \right|$ columns from $\mathbf{\Lambda }$ and † is the Moore-Penrose inverse. 12:  end while 13:  ${\widehat{\mathit{u}}}_{\rho }={\mathbf{\Lambda }}_{\rho }^{†}\mathit{r}$ with other entries of $\widehat{\mathit{u}}$ being 0. 14:  Obtain ${\mathit{\nu }}_{\mathrm{T}}^t={\left.\frac{\partial f_{\mathrm{T}}\left({\mathit{c}}_{\mathrm{T}}\right)}{\partial {\mathit{c}}_{\mathrm{T}}^{\ast }}\right|}_{{\mathit{c}}_{\mathrm{T}}={\widehat{\mathit{c}}}_{\mathrm{T}}^{t-1}}$ from (46). 15:  Obtain ${\mathit{\nu }}_{\mathrm{R}}^t={\left.\frac{\partial f_{\mathrm{R}}\left({\mathit{c}}_{\mathrm{R}}\right)}{\partial {\mathit{c}}_{\mathrm{R}}^{\ast }}\right|}_{{\mathit{c}}_{\mathrm{R}}={\widehat{\mathit{c}}}_{\mathrm{R}}^{t-1}}$ from (49). 16:  $\mathit{\nu }=\left[{\mathit{\nu }}_{\mathrm{T}}^{t\mathrm{T}},{\mathit{\nu }}_{\mathrm{R}}^{t\mathrm{T}}\right]$. 17:  $\left[{\widehat{\mathit{c}}}_{\mathrm{T}}^{t,\mathrm{T}},{\widehat{\mathit{c}}}_{\mathrm{R}}^{t,\mathrm{T}}\right]\leftarrow \left[{\widehat{\mathit{c}}}_{\mathrm{T}}^{t-1,\mathrm{T}}{\widehat{\mathit{c}}}_{\mathrm{R}}^{t-1,\mathrm{T}}\right]-\delta {\mathit{\nu }}^{t,\mathrm{T}}$. 18: end while 19: Output: the sparse vector $\widehat{\mathit{u}}$.

4. Simulation Results

In this section, the simulation results are given, and the simulation parameters are given in

Table 1. Simulation Parameters.

Parameter	Value
The number of sampled signals P	50
The number of transmitting antennas M	20
The number of receiving antennas N	10
The number of paths K	3
The space between antennas d	0.5 wavelength
The grid space $\delta$	0.2°
The detection DoA range	[−45°, 45°]

, where the grids in Table 1 are for the 1-D arrangement. MATLAB codes have been made available online at https://drive.google.com/drive/folders/1dFw-XktTZaPQeCZnQMr6Gr_igRBdH0Rk?usp=sharing. All experiments are carried out in Matlab R2017b on a PC with a 2.9 GHz Intel Core i5 and 8 GB of RAM. The beamforming vectors $\mathit{a}\left(t\right)$ and $\mathit{b}\left(t\right)$ are uniformly chosen from a unit circle, and this scheme is referred to as a random coding scheme. The DoD/DoA estimation performance is measured by root-mean-square error (RMSE)

$$\begin{array}{c}\hfill e=\sqrt{\frac{1}{2K{N}_{\mathrm{p}}}{\displaystyle \sum _{n=0}^{N_{\mathrm{p}}-1}}\left(\parallel \widehat{\mathit{\varphi }}{-\mathit{\varphi }\parallel }_2^2+{\parallel \widehat{\mathit{\psi }}-\mathit{\psi }\parallel }_2^2\right)}\left(\mathrm{deg}\right),\end{array}$$

(50)

where $\widehat{\mathit{\varphi }}$ and $\widehat{\mathit{\varphi }}$ denote the estimated DoA and DoD, respectively. $N_{\mathrm{p}}$ denotes the number of pairs to simulate DoD/DoA. The DoD and DoA are randomly chosen from [−45°, 45°] and the minimum space of DoD/DoA for different path is greater than 10°. The signal-to-noise (SNR) ratio is defined as

$$\begin{array}{c}\hfill \mathrm{SNR}\triangleq \frac{{\mathit{y}}^{\mathrm{H}}\mathit{y}}{\mathcal{E}\left\{{\mathit{n}}^{\mathrm{H}}\mathit{n}\right\}}\end{array}$$

(51)

where $\mathit{y}$ denotes the signal $\mathit{y}\triangleq \mathbf{\Psi }\left[T^{\mathrm{T}}\left({\mathit{c}}_{\mathrm{T}}\right)\otimes T^{\mathrm{H}}\left({\mathit{c}}_{\mathrm{R}}\right)\right]({\mathit{D}}^{\ast }\otimes \mathit{C})\mathit{g}$, and $\mathit{n}$ denotes the additive white Gaussian noise (AWGN), and the entries of $\mathit{n}$ follow the zero-mean complex Gaussian distribution $\mathit{n}\sim \mathcal{CN}(\mathbf{0},{\sigma }_{\mathrm{n}}^2\mathit{I})$.

First, the DoD/DoA estimation performance with different SNRs is given in Figure 2, where the MC effect between adjacent antennas is $-15$ dB. The curve “Without MC effect” is the DoD/DoA estimation performance of OMP method in the scenario without the MC effect. The curve “Perfect MC information” denotes the DoD/DoA estimation performance of the proposed method with perfect MC information, so the MC vectors are not updated in the proposed method. The curve “Ignoring MC information” denotes the DoD/DoA estimation performance of traditional OMP method without considering the MC effect. The curve “Proposed method” denotes the DoD/DoA estimation performance using the proposed method. As shown in this figure, the DoD/DoA estimation performance can be significantly improved by the proposed method with the additional estimation for MC effect. When SNR is 20 dB, the RMSE of the traditional OMP method is 0.889°, but the RMSE can be decreased to 0.800° (10% improvement). The same estimation performance can be achieved by the proposed method when SNR is 12 dB, so the estimation performance is improved by 8 dB.

Then, with the MC effect being −10 dB, we show the DoD/DoA estimation performance in Figure 3. When SNR is 20 dB, the RMSE of the traditional OMP method is 1.276°, and that of the proposed method is 0.948° (25.7% improvement). Therefore, with worse MC effect, the DoD/DoA estimation performance is improved more efficiently using the proposed method.

Figure 4 shows the DoD/DoA estimation performance with different MC effect. When the MC effect is greater than $-10$ dB, the estimation performance will be worse significantly. When the MC effect is less than $-10$ dB, the MC effect can be reduced by about 4 dB using the proposed method. Therefore, the proposed method is efficient for the mmWave channel estimation in the scenario with the MC effect.

5. Conclusions

In this paper, the channel estimation problem in the mmWave communication system has been investigated. The unknown MC effect is described by a symmetric Toeplitz matrix, and the sparse-based mmWave system model with MC coefficients has been formulated. Then, by exploiting the channel sparsity in the spatial domain, the two-stage method based CS has been proposed by estimating the DoD/DoA and MC coefficients iteratively. Simulation results show that the proposed method can improve the estimation performance of mmWave channel significantly. Future work will focus on mmWave channel estimation with moving users.