Robust Angle Estimation for MIMO Radar with the Coexistence of Mutual Coupling and Colored Noise

Abstract

This paper deals with joint estimation of direction-of-departure (DOD) and direction-of- arrival (DOA) in bistatic multiple-input multiple-output (MIMO) radar with the coexistence of unknown mutual coupling and spatial colored noise by developing a novel robust covariance tensor-based angle estimation method. In the proposed method, a third-order tensor is firstly formulated for capturing the multidimensional nature of the received data. Then taking advantage of the temporal uncorrelated characteristic of colored noise and the banded complex symmetric Toeplitz structure of the mutual coupling matrices, a novel fourth-order covariance tensor is constructed for eliminating the influence of both spatial colored noise and mutual coupling. After a robust signal subspace estimation is obtained by using the higher-order singular value decomposition (HOSVD) technique, the rotational invariance technique is applied to achieve the DODs and DOAs. Compared with the existing HOSVD-based subspace methods, the proposed method can provide superior angle estimation performance and automatically jointly perform the DODs and DOAs. Results from numerical experiments are presented to verify the effectiveness of the proposed method.

1. Introduction

As an innovative radar framework, multiple-input multiple-output (MIMO) radar system plays an important role in improving detection and estimation performance. MIMO radar uses multiple antennas to emit mutual orthogonal waveforms, and attempts to detect the locations of interest targets from the echoes received by multiple antennas. By exploiting matched filters, the information from an individual transmitter-to-receiver path can be extracted, and a virtual array with a large aperture is formed. Thus, MIMO radar can achieve superior performance compared with the traditional phase-array radar [1]. In terms of the antennas configuration, MIMO radar can be generally divided into two classes: statistical MIMO radar and collocated MIMO radar [2,3]. The former can deal with the scintillation problem by taking advantage of widely separated antennas and the latter can achieve unambiguous angle estimation with closely placed antenna arrays. In this paper, we focus on the bistatic MIMO radar, which is a kind of the collocated MIMO radar.

In bistatic MIMO radar, joint estimation of direction-of-departure (DOD) and direction-of-arrival (DOA) is a key issue. Many excellent algorithms, such as spatial spectrum searching methods [4,5], estimate of signal parameter via rotational invariance techniques (ESPRIT) based methods [6,7], propagator methods (PM) [8,9,10], optimization methods [11,12] and tensor methods [13,14,15], have been investigated. As suggested in these literatures, all the methods can perform well under the desired conditions, such as the well-calibrated transmit and receive arrays, and Gaussian white noise. Nevertheless, in practice, the transmit and receive arrays may suffer from mutual coupling due to the radiation effect from antenna elements [16,17,18,19,20]. On the other hand, the noise model is often unknown spatial colored noise, i.e., the covariance of the noise is no longer a scaled identical matrix. Thus, the high-resolution estimation methods usually have some performance degradation or fail to work. The mutual coupling problem in MIMO radar has been investigated [21,22]. In [23], an improved multiple signal classification (MUSIC) algorithm was proposed to transform the estimation of DOD, DOA and mutual coupling matrices into a linear constrained quadratic problem. However, it is computationally inefficient since two one-dimensional peaking searches are required, and angle ambiguity would appear because the columns in the direction matrices may be linearly dependent. By using the the banded complex symmetric Toeplitz structure in [24], the ESPRIT-Like algorithm was introduced to eliminate the effect of mutual coupling for a closed solution to angle estimation, which avoids the peaking search. A PM-Like method in [25] avoided requiring eigenvalue decomposition (EVD) of the array covariance matrix. Compared with ESPRIT-Like algorithm, it provides similar performance with lower computational complexity. The above matrix-based algorithms ignore the multidimensional nature that is inherent in the received data. Taking the multidimensional structure of the received data into consideration, the higher-order singular value decomposition (HOSVD) method and the trilinear decomposition method were derived in [26,27], respectively to achieve better performance than the conventional subspace methods. On the other hand, efforts have been devoted to solve the angle estimation problem in spatial colored noise environment. For instance, the spatial cross-correlation methods [28,29,30,31], the temporal cross-correlation methods [32,33] and the differencing covariance method [34] have been reported. Although these methods can achieve satisfied performance with spatial colored noise, their performance may have obvious degradation or fail to work. The above mentioned angle estimation methods investigate the mutual coupling and spatial colored noise independently.

In this paper, we attempt to estimate the DODs and DOAs in the environment of mutual coupling and spatial colored noise. To the best knowledge of the authors, there has been little attention to this case in the open literature. Towards this end, a novel robust covariance tensor-based angle estimation method is developed, which can be viewed as a generalized HOSVD approach. The main contribution of this paper can be summarized as: (1) Unlike solving the influence of mutual coupling and spatial colored noise independently in [26,27,28,29,30,31,32,33,34], the proposed method can solve the joint DOD and DOA estimation issue with the coexistence of unknown mutual coupling and spatial colored noise in MIMO radar for the first time. (2) The proposed method formulates a novel fourth-order covariance tensor for eliminating the influence of both spatial colored noise and mutual coupling in tensor domain. (3) The rotational invariance technique is utilized to obtain automatically paired DODs and DOAs. (4) The proposed method provides better angle estimation performance than the existing HOSVD-based subspace methods, and simulation results are performed to show the advantages of the proposed algorithm .

The paper outline is as follows. Some necessary tensor preliminaries and the signal model are given in Section 2. The proposed scheme is established in Section 3. Some remarks and detailed analysis of the proposed algorithm are discussed in Section 4. Simulation results are given in Section 5. Finally, we give the conclusions of the proposed method in Section 6.

Notation: The bold face capital letter $\mathbf{X}$ and lower case $\mathbf{x}$ denote matrices and vectors, respectively. The $M\times M$ identity matrix is denoted by ${\mathbf{I}}_M$, and a $M\times N$ zero matrix (all the elements are zeros) is denoted by ${\mathbf{0}}_{M\times N}$. For $\mathbf{X}$, the expressions, ${\left(\mathbf{X}\right)}^T$,${\left(\mathbf{X}\right)}^H$,${\left(\mathbf{X}\right)}^{-1}$ and ${\left(\mathbf{X}\right)}^{†}$, represent the operations of transpose, Hermitian transpose, inverse and pseudo-inverse, respectively and ⊗ stands for the Kronecker product. The Khatri-Rao product (column-wise Kronecker product) is denoted by ⊙, i.e., $\left[{\mathbf{a}}_1,{\mathbf{a}}_2,\dots ,{\mathbf{a}}_K\right]\odot \left[{\mathbf{b}}_1,{\mathbf{b}}_2,\dots ,{\mathbf{b}}_K\right]=\left[{\mathbf{a}}_1\otimes {\mathbf{b}}_1,{\mathbf{a}}_2\otimes {\mathbf{b}}_2,\dots ,{\mathbf{a}}_K\otimes {\mathbf{b}}_K\right]$; $\mathrm{Toeplitz}\left[\mathbf{r}\right]$ denotes the symmetric Toeplitz matrix constructed by the vector $\mathbf{r}$; $\mathrm{diag}\left(\mathbf{r}\right)$ denotes the diagonalization operation; $\mathrm{angle}(·)$ returns the phase angles of an element in radians.

2. Tensor Preliminaries and Signal Model

2.1. Tensor Preliminaries

Let us briefly review some necessary operations in tensor domain. For more details about tensor algebra, the readers can refer to the review article [35].

Definition 1

(Unfolding or Matricization). An $\left(I_1\times I_2\times \cdots \times I_N\right)$-dimensional tensor has N indices. The mode-n unfolding of a tensor $\mathcal{X}\in {\mathrm{C}}^{I_1\times I_2\times \cdots \times I_N}$ is denoted by ${\left[\mathcal{X}\right]}_{\left(n\right)}$, where the $(i_1,i_2,\dots ,i_N)$-element of $\mathcal{X}$ maps to the $(i_n,j)$-th element of ${\left[\mathcal{X}\right]}_{\left(n\right)}$, where $j=1+{\sum }_{k=1,k\ne n}^N(i_k-1)J_k$ with $J_k={\prod }_{m=1,m\ne n}^{k-1}{I}_m$.

Definition 2

(Mode-n Tensor-Matrix Product). The mode-n product of an N-order tensor $\mathcal{X}\in {\mathrm{C}}^{I_1\times I_2\times \cdots \times I_N}$ and a matrix $\mathbf{A}\in {\mathrm{C}}^{J_n\times I_n}$, is denoted by $\mathcal{Y}={\mathcal{X}}_{\times n}\mathbf{A}$. In the matrix form, it can be expressed as

$${\left[\mathcal{Y}\right]}_{\left(n\right)}=\mathbf{A}{\left[\mathcal{X}\right]}_{\left(n\right)}$$

(1)

Moreover, the mode-n tensor-matrix product satisfies the following properties

$$\left\{\begin{array}{c}{\mathcal{X}}_{\times n}{\mathbf{A}}_{\times m}\mathbf{B}={\mathcal{X}}_{\times m}{\mathbf{B}}_{\times n}\mathbf{A},m\ne n\hfill \\ {\mathcal{X}}_{\times n}{\mathbf{A}}_{\times n}\mathbf{B}={\mathcal{X}}_{\times n}\left(\mathbf{B}\mathbf{A}\right),m=n\hfill \end{array}\right.$$

(2)

$${\left[{\mathcal{X}}_{\times 1}{\mathbf{A}}_{1\times 2}{\mathbf{A}}_{2\times }{\dots }_{\times N}{\mathbf{A}}_N\right]}_{\left(n\right)}={\mathbf{A}}_n·{\left[\mathcal{X}\right]}_{\left(n\right)}·\left[{\mathbf{A}}_{n+1}\otimes \cdots \otimes {\mathbf{A}}_N\otimes {\mathbf{A}}_1\cdots \otimes {\mathbf{A}}_{n-1}\right]$$

(3)

Definition 3

(Tensor Decomposition). The HOSVD of a tensor $\mathcal{X}\in {\mathrm{C}}^{I_1\times I_2\times \cdots \times I_N}$ is given by

$$\mathcal{X}={\mathcal{G}}_{\times 1}{\mathbf{U}}_{1\times 2}{\mathbf{U}}_{2\times 3}{\dots }_{\times N}{\mathbf{U}}_N$$

(4)

where $\mathcal{G}\in {\mathrm{C}}^{I_1\times I_2\times \cdots \times I_N}$ is the core tensor, and ${\mathbf{U}}_n\in {\mathrm{C}}^{I_n\times I_n}(n=1,2,3,\dots ,N)$ is a unitary matrix, which is consist of the left singular vectors of ${\left[\mathcal{X}\right]}_{\left(n\right)}$.

2.2. Signal Model

Now we consider a general bistatic MIMO radar system, as shown in Figure 1. There are M transmitters and N receivers, and both of which are uniform linear arrays (ULAs) with half-wavelength spacing. The transmitters emit M mutual orthogonal waveforms and illuminate a given area. Suppose that there are K point targets located in the far field. The signals are reflected by the slow-moving targets and the echoes are collected by the receivers. With the assumtion that each pulse period of the transmitted waveform is consist of Q coded symbols, let us consider a coherent processing interval (CPI) consisting of L pulses. Then the received data during the l-th ($l=1,2,\dots ,L$) pulse period can be expressed as

$${\mathbf{X}}_l={\mathbf{A}}_r\mathrm{diag}\left({\mathbf{s}}_l\right){\mathbf{A}}_t^T\mathbf{W}+{\mathbf{F}}_l$$

(5)

where ${\mathbf{A}}_r\in {\mathrm{C}}^{N\times K}$ is the receive direction matrix, ${\mathbf{A}}_t\in {\mathrm{C}}^{M\times K}$ is the transmit direction matrix, ${\mathbf{s}}_l\in {\mathrm{C}}^{K\times 1}$ is the echo coefficient vector, $\mathbf{W}\in {\mathrm{C}}^{M\times Q}$ is the baseband code matrix, and ${\mathbf{F}}_l\in {\mathrm{C}}^{N\times Q}$ is the received spatial colored noise. The details of the above mentioned matrices are given below

$$\begin{array}{c}{\mathbf{A}}_r=\left[{\mathbf{a}}_r\left({\theta }_1\right),{\mathbf{a}}_r\left({\theta }_2\right),\dots ,{\mathbf{a}}_r\left({\theta }_K\right)\right]\\ {\mathbf{a}}_r\left({\theta }_k\right)={\left[1,e^{-j\pi sin{\theta }_k},\dots ,e^{-j\pi \left(N-1\right)sin{\theta }_k}\right]}^T,k=1,2,\dots ,K\\ {\mathbf{A}}_t=\left[{\mathbf{a}}_t\left({\phi }_1\right),{\mathbf{a}}_t\left({\phi }_2\right),\dots ,{\mathbf{a}}_t\left({\phi }_K\right)\right]\\ {\mathbf{a}}_t\left({\phi }_k\right)={\left[1,e^{-j\pi sin{\phi }_k},\dots ,e^{-j\pi \left(M-1\right)sin{\phi }_k}\right]}^T,k=1,2,\dots ,K\\ {\mathbf{s}}_l={\left[{\alpha }_1{e}^{j2\pi l{f}_1/f_s},{\alpha }_2{e}^{j2\pi l{f}_2/f_s},\dots ,{\alpha }_K{e}^{j2\pi l{f}_K/f_s}\right]}^T\\ \mathbf{W}=\left[{\mathbf{w}}_1,{\mathbf{w}}_2,\dots ,{\mathbf{w}}_M\right]\end{array}$$

where ${\mathbf{a}}_r\left({\theta }_k\right)$ and ${\mathbf{a}}_t\left({\phi }_k\right)$ denote the receive steering vector and the transmit steering vector, respectively; ${\alpha }_k$, $f_k$ and $f_s$ represent the radar cross section (RCS) amplitude, the Doppler frequency and the pulse repeat frequency, respectively and ${\mathbf{w}}_m\in {\mathrm{C}}^{Q\times 1}$ is the m-th transmitted baseband code with ${\mathbf{w}}_m^H{\mathbf{w}}_n=\left\{\begin{array}{c}0,m\ne n\hfill \\ P,m=n\hfill \end{array}\right.m,n\in \left\{1,2,\dots ,M\right\}$. The received noise is spatially colored, which indicates that the columns of ${\mathbf{F}}_l$ are independently and identically distributed complex Gaussian random vectors with zero mean and unknown covariance matrix $\mathbf{C}$, i.e., for any $a,b\in \left\{1,2,\dots ,L\right\}$, there exist

$$\mathrm{E}\left\{\mathrm{vec}\left({\mathbf{F}}_a\right)\mathrm{vec}\left({\mathbf{F}}_b^H\right)\right\}=\left\{\begin{array}{cc}\mathbf{0},\hfill & a\ne b\hfill \\ {\mathbf{I}}_Q\otimes \mathbf{C},\hfill & a=b\hfill \end{array}\right.$$

(6)

0.98,0.00,0.00 where ${\mathbf{F}}_a$ and ${\mathbf{F}}_b$ are the columns of ${\mathbf{F}}_l$. At the receive array, the received data ${\mathbf{X}}_l$ is matched by ${\mathbf{w}}_m/Q$, $m=1,2,\dots ,M$, respectively. After this operation, the matched data can be stacked in to a matrix $\tilde{\mathbf{Y}}$ according to the spatial-temporal order. Similar to [24], we have

$$\tilde{\mathbf{Y}}=\left[{\mathbf{A}}_t\odot {\mathbf{A}}_r\right]{\mathbf{S}}^T+\frac{1}{Q}\mathbf{N}$$

(7)

where $\mathbf{S}={\left[{\mathbf{s}}_1,{\mathbf{s}}_2,\dots ,{\mathbf{s}}_L\right]}^T$ is the echo coefficient matrix and $\mathbf{N}=\left[{\mathbf{n}}_1,{\mathbf{n}}_2,\dots ,{\mathbf{n}}_L\right]$ denotes the matched noise matrix with ${\mathbf{n}}_l=\mathrm{vec}\left({\mathbf{F}}_l{\mathbf{W}}^H\right)$, ($l=1,2,\dots ,L$). Generally speaking, mutual coupling effect would exist in the antenna array due to the radiation effects of the antenna elements [16]. The mutual coupling between antenna elements of a ULA can be described as a banded symmetric Toeplitz matrix, known as the mutual coupling matrix. The mutual coupling coefficient between two antennas in a ULA is opposite to their distance and is approximated as zeros when the distance is large enough [24]. Assuming that the number of nonzero mutual coupling coefficients is $P+1$ for both transmit and receive arrays, and P is satisfied with $\min \{M,N\}>2P$. Taking the mutual coupling effect into consideration, the data model in Equation (7) becomes

$$\mathbf{Y}=\left[\left({\mathbf{C}}_t{\mathbf{A}}_t\right)\odot \left({\mathbf{C}}_r{\mathbf{A}}_r\right)\right]{\mathbf{S}}^T+\frac{1}{Q}\mathbf{N}=\left[{\tilde{\mathbf{A}}}_t\odot {\tilde{\mathbf{A}}}_r\right]{\mathbf{S}}^T+\frac{1}{Q}\mathbf{N}$$

(8)

where ${\mathbf{C}}_t=\mathrm{toeplitz}\left(\left[{\mathbf{c}}_t^T,{\mathbf{0}}_{1\times \left(M-P-1\right)}\right]\right)\in {\mathrm{C}}^{M\times M}$ and ${\mathbf{C}}_r=\mathrm{toeplitz}\left(\left[{\mathbf{c}}_r^T,{\mathbf{0}}_{1\times \left(N-P-1\right)}\right]\right)\in {\mathrm{C}}^{N\times N}$ are the mutual coupling matrices with ${\mathbf{c}}_t=[c_{t0},c_{t1},\dots ,c_{tP}]$ and ${\mathbf{c}}_r=[c_{r0},c_{r1},\dots ,c_{rP}]$, and $c_{ip}(i=r,t;p=0,1,2,\dots ,P)$ is the $P+1$ nonzero mutual coupling coefficients, which satisfy with $0<|c_{iP}|<,\dots ,<|c_{i1}|<|c_{i0}|=1$. ${\tilde{\mathbf{A}}}_t=\left[{\tilde{\mathbf{a}}}_t\left({\phi }_1\right),{\tilde{\mathbf{a}}}_t\left({\phi }_2\right),\dots ,{\tilde{\mathbf{a}}}_t\left({\phi }_K\right)\right]={\mathbf{C}}_t{\mathbf{A}}_t$, ${\tilde{\mathbf{A}}}_r=\left[{\tilde{\mathbf{a}}}_r\left({\phi }_1\right),{\tilde{\mathbf{a}}}_r\left({\phi }_2\right),\dots ,{\tilde{\mathbf{a}}}_r\left({\phi }_K\right)\right]={\mathbf{C}}_r{\mathbf{A}}_r$.

Since each element in $\mathbf{Y}$ is formed by multiplying the unique entity in ${\tilde{\mathbf{A}}}_t$, ${\tilde{\mathbf{A}}}_r$ and $\mathbf{S}$, the matched data $\mathbf{Y}$ exhibits three diversities. However, these diversities have been ignored by the traditional matrix-based estimation methods. To further exploiting the inherent multidimensional structure, $\mathbf{Y}$ is rearranged into a third-order tensor $\mathcal{Y}$ shown as Figure 2. Then the ($m,n,l$)-th ($m=1,\dots ,M;n=1,\dots ,N;l=1,\dots ,L$) element is given by

$$\begin{array}{c}\hfill \mathcal{Y}\left(m,n,l\right)=\sum _{k=1}^K{\tilde{\mathbf{A}}}_t\left(m,k\right){\tilde{\mathbf{A}}}_r\left(n,k\right)\mathbf{S}\left(l,k\right)+\frac{1}{Q}\mathcal{N}\left(m,n,l\right)\end{array}$$

(9)

where $\mathcal{Y}\left(m,n,l\right)$ denotes the $\left(m,n,l\right)$-th element in $\mathcal{Y}$ and similar to others, and $\mathcal{N}\in {\mathrm{C}}^{M\times N\times L}$ is the rearranged noise tensor. According to Definition 1, it is obvious that $\mathbf{Y}={\left[\mathcal{Y}\right]}_{\left(3\right)}^T$ and $\mathbf{N}={\left[\mathcal{N}\right]}_{\left(3\right)}^T$. In HOSVD-wise format, Equation (9) can be expressed as

$$\mathcal{Y}={\mathcal{I}}_{K\times 1}{\tilde{\mathbf{A}}}_{r\times 2}{\tilde{\mathbf{A}}}_{t\times 3}\mathbf{S}+\frac{1}{Q}\mathcal{N}$$

(10)

where ${\mathcal{I}}_K$ is the $K\times K\times K$ identity tensor.

3. Robust Covariance Tensor-based Angle Estimation Method

Because the conventional subspace-based methods or tensor decomposition technique [24,25,26,27,28,29,31,33,34] considers the influence of mutual coupling and spatial colored noise independently, they unavoidably have performance degradation or fail to work with the coexistence of unknown mutual coupling and spatial colored noise. Let us formulate a novel robust covariance tensor-based angle estimation method to remove the influence of both mutual coupling and spatial colored noise in tensor domain.

3.1. Spatial Colored Noise Suppression and Mutual Coupling Elimination

In Equation (10), the mutual coupling has an impacts on the transmit direction matrix ${\tilde{\mathbf{A}}}_t$ and receive direction matrix ${\tilde{\mathbf{A}}}_r$, i.e, ${\tilde{\mathbf{A}}}_t$ and ${\tilde{\mathbf{A}}}_r$ are no longer Vandermonde matrices. Fortunately, the mutual coupling matrices are banded symmetric Toeplitz, and two sub-matrices can be extracted from transmit and receive direction matrices for decoupling. By defining two selection matrices as ${\mathbf{J}}_3=\left[{\mathbf{0}}_{\left(N-2P\right)\times P},{\mathbf{I}}_{\left(N-2P\right)},{\mathbf{0}}_{\left(N-2P\right)\times P}\right]$ and ${\mathbf{J}}_4=\left[{\mathbf{0}}_{\left(M-2P\right)\times P},{\mathbf{I}}_{\left(M-2P\right)},{\mathbf{0}}_{\left(M-2P\right)\times P}\right]$, we have

$$\left\{\begin{array}{c}{\widehat{\mathbf{a}}}_r\left({\theta }_k\right)={\mathbf{J}}_3{\tilde{\mathbf{a}}}_r\left({\theta }_k\right)={\beta }_{rk}{\overline{\mathbf{a}}}_r\left({\theta }_k\right)\\ {\widehat{\mathbf{a}}}_t\left({\theta }_k\right)={\mathbf{J}}_4{\tilde{\mathbf{a}}}_t\left({\phi }_k\right)={\beta }_{tk}{\overline{\mathbf{a}}}_t\left({\phi }_k\right)\end{array}\right.$$

(11)

where ${\beta }_{tk}=1+{\sum }_{p=1}^{P}2c_{tp}cos\left(p\pi sin{\phi }_k\right)$, ${\beta }_{rk}=1+{\sum }_{p=1}^{P}2c_{rp}cos\left(p\pi sin{\theta }_k\right)$, ${\overline{\mathbf{a}}}_r\left({\theta }_k\right)$ and ${\overline{\mathbf{a}}}_t\left({\phi }_k\right)$ are column vectors composed of the first $N-2P$ and $M-2P$ elements of ${\mathbf{a}}_r\left({\theta }_k\right)$ and ${\mathbf{a}}_t\left({\phi }_k\right)$, respectively. From Equation (11), it is easy to know that for each target, ${\beta }_{tk}$ and ${\beta }_{rk}$ are constant, which indicates that the direction matrices ${\widehat{\mathbf{A}}}_r\left(\theta \right)=[{\widehat{\mathbf{a}}}_r\left({\theta }_1\right),{\widehat{\mathbf{a}}}_r\left({\theta }_2\right),\dots ,{\widehat{\mathbf{a}}}_r\left({\theta }_K\right)]$ and ${\widehat{\mathbf{A}}}_t\left(\theta \right)=[{\widehat{\mathbf{a}}}_t\left({\theta }_1\right),{\widehat{\mathbf{a}}}_t\left({\theta }_2\right),\dots ,{\widehat{\mathbf{a}}}_t\left({\theta }_K\right)]$ have the Vandermonde structure. Thus, the influence of mutual coupling is eliminated after the decoupling operation. The decoupling operation in Equation (11) can be extended to Equation (10) in tensor domain, which shows

$$\widehat{\mathcal{Y}}={\mathcal{Y}}_{\times 1}{\mathbf{J}}_{3\times 2}{\mathbf{J}}_4={\mathcal{I}}_{K\times 1}{\widehat{\mathbf{A}}}_{r\times 2}{\widehat{\mathbf{A}}}_{t\times 3}\mathbf{S}+\frac{1}{Q}\widehat{\mathcal{N}}$$

(12)

where $\widehat{\mathcal{N}}={\mathcal{N}}_{\times 1}{\mathbf{J}}_{3\times 2}{\mathbf{J}}_4$ is a part of $\widehat{\mathcal{N}}$, which corresponds to the spatial colored noise of $(M-2P)(N-2P)$ elements after the decoupling operation in matrix domain. According to Equation (12), it can be indicated that the effect of mutual coupling has been eliminated in tensor domain, and the tensor noise $\widehat{\mathcal{N}}$ also holds the same characteristic with $\mathcal{N}$. Thus, the decoupling in Equation (12) also hold the influence of spatial colored noise, which makes the performance degradation. In order to further eliminate the spatial colored noise, it is necessary to analysis the covariance of the colored noise. Let $a,b\in \left\{1,2,\dots ,L\right\}$, we get

$$\begin{array}{ccc}\hfill \mathrm{E}\left\{{\mathbf{n}}_a{\mathbf{n}}_b^H\right\}& =& \mathrm{E}\left\{\mathrm{vec}\left({\mathbf{F}}_a{\mathbf{W}}^H\right){\mathrm{vec}}^H\left({\mathbf{F}}_b{\mathbf{W}}^H\right)\right\}=\mathrm{E}\left\{\mathrm{vec}\left({\mathbf{I}}_N{\mathbf{F}}_a{\mathbf{W}}^H\right){\mathrm{vec}}^H\left({\mathbf{I}}_N{\mathbf{F}}_b{\mathbf{W}}^H\right)\right\}\hfill \\ & =& \mathrm{E}\left\{\left[{\mathbf{W}}^{\ast }\otimes {\mathbf{I}}_N\right]\left[\mathrm{vec}\left({\mathbf{F}}_a\right){\mathrm{vec}}^H\left({\mathbf{F}}_b\right)\right]\left[{\mathbf{W}}^T\otimes {\mathbf{I}}_N\right]\right\}\hfill \\ & =& \{\begin{array}{cc}\mathbf{0},\hfill & \hfill a\ne b\\ \mathrm{E}\left[{\mathbf{W}}^{\ast }\otimes {\mathbf{I}}_N\right]\left[{\mathbf{I}}_M\otimes \mathbf{C}\right]\left[{\mathbf{W}}^T\otimes {\mathbf{I}}_N\right],\hfill & \hfill a=b\end{array}\hfill \\ & =& \{\begin{array}{cc}\mathbf{0},\hfill & \hfill a\ne b\\ Q({\mathbf{I}}_M\otimes \mathbf{C}),& \hfill a=b\end{array}\hfill \end{array}$$

(13)

Based on Equation (13), it can be concluded that the covariance matrix of the spatial colored noise with different pulse period is $\mathbf{0}$, i.e., the spatial colored noise is temporal uncorrelated. Motivated by this feature, the tensor $\widehat{\mathcal{Y}}$ is divided into two sub-tensors ${\mathcal{Z}}_1$ and ${\mathcal{Z}}_2$, which are shown as

$$\begin{array}{c}\hfill {\mathcal{Z}}_1={\widehat{\mathcal{Y}}}_{\times 3}{\mathbf{J}}_3={\mathcal{I}}_{K\times 1}{\widehat{\mathbf{A}}}_{r\times 2}{\widehat{\mathbf{A}}}_{t\times 3}{\mathbf{S}}_1+\frac{1}{Q}{\underline{\mathcal{N}}}_1\\ \hfill {\mathcal{Z}}_2={\widehat{\mathcal{Y}}}_{\times 3}{\mathbf{J}}_4={\mathcal{I}}_{K\times 1}{\widehat{\mathbf{A}}}_{r\times 2}{\widehat{\mathbf{A}}}_{t\times 3}{\mathbf{S}}_2+\frac{1}{Q}{\underline{\mathcal{N}}}_2\end{array}$$

(14)

where ${\mathbf{J}}_3=\left[{\mathbf{I}}_{L-1},{\mathbf{0}}_{L-1\times 1}\right]$, ${\mathbf{J}}_4=\left[{\mathbf{0}}_{L-1\times 1},{\mathbf{I}}_{L-1}\right]$, ${\mathbf{S}}_1={\mathbf{J}}_3\mathbf{S}$, ${\underline{\mathcal{N}}}_1={\widehat{\mathcal{N}}}_{\times 3}{\mathbf{J}}_3$, ${\mathbf{S}}_2={\mathbf{J}}_4\mathbf{S}$, and ${\underline{\mathcal{N}}}_2={\widehat{\mathcal{N}}}_{\times 3}{\mathbf{J}}_4$. According to the characteristic of the spatial colored noise in Equation (13), we have

$${\mathbf{R}}_{\mathbf{Z}}=\mathrm{E}\left\{{\left[{\mathcal{Z}}_1\right]}_{\left(3\right)}^T{\left[{\mathcal{Z}}_2\right]}_{\left(3\right)}^{\ast }\right\}=\left[{\widehat{\mathbf{A}}}_t\odot {\widehat{\mathbf{A}}}_r\right]{\mathbf{R}}_{\widehat{\mathbf{S}}}{\left[{\widehat{\mathbf{A}}}_t\odot {\widehat{\mathbf{A}}}_r\right]}^H+\frac{1}{Q^2}{\mathbf{R}}_{\underline{N}}=\left[{\widehat{\mathbf{A}}}_t\odot {\widehat{\mathbf{A}}}_r\right]{\mathbf{R}}_{\widehat{\mathbf{S}}}{\left[{\widehat{\mathbf{A}}}_t\odot {\widehat{\mathbf{A}}}_r\right]}^H$$

(15)

where ${\mathbf{R}}_{\widehat{\mathbf{S}}}=E\left[{\mathbf{S}}_1^T{\mathbf{S}}_2^{\ast }\right]$, and according to Equation (13), ${\mathbf{R}}_{\underline{\mathbf{N}}}=\mathrm{E}\left\{{\left[{\underline{\mathcal{N}}}_1\right]}_{\left(3\right)}^T{\left[{\underline{\mathcal{N}}}_2\right]}_{\left(3\right)}^{\ast }\right\}=\mathrm{E}\left\{\left[{\widehat{\mathbf{n}}}_1,{\widehat{\mathbf{n}}}_2,\dots ,{\widehat{\mathbf{n}}}_{L-1}\right]{\left[{\widehat{\mathbf{n}}}_2,{\widehat{\mathbf{n}}}_3,\dots ,{\widehat{\mathbf{n}}}_L\right]}^H\right\}=0$, which indicates that the spatial colored noise is eliminated. Thus, similar to [26], the de-noising method in Equation (15) can be extended in tensor format. Using the characteristic of the spatial colored noise, a new fourth-order tensor $\mathcal{R}\in {\mathrm{C}}^{N\times M\times N\times M}$ is constructed with the ($n,m,q,p$)-th ($n,q\in \left\{1,2,\dots ,N-2P\right\};m,p\in \left\{1,2,\dots ,M-2P\right\}$) element as

$$\mathcal{R}\left(n,m,q,p\right)=\sum _{l=1}^{L-1}{\mathcal{Z}}_1\left(n,m,l\right){\mathcal{Z}}_2^{\ast }\left(q,p,l\right)$$

(16)

According to the structure of $\mathcal{R}$, the relationship between ${\mathbf{R}}_{\mathbf{Z}}$ and $\mathcal{R}$ can be written as

$${\mathbf{R}}_{\mathbf{Z}}=\left[\begin{array}{ccccccc}\mathcal{R}\left(1,1,1,1\right)\hfill & \mathcal{R}\left(1,1,2,1\right)\hfill & \dots \hfill & \mathcal{R}\left(1,1,N,1\right)\hfill & \mathcal{R}\left(1,1,1,2\right)\hfill & \dots \hfill & \mathcal{R}\left(1,1,N,M\right)\hfill \\ \mathcal{R}\left(2,1,1,1\right)\hfill & \mathcal{R}\left(2,1,2,1\right)\hfill & \dots \hfill & \mathcal{R}\left(2,1,N,1\right)\hfill & \mathcal{R}\left(2,1,1,2\right)\hfill & \dots \hfill & \mathcal{R}\left(2,1,N,M\right)\hfill \\ ⋮\hfill & ⋮\hfill & \ddots \hfill & ⋮\hfill & ⋮\hfill & \ddots \hfill & ⋮\hfill \\ \mathcal{R}\left(N,1,1,1\right)\hfill & \mathcal{R}\left(N,1,2,1\right)\hfill & \dots \hfill & \mathcal{R}\left(N,1,N,1\right)\hfill & \mathcal{R}\left(N,1,1,2\right)\hfill & \dots \hfill & \mathcal{R}\left(N,1,N,M\right)\hfill \\ \mathcal{R}\left(1,2,1,1\right)\hfill & \mathcal{R}\left(1,2,2,1\right)\hfill & \dots \hfill & \mathcal{R}\left(1,2,N,1\right)\hfill & \mathcal{R}\left(1,2,1,2\right)\hfill & \dots \hfill & \mathcal{R}\left(1,2,N,M\right)\hfill \\ \mathcal{R}\left(2,2,1,1\right)\hfill & \mathcal{R}\left(2,2,2,1\right)\hfill & \dots \hfill & \mathcal{R}\left(2,2,N,1\right)\hfill & \mathcal{R}\left(2,2,1,2\right)\hfill & \dots \hfill & \mathcal{R}\left(2,2,N,M\right)\hfill \\ ⋮\hfill & ⋮\hfill & \ddots \hfill & ⋮\hfill & ⋮\hfill & \ddots \hfill & ⋮\hfill \\ \mathcal{R}\left(N,M,1,1\right)\hfill & \mathcal{R}\left(N,M,2,1\right)\hfill & \dots \hfill & \mathcal{R}\left(N,M,N,1\right)\hfill & \mathcal{R}\left(N,M,1,2\right)\hfill & \dots \hfill & \mathcal{R}\left(N,M,N,M\right)\hfill \end{array}\right]$$

(17)

0.98,0.00,0.00 From Equation (17), it is clearly seen that the spatial colored noise is suppressed in $\mathcal{R}$, which indicates that the covariance tensor $\mathcal{R}$ is without the influence of both mutual coupling and spatial colored noise. Similar to the model in Equation (10), $\mathcal{R}$ can be rewritten as

$$\mathcal{R}={\mathcal{R}}_{\mathbf{S}\times 1}{\widehat{\mathbf{A}}}_{r\times 2}{\widehat{\mathbf{A}}}_{t\times 3}{\widehat{\mathbf{A}}}_{r\times 3}^{\ast }{\widehat{\mathbf{A}}}_t^{\ast }$$

(18)

where ${\mathcal{R}}_{\mathbf{S}}\in {\mathrm{C}}^{K\times K\times K\times K}$ is the Hermitian folding of ${\mathbf{R}}_{\tilde{\mathbf{S}}}$ in the same way as Equation (17), i.e., ${\mathcal{R}}_{\mathbf{S}}\left(:,k_1,:,k_2\right)=\left\{\begin{array}{c}0,k_1\ne k_2\hfill \\ {\mathbf{R}}_{\tilde{\mathbf{S}}}\left(k_1,k_2\right),k_1=k_2\hfill \end{array}\right.$, $k_1,k_2\in \left\{1,2,\dots ,K\right\}$. The decomposition of fourth-order tensor in Equation (18) is commonly known as the ’Tucker4’ decomposition, where the corresponding core tensor is ${\mathcal{R}}_{\mathbf{S}}$.

3.2. Tensor-Based Signal Subspace Estimation and Angle Estimation

In order to exploit the subspace technique for angle estimation, the signal/noise subspace is required to estimate firstly. Although the traditional SVD technique is easy to implement for estimating the signal/noise subspace, the multidimensional structure inherent in $\mathcal{R}$ is ignored. Fortunately, the HOSVD technique is an effective way to estimate the signal subspace with capturing the multidimensional structure information in tensor. The HOSVD of the measurement $\mathcal{R}$ is given by [31]

$$\mathcal{R}={\mathcal{G}}_{\times 1}{\mathbf{U}}_{1\times 2}{\mathbf{U}}_{2\times 3}{\mathbf{U}}_{3\times 4}{\mathbf{U}}_4$$

(19)

where $\mathcal{G}\in {\mathrm{C}}^{(N-2P)\times (M-2P)\times (N-2P)\times (M-2P)}$ is the associated core tensor, ${\mathbf{U}}_1\in {\mathrm{C}}^{(N-2P)\times (N-2P)}$, ${\mathbf{U}}_2\in {\mathrm{C}}^{(M-2P)\times (M-2P)}$, ${\mathbf{U}}_3\in {\mathrm{C}}^{(N-2P)\times (N-2P)}$, ${\mathbf{U}}_4\in {\mathrm{C}}^{(M-2P)\times (M-2P)}$ are unitary matrices, which are composed of the left singular vectors of the mode-n ($n\in \left\{1,2,3,4\right\}$) matrix unfolding of $\mathcal{R}$ as ${\left[\mathcal{R}\right]}_{\left(n\right)}={\mathbf{U}}_n{\mathsf{\Sigma }}_n{\mathbf{V}}_n^H$. Note that the rank of $\mathcal{R}$ is K, hence $\mathcal{R}$ can be approximated by its truncated HOSVD as

$${\mathcal{R}}_s={\mathcal{G}}_{s\times 1}{\mathbf{U}}_{1s\times 2}{\mathbf{U}}_{2s\times 3}{\mathbf{U}}_{3s\times 4}{\mathbf{U}}_{4s}$$

(20)

where ${\mathbf{U}}_{is}$ ($i=1,2,3,4$) is consist of the singular vectors corresponding to the K dominant singular values of ${\mathbf{U}}_i$. Note that $\mathcal{R}$ is Hermitian, we have ${\mathbf{U}}_{1s}={\mathbf{U}}_{3s}^{\ast }$, ${\mathbf{U}}_{2s}={\mathbf{U}}_{4s}^{\ast }$. ${\mathcal{G}}_s={\mathcal{R}}_{\times 1}{\mathbf{U}}_{1s\times 2}^H{\mathbf{U}}_{2s\times 3}^H{\mathbf{U}}_{3s\times 4}^H{\mathbf{U}}_{4s}^H$ accounting for the reduced core tensor. Insertion of ${\mathcal{G}}_s$ into Equation (20) yields

$${\mathcal{R}}_s={\mathcal{R}}_{\times 1}{\left({\mathbf{U}}_{1s}{\mathbf{U}}_{1s}^H\right)}_{\times 2}{\left({\mathbf{U}}_{2s}{\mathbf{U}}_{2s}^H\right)}_{\times 3}{\left({\mathbf{U}}_{1s}{\mathbf{U}}_{1s}^H\right)}_{\times 4}^{\ast }{\left({\mathbf{U}}_{2s}{\mathbf{U}}_{2s}^H\right)}^{\ast }$$

(21)

By Hermitian unfolding of ${\mathcal{R}}_s$, a new cross-correlation matrix $\tilde{\mathbf{R}}$ is formed to be

$$\widehat{\mathbf{R}}=\left[\left({\mathbf{U}}_{1s}{\mathbf{U}}_{1s}^H\right)\otimes \left({\mathbf{U}}_{2s}{\mathbf{U}}_{2s}^H\right)\right]{\mathbf{R}}_{\mathbf{Z}}{\left[\left({\mathbf{U}}_{1s}{\mathbf{U}}_{1s}^H\right)\otimes \left({\mathbf{U}}_{2s}{\mathbf{U}}_{2s}^H\right)\right]}^H$$

(22)

Since the rank of ${\mathbf{R}}_Z$ is K, it can be approximated by its truncated EVD as ${\mathbf{R}}_{\mathbf{Z}}\approx {\mathbf{U}}_s{\mathbf{\Lambda }}_s{\mathbf{U}}_s^H$, where ${\mathbf{U}}_s$ contains the singular vectors corresponding to the K-dominate singular values. Obviously, ${\mathbf{U}}_s$ spans the same subspace by the columns of $\left[{\tilde{\mathbf{A}}}_t\odot {\tilde{\mathbf{A}}}_r\right]$. Inserting ${\mathbf{R}}_{\tilde{\mathbf{Y}}}$ into Equation (22), we get

$$\widehat{\mathbf{R}}=\left[\left({\mathbf{U}}_{1s}{\mathbf{U}}_{1s}^H\right)\otimes \left({\mathbf{U}}_{2s}{\mathbf{U}}_{2s}^H\right){\mathbf{U}}_s\right]\mathsf{\Sigma }{\left[\left({\mathbf{U}}_{1s}{\mathbf{U}}_{1s}^H\right)\otimes \left({\mathbf{U}}_{2s}{\mathbf{U}}_{2s}^H\right){\mathbf{U}}_s\right]}^H$$

(23)

After truncating EVD of ${\mathbf{R}}_s$, a new signal subspace ${\widehat{\mathbf{U}}}_s$ is obtained as

$${\widehat{\mathbf{U}}}_s=\left[\left({\mathbf{U}}_{1s}{\mathbf{U}}_{1s}^H\right)\otimes \left({\mathbf{U}}_{2s}{\mathbf{U}}_{2s}^H\right)\right]{\mathbf{U}}_s$$

(24)

Equation (24) describes the relationship between the signal subspace obtained via matrix-based method and that estimated via tensor-based method, from which we can observe that ${\mathbf{U}}_s$ is weighted by $\left[\left({\mathbf{U}}_{1s}{\mathbf{U}}_{1s}^H\right)\otimes \left({\mathbf{U}}_{2s}{\mathbf{U}}_{2s}^H\right)\right]$. Due to the weighting effect, the noise in the tensor measurement is suppressed to obtain an enhanced signal subspace. Obviously, ${\widehat{\mathbf{U}}}_s$ spans the same space as ${\mathbf{U}}_s$. Therefore, there exists a full-rank matrix $\mathbf{T}$ that

$${\widehat{\mathbf{U}}}_s=\left[{\widehat{\mathbf{A}}}_t\odot {\widehat{\mathbf{A}}}_r\right]\mathbf{T}$$

(25)

3.3. Joint DOD and DOA Estimation

After obtaining the tensor-based signal subspace without the influence of both mutual coupling and spatial colored noise, the conventional ESPRIT method can be applied to estimate the DODs and DOAs [6,7]. Let $\widehat{\mathbf{a}}={\widehat{\mathbf{a}}}_t\left({\phi }_k\right)\odot {\widehat{\mathbf{a}}}_r\left({\theta }_k\right)$, and define ${\mathbf{J}}_{t1}=\left[{\mathbf{I}}_{\left(M-2P-1\right)},{\mathbf{0}}_{\left(M-2P-1\right)\times 1}\right]$, ${\mathbf{J}}_{t2}=\left[{\mathbf{0}}_{\left(M-2P-1\right)\times 1},{\mathbf{I}}_{\left(M-2P-1\right)}\right]$, ${\mathbf{J}}_{r1}=\left[{\mathbf{I}}_{\left(N-2P-1\right)},{\mathbf{0}}_{\left(N-2P-1\right)\times 1}\right]$ and ${\mathbf{J}}_{r2}=\left[{\mathbf{0}}_{\left(N-2P-1\right)\times 1},{\mathbf{I}}_{\left(N-2P-1\right)}\right]$. Then we can get the following rotational invariance properties

$$\left\{\begin{array}{c}{e}^{-j\pi sin{\phi }_k}\left[{\mathbf{J}}_{t1}\otimes {\mathbf{I}}_{N-2P}\right]\widehat{\mathbf{a}}=\left[{\mathbf{J}}_{t2}\otimes {\mathbf{I}}_{N-2P}\right]\widehat{\mathbf{a}}\hfill \\ e^{-j\pi sin{\theta }_k}\left[{\mathbf{I}}_{M-2P}\otimes {\mathbf{J}}_{r1}\right]\widehat{\mathbf{a}}=\left[{\mathbf{I}}_{M-2P}\otimes {\mathbf{J}}_{r2}\right]\widehat{\mathbf{a}}\hfill \end{array}\right.$$

(26)

Let ${\mathbf{\Phi }}_t=\mathrm{diag}\left\{e^{-j\pi sin{\phi }_1}and{e}^{-j\pi sin{\phi }_2},\dots ,e^{-j\pi sin{\phi }_K}\right\}$, ${\mathbf{\Phi }}_r=\mathrm{diag}\left\{e^{-j\pi sin{\theta }_1},e^{-j\pi sin{\theta }_2},\dots ,e^{-j\pi sin{\theta }_K}\right\}$. Combined with Equation (25) and Equation (26), we can get the least squares (LS) estimation of ${\mathbf{\Phi }}_t$ and ${\mathbf{\Phi }}_r$ via

$$\left\{\begin{array}{c}{\widehat{\mathbf{\Phi }}}_t={\left(\left[{\mathbf{J}}_{t1}\otimes {\mathbf{I}}_{N-2P}\right]{\widehat{\mathbf{U}}}_s\right)}^{†}\left[{\mathbf{J}}_{t2}\otimes {\mathbf{I}}_{N-2P}\right]{\widehat{\mathbf{U}}}_s\hfill \\ {\widehat{\mathbf{\Phi }}}_r={\left(\left[{\mathbf{I}}_{M-2P}\otimes {\mathbf{J}}_{r1}\right]{\widehat{\mathbf{U}}}_s\right)}^{†}\left[{\mathbf{I}}_{M-2P}\otimes {\mathbf{J}}_{r1}\right]{\widehat{\mathbf{U}}}_s\hfill \end{array}\right.$$

(27)

where the relations between ${\widehat{\mathbf{\Phi }}}_t,{\mathbf{\Phi }}_t$ and ${\widehat{\mathbf{\Phi }}}_r,{\mathbf{\Phi }}_r$ are ${\widehat{\mathbf{\Phi }}}_t=\mathbf{T}{\mathbf{\Phi }}_t{\mathbf{T}}^{-1}$, and ${\widehat{\mathbf{\Phi }}}_r=\mathbf{T}{\mathbf{\Phi }}_r{\mathbf{T}}^{-1}$, respectively. Once ${\widehat{\mathbf{\Phi }}}_t$ has been estimated, the estimation $e^{-j\pi sin{\widehat{\phi }}_k}$ can be obtained via the eigenvalue of ${\widehat{\mathbf{\Phi }}}_t$. Let $\widehat{\mathbf{T}}$ be the estimated eigenvectors of ${\widehat{\mathbf{\Phi }}}_t$, then the estimation $e^{-j\pi sin{\widehat{\theta }}_k}$ can be obtained via the diagonal element of $\widehat{\mathbf{T}}{\widehat{\mathbf{\Phi }}}_r{\widehat{\mathbf{P}}}^{-1}$. Finally, the k-th ($k=1,2,\dots ,K$) DOD and DOA are given by

$$\left\{\begin{array}{c}{\widehat{\phi }}_k=\arcsin [-\mathrm{angle}\left({\upsilon }_k\right)/\pi ]\hfill \\ {\widehat{\theta }}_k=\arcsin [-\mathrm{angle}\left({\nu }_k\right)/\pi ]\hfill \end{array}\right.$$

(28)

Because ${\widehat{\mathbf{\Phi }}}_t$ and ${\widehat{\mathbf{\Phi }}}_r$ share the same eigenvectors, the estimated DODs and DOAs are paired automatically.

The proposed method is summarized as follows.

Step.1

Stack the matched data into a third-order tensor $\mathcal{Y}$ as Equation (9);

Step.2

Construct two sub-tensors ${\mathcal{Y}}_1$ and ${\mathcal{Y}}_2$ through Equation (12) and Equation (14), then form the cross-covariance tensor $\mathcal{R}$ through Equation (16);

Step.3

Perform HOSVD of $\mathcal{R}$ to get ${\mathbf{U}}_{ns}$, ($n=1,2,3,4$), and get $\tilde{\mathbf{R}}$ via Equation (22);

Step.4

Perform EVD of ${\mathbf{R}}_s$ to get ${\tilde{\mathbf{U}}}_s$, obtain ${\widehat{\mathbf{\Phi }}}_t$ and ${\widehat{\mathbf{\Phi }}}_r$ via Equation (27), and compute EVD of ${\widehat{\mathbf{\Phi }}}_t$ to get ${\mathbf{\Phi }}_t$ and $\widehat{\mathbf{T}}$.

Step.5

Calculate ${\mathbf{\Phi }}_r$ via $\widehat{\mathbf{T}}{\widehat{\mathbf{\Phi }}}_r{\widehat{\mathbf{T}}}^{-1}$ and finally get the DODs and DOAs via Equation (28).

4. Remarks and Algorithm Analysis

4.1. Related Remarks

Remark 1.

The estimation algorithms in [24,25,26] are only effective with uniform white noise, while the denoising methods in [28,29,30,31,32,33,34] may suffer from mutual coupling. Consequently, their performances would degrade to deal with the co-existence mutual coupling and spatial colored noise. However, the proposed method can circumvent the limitations above to achieve better performance than these early reported algorithms.

Remark 2.

The main drawback of the MUSIC-Like algorithm is the ambiguity [23], since the columns in the mutual coupling matrices may be linear dependent. From Equation (12) we can observe that the direction submatrices exhibit a Vandermonde-Like structure to remove the ambiguity in the proposed method.

Remark 3.

The HOSVD of a tensor is calculated by SVDs of all the unfolded matrices. However, since ${\mathbf{U}}_{1s}={\mathbf{U}}_{3s}^{\ast }$, ${\mathbf{U}}_{2s}={\mathbf{U}}_{4s}^{\ast }$, we only need to compute the truncated SVDs of the mode-1 and mode-2 unfolding of $\mathcal{R}$, resulting in a significant computational saving.

4.2. Computation Complexity

Now we analysis the complexity of the proposed method in terms of the number of complex multiplications. The complexity of forming $\mathcal{R}$ is $M^2{N}^2\left(L-1\right)$ and performing truncated HOSVD of $\mathcal{R}$ requires $2{(M-2P)}^3{(N-2P)}^3$. The computation load of calculating $\widehat{\mathbf{R}}$ is $2{(M-2P)}^{2}K+2{(N-2P)}^{2}K+2{(M-2P)}^2{(N-2P)}^2$. Computing EVD of $\widehat{\mathbf{R}}$ needs $({(M-2P)}^3{(N-2P)}^3)$ complex multiplications. The computational complexity of Equation (27) is $2\left(M-2P-1\right)\left(N-2P\right)K^2+2\left(N-2P-1\right)\left(M-2P\right)K^2+2\left(K^3\right)$. Estimating DOD and DOA through ${\widehat{\mathbf{\Phi }}}_t$ and ${\widehat{\mathbf{\Phi }}}_r$ needs $K^3$ complex multiplications. I summary, the total complexities of the proposed method, the ESPRIT-Like method [24], the PM-Like method [25] and the HOSVD method [26] are presented in

Table 1. Comparison of the complexity in joint DOD and DOA estimation.

Method	Computational Complexity
ESPRIT-Like	$\begin{array}{c}{\left(M-2P\right)}^2{\left(N-2P\right)}^{2}L+2\left(M-2P-1\right)\left(N-2P\right)K^2+2\left(N-2P-1\right)\left(M-2P\right)K^2\hfill \\ +3K^3+2{\left(M-2P\right)}^3{\left(N-2P\right)}^3\hfill \end{array}$
PM-Like	$\begin{array}{c}{\left(M-2P\right)}^2{\left(N-2P\right)}^{2}L+2\left(M-2P\right)\left(N-2P\right)K^2+2\left(M-2P-1\right)\left(N-2P\right)K^2\hfill \\ +\left(M-2P\right)\left(N-2P\right)K\left(\left(M-2P\right)\left(N-2P\right)-K\right)+2\left(N-2P-1\right)\left(M-2P\right)K^2+3K^3\hfill \end{array}$
HOSVD	$\begin{array}{c}{\left(M-2P\right)}^2{\left(N-2P\right)}^{2}K+0.5\left(M-2P-1\right)\left(N-2P\right)K^2\hfill \\ +0.5\left(N-2P-1\right)\left(M-2P\right)K^2+K^3+2{\left(N-2P\right)}^3{\left(M-2P\right)}^3\hfill \end{array}$
Proposed	$\begin{array}{c}{M}^2{N}^2\left(L-1\right)+2{(M-2P)}^{2}K+2{(N-2P)}^{2}K+2{(M-2P)}^2{(N-2P)}^2+\hfill \\ 2\left(M-2P-1\right)\left(N-2P\right)K^2+2\left(N-2P-1\right)\left(M-2P\right)K^2+\hfill \\ 3K^3+3{(M-2P)}^3{(N-2P)}^3+{(M-2P)}^3{(N-2P)}^3\hfill \end{array}$

. Clearly, the complexity of the proposed method is a slight heaver than the HOSVD method, the ESPRIT-Like method and the PM-Like method. However, it provides much better estimation performance than all these other methods, which will be shown in the simulation section.

5. Simulation Results

In the simulation, the bistatic MIMO radar is configured with $M=10$ transmit elements and $N=12$ receive elements. The transmit code length is $Q=256$ and the pulse repeat frequency is $f_s=20$ KHz. Suppose that there are $K=3$ uncorrelated targets located at the directions $({\theta }_1,{\phi }_1)=\left({-30}^{°},{10}^{°}\right)$, $({\theta }_2,{\phi }_2)=\left({20}^{°},0^{°}\right)$, $({\theta }_3,{\phi }_3)=\left(0^{°},and{-20}^{°}\right)$. The corresponding RCS are ${\alpha }_1={\alpha }_2={\alpha }_3=1$ and the Doppler shifts are ${\left\{f_k\right\}}_{k=1}^3=\{200,400,850\}$Hz, respectively. The spatial colored noise is modeled as a second-order autoregressive (AR) process with the coefficient $z=\left[1,-1,0.8\right]$ [31,33]. The SNR in the simulation is defined by $\mathrm{SNR}=10{log}_{10}{∥{\mathbf{X}}_l-{\mathbf{F}}_l∥}^2/{∥{\mathbf{F}}_l∥}^2\left[\mathrm{dB}\right]$, where ${\mathbf{X}}_l$ and ${\mathbf{F}}_l$ are the matrices in Equation (5). In the following simulations, two measures are applied for performance assessment [33]. The first one is the root mean square error (RMSE), defined as

$$\mathrm{RMSE}=\frac{1}{K}\sum _{k=1}^K\sqrt{\frac{1}{\zeta }\sum _{i=1}^{\zeta }\left\{{({\widehat{\theta }}_{i,k}-{\theta }_k)}^2+{({\widehat{\phi }}_{i,k}-{\phi }_k)}^2\right\}}$$

where ${\widehat{\theta }}_{i,k}$ and ${\widehat{\phi }}_{i,k}$ represent the estimations of ${\theta }_k$ and ${\phi }_k$ for the i th Monte Carlo trial, respectively. $\zeta$ is the total number of Monte Carlo trials. The other one is the probability of the successful detection (PSD) defined as $\mathrm{PSD}=\frac{D}{\zeta }\times 100\%$, where D denotes the total numbers of successful trial and a successful trial is recognized if the absolute errors of all the estimated angles are smaller than 0.1°.

In the first step simulation results from Figure 3, Figure 4 and Figure 5, we consider the transmit and receive arrays with a weak mutual coupling defined by the mutual coefficients ${\mathbf{c}}_t=[1,0.1174+j0.0577]$ and ${\mathbf{c}}_r=[1,-0.0121-j0.1029]$, respectively. The number of snapshots is 50. Figure 3 illustrates the estimation results of the proposed method with $\mathrm{SNR}=0\mathrm{dB}$, from which we can observe that the DODs and DOAs are accurately obtained and correctly paired, which proves the effectiveness of the proposed method.

Figure 4 and Figure 5 show the $\mathrm{RMSE}$ and the $\mathrm{PSD}$ performance comparisons of the proposed method with the ESPRIT-Like method [24], the PM-Like method [25] and the HOSVD method [26], respectively. It can be concluded from Figure 4 that the proposed method provides much better estimation perfprmance than the ESPRIT-Like method, the PM-Like method and the HOSVD method in the low SNR region. The reason is that the proposed method can eliminate the effect of the mutual coupling and spatially colored noise simultaneously, and the other methods only consider the effect of the mutual coupling. It is shown in Figure 5 that all the methods exhibit a $100\%$ successful detection in the high SNR region. As the SNR decreases, the PSD of each method starts to drop at a certain point, which is defined as the SNR threshold [33]. The proposed method provides lower PSD threshold and better detection performance than all the compared methods.

In the second step simulation results shown in Figure 6 to Figure 7, we consider the transmit and receive arrays with a strong mutual coupling, and the mutual coupling coefficients are fixed at ${\mathbf{c}}_t=[1,0.8+j0.5,0.2+j0.1]$, and ${\mathbf{c}}_r=[1,0.6+j0.4,0.1-j0.3]$. The number of snapshots is 50. Figure 6 and Figure 7 illustrate the $\mathrm{RMSE}$ and $\mathrm{PSD}$ curves of various methods versus $\mathrm{SNR}$. According to the results, the RMSE and PSD performances of all the methods gradually improved with the increasing SNR. However, the ESPRIT-Like method, the PM-Like method and the HOSVD method provide higher RMSE than the proposed method in low SNR region. On the other hand, the proposed method has estimation performance similar to that achieved by the HOSVD method in high SNR region. This is because that the spatially colored noise is not the important factor to effect the estimation performance when the SNR is high enough. From Figure 7, it is obvious that the proposed method has better PSD than the other methods, i.e., it outperforming all the compared methods.

6. Conclusions

In this paper, a robust covariance tensor-based angle estimation method is developed for bistatic MIMO radar with the co-existence mutual coupling and spatial colored noise. The proposed method can capture the multidimensional nature of the matched array data. At the same time, it can eliminate the effect of mutual coupling and colored noise in the tensor domain. As a result, the proposed algorithm provides better estimation performance than the existing estimation algorithms. The robustness and superiority of the proposed method are clearly verified by simulation results.