A Sparse-Based Off-Grid DOA Estimation Method for Coprime Arrays

Abstract

Recently, many sparse-based direction-of-arrival (DOA) estimation methods for coprime arrays have become popular for their excellent detection performance. However, these methods often suffer from grid mismatch problem due to the discretization of the potential angle space, which will cause DOA estimation performance degradation when the target is off-grid. To this end, we proposed a sparse-based off-grid DOA estimation method for coprime arrays in this paper, which includes two parts: coarse estimation process and fine estimation process. In the coarse estimation process, the grid points closest to the true DOAs, named coarse DOAs, are derived by solving an optimization problem, which is constructed according to the statistical property of the vectorized covariance matrix estimation error. Meanwhile, we eliminate the unknown noise variance effectively through a linear transformation. Due to finite snapshots effect, some undesirable correlation terms between signal and noise vectors exist in the sample covariance matrix. In the fine estimation process, we therefore remove the undesirable correlation terms from the sample covariance matrix first, and then utilize a two-step iterative method to update the grid biases. Combining the coarse DOAs with the grid biases, the final DOAs can be obtained. In the end, simulation results verify the effectiveness of the proposed method.

1. Introduction

Direction of arrival (DOA) estimation of multiple far-field narrowband sources is a vital and interesting topic in a signal processing field, which has wide application in many areas such as radar, sonar, radio astronomy and wireless communications, etc. [1,2,3,4]. As we all know, the DOA estimation problem can be solved efficiently by classical subspace-based methods such as multiple signal classification (MUSIC) [5] and estimation of signal parameters via rotational invariance technique (ESPRIT) [6] in conditions where the receiving arrays are mostly uniform arrays, such as uniform linear array (ULA) [7], uniform rectangular array (URA) [8] and uniform circular array (UCA) [9]. The inter-element spacing of these receiving arrays is required to be less than or equal to the half-wavelength of received signal. Therefore, the detection performance of these DOA estimation methods is greatly confined by the number of physical sensors. For example, a maximum of $M-1$ sources can be detected when these algorithms are applied to a ULA with M sensors. Accordingly, more sensors are needed in order to detect more sources and acquire the desirable estimation accuracy. However, a large number of sensors will increase the hardware cost and the difficulty of array calibration in practical applications. To overcome this challenge, some non-uniform array geometries, called sparse arrays, have been proposed, such as minimum redundancy arrays (MRAs) [10], nested arrays (NAs) [11,12] and coprime arrays (CPAs) [13,14,15]. Though MRAs can obtain more degrees of freedom (DOFs) through constructing an augmented covariance matrix, they have no closed form expressions for the optimal array configurations as well as the achievable DOFs for an arbitrary number of sensor elements. Unlike MRAs, the NAs with $O\left(N\right)$ physical sensors can achieve $O\left(N^2\right)$ DOFs and they have closed form expressions for the optimal sensors positions. However, mutual coupling arising from the dense subarray of NAs may exist, which will cause the DOA estimation performance degradation. To circumvent this problem, coprime arrays which can detect $O\left(MN\right)$ sources with only $O\left(M+N\right)$ physical sensors were proposed in [13], where M and N are coprime integers.

When the incident signals are received by the CPAs, a virtual array with extended aperture can be achieved by vectorizing the receiving data covariance matrix. Thus, more DOFs than sensors can be obtained. In order to exploit the extended DOFs for DOA estimation, many algorithms have been proposed. The spatial smoothing MUSIC (SS-MUSIC) algorithm for comprime arrays was first proposed in [16], which has robust estimation performance and can detect more sources than sensors in coarray domain. Unfortunately, it needs to know the number of sources in advance. To this end, a MUSIC-like subspace method was proposed in [17], in which the number of sources is a by-product of low rank denoising stage. In [18], a novel coarray ESPRIT-based DOA estimation algorithm was proposed to balance the contradiction between estimation performance and computation complexity. However, all of the aforementioned subspace methods can only utilize the consecutive difference coarray. Additionally, due to the application of spatial smoothing technique [19], half of the continuous DOFs are lost, which will result in a significant degradation of detection performance. Accordingly, in order to maximize the utilization of extended DOFs, some sparse representation methods for coprime arrays [20,21,22] were proposed, which can be directly applied to derive the signal power vector without requiring decorrelation and covariance matrix rank recovery operation. Generally speaking, the extended virtual array aperture can be fully used for DOA estimation in these sparse-based methods. Consequently, they can detect more sources than those subspace methods and own better detection performance. On the basis of [16], a least absolute shrinkage and selection operator (LASSO) algorithm with extended coprime array geometry [23] was proposed, which can provide more DOFs than [16]. Now, this extended coprime array geometry is used very widely due to its excellent performance. Though there are so many advantages for these sparse-based methods, there is still a disadvantage that they need to discretize the angle space and preset enough grid points. The precondition for estimating the DOAs successfully is that the locations of the sources must fall on the predefined grid points. However, in practical situations, no matter how fine the grids are, it is impossible that the sources always lie on the predefined grid points. Thus, the off-grid sources may cause dictionary mismatch problems. As a result, the recovery performance of these methods will be deteriorated.

In recent years, lots of methods [24,25,26,27,28,29,30,31,32] have been proposed for solving the off-grid problems. Zhang et al. [24] presented a block-sparse Bayesian algorithm to solve the grid mismatch problem, in which the noise variance can be normalized to 1 and thus its effect on the estimation performance can be reduced. In references [25,26], the joint sparsity between the original signals and the grid biases was used to solve the off-grid problem. In a previous study [27], a gridless sparse method was developed for off-grid/continuous DOA estimation by minimizing the reweighted atomic norm. In reference [28], by first-order Lagrange approximation, an off-grid signal model with approximate dictionary matrix is established. Then, the grid biases are derived in sparse Bayesian learning approach by using the joint sparsity between different snapshots of the received signal. In reference [29], a root sparse Bayesian learning method was proposed, in which there is no need to construct off-grid model. The grid points are iteratively updated by finding the roots of a polynomial. Based on Dai et al. [29], a grid evolution direction of arrival (GEDOA) estimation method [30], which includes two sub-processes, i.e., learning process and fission process, was proposed to solve the problem of more than one DOA in one initial grid interval and speed up the method proposed in reference [29]. Wu et al. [31] proposed two iterative methods, both of which update the signal power vector and off-grid biases alternately. Wang et al. [32] proposed a real-valued formulation of covariance vector-based relevance vector machine (CVRVM) technique, which is implemented in a real domain and has low computation complexity. However, these methods are all applied on the traditional uniform linear array and they do not utilize the increased DOFs provided by the difference coarray of coprime arrays. Thus, they cannot detect more sources than sensors. As mentioned before, virtual array aperture extension can be described effectively by the vectorized covariance matrix model. However, it is not rational to directly apply most sparse-based methods to this model as those methods are usually designed to work in the raw data domain. Due to the superiority of coprime arrays, some off-grid DOA estimation methods have been proposed for coprime arrays. In reference [33], an off-grid DOA estimation method based on the framework of sparse Bayesian learning was proposed for coprime arrays. The predefined grid points are directly updated by iteratively decreasing a surrogate function majorizing the given objective function. Sun et al. [34] proposed an iterative method to obtain the final DOA estimation by gradually amending the offset vector with introducing a convex function majorizing the objective function. Both of the methods in references [33,34] need to use the gradient descent method, in which the gradient descent coefficient is difficult to choose. If the gradient descent coefficient is too large, the DOA estimation accuracy will be reduced. On the contrary, if the gradient descent coefficient is too small, the convergence speed of the algorithms will be limited.

To deal with the above difficulties, a sparse-based off-grid DOA estimation method for coprime arrays is proposed in this paper. The proposed method includes two processes, i.e., coarse estimation process and fine estimation process. In the coarse estimation process, we consider the estimation error caused by finite snapshots effect and remove the redundant elements in the covariance vector (i.e., vectorized covariance matrix). In addition, we circumvent estimating the noise variance by using a linear transformation to remove the noise-related part in the covariance vector. This is due to the fact that the noise variance is not so easy to evaluate especially under the underdetermined scenarios. Through a series of linear transformations, the covariance matrix estimation error can be normalized to an identity matrix. Then, according to the statistical property of the vectorized covariance matrix estimation error, DOA estimation can be cast as a problem of recovering a nonnegative sparse vector with an extended steering vector. The grid points corresponding to the non-zero elements in the recovered sparse vector are coarse DOA estimation results for the true signals, which are taken as the predefined grid points required for the fine estimation process. In practical applications, the number of snapshots is finite, which will cause the sample covariance matrix deviating from the actual covariance matrix largely [35]. Therefore, in the fine estimation process, the sample covariance matrix is modified firstly and then the grid biases are derived by applying a two-step iterative technique to the vectorized modified covariance matrix. The final estimated DOAs can be obtained by integrating the coarse DOA estimation results and the grid biases.

The rest of the paper is organized as follows. The far-field narrowband signal model for coprime arrays is briefly introduced in Section 2. Section 3 formulates the proposed sparse DOA estimation method. Section 4 provides several numerical simulation results to show the superior estimation performance for the proposed method and Section 5 concludes this paper.

Notations: In this paper, italic lower-case (upper-case) bold characters are used to denote vectors (matrices). In particular, ${\mathit{I}}_K$ denotes the $K\times K$ identity matrix. Sets are denoted by capital letters in blackboard boldface. Specifically, the symbols $\mathbb{R}$, $\mathbb{C}$ and $\mathbb{Z}$ represent the sets of real numbers, complex numbers and integers, respectively. ${\left(·\right)}^{\ast }$, ${\left(·\right)}^{\mathrm{T}}$, ${\left(·\right)}^{\mathrm{H}}$ and ${\left(·\right)}^{-1}$ imply conjugate, transpose, conjugate transpose and inverse, respectively. $\mathrm{vec}\left(·\right)$ denotes the vectorization operator. The symbols ∘, ⊙ and ⊗ represent Hadamard product, Khatri–Rao product and Kronecker product, respectively. $\mathrm{diag}\left(\mathit{a}\right)$ stands for a diagonal matrix whose diagonal elements are taken from the given vector $\mathit{a}$, while $\mathrm{diag}\left(\mathit{A}\right)$ represents a column vector whose elements are the diagonal elements of the given matrix $\mathit{A}$. $\Re \left(·\right)$ means taking the real part of a complex value. The $l_1$-norm and $l_2$-norm are respectively denoted by ${∥·∥}_1$ and ${∥·∥}_2$. $\mathrm{tr}\left(·\right)$ is the trace operator. $\mathrm{E}\left[·\right]$ denotes the statistical expectation operator. $\mathcal{C}\mathcal{N}\left(\mathit{a},\mathit{B}\right)$ denotes a complex Gaussian distribution with mean vector $\mathit{a}$ and covariance matrix $\mathit{B}$.

2. Signal Model

As illustrated in Figure 1, consider the coprime arrays which are composed of two uniform linear subarrays. One includes N sensors with inter-element spacing $M\lambda /\phantom{M\lambda 2}2$, while the other owns $2M$ sensors with inter-element spacing $N\lambda /\phantom{N\lambda 2}2$, where $\lambda$ is the wavelength of the received signal. As described in Section 1, M and N are coprime integers. Without loss of generality, we assume that M is less than N. The two subarrays share the zeroth sensor, which is also the reference sensor. Thus, the total number of sensors in the coprime arrays is $2M+N-1$, which are located at

$$\begin{array}{cc}\hfill \mathbb{L}& =\left\{nM\lambda /\phantom{\lambda 2}2\left|n\in \mathbb{Z},0\le n\le N-1\right.\right\}\cup \left\{mN\lambda /\phantom{\lambda 2}2\left|m\in \mathbb{Z},0\le m\le 2M-1\right.\right\}\hfill \\ & =\left\{l_i\lambda /\phantom{\lambda 2}2,i=1,2,\cdots ,2M+N-1\right\},\hfill \end{array}$$

(1)

where $\mathbb{L}$ is sorted in ascending order with $l_1=0$ and $l_{2M+N-1}=\left(2M-1\right)N$.

Suppose K far-field narrowband signals are incident on the coprime arrays from directions $\left({\theta }_1,{\theta }_2,\cdots {\theta }_K\right)$, where ${\theta }_k\in \left[-\frac{\pi }{2},\frac{\pi }{2}\right],k=1,2,\cdots ,K$ is the direction of the k-th signal. Then, the received signal at time t can be expressed as

$$\mathit{x}\left(t\right)=\sum _{k=1}^K\mathit{a}\left({\theta }_k\right){\mathit{s}}_k\left(t\right)+\mathit{n}\left(t\right)=\mathit{A}\mathit{s}\left(t\right)+\mathit{n}\left(t\right).$$

(2)

In Equation (2), $\mathit{s}\left(t\right)={\left[s_1\left(t\right),s_2\left(t\right),\cdots ,s_K\left(t\right)\right]}^{\mathrm{T}}\in {\mathbb{C}}^{K\times 1}$ is the signal vector at time $t\left(0\le t\le T\right)$, which is independent from the signal vectors at other time instances. T is the number of snapshots. $\mathit{n}\left(t\right)={\left[n_1\left(t\right),n_2\left(t\right),\cdots ,n_{2M+N-1}\left(t\right)\right]}^{\mathrm{T}}$ is the noise vector at time t, the elements of which are assumed to be temporally and spatially independent and identically distributed (i.i.d.) random variables which satisfy the complex Gaussian distribution with zero mean and variance ${\sigma }_{}^2$. When the number of sampled snapshots is sufficiently large, the signal vectors can be assumed to be independent from the noise vectors. $\mathit{A}=\left[\mathit{a}\left({\theta }_1\right),\mathit{a}\left({\theta }_2\right),\cdots ,\mathit{a}\left({\theta }_K\right)\right]\in {\mathbb{C}}^{\left(2M+N-1\right)\times K}$ is the array manifold matrix, where $\mathit{a}\left({\theta }_k\right)$ is the steering vector for source k and it can be written as

$$\mathit{a}\left({\theta }_k\right)={\left[e^{j\pi l_{1}sin{\theta }_k},e^{j\pi l_{2}sin{\theta }_k},\cdots ,e^{j\pi l_{2M+N-1}sin{\theta }_k}\right]}^{\mathrm{T}}.$$

(3)

Since the number of sampled snapshots is finite in practical applications, the sample covariance matrix can be approximated as

$${\widehat{\mathit{R}}}_{xx}=\frac{1}{T}\sum _{t=1}^{T}x\left(t\right)x^{\mathrm{H}}\left(t\right).$$

(4)

3. Proposed Sparse-Based Off-Grid Direction of Arrival Estimation Method

For the sake of solving the problem that the incident signals are not always located on the predefined grid points, an off-grid DOA estimation method is proposed in this section. The method includes two parts: coarse estimation process and fine estimation process. The coarse estimation process is to determine the nearest grid point to the direction of each incident signal, while the fine estimation process is to find the bias between the nearest grid point and the true DOA.

3.1. Coarse Estimation Process

One of the tasks for the coarse estimation process is to estimate the positions of the grid points closest to the incident signals. In addition, it also provides the parameters needed for the fine estimation process.

Under the situations where the number of sampled snapshots is sufficiently large, the received data covariance matrix can be denoted as

$${\mathit{R}}_{xx}=\mathrm{E}\left[\mathit{x}\left(t\right){\mathit{x}}^{\mathrm{H}}\left(t\right)\right]=\mathit{A}{\mathit{R}}_s{\mathit{A}}^{\mathrm{H}}+{\sigma }_{}^2{\mathit{I}}_{2M+N-1}.$$

(5)

Here, ${\mathit{R}}_s=\mathrm{E}\left[\mathit{s}\left(t\right){\mathit{s}}^{\mathrm{H}}\left(t\right)\right]=\mathrm{diag}\left({\sigma }_1^2,{\sigma }_2^2,\cdots ,{\sigma }_K^2\right)$ is the theoretical signal covariance matrix and ${\sigma }_k^2\left(k=1,2,\cdots ,K\right)$ denotes the power of the k-th signal. By vectorizing the covariance matrix ${\mathit{R}}_{xx}$, the covariance vector can be obtained as

$$\mathit{r}=\mathrm{vec}\left({\mathit{R}}_{xx}\right)=\tilde{\mathit{A}}\mathit{p}+{\sigma }^2{\mathit{1}}_n,$$

(6)

where $\tilde{\mathit{A}}=\left[{\mathit{a}}^{\ast }\left({\theta }_1\right)\otimes \mathit{a}\left({\theta }_1\right),{\mathit{a}}^{\ast }\left({\theta }_2\right)\otimes \mathit{a}\left({\theta }_2\right),\cdots ,{\mathit{a}}^{\ast }\left({\theta }_K\right)\otimes \mathit{a}\left({\theta }_K\right)\right]$, $\mathit{p}={\left[{\sigma }_1^2,{\sigma }_2^2,\cdots ,{\sigma }_K^2\right]}^{\mathrm{T}}$ and ${\mathit{1}}_n=\mathrm{vec}\left({\mathit{I}}_{2M+N-1}\right)={\left[{\mathit{e}}_1^{\mathrm{T}},{\mathit{e}}_2^{\mathrm{T}},\cdots ,{\mathit{e}}_{2M+N-1}^{\mathrm{T}}\right]}^{\mathrm{T}}$. ${\mathit{e}}_i$ denotes a column vector whose elements are all zeros except that the i-th element is one. $\mathit{r}$ can be seen as the received data from the virtual array with extended array aperture, where the virtual sensors are located at

$$\mathbb{D}=\left\{d_i-d_j\left|d_i,d_j\in \mathbb{L},1\le i,j\le 2M+N-1\right.\right\}.$$

(7)

The sample covariance matrix is as described in Equation (4). Owing to the finite snapshots, the estimation error between the actual and sample covariance matrix needs to be considered. Thus, vectorizing the sample covariance matrix, we can get

$$\widehat{\mathit{r}}=\mathrm{vec}\left({\widehat{\mathit{R}}}_{xx}\right)=\tilde{\mathit{A}}\mathit{p}+{\sigma }^2{\mathit{1}}_n+\mathit{\epsilon },$$

(8)

where $\mathit{\epsilon }$ is the vectorized covariance matrix estimation error and it satisfies an asymptotic Gaussian distribution [36,37]

$$\mathit{\epsilon }=\mathrm{vec}\left({\widehat{\mathit{R}}}_{xx}^{}-{\mathit{R}}_{xx}\right)\sim \mathcal{C}\mathcal{N}\left(\mathbf{0},\frac{1}{T}{\mathit{R}}_{xx}^{\mathrm{T}}\otimes {\mathit{R}}_{xx}^{}\right).$$

(9)

The number of extended virtual sensors in Equation (6) is ${\left(2M+N-1\right)}^2$, while the number of non-redundant virtual sensors is $3MN+M-N$. These duplicate virtual sensors only provide unnecessary redundant information. Thus, for the sake of saving computation complexity, we remove the repeated rows of $\tilde{\mathit{A}}$ corresponding to the redundant virtual sensors and sort the remaining rows to construct a new virtual array manifold matrix $\mathit{B}$. Similarly, the same transformation is performed on $\widehat{r}$ to obtain the corresponding received data vector $\mathit{z}$, i.e.,

$$\mathit{z}=\mathit{B}\mathit{p}+{\sigma }^2\mathit{e}+\overline{\mathit{\epsilon }},$$

(10)

where $\mathit{e}$ is a column vector whose elements are all zeros except that the element at the center is one. It is obvious that $\overline{\mathit{\epsilon }}$ is different from $\mathit{\epsilon }$. The relationship between $\overline{\mathit{\epsilon }}$ and $\mathit{\epsilon }$ will be given later.

We define $\overline{\mathbb{D}}$ as the set, the elements of which are the locations of the virtual sensors corresponding to array manifold matrix $\mathit{B}$. It can be directly derived by subtracting the duplicates from the difference $\mathbb{D}$ and sorting the remaining entries in ascending order and can be denoted as

$$\overline{\mathbb{D}}=\left\{{\overline{d}}_i\lambda /\phantom{\lambda 2}2,i=1,2,\cdots ,3MN+M-N\right\},$$

(11)

where ${\overline{d}}_1=-\left(2M-1\right)N$ and ${\overline{d}}_{3MN+M-N}=\left(2M-1\right)N$. Let $\mathit{B}=\left[\mathit{b}\left({\theta }_1\right),\mathit{b}\left({\theta }_2\right),\cdots ,\mathit{b}\left({\theta }_K\right)\right]\in {\mathbb{C}}^{\left(3MN+M-N\right)\times K}$ and $\mathit{b}\left({\theta }_k\right)$ is the virtual steering vector for the k-th signal with the i-th element $e^{j\pi {\overline{d}}_{i}sin{\theta }_k}$, ${\overline{d}}_i\in \overline{\mathbb{D}}$.

According to [33], we can get the relationship between $\mathit{\epsilon }$ and $\overline{\mathit{\epsilon }}$:

$$\mathit{\epsilon }=\left(\mathit{\Phi }\otimes \mathit{\Phi }\right)\mathit{G}{\mathit{\Psi }}^{-1}\overline{\mathit{\epsilon }}.$$

(12)

For the convenience of notation expression, let $\mathit{F}=\left(\mathit{\Phi }\otimes \mathit{\Phi }\right)\mathit{G}{\mathit{\Psi }}^{-1}$ in Equation (12), where $\mathit{\Phi }\in {\left\{0,1\right\}}^{\left(2M+N-1\right)\times \left[\left(2M-1\right)N+1\right]}$ is a selection matrix, the i-th row of which contains all zeros but a single one at the $\left(l_i+1\right)$-th position. $\mathit{\Psi }\in {\left\{0,1\right\}}^{\left(3MN+M-N\right)\times \left[2N\left(2M-1\right)+1\right]}$ is also a selection matrix, the elements of which are all zeros in the i-th row except a single one at the $\left({\overline{d}}_i+2MN-N+1\right)$-th position. $\mathit{G}\in {\mathbb{R}}^{{\left[N\left(2M-1\right)+1\right]}^2\times \left[2N\left(2M-1\right)+1\right]}$ can be denoted as

$$\mathit{G}=\left[\begin{array}{lll}{\mathbf{0}}_{\left[N\left(2M-1\right)+1\right]\times \left[N\left(2M-1\right)\right]}\hfill & & {\mathit{I}}_{N\left(2M-1\right)+1}\\ {\mathbf{0}}_{\left[N\left(2M-1\right)+1\right]\times \left[N\left(2M-1\right)-1\right]}\hfill & {\mathit{I}}_{N\left(2M-1\right)+1}& {\mathbf{0}}_{\left[N\left(2M-1\right)+1\right]\times 1}\\ ⋮\hfill & ⋮& ⋮\\ {\mathit{I}}_{N\left(2M-1\right)+1}\hfill & & {\mathbf{0}}_{\left[N\left(2M-1\right)+1\right]\times \left[N\left(2M-1\right)\right]}\end{array}\right].$$

(13)

Let $\mathbf{\vartheta }=\left({\vartheta }_1,{\vartheta }_2,\cdots ,{\vartheta }_L\right)$ be the grid points sampled from $\left[-\frac{\pi }{2},\frac{\pi }{2}\right]$ with appropriate step size, where L is the total number of grid points and $L\gg K$ in general. Then, the virtual received signal model in Equation (10) can be sparsely represented as

$$\mathit{z}=\overline{\mathit{B}}\overline{\mathit{p}}+{\sigma }^2\mathit{e}+\overline{\mathit{\epsilon }},$$

(14)

where $\overline{\mathit{B}}=\left[\mathit{b}\left({\vartheta }_1\right),\mathit{b}\left({\vartheta }_2\right),\cdots ,\mathit{b}\left({\vartheta }_L\right)\right]\in {\mathbb{C}}^{\left(3MN+M-N\right)\times L}$ is the overcomplete dictionary matrix and $\overline{\mathit{p}}$ is the extension of $\mathit{p}$ with zeros filling in the positions where no signals stay in.

As mentioned before, ${\sigma }^2\mathit{e}$ has only one non-zero element at its own center. Therefore, to avoid calculating the noise variance, we remove the corresponding row in the received data vector $\mathit{z}$ by a linear transformation, so that Equation (14) can be transformed to a noiseless received signal model, which can be expressed as

$$\overline{\mathit{z}}=\mathit{J}\mathit{z}=\mathit{J}\overline{\mathit{B}}\overline{\mathit{p}}+\mathit{J}\overline{,}\mathit{\epsilon },$$

(15)

where $\mathit{J}\in {\mathbb{R}}^{\left(3MN+M-N-1\right)\times \left(3MN+M-N\right)}$ can be denoted as

$$\mathit{J}=\left[\begin{array}{ll}{\mathit{I}}_{\left[\left(3MN+M-N-1\right)/\phantom{\left(3MN+M-N-1\right)2}2\right]}& {\mathbf{0}}_{\left[\left(3MN+M-N-1\right)/\phantom{\left(3MN+M-N-1\right)2}2\right]\times \left[\left(3MN+M-N-1\right)/\phantom{\left(3MN+M-N-1\right)2}2+1\right]}\\ {\mathbf{0}}_{\left[\left(3MN+M-N-1\right)/\phantom{\left(3MN+M-N-1\right)2}2\right]\times \left[\left(3MN+M-N-1\right)/\phantom{\left(3MN+M-N-1\right)2}2+1\right]}\hfill & {\mathit{I}}_{\left[\left(3MN+M-N-1\right)/\phantom{\left(3MN+M-N-1\right)2}2\right]}\end{array}\right].$$

(16)

Combining Equations (9) with (12) and (15), we can get

$$\mathit{J}\overline{\mathit{\epsilon }}\sim \mathcal{C}\mathcal{N}\left(\mathbf{0},\mathit{J}{\mathit{F}}^{-1}\left({\mathit{R}}_{xx}^{\mathrm{T}}\otimes {\mathit{R}}_{xx}^{}\right){\left(\mathit{J}{\mathit{F}}^{-1}\right)}^{\mathrm{T}}/\phantom{\mathit{J}{\mathit{F}}^{-1}\left({\mathit{R}}_{xx}^{\mathrm{T}}\otimes {\mathit{R}}_{xx}^{}\right){\left(\mathit{J}{\mathit{F}}^{-1}\right)}^{\mathrm{T}}T}T\right).$$

(17)

From Equations (15) and (17), we can derive

$${\mathit{W}}^{-1/2}\left(\overline{\mathit{z}}-\mathit{J}\overline{\mathit{B}}\overline{\mathit{p}}\right)\sim \mathcal{C}\mathcal{N}\left(\mathbf{0},{\mathit{I}}_{3MN+M-N-1}\right),$$

(18)

where $\mathit{W}=\mathit{J}{\mathit{F}}^{-1}\left({\mathit{R}}_{xx}^{\mathrm{T}}\otimes {\mathit{R}}_{xx}^{}\right){\left(\mathit{J}{\mathit{F}}^{-1}\right)}^{\mathrm{T}}/T$ and ${\mathit{W}}^{-1/2}$ is the Hermitian square root of ${\mathit{W}}^{-1}$. Then,

$${∥{\mathit{W}}^{-1/2}\left(\overline{\mathit{z}}-\mathit{J}\overline{\mathit{B}}\overline{\mathit{p}}\right)∥}_2^2\sim As{\chi }^2\left(3MN+M-N-1\right).$$

(19)

Here, $As{\chi }^2\left(3MN+M-N-1\right)$ denotes asymptotic chi-square distribution with $\left(3MN+M-N-1\right)$ DOFs. Thus, coarse DOA estimation results can be derived by solving the optimization problem:

$$\underset{\overline{\mathit{p}}\ge 0}{min}{∥\overline{\mathit{p}}∥}_{1}s.t.{∥{\widehat{\mathit{W}}}^{-1/2}\left(\overline{\mathit{z}}-\mathit{J}\overline{\mathit{B}}\overline{\mathit{p}}\right)∥}_2^{}\le \xi ,$$

(20)

where $\widehat{\mathit{W}}=\mathit{J}{\mathit{F}}^{-1}\left({\widehat{\mathit{R}}}_{xx}^{\mathrm{T}}\otimes {\widehat{\mathit{R}}}_{xx}^{}\right){\left(\mathit{J}{\mathit{F}}^{-1}\right)}^{\mathrm{T}}/T$ is the estimation of $\mathit{W}$. $\xi$ can be uniquely determined by $\xi =\sqrt{chi2inv\left(1-\beta ,3MN+M-N-1\right)}$. According to experience, $\beta$ is usually set as ${10}^{-4}$.

After attaining $\overline{\mathit{p}}$, the grid points corresponding to the K maximum values of $\overline{\mathit{p}}$ are the coarse DOA estimation results, called coarse DOAs, which are denoted as ${\mathbf{\theta }}_{coarse}=\left({\widehat{\theta }}_1^{\left(1\right)},{\widehat{\theta }}_2^{\left(1\right)},\cdots ,{\widehat{\theta }}_K^{\left(1\right)}\right)$.

3.2. Fine Estimation Process

The following is the fine estimation process where the grid biases can be derived according to the coarse DOAs. First of all, we expand the sample covariance matrix ${\widehat{\mathit{R}}}_{xx}$ in Equation (4) as

$$\begin{array}{ll}{\widehat{\mathit{R}}}_{xx}\hfill & =\frac{1}{T}{\displaystyle \sum _{t=1}^T}\left(\mathit{A}\mathit{s}\left(t\right)+\mathit{n}\left(t\right)\right){\left(\mathit{A}\mathit{s}\left(t\right)+\mathit{n}\left(t\right)\right)}^{\mathrm{H}}\hfill \\ & =\mathit{A}\left\{\frac{1}{T}{\displaystyle \sum _{t=1}^T}\mathit{s}\left(t\right){\mathit{s}}^{\mathrm{H}}\left(t\right)\right\}{\mathit{A}}^{\mathrm{H}}+\frac{1}{T}{\displaystyle \sum _{t=1}^T}\mathit{n}\left(t\right){\mathit{n}}^{\mathrm{H}}\left(t\right)\hfill \\ & +\mathit{A}\left\{\frac{1}{T}{\displaystyle \sum _{t=1}^T}\mathit{s}\left(t\right){\mathit{n}}^{\mathrm{H}}\left(t\right)\right\}+\left\{\frac{1}{T}{\displaystyle \sum _{t=1}^T}\mathit{n}\left(t\right){\mathit{s}}^{\mathrm{H}}\left(t\right)\right\}{\mathit{A}}^{\mathrm{H}}.\end{array}$$

(21)

Comparing (5) with (21), we can easily find that the expansion of ${\widehat{\mathit{R}}}_{xx}$ in (21) contains four terms while ${\mathit{R}}_{xx}$ in (5) only contains two. The first two terms in (21) can be viewed as the estimates of ${\mathit{R}}_{xx}$, which correspond to the signal and noise components, respectively. The last two terms in (21) can be seen as the estimates of the correlation parts between signal and noise vectors. Generally, the signal vector $\mathit{s}\left(t\right)$ is assumed to be independent from the noise vector $\mathit{n}\left(t\right)$. Therefore, when the number of snapshots is sufficiently large, the last two terms in (21) are equal to zeros in theory. However, in practical applications, the sampled snapshots available are finite so that the undesirable correlation terms between signal and noise vectors exist, which may have significant values. The existence of these correlation terms will cause the estimate of the received data covariance matrix deviating from the actual one and further result in DOA estimation performance degradation. Accordingly, in order to improve the DOA estimation performance, we should estimate and remove these correlation terms first to construct a new modified sample covariance matrix.

From the signal model (2), we can derive the estimate of signal vector by minimizing the residual between observations and estimators, i.e.,

$$\widehat{\mathit{s}}\left(t\right)=arg\underset{\mathit{s}}{min}{∥x\left(t\right)-\widehat{\mathit{A}}\mathit{s}\left(t\right)∥}_2^2.$$

(22)

Here, $\widehat{\mathit{A}}=\left[\mathit{a}\left({\widehat{\theta }}_1^{\left(1\right)}\right),\mathit{a}\left({\widehat{\theta }}_2^{\left(1\right)}\right),\cdots ,\mathit{a}\left({\widehat{\theta }}_K^{\left(1\right)}\right)\right]\in {\mathbb{C}}^{\left(2M+N-1\right)\times K}$ is the new estimated array manifold matrix based on the coarse DOA estimation results with steering vector $\mathit{a}\left({\widehat{\theta }}_k^{\left(1\right)}\right)={\left[e^{j\pi l_{1}sin{\widehat{\theta }}_k^{\left(1\right)}},e^{j\pi l_{2}sin{\widehat{\theta }}_k^{\left(1\right)}},\cdots ,e^{j\pi l_{2M+N-1}sin{\widehat{\theta }}_k^{\left(1\right)}}\right]}^{\mathrm{T}}$ for source k, where $l_1,l_2,\cdots ,l_{2M+N-1}\in \mathbb{L}$.

Using the Least Square (LS) method to solve the optimization problem in (22), we can obtain the estimate of signal vector $\mathit{s}\left(t\right)$ as

$$\widehat{\mathit{s}}\left(t\right)={\left({\left(\widehat{\mathit{A}}{\widehat{\mathit{A}}}^{\mathrm{H}}\right)}^{-1}\widehat{\mathit{A}}\right)}^{\mathrm{H}}\mathit{x}\left(t\right).$$

(23)

Then, the estimate of noise vector $\mathit{n}\left(t\right)$ can be represented as

$$\widehat{\mathit{n}}\left(t\right)=\mathit{x}\left(t\right)-\widehat{\mathit{A}}\widehat{\mathit{s}}\left(t\right)={\mathit{P}}_{\widehat{\mathit{A}}}^{\perp }\mathit{x}\left(t\right),$$

(24)

where ${\mathit{P}}_{\widehat{\mathit{A}}}^{}=\widehat{\mathit{A}}{\left({\left(\widehat{\mathit{A}}{\widehat{\mathit{A}}}^{\mathrm{H}}\right)}^{-1}\widehat{\mathit{A}}\right)}^{\mathrm{H}}$ is an estimation for the projection matrix of signal component and ${\mathit{P}}_{\widehat{\mathit{A}}}^{\perp }=\mathit{I}-{\mathit{P}}_{\widehat{\mathit{A}}}^{}$ is an estimation for the projection matrix of noise component. Combining Equations (23) with (24), the third term in (21) can be written as

$$\begin{array}{c}\mathit{V}=\widehat{\mathit{A}}\left\{\frac{1}{T}{\displaystyle \sum _{t=1}^T}\widehat{\mathit{s}}\left(t\right){\widehat{\mathit{n}}}^{\mathrm{H}}\left(t\right)\right\}={\mathit{P}}_{\widehat{\mathit{A}}}^{}{\widehat{\mathit{R}}}_{xx}{\left({\mathit{P}}_{\widehat{\mathit{A}}}^{\perp }\right)}^{\mathrm{H}}.\end{array}$$

(25)

The fourth term in (21) is the conjugate transposition of the third one. Thus, the modified sample covariance matrix can be expressed as

$$\tilde{\mathit{R}}={\widehat{\mathit{R}}}_{xx}-\gamma \left(\mathit{V}+{\mathit{V}}^{\mathrm{H}}\right),$$

(26)

where $\gamma$ is the scale factor with a value between zero and one [35]. The value of $\gamma$ depends on the estimation accuracy of the undesirable correlation terms. Ideally, $\gamma$ should be equal to one when there are no errors in the estimates of the correlation terms. However, estimation errors are inevitable. Therefore, $\gamma$ takes a value close to one when $\mathit{V}$ in (25) has a small estimation error and a small value conversely if the estimate of $\mathit{V}$ is inaccurate. The modified covariance vector $\tilde{r}$ can be generated by vectorizing $\tilde{\mathit{R}}$ in (26). By extracting the entries in positions corresponding to the set $\overline{\mathbb{D}}$ from $\tilde{r}$, we can attain the covariance vector $\tilde{\mathit{z}}$ without redundant information. Here, the extended virtual array is consistent with that in (11).

Generally, no matter how dense the grid points are, the true signals are impossible to always fall on the predefined grid points. Therefore, it is necessary to introduce off-grid problem for improving the DOA estimation performance. The coarse DOAs are the grid points closest to the true signals. Let ${\delta }_k$ denote the grid bias for the k-th signal. Then, the grid biases for all K narrowband signals can be expressed as $\mathbf{\Delta }=\mathrm{diag}\left(\mathbf{\delta }\right)$, where $\mathbf{\delta }={\left[{\delta }_1,{\delta }_2,\cdots ,{\delta }_K\right]}^{\mathrm{T}}$. The interval between two adjacent grid points in the coarse estimation process is $r_g$, which determines the range of the grid bias is $\left[-\frac{r_g}{2},\frac{r_g}{2}\right]$. By first-order Lagrangian approximation, the steering vector of the kth signal can be approximated as $\tilde{\mathit{a}}\left({\theta }_k\right)\approx \tilde{\mathit{a}}\left({\widehat{\theta }}_k^{\left(1\right)}\right)+\tilde{\mathit{b}}\left({\widehat{\theta }}_k^{\left(1\right)}\right)\left({\theta }_k-{\widehat{\theta }}_k^{\left(1\right)}\right)$. Consequently, the off-grid model can be denoted as

$$\tilde{\mathit{z}}=\left(\tilde{\mathit{A}}+\tilde{\mathit{B}}\mathbf{\Delta }\right)\tilde{\mathit{p}}+{\sigma }^2{\mathbf{1}}_{3MN+M-N},$$

(27)

where $\tilde{\mathit{A}}=\left[\tilde{\mathit{a}}\left({\widehat{\theta }}_1^{\left(1\right)}\right),\tilde{\mathit{a}}\left({\widehat{\theta }}_2^{\left(1\right)}\right),\cdots ,\tilde{\mathit{a}}\left({\widehat{\theta }}_K^{\left(1\right)}\right)\right]\in {\mathbb{C}}^{\left(3MN+M-N\right)\times K}$ is the array manifold matrix for extended virtual array and $\tilde{\mathit{a}}\left({\widehat{\theta }}_k^{\left(1\right)}\right)$ is the steering vector for the grid point closest to the k-th signal with the i-th element $e^{j\pi {\overline{d}}_{i}sin{\widehat{\theta }}_k^{\left(1\right)}}$, ${\overline{d}}_i\in \overline{\mathbb{D}}$. $\tilde{\mathit{B}}=\tilde{\mathit{B}}\left({\theta }_{coarse}\right)=\left[\tilde{\mathit{b}}\left({\widehat{\theta }}_1^{\left(1\right)}\right),\tilde{\mathit{b}}\left({\widehat{\theta }}_2^{\left(1\right)}\right),\cdots ,\tilde{\mathit{b}}\left({\widehat{\theta }}_K^{\left(1\right)}\right)\right]\in {\mathbb{C}}^{\left(3MN+M-N\right)\times K}$ is the derivative of $\tilde{\mathit{A}}$ with respect to ${\mathbf{\theta }}_{coarse}$. $\tilde{\mathit{p}}\in {\mathbb{R}}^{K\times 1}$ is a nonnegative signal power vector.

Based on (27), a two-step iterative method is proposed for solving the grid biases. The first step is to fix the grid bias vector $\mathbf{\delta }$ and solve the optimization problem below:

$${\tilde{\mathit{p}}}^{\left(j+1\right)}=arg\underset{\tilde{\mathit{p}}\ge 0}{min}{∥\tilde{\mathit{p}}∥}_{1}s.t.{∥\tilde{\mathit{z}}-\left(\tilde{\mathit{A}}+\tilde{\mathit{B}}{\mathbf{\Delta }}^{\left(j\right)}\right)\tilde{\mathit{p}}∥}_2\le {\eta }_0.$$

(28)

The unconstrained optimization problem [38] of (28) can be expressed as

$${\tilde{\mathit{p}}}^{\left(j+1\right)}=arg\underset{\tilde{\mathit{p}}\ge 0}{min}\left[\frac{1}{2}{∥\tilde{\mathit{z}}-\left(\tilde{\mathit{A}}+\tilde{\mathit{B}}{\mathbf{\Delta }}^{\left(j\right)}\right)\tilde{\mathit{p}}∥}_2+{\eta }_0{∥\tilde{\mathit{p}}∥}_1\right],$$

(29)

where ${\eta }_0$ is a regularization parameter that can be adjustable to trade off the sparsity of $\tilde{\mathit{p}}$ and the least-squares error between the observations and estimators. The superscript $\left(j+1\right)$ represents the $\left(j+1\right)$-th iteration.

The optimization problem in (29) can be solved by CVX toolbox [39]. Then, turning to the second step, fix the obtained nonnegative vector ${\tilde{\mathit{p}}}^{\left(j+1\right)}$ and update the grid bias vector $\mathbf{\delta }$ by

$${\mathbf{\delta }}^{\left(j+1\right)}=arg\underset{\mathbf{\delta }}{min}{∥\tilde{\mathit{z}}-\left(\tilde{\mathit{A}}+\tilde{\mathit{B}}\mathbf{\Delta }\right){\tilde{\mathit{p}}}^{\left(j+1\right)}∥}_2^2.$$

(30)

To solve the optimization problem about $\mathbf{\delta }$, we first expand (30) as

$$\begin{array}{c}{∥\tilde{\mathit{z}}-\left(\tilde{\mathit{A}}+\tilde{\mathit{B}}\mathbf{\Delta }\right){\tilde{\mathit{p}}}^{\left(j+1\right)}∥}_2^2\hfill \\ ={\left(\tilde{\mathit{z}}-\tilde{\mathit{A}}{\tilde{\mathit{p}}}^{\left(j+1\right)}\right)}^{\mathrm{H}}\left(\tilde{\mathit{z}}-\tilde{\mathit{A}}{\tilde{\mathit{p}}}^{\left(j+1\right)}\right)-{\left(\tilde{\mathit{z}}-\tilde{\mathit{A}}{\tilde{\mathit{p}}}^{\left(j+1\right)}\right)}^{\mathrm{H}}\tilde{\mathit{B}}\mathbf{\Delta }{\tilde{\mathit{p}}}^{\left(j+1\right)}\hfill \\ \hspace{1em}\hspace{1em}-{\left(\tilde{\mathit{B}}\mathbf{\Delta }{\tilde{\mathit{p}}}^{\left(j+1\right)}\right)}^{\mathrm{H}}\left(\tilde{\mathit{z}}-\tilde{\mathit{A}}{\tilde{\mathit{p}}}^{\left(j+1\right)}\right)+{\left(\tilde{\mathit{B}}\mathbf{\Delta }{\tilde{\mathit{p}}}^{\left(j+1\right)}\right)}^{\mathrm{H}}\tilde{\mathit{B}}\mathbf{\Delta }{\tilde{\mathit{p}}}^{\left(j+1\right)}.\end{array}$$

(31)

Removing the terms that are not related to $\mathbf{\delta }$ in (31) and letting ${\tilde{\mathit{p}}}^{\left(j+1\right)}=\tilde{\mathit{p}}$, we can obtain

$$\begin{array}{c}-{\left(\tilde{\mathit{z}}-\tilde{\mathit{A}}\tilde{\mathit{p}}\right)}^{\mathrm{H}}\tilde{\mathit{B}}\mathbf{\Delta }\tilde{\mathit{p}}-{\left(\tilde{\mathit{B}}\mathbf{\Delta }\tilde{\mathit{p}}\right)}^{\mathrm{H}}\left(\tilde{\mathit{z}}-\tilde{\mathit{A}}\tilde{\mathit{p}}\right)+{\left(\tilde{\mathit{B}}\mathbf{\Delta }\tilde{\mathit{p}}\right)}^{\mathrm{H}}\tilde{\mathit{B}}\mathbf{\Delta }\tilde{\mathit{p}}\hfill \\ =-2\Re \left\{{\left(\tilde{\mathit{z}}-\tilde{\mathit{A}}\tilde{\mathit{p}}\right)}^{\mathrm{H}}\tilde{\mathit{B}}\mathbf{\Delta }\tilde{\mathit{p}}\right\}+{\tilde{\mathit{p}}}^{\mathrm{H}}{\mathbf{\Delta }}^{\mathrm{H}}{\tilde{\mathit{B}}}^{\mathrm{H}}\tilde{\mathit{B}}\mathbf{\Delta }\tilde{\mathit{p}}\hfill \\ =-2\Re \left\{\mathrm{tr}\left(\tilde{\mathit{p}}{\left(\tilde{\mathit{z}}-\tilde{\mathit{A}}\tilde{\mathit{p}}\right)}^{\mathrm{H}}\tilde{\mathit{B}}\mathbf{\Delta }\right)\right\}+\mathrm{tr}\left(\mathbf{\Delta }{\tilde{\mathit{B}}}^{\mathrm{H}}\tilde{\mathit{B}}\mathbf{\Delta }\tilde{\mathit{p}}{\tilde{\mathit{p}}}^{\mathrm{H}}\right)\hfill \\ =-2\Re \left\{{\left(\mathrm{diag}\left(\tilde{\mathit{p}}{\left(\tilde{\mathit{z}}-\tilde{\mathit{A}}\tilde{\mathit{p}}\right)}^{\mathrm{H}}\tilde{\mathit{B}}\right)\right)}^{\mathrm{T}}\mathbf{\delta }\right\}+{\mathbf{\delta }}^{\mathrm{T}}\left({\tilde{\mathit{B}}}^{\mathrm{H}}\tilde{\mathit{B}}\circ {\left(\tilde{\mathit{p}}{\tilde{\mathit{p}}}^{\mathrm{H}}\right)}^{\ast }\right)\mathbf{\delta }.\end{array}$$

(32)

For convenience, let ${\mathit{h}}^{\mathrm{T}}={\left(\mathrm{diag}\left(\tilde{\mathit{p}}{\left(\tilde{\mathit{z}}-\tilde{\mathit{A}}\tilde{\mathit{p}}\right)}^{\mathrm{H}}\tilde{\mathit{B}}\right)\right)}^{\mathrm{T}}$ and $\mathit{C}={\tilde{\mathit{B}}}^{\mathrm{H}}\tilde{\mathit{B}}\circ {\left(\tilde{\mathit{p}}{\tilde{\mathit{p}}}^{\mathrm{H}}\right)}^{\ast }$. Then, Equation (32) can be written as

$$-2\Re \left({\mathit{h}}^{\mathrm{T}}\mathbf{\delta }\right)+{\mathbf{\delta }}^{\mathrm{T}}\mathit{C}\mathbf{\delta }=-2\Re \left({\mathit{h}}^{\mathrm{T}}\right)\mathbf{\delta }+{\mathbf{\delta }}^{\mathrm{T}}\mathit{C}\mathbf{\delta }.$$

(33)

Let the derivative of Equation (33) with respect to $\mathbf{\delta }$ be equal to zero. Then, we can get the updated grid bias vector ${\mathbf{\delta }}^{\left(j+1\right)}$ as

$${\mathbf{\delta }}^{\left(j+1\right)}=\Re \left({\mathit{C}}^{-1}\mathit{h}\right).$$

(34)

As described before, the grid bias ${\delta }_k\left(k=1,2,\cdots ,K\right)$ is constrained in $\left[-\frac{r_g}{2},\frac{r_g}{2}\right]$, so we have

$${\delta }_k^{\left(j+1\right)}=\left\{\begin{array}{cc}-\frac{r_g}{2}& {\delta }_k^{\left(j+1\right)}<-\frac{r_g}{2},\\ \frac{r_g}{2}& {\delta }_k^{\left(j+1\right)}>\frac{r_g}{2},\\ {\delta }_k^{\left(j+1\right)}& otherwise.\end{array}\right.$$

(35)

After the iteration converges, the final estimate of the grid bias vector is obtained and can be denoted as ${\mathbf{\delta }}_{final}$. Consequently, the final DOA estimation results can be expressed as

$$\widehat{\mathbf{\theta }}={\mathbf{\theta }}_{coarse}+{\mathbf{\delta }}_{final}.$$

(36)

3.3. Summary of the Steps about the Proposed Method

Synthesizing the foregoing analysis, the steps of the proposed method can be summarized as follows:

Step 1:

Compute the sample covariance matrix ${\widehat{\mathit{R}}}_{xx}$ according to Equation (4);

Step 2:

By vectorizing ${\widehat{\mathit{R}}}_{xx}$, get $\widehat{\mathit{r}}$ according to Equation (8). Then, remove the repeated entries of $\widehat{\mathit{r}}$ and sort the remaining entries to get $\mathit{z}$ as Equation (10);

Step 3:

Represent $\mathit{z}$ sparsely according to Equation (14) and get the noiseless received signal model according to Equations (15) and (16);

Step 4:

Normalize the sample covariance matrix estimation error according to Equations (9), (12), (17) and (18). Then, solve the optimization problem (20) to get $\overline{\mathit{p}}$. The grid points corresponding to K maximum values of $\overline{\mathit{p}}$ are the coarse DOAs ${\mathbf{\theta }}_{coarse}$;

Step 5:

Get modified sample covariance matrix $\tilde{\mathit{R}}$ according to Equation (26);

Step 6:

Vectorize $\tilde{\mathit{R}}$ and construct the off-grid model according to Equation (27);

Step 7:

Fix the grid bias vector ${\mathbf{\delta }}^{\left(j\right)}$ and solve the optimization problem (29) to get ${\tilde{\mathit{p}}}^{\left(j+1\right)}$;

Step 8:

Fix ${\tilde{\mathit{p}}}^{\left(j+1\right)}$ and update grid bias vector ${\mathbf{\delta }}^{\left(j+1\right)}$ according to Equations (34) and (35);

Step 9:

Terminate the iterative process if convergence criteria $∥{\mathbf{\delta }}^{\left(j+1\right)}-{\mathbf{\delta }}^{\left(j\right)}∥/∥{\mathbf{\delta }}^{\left(j\right)}∥\le \tau$ is satisfied or the number of iterations exceeds the maximum one. Otherwise, return to Step 7 and continue the iteration process.

Output: 

The final grid bias vector ${\mathbf{\delta }}_{final}$ and DOA estimation results $\widehat{\mathbf{\theta }}={\mathbf{\theta }}_{coarse}+{\mathbf{\delta }}_{final}$.

4. Numerical Simulations

In this section, we perform several simulation experiments to demonstrate the excellent performance of the proposed method for DOA estimation, and also compare it with SS-MUSIC [16], low rank matrix denoising (LRD) [17], LASSO [23] and off-grid sparse Bayesian inference (OGSBI) [28]. In particular, for estimation accuracy comparision, the Cramer–Rao bound (CRB) for coprime arrays [40] is included. The coprime arrays considered here are composed of ten sensors with two coprime integers $M=3$ and $N=5$. The positions of sensors in the first subarray are $\left[0,3,6,9,12\right]\frac{\lambda }{2}$ and the others in the second subarray are $\left[0,5,10,15,20,25\right]\frac{\lambda }{2}$. The zeroth sensor is assumed to be the reference element, which is shared by the two subarrays. The scale factor $\gamma$ in (26) is determined by traversal simulation experiments [41]. The regularization parameter ${\eta }_0$ in (29) is set as 0.3. The iterative convergence criterion is $∥{\mathbf{\delta }}^{\left(j+1\right)}-{\mathbf{\delta }}^{\left(j\right)}∥/∥{\mathbf{\delta }}^{\left(j\right)}∥\le {10}^{-4}$ and the maximum number of iterations is 60. The predefined grid points for LASSO, OGSBI and the coarse estimation process in the proposed method are sampled from $[-{90}^{\circ },{90}^{\circ }]$ with sampling interval being $1^{\circ }$. For fair comparison, the peak search process in SS-MUSIC and LRD is performed with the same step size of $1^{\circ }$ in $[-{90}^{\circ },{90}^{\circ }]$.

4.1. Detection Performance

In this subsection, the detection performance of the proposed method is compared with that of the methods OGSBI, LASSO and LRD. All of these methods can detect more sources than sensors with using coprime arrays. In theory, LRD can only identify at most 17 sources under the coprime arrays set above. Therefore, for convenience of comparison, we consider 17 randomly distributed narrowband uncorrelated sources here, which are located at $-{50.88}^{\circ }$, $-{45.36}^{\circ }$, $-{40.35}^{\circ }$, $-{33.64}^{\circ }$, $-{27.60}^{\circ }$, $-{22.12}^{\circ }$, $-{15.46}^{\circ }$, $-{9.80}^{\circ }$, $-{4.18}^{\circ }$, ${1.80}^{\circ }$, ${9.23}^{\circ }$, ${14.74}^{\circ }$, ${20.80}^{\circ }$, ${25.96}^{\circ }$, ${34.46}^{\circ }$, ${38.39}^{\circ }$ and ${44.51}^{\circ }$. The number of sampled snapshots is 800 and the Signal-to-Noise Ratio (SNR) is set as 0 dB. The scale factor $\gamma$ is 0.5. The spatial spectra of the four algorithms are shown in Figure 2a–d, respectively. In these figures, the red dashed line indicates the true direction of the incident signal, and the solid blue line indicates the estimated spectral line.

From Figure 2, we can see that OGSBI, LASSO and the proposed method can detect all 17 sources successfully. However, there are still some small spurious peaks in Figure 2a for OGSBI and Figure 2c for LASSO. Furthermore, from Figure 2b, it is easy to find that LRD fail to detect all sources. In its spatial spectrum, several peaks are missed and some of the existing spectral peaks deviate from the true signals a lot. Therefore, it can be concluded that the detection performance of the proposed method is superior to the other three algorithms.

4.2. Resolution Ability

In this subsection, the resolution ability of the proposed method is still compared with that of the three algorithms mentioned in Section 4.1. We test the resolution abilities of these algorithms by detecting two closely spaced sources which are located at $-{31.25}^{\circ }$ and $-{29.64}^{\circ }$, respectively. Here, the SNR is set as 0 dB and the number of sampled snapshots is set as 800. Additionally, the scale factor $\gamma$ is identical with that in Section 4.1.

From Figure 3, it is obvious to find that there is only one peak in the spatial spectrums for LRD and LASSO, which means that both of them can not identify the two signals successfully. In addition, from Figure 3d, we can find that the proposed method identifies the two signals successfully and its estimated DOAs about the two signals are $-{31.05}^{\circ }$ and $-{29.3}^{\circ }$, respectively. Though there are errors between the estimated DOAs and the true DOAs, the errors are very small and within an acceptable range. From Figure 3a, it can be seen that OGSBI can also identify the two signals, but the estimation error for OGSBI is larger than the proposed method. Therefore, it can be concluded that the resolution ability of the proposed method is the best among all four of the algorithms.

4.3. Estimation Accuracy

In this subsection, the estimation accuracy of each algorithm is reflected by its root-mean-square error (RMSE), which is defined as

$$\mathrm{RMSE}=\sqrt{\frac{1}{QK}\sum _{q=1}^Q\sum _{k=1}^K{\left({\widehat{\theta }}_{k,q}-{\theta }_k\right)}^2},$$

(37)

where Q is the total number of Monte Carlo trials, ${\widehat{\theta }}_{k,q}$ represents the estimate of the k-th source in the q-th trial. For each scenario, we conduct 1000 Monte Carlo trials to get the corresponding RMSE. The estimation accuracy of the proposed method is compared with that of SS-MUSIC, LRD, LASSO, OGSBI and CRB. Although SS-MUSIC and LRD can theoretically identify at most 17 sources, in reality, the peak leakage phenomenon often occurs when applying them to estimate 16 or 17 sources impinging on the coprime arrays. Therefore, we consider 15 narrowband uncorrelated sources that come from the directions of $-{55}^{\circ }+\Delta \theta$, $-{47}^{\circ }+\Delta \theta$, $-{39}^{\circ }+\Delta \theta$, $-{31}^{\circ }+\Delta \theta$, $-{23}^{\circ }+\Delta \theta$, $-{15}^{\circ }+\Delta \theta$, $-7^{\circ }+\Delta \theta$, $1^{\circ }+\Delta \theta$, $9^{\circ }+\Delta \theta$, ${17}^{\circ }+\Delta \theta$, ${25}^{\circ }+\Delta \theta$, ${33}^{\circ }+\Delta \theta$, ${41}^{\circ }+\Delta \theta$, ${49}^{\circ }+\Delta \theta$ and ${57}^{\circ }+\Delta \theta$, where $\Delta \theta$ is chosen randomly from the interval $\left[-{0.5}^{\circ },{0.5}^{\circ }\right]$ in each trial to remove the possible prior information contained in the predefined direction set in the coarse estimation process.

In the first experiment of this subsection, we fix the number of snapshots at 800 and vary SNR from $-10\mathrm{dB}$ to $20\mathrm{dB}$. When SNR is less than or equal to $2\mathrm{dB}$, $\gamma$ is set as 0.5. Otherwise, $\gamma$ is set as 0.7. Figure 4 shows the RMSE curves of the five algorithms and CRB as a function of SNR, respectively.

From Figure 4, we can see that the estimation accuracy is improved with the increase of SNR for all algorithms. It is obvious that the estimation performance of the proposed method outperforms the other four methods. The reason is that spatial smoothing technique causes the extended DOF loss for SS-MUSIC and LRD. Though LASSO can utilize all of the extended DOFs, the assumption that incident signals must fall on the predefined grid points causes its estimation performance degradation. The algorithm OGSBI needs to update the noise variance in each iteration, which makes it sensitive to the noise. In addition, the covariance matrix estimation error and the correlation terms between signal and noise vectors are not considered in OGSBI, which is another reason that the proposed method has better estimation performance than OGSBI.

For the second experiment, the SNR is fixed at $0\mathrm{dB}$ and RMSE is a function of snapshots, the number of which varies from 10 to 1500. When the number of snapshots is less than or equal to 800, $\gamma$ is set as 0.5. Otherwise, $\gamma$ is set as 0.8. The RMSE values for all algorithms and CRB versus snapshots are shown in Figure 5.

It is obvious from Figure 5 that the estimation accuracy is improved with the increase of snapshots for all algorithms. For the same reason as described above, the estimation performance of SS-MUSIC, LRD, LASSO and OGSBI is inferior to the proposed method. It can be seen from Figure 5 that when the number of snapshots changes from 10 to 1500, the estimation accuracy of the proposed method is getting better and better than the other four algorithms and finally outperforms the other four algorithms notably by a large margin.

In addition, to have an intuitive understanding on the computation complexity, we compute the average execution time of each algorithm on an Intel Core i7-7700@3.6GHz, 16G RAM PC. Under the same conditions, it takes 0.006 s for SS-MUSIC, 0.670 s for LRD, 0.285 s for LASSO, 1.343 s for OGSBI and 1.599 s for the proposed method. It can be found that SS-MUSIC has the shortest running time. The algorithm OGSBI and the proposed method are a little time-consuming. However, it is worth noting that the simulation results show that the estimation performance of the proposed method is the best among all five of the algorithms. According to the characteristics of each algorithm, they can be applied in different scenarios, respectively. When applied in scenarios where the real-time demand is not very high but the estimation accuracy demand is high, the proposed method is the preferred choice. Conversely, the other several comparison algorithms can be used in the scenarios with high real-time requirements but low estimation accuracy requirements. As we all know, the trade-off between the estimation accuracy and the computation complexity has always been an intractable problem for nearly all algorithms. Therefore, reducing the computation complexity on the basis of the proposed method will be a research direction for us in the future.

5. Conclusions

In this paper, we have proposed a sparse-based off-grid DOA estimation method for coprime arrays. By using coprime arrays, we can obtain extended virtual array aperture and increased DOFs. Accordingly, the proposed method can detect more sources than sensors. In practical applications, the existence of the estimation error between the sample and actual covariance matrix is inevitable due to finite snapshots. According to the statistical property of the vectorized covariance matrix estimation error, we construct an optimization problem about the signal power vector in the coarse estimation process so as to obtain the positions of the grid points closest to the true DOAs. In the fine estimation process, we first remove the undesirable correlation terms between the signal and noise vectors to get the modified sample covariance matrix. Then, for solving the grid mismatch problem, an off-grid model with vectorized modified sample covariance matrix is constructed. By applying the two-step iterative method to the off-grid model, the grid biases can be estimated precisely. Combining the coarse estimation results with the estimated grid biases, the final DOA estimation results can be accurately acquired. By considering the covariance matrix estimation error and the correlation terms between signal and noise vectors, the proposed method can achieve better DOA estimation performance than those traditional algorithms that do not consider the finite snapshots effect. In addition, the introduction of an off-grid model solves the grid mismatch problem existing in the traditional on-grid model successfully. Numerical simulation experiments confirmed the superiority and effectiveness of the proposed method.