Gridless Sparse Direction Finding Method for Correlated Signals with Gain-Phase Errors

Abstract

In practical array systems, the gain-phase errors among antennas degrade the performance of direction finding significantly. In this paper, a novel sparse system model for direction of arrival (DOA) estimation in the scenario with gain-phase errors is proposed by exploiting the signal sparsity in the spatial domain. In contrast to the existing sparse-based methods using the grids to construct the dictionary matrix, a novel gridless method based on atomic norm and convex optimization is proposed, where the gain-phase errors are described by a diagonal matrix. With the Schur complement, a semidefinite programming is formulated from the optimization problem, and can be solved efficiently. With the gain-phase errors, the corresponding Cram’er-Rao lower bound (CRLB) of direction finding is derived as an estimation benchmark. Simulation results show that the proposed method performs better than the state-of-the-art methods in the scenario with correlated signals and gain-phase errors.

1. Introduction

Direction of arrival (DOA) estimation, which is of great significance in array signal processing [1,2], is solved by estimating the spatial spectrum, such as conventional beamforming (CBF) method [3]. However, the “Rayleigh limit” [4] restrains the angular resolution and the targets within one beam cannot be distinguished by the CBF method. Many super-resolution methods such as Pisarenko spectral analysis method [5], Capon minimum variance method [6] and Burg maximum entropy method [7] have been proposed to break through the Rayleigh limit [4] and obtain a higher angular resolution. To improve the estimation performance, some subspace-based methods, such as multiple signal classification (MUSIC) [8] and estimating signal parameters via rotational invariant techniques (ESPRIT) [9] have also been proposed. Additionally, an asymptotically unbiased and efficient method named the least squares-ESPRIT [10] is used widely and the maximum likelihood (ML) [11,12] method achieves a better performance in the scenario with low signal-to-noise ratio (SNR).

By exploiting the signal sparsity, the DOA estimation performance can be improved further. With the knowledge of compressed sensing (CS), unknown signals can be acquired and reconstructed by dealing with underdetermined linear systems. Correspondingly, some CS-based DOA estimation methods are proposed and can obtain a better performance. Focal underdetermined system solver (FOCUSS) method [13] converts the problem of DOA estimation into that of sparse recovery firstly. Then, combining the CS theory and subspace algorithm, ${\ell }_1$-SVD method [14,15,16,17] obtains the signal subspace using SVD decomposition. A sparse iterative covariance-based estimation (SPICE) method is proposed in [18], and the method minimizes a covariance matrix fitting criterion to obtain the DOAs. ${\ell }_1$ penalty also provides an approach to solve the DOA estimation problem, and develops many methods. Basis pursuit (BP) method [19] and match pursuit (MP) [20] are 2 typical ${\ell }_1$-based methods. However, the sparse-based methods cannot deal with the scenario with low SNR.

When correlations exist between signals, the DOA estimation performance degrades seriously [21,22]. At concerns the cost of losing the array aperture, the forward/backward spatial smoothing (FBSS) method proposed by [23,24] can solve the problem efficiently. To avoid the aperture loss, non-dimensionality reduction processing [25] introduces a toeplitz matrix composed of the received data, which can also achieve decoherence. However, the computational complexity increases. Then, ref. [26] proposes an ESPRIT-like method, which is computational efficient and can be applied to any single row of an array output covariance matrix. In the ESPRIT-like method, the number of signals determines the rank of the toeplitz matrix.

Specifically, the spatial frequency of an array is a traditional representation of DOA. As CS theory shown in [27], without knowing any locations and gains of grids, the signals can be exactly recovered by using a convex optimization method in the scenario with the frequencies laying on the discretized grids, and convex optimization is firstly introduced in [28] as near-optimal way to obtain frequency information. However, in most cases, the sparse signals are continuous and the off-grid errors are introduced by applying the method with discretized grids. When denser grids are used to fit the continuous characteristic and to solve the DOA estimation problem, the correlation between adjacent steering vectors (or atoms) becomes higher which further degrades the performance of sparse signal recovery [29,30].

Thus, to avoid describing the DOA problem by discretizing the spatial domain, another general framework of converting the DOA estimation problem into a convex optimization method, which appeared in [31,32], is proposed as the atomic norm-based method. In the problem of sparse signal estimation, the atomic norm and a ${\ell }_1$ norm are considered equivalent [33], and nuclear norm are generalized to reconstruct the low-rank matrix in [34]. In [35], the atomic norm is applied to solve the problem of a spectrally-sparse signal recovery in case of a small number of consecutive samples from the time domain. Besides, known as the reweighted atomic norm minimization, the nonconvex sparse method is proposed in [36]. Without the acknowledge of model order, the enhanced matrix completion (EMaC) method based on the structured matrix completion is proposed in [37]. An enhanced covariance matrix in sparse representation method is used in [38] to dealing with the coherent signals for the DOA estimation problem. Ref. [39,40,41] developed a toeplitz matrix constraint for the semidefinite program (SDP) of the atomic norm-based method with single sampled signals. Then, the problem with multiple sampled signals is discussed in ref. [42].

Additionally, after establishing a model of atomic norm, ref. [43] further defines an infinite atom set, and applies the dual problem to estimate the signal frequency. Without using the toeplitz matrix, a new method is proposed in. [35], and the dual problem and SDP method are also used to recover the sparse signal. Based on existing work, with an order of $rlognlogr$ random samples, sparse signals can be recovered with high probability in the case that the spatial frequencies are separated by no more than $4/(n-1)$, where the number of sampled signals is denoted by n, and r denotes the sparsity.

The inconsistence of antennas degrades the DOA estimation performance too [44]. Some work has been done to deal with the unknown mutual coupling effect [10,45,46]. In this paper, we concentrate on compensating for the effects of the gain and phase errors. Before our study, ref. [47] introduced a calibration matrix composed of the unknown gain, phase and mutual-coupling coefficients, the eigen structure [48] and eigen decomposition [49], consisting of the covariance and conjugate matrices is used to get better estimation performance in the scenario with gain and phase errors. Later, a simultaneous orthogonal matching pursuit (SOMP)-based method has been proposed [50], which introduces a diagonal matrix composed of phase errors and estimate the DOAs and phase errors iteratively. However, the computational complexity is so high that the above methods are not applicable to a practical scenario. Some articles work to solve the gain and phase errors disturb at the same time. Ref. [51] exploits the structural features of the uniform circular array covariance matrix and ref. [52] introduces a universal method for self-calibration based on the minimize cost functionto obtain a better estimation performance.

In this paper, the DOA estimation problem with unknown gain and phase errors among antennas is investigated. With gain-phase errors being described as a diagonal matrix, the CS-based system model is formulated and the corresponding DOA estimation problem is transmitted into sparse reconstruction problem.Then, a novel atomic norm minimization approach is proposed to improve performance of the DOA estimation in the scenario with gain-phase uncertainties and can deal with correlated signals. We focus on the dual problem of the atomic norm and solve the associate convex optimization problem using SDP without calculating the gain and phase errors. In contrast to the traditional atomic norm-based method, a new matrix constraint is proposed in the dual problem instead of the toeplitz matrix in the existing methods. Then, we derive the Cram’er-Rao lower bound (CRLB) [53] of the system model. Finally, the proposed method is compared with the classical methods including SOMP, MUSIC, the off-grid sparse Bayesian inference method (OGSBI) and toeplitz-based atomic norm (TAN). To conclude, our contributions in this papers can be summarized as follows:

• The system model with gain-phase uncertainties: The system model is formulated, where the gain-phase uncertainties among antennas is formulated by a diagonal matrix.
• The proposed method for direction finding with gain-phase uncertainties: A novel atomic norm-based gridless method is proposed. In this method, the DOAs are estimated without knowing the gain and phase errors. The proposed method focus on the dual problem of the atomic norm and using SDP method to recover the sparse signals. The proposed method behaves better than the existing methods in the case with correlated signals and gain-phase uncertainties.
• The theoretical expression of SDP: The convex optimization problem is solved using SDP in the proposed method. By analyzing the structure of the dual problem, we theoretically derive the expression of the SDP.

The organization of the paper is as follows. Section 2 introduces the CS-based system model with gain-phase uncertainties. Section 3 describes the proposed gridless method with gain-phase uncertainties. Section 4 derives the CRLB of DOA estimation with correlated signals. Section 5 gives the simulation results. Finally, Section 6 concludes the paper.

Notations: Matrices are denoted by capital letters in boldface, and vectors are denoted by lowercase letters in boldface. ${\parallel ·\parallel }_F$, ${∥·∥}_{\mathcal{A}}$, ${\parallel ·\parallel }_2$ and ${\left(·\right)}_{\mathrm{H}}$ denote the Frobenius norm, Atomic norm, ${\ell }_2$ norm and Hermitian transpose, respectively. A $N\times M$ matrix with complex entries is denoted by ${\mathbb{C}}^{N\times M}$. We use ${\mathit{I}}_N$ to denote the $N\times N$ identity matrix. The symbol $diag\left\{\mathit{a}\right\}$ denotes a diagonal matrix which diagonal entry is the vector $\mathit{a}$, and $a_n$ is the $n$-th entry of the vector $\mathit{a}$. The symbol $vec\left\{\mathit{B}\right\}$ denotes linearly transmitting the matrix $\mathit{B}$ into a column vector, and $Tr\left\{\mathit{B}\right\}$ denotes the trace of $\mathit{B}$. For a complex $c+jd$, $\mathcal{R}\{c+jd\}$ denotes the real part c. $E\left\{·\right\}$ denotes the mathematical expectation. ⊗ denotes the Kronecker product.

2. Problem Formulation

2.1. Ideal System Model

The ULA system for direction finding shown in Figure 1 has N antennas and K target signals, d denotes the intersensor spaces of receivers. The antenna considered in our article is an omnidirectional antenna. The target signals are reflected by K slowly moving targets. The DOA of the k-th target $k=1,2,\dots ,K$ is described by ${\theta }_k$. We consider the situation with M samples, and the received signal are sampled with a sampling interval $T_s$. For all antennas, the sampled signals during the $m{T}_s$ sampling time can be expressed as ${\mathit{y}}_m\triangleq {[y_{0,m},y_{1,m},\dots ,y_{N-1,m}]}^T$, and the noise vector is defined as ${\mathit{w}}_m\triangleq {[w_{0,m},w_{1,m},\dots ,w_{N-1,m}]}^T$, while for all target signals, the signal vector is denoted by ${\mathit{s}}_m\triangleq {[s_{0,m},s_{1,m},\dots ,s_{K-1,m}]}^T$. The DOA vectors are given by $\mathit{\theta }={[{\theta }_1,{\theta }_2\dots {\theta }_K]}^T$ (${\theta }_k\in [-\pi ,\pi )$) for the k-th antenna. In contrast to the independent signals in the traditional methods, we consider the DOA estimation problem for the correlated signals.

Assuming that our target signals are far-field, the signals received can be written as:

$$\begin{array}{c}\hfill \mathit{Y}=\mathit{A}\mathit{S}+\mathit{W},\end{array}$$

(1)

where $\mathit{A}\triangleq \left[\begin{array}{c}\mathit{a}\left({\theta }_0\right),\mathit{a}\left({\theta }_1\right),\dots ,\mathit{a}\left({\theta }_{K-1}\right)\end{array}\right]$, and the k-th signal is weighted by corresponding directional vector $\mathit{a}\left({\theta }_k\right)=[1e^{-j2\pi (d/\lambda )sin{\theta }_k}\dots e^{-j2\pi (N-1)(d/\lambda )sin{\theta }_k}]$. The correlated target signal $\mathit{S}\triangleq \left[\begin{array}{c}{\mathit{s}}_0,{\mathit{s}}_1,\dots ,{\mathit{s}}_{M-1}\end{array}\right]\in {\mathbb{C}}^{K\times M}$, and the received signal $\mathit{Y}\triangleq \left[\begin{array}{c}{\mathit{y}}_0,{\mathit{y}}_1,\dots ,{\mathit{y}}_{M-1}\end{array}\right]\in {\mathbb{C}}^{N\times M}$. Besides, $\mathit{W}\triangleq \left[\begin{array}{c}{\mathit{w}}_0,{\mathit{w}}_1,\dots ,{\mathit{w}}_{M-1}\end{array}\right]\in {\mathbb{C}}^{N\times M}$ describes the additive white Gaussian noise (AWGN).

2.2. System Considering Gain-Phase Uncertainties

In practice, gain-phase uncertainties among antennas affect the ideal steering vector significantly. Thus, a matrix $\mathit{\Gamma }$ is introduced to modify the ideal system model in Equation (1). $\mathit{\Gamma }$ is a diagonal matrix, and the element of the diagonal is associated with the gain and phase errors. We define $\mathit{\Gamma }\triangleq diag\left\{\gamma \right\}$, where $\gamma \triangleq {\left[\begin{array}{c}{z}_0{e}^{-j{\varphi }_0},z_1{e}^{-j{\varphi }_1},\dots ,z_{N-1}{e}^{-j{\varphi }_{N-1}}\end{array}\right]}^{\mathrm{T}}$, and $z_n$ ($z_n\ge 0$) and ${\varphi }_n$ (${\varphi }_n\in (-\pi ,\pi ]$) denote the gain and phase error for the n-th receiving antenna respectively. Accordingly, the received signals considering the effects of gain and phase errors can be described as:

$$\mathit{Y}=\mathit{\Gamma }\mathit{A}\mathit{S}+\mathit{W}.$$

(2)

Thus, we express the system model for direction finding with gain-phase uncertainties in Equation (2). DOA vector $\mathit{\theta }$ will be estimated by using the method based on atomic norm with unknown parameters $\mathsf{\Gamma }$ and correlated signals $\mathit{S}$ in the next section.

3. Atomic Norm-Based Algorithm for DOA Estimation

We use Equation (2) to describe the system model for direction finding with gain and phase uncertainties among antennas. In this section, a novel gridless DOA estimation approach is proposed. The new method is based on atomic norm and can obtain an estimate $\widehat{\mathit{S}}$ of signal $\mathit{S}$. By treating the DOA as a continuous parameter, the method is capable of recovering the sparse signal without requirement of signals falling on the discretized frequency grids. We will give more particulars about the proposed direction finding approach in the scenario with gain-phase uncertainties.

An atomic norm is introduced to solve the direction finding problem with antenna inconsistent. The atomic norm for $\mathit{Y}$ is defined as

$${∥\mathit{Y}∥}_{\mathcal{A},0}\triangleq \underset{K}{inf}\left\{\mathit{Y}=\sum _{k=0}^{K-1}{s}_k\mathsf{\Gamma }\mathit{A}({\theta }_k,{\mathit{b}}_k),c_k\ge 0,\mathsf{\Gamma }\triangleq diag\left(\gamma \right)\right\},$$

(3)

where $\mathit{A}(\theta ,\mathit{b})\triangleq \mathit{a}\left(\theta \right){\mathit{b}}^H$ ($\theta \in [-\pi ,\pi ),\mathit{b}\in {\mathbb{C}}^M$) with ${\parallel \mathit{b}\parallel }_2=1$ is defined as an atom for representing $\mathit{Y}$. We use the diagonal matrix $\mathsf{\Gamma }$ to denote the phase errors among antennas. $\gamma \triangleq \left\{c_0{e}^{j{\varphi }_0},c_1{e}^{j{\varphi }_1},\dots ,c_{N-1}{e}^{j{\varphi }_{N-1}}\right\}$, and $c_n$ and ${\varphi }_n$ denote the gain error and phase error of the n-th receiving antenna respectively.

However, it is difficult to solve ${\parallel \mathit{Y}\parallel }_{\mathcal{A},0}$, which is an NP-hard problem. We convert it into a convex optimization norm

$$\begin{array}{cc}\hfill {∥\mathit{Y}∥}_{\mathcal{A}}& \triangleq inf\left\{t>0:\mathit{Y}\in t\mathrm{conv}\left(\mathcal{A}\right)\right\}\hfill \\ & =inf\left\{\sum _k{s}_k|\mathit{Y}=\sum _{k=0}^{K-1}{s}_k\mathsf{\Gamma }\mathit{A}({\theta }_k,{\mathit{b}}_k),s_k\ge 0\right\},\hfill \end{array}$$

(4)

where $conv\left(\mathcal{A}\right)$ denotes the convex hull of elements in $\mathcal{A}$. We use the atomic norm to enforce sparsity in $\mathcal{A}$ and solve denoising estimation problem. Ref. [39] firstly introduced concept of applying the atomic norm to encourage sparsity for a universal set of atoms. As prescribed in [42], the atomic norm minimization method is a solution of line spectrum estimation problem and denoising from noisy observations of multiple spectral-sparse signals. We can convert the DOA estimation problem into an optimization one

$$\underset{\mathit{Z}}{min}\frac{1}{2}{∥\mathit{Y}-\mathit{Z}∥}_F^2+\tau {∥\mathit{Z}∥}_{\mathcal{A}},$$

(5)

where $\tau$ $(\tau >0)$ is a properly chosen regularization parameter. With already known noise, an upper bound of mean square error (MSE) is provided.

Next, the framework of the dual problem will be analyzed. We can identify the DOA vector $\mathit{\theta }$ that comprises the optimal $\mathit{Y}$ using the dual optimal solution.

We construct the dual problem of Equation (5). Firstly, we define dual norm as

$${∥\mathit{Z}∥}_{\mathcal{A}}^{*}\triangleq \underset{{∥\mathit{X}∥}_{\mathcal{A}}\le 1}{sup}{〈\mathit{X},\mathit{Z}〉}_R,$$

(6)

where the inner product is $〈\mathit{X},\mathit{Z}〉\triangleq \mathcal{R}\left\{\mathrm{Tr}\left({\mathit{Z}}^{\mathrm{H}}\mathit{X}\right)\right\}$.

By introducing the Lagrangian variable $\mathit{U}$, primal variable $\mathit{Z}$ and $\mathit{X}$, the Lagrangian function of Equation (5) can be expressed as

$$L(\mathit{Z},\mathit{X},\mathit{U})\triangleq \frac{1}{2}{∥\mathit{Y}-\mathit{X}∥}_F^2+\tau {∥\mathit{Z}∥}_{\mathcal{A}}+〈\mathit{X}-\mathit{Z},\mathit{U}〉.$$

(7)

We minimize the over all $\mathit{X}$ and $\mathit{Z}$ to get the dual function to be the function of $\mathit{U}$, then maximize the $\mathit{U}$ to get the result. So we can get the dual problem of the optimization problem

$$\underset{\mathit{U}}{max}\underset{\mathit{Z},\mathit{X}}{min}L(\mathit{Z},\mathit{X},\mathit{U}).$$

(8)

The Lagrangian can be minimized over all $\mathit{X}$ and $\mathit{Z}$ by breaking it up and which can be simplified as

$$\begin{array}{cc}\hfill l\underset{\mathit{U}}{max}\underset{\mathit{Z},\mathit{X}}{min}L(\mathit{Z},\mathit{X},\mathit{U})& =\underset{\mathit{U}}{max}\underset{\mathit{Z},\mathit{X}}{min}\left\{\frac{1}{2}{∥\mathit{Y}-\mathit{X}∥}_F^2+\tau {∥\mathit{Z}∥}_{\mathcal{A}}+〈\mathit{X}-\mathit{Z},\mathit{U}〉\right\}\hfill \\ & =\underset{\mathit{U}}{max}\underset{\mathit{X}}{min}\left\{\frac{1}{2}{∥\mathit{Y}-\mathit{X}∥}_F^2+〈\mathit{X},\mathit{U}〉\right\}+\underset{\mathit{Z}}{min}\left\{\tau {∥\mathit{Z}∥}_{\mathcal{A}}-〈\mathit{Z},\mathit{U}〉\right\}\hfill \\ & =\underset{\mathit{U}}{max}\frac{1}{2}\left({∥\mathit{Y}∥}_F^2-{∥\mathit{Y}-\mathit{U}∥}_F^2\right)-I\left\{{∥\mathit{U}∥}_{\mathcal{A}}^{*}\le \tau \right\},\hfill \end{array}$$

(9)

where $I\left\{{∥\mathit{U}∥}_{\mathcal{A}}^{*}\le \tau \right\}$ means that the value is 0 when the condition in the set is satisfied, otherwise the value is ∞. Therefore, the dual problem of (5) can be simplified as

$$\begin{array}{c}\underset{\mathit{U}}{max}\frac{1}{2}\left({∥\mathit{Y}∥}_F^2-{∥\mathit{Y}-\mathit{U}∥}_F^2\right)\hfill \\ \mathrm{s}.\mathrm{t}.{∥\mathit{U}∥}_{\mathcal{A}}^{*}\le \tau .\hfill \end{array}$$

(10)

Furthermore, according to (6), ${∥\mathit{U}∥}_{\mathcal{A}}^{*}$ can be expressed as

$$\begin{array}{cc}\hfill {∥\mathit{U}∥}_{\mathcal{A}}^{*}& =\underset{{∥\mathit{Y}∥}_{\mathcal{A}}\le 1}{sup}〈\mathit{U},\mathit{Y}〉\hfill \\ & =\underset{f\in \left[0,1\right),{∥\mathit{b}∥}_2=1,\mathsf{\Gamma }=diag\left(\gamma \right),{\parallel \gamma \parallel }_2^2\le 1}{sup}〈\mathit{U},\mathsf{\Gamma }\mathit{a}\left(f\right){\mathit{b}}^H〉\hfill \\ & =\underset{f\in \left[0,1\right),{∥\mathit{b}∥}_2=1,\mathsf{\Gamma }=diag\left(\gamma \right),{\parallel \gamma \parallel }_2^2\le 1}{sup}\left|〈\mathit{b},{\mathit{U}}^H\mathsf{\Gamma }\mathit{a}\left(f\right)〉\right|\hfill \\ & =\underset{f\in [0,1],\mathsf{\Gamma }={diag\left(\gamma \right),\parallel \gamma \parallel }_2\le 1}{sup}{∥{\mathit{U}}^{\mathrm{H}}\mathsf{\Gamma }\mathit{a}\left(f\right)∥}_2\hfill \\ & =\underset{f\in [0,1]}{sup}{∥{\mathit{U}}^{\mathrm{H}}\mathit{a}\left(f\right)∥}_2,\hfill \end{array}$$

(11)

where $\mathit{b}=\mathit{a}{\left(f\right)}^{\mathrm{H}}{\mathsf{\Gamma }}^{\mathrm{H}}\mathit{U}$. Equation (11) accludes the dual atomic norm equals to the upper bound of ${∥{\mathit{U}}^{\mathrm{H}}\mathit{a}\left(f\right)∥}_2$, which means, the constraint ${∥\mathit{U}∥}_{\mathcal{A}}^{*}\le \tau$ equals to

$${∥{\mathit{U}}^{\mathrm{H}}\mathit{a}\left(f\right)∥}_2\le \tau \iff {∥{\mathit{U}}^{\mathrm{H}}\mathit{a}\left(f\right)∥}_2^2\le {\tau }^2.$$

(12)

Semidefinite matrix theorem [54]: if there exists a semidefinite matrix $\mathit{M}=\left[\begin{array}{cc}\mathit{A}& \mathit{B}\\ {\mathit{B}}^H& \mathit{C}\end{array}\right]$, the necessary and sufficient condition for $\mathit{M}⪰0$ is $\mathit{C}⪰0$ and $\mathit{A}-\mathit{B}{\mathit{C}}^{-1}{\mathit{B}}^H⪰0$.

By the above theorem, we can build a semidefinite matrix

$$\left[\begin{array}{cc}\mathit{Q}& \mathit{U}\\ {\mathit{U}}^H& \mathit{G}\end{array}\right]⪰0,$$

(13)

where $\mathit{G}⪰0$. Then according to Schur complement, we know that

$$\mathit{Q}-\mathit{U}{\mathit{G}}^{-1}{\mathit{U}}^H⪰0,$$

(14)

which means for any $\mathit{t}\triangleq {[t_0,t_1,\dots ,t_{N-1}]}^T\in {\mathbb{C}}^N(t_n\triangleq e^{-j2\pi n(d/\lambda )sin\theta }$), we have

$${\mathit{t}}^H\mathit{Qt}\ge {\mathit{t}}^H\mathit{U}{\mathit{G}}^{-1}{\mathit{U}}^H\mathit{t}=\mathrm{Tr}\left\{{\mathit{G}}^{-1}{\mathit{U}}^H\mathit{t}{\mathit{t}}^H\mathit{U}\right\}.$$

(15)

Observing Equation (15), it is easy to find that when $\mathit{G}=\mathit{I}$, the equation can be rewritten as

$${\mathit{t}}^H\mathit{Qt}\ge Tr\left\{{\mathit{U}}^H{\mathit{tt}}^H\mathit{U}\right\}={∥{\mathit{U}}^H\mathit{t}∥}_2^2.$$

(16)

Thus, when ${\mathit{t}}^H\mathit{Q}\mathit{t}={\tau }^2$, we can meet the constraint of Equation (12).

Next, we build a matrix as

$$\sum _n{Q}_{n,n+k}=\left\{\begin{array}{cc}{\tau }^2,\hfill & k=0\hfill \\ 0,\hfill & k\ne 0\&-n\le k\le N-1-n\hfill \end{array},\right.$$

(17)

and substitute Equation (17) into ${\mathit{t}}^{\mathrm{H}}\mathit{Qt}$ as

$$\begin{array}{cc}\hfill t^H\mathbf{Q}t& =\sum _n\sum _m{t}_n^H{t}_m{Q}_{nm}=\sum _{n=0}^{N-1}\sum _{k=-n}^{N-1-n}{t}_n^H{t}_{n+k}{Q}_{n,n+k}\hfill \\ & =\sum _n\sum _{k=-n}^{N-1-n}{e}^{-j2\pi n(d/\lambda )sin\theta -j2\pi (n+k)(d/\lambda )sin\theta }{Q}_{n,n+k}\hfill \\ & =\sum _n\sum _{k=-n}^{N-1-n}{e}^{-j2\pi k(d/\lambda )sin\theta }{Q}_{n,n+k}=\sum _n{Q}_{n,n}+\sum _{k=-n,k\ne 0}^{N-1-n}{e}^{-j2\pi k(d/\lambda )sin\theta }{Q}_{n,n+k}\hfill \\ & ={\tau }^2.\hfill \end{array}.$$

(18)

Therefore, with matrix satisfying Equation (17), the constraint of Equation (15) can be met. Then the optimization problem Equation (10) can be expressed as

$$\begin{array}{c}\underset{\mathit{U},\mathit{Q}}{max}\frac{1}{2}\left({∥\mathit{Y}∥}_F^2-{∥\mathit{Y}-\mathit{U}∥}_F^2\right)\hfill \\ \mathrm{s}.\mathrm{t}.\left[\begin{array}{cc}\mathit{Q}& \mathit{U}\\ {\mathit{U}}^H& {\tau }^2\mathit{I}\end{array}\right]⪰0\hfill \\ {\displaystyle \sum _n}{Q}_{n,n+k}=\left\{\begin{array}{cc}1,\hfill & k=0\hfill \\ 0,\hfill & k\ne 0\&-n\le k\le N-1-n\hfill \end{array}\right.\hfill \\ \mathit{Q}\mathrm{is}\mathrm{Hermitan}.\hfill \end{array}$$

(19)

Here, the three linear inequalities of Equation (19) are equal to the dual norm constraint Equation (17). By solving the SDP optimization problem Equation (19), the variable $\mathit{U}$ can be obtained and we can get the DOA vector $\mathit{\theta }$ after substituting $\mathit{U}$ into Equation (12).

4. CRLB of DOA Estimation with Correlated Signals

For system model (1), the following equation can obtain

$$\begin{array}{cc}\hfill vec\left(\mathit{Y}\right)& =(\mathit{I}\otimes \mathit{A})vec\left(\mathit{S}\right)+vec\left(\mathit{W}\right),\hfill \end{array}$$

(20)

which can be written as

$$\mathit{y}=(\mathit{I}\otimes \mathit{A})\mathit{s}+\mathit{w},$$

(21)

where $\mathit{s}$ denotes the target signals, and $\mathit{s}\sim \mathcal{CN}$ $(0,{\mathit{C}}_s)$. AWGN is denoted by $\mathit{w}$ with $\mathit{w}\sim \mathcal{CN}$ $(0,{\mathit{C}}_w)$. ${\mathit{C}}_s$ and ${\mathit{C}}_w$ denote covariance of $\mathit{s}$ and $\mathit{w}$, respectively. Obviously, the mean of $\mathit{y}$ is $\mathbf{0}$. The covariance of $\mathit{y}$ is defined as $\mathsf{\Sigma }$ and $\mathsf{\Sigma }$ can be expressed as

$$\begin{array}{cc}\hfill \mathsf{\Sigma }& =E\left\{\mathit{y}{\mathit{y}}^H\right\}\hfill \\ & =(\mathit{I}\otimes \mathit{A}){\mathit{C}}_s{(\mathit{I}\otimes \mathit{A})}^H+{\mathit{C}}_w.\hfill \end{array}$$

(22)

As a result, $\mathit{y}\sim \mathcal{CN}$ $(0,\mathsf{\Sigma })$, and $\mathsf{\Sigma }=(\mathit{I}\otimes \mathit{A}){\mathit{C}}_s{(\mathit{I}\otimes \mathit{A})}^H+{\mathit{C}}_w$. Based on the above derivation, the likelihood function of $\mathit{y}$ can be written as

$$f(\mathit{y};\mathit{\theta })=\frac{1}{{\pi }^{NP}\left|\mathsf{\Sigma }\right|}{e}^{-{\mathit{y}}^H{\mathsf{\Sigma }}^{-1}\mathit{y}}.$$

(23)

Then, we get the natural logarithm of the likelihood function $f(\mathit{y};\mathit{\theta })$ as follows

$$\begin{array}{cc}\hfill \ell (\mathit{y};\mathit{\theta })& =lnf(\mathit{y};\mathit{\theta })\hfill \\ & =const-ln\left|\mathsf{\Sigma }\right|-{\mathit{y}}^H{\mathsf{\Sigma }}^{-1}\mathit{y}.\hfill \end{array}$$

(24)

The Fisher information is defined as

$${\displaystyle I_{m,n}=E\left[\frac{\partial }{\partial {\theta }_m}lnf\left(\mathit{y};\mathit{\theta }\right)\frac{\partial }{\partial {\theta }_n}lnf\left(\mathit{y};\mathit{\theta }\right)\right]=-E\left[\frac{{\partial }^2}{\partial {\theta }_m\partial {\theta }_n}lnf\left(\mathit{y};\mathit{\theta }\right)\right],}$$

(25)

where

$$\frac{{\partial }^2}{\partial {\theta }_m\partial {\theta }_n}lnf\left(\mathit{y};\mathit{\theta }\right)=\frac{{\partial }^2(ln\left|\mathsf{\Sigma }\right|+{\mathit{y}}^H{\mathsf{\Sigma }}^{-1}\mathit{y})}{\partial {\theta }_m\partial {\theta }_n}=\frac{{\partial }^2(ln\left|\mathsf{\Sigma }\right|)}{\partial {\theta }_m\partial {\theta }_n}+\frac{{\partial }^2\left({\mathit{y}}^H{\mathsf{\Sigma }}^{-1}\mathit{y}\right)}{\partial {\theta }_m\partial {\theta }_n}.$$

(26)

The former part of Equation (26) can be expressed as

$$\begin{array}{cc}\hfill \frac{{\partial }^2(ln\left|\mathsf{\Sigma }\right|)}{\partial {\theta }_m\partial {\theta }_n}=& \frac{\partial (Tr\left[{\mathsf{\Sigma }}^{-1}\frac{\partial \mathsf{\Sigma }}{\partial {\theta }_m}\right])}{\partial {\theta }_n}\hfill \\ & =Tr\left[\frac{\partial {\mathsf{\Sigma }}^{-1}}{\partial {\theta }_n}{\frac{\partial \mathsf{\Sigma }}{\partial \theta }}_m+{\mathsf{\Sigma }}^{-1}\frac{{\partial }^2\mathsf{\Sigma }}{\partial {\theta }_m\partial {\theta }_n}\right]\hfill \\ & =Tr\left[-{\mathsf{\Sigma }}^{-1}\frac{\partial \mathsf{\Sigma }}{\partial {\theta }_n}{\mathsf{\Sigma }}^{-1}\frac{\partial \mathsf{\Sigma }}{\partial {\theta }_m}+{\mathsf{\Sigma }}^{-1}\frac{{\partial }^2\mathsf{\Sigma }}{\partial {\theta }_m\partial {\theta }_n}\right],\hfill \end{array}$$

(27)

and the latter part is

$$\begin{array}{cc}\hfill \frac{{\partial }^2\left({\mathit{y}}^H{\mathsf{\Sigma }}^{-1}\mathit{y}\right)}{\partial {\theta }_m\partial {\theta }_n}& =\frac{\partial }{\partial {\theta }_n}\left(\frac{\partial {\mathit{y}}^H}{\partial {\theta }_m}{\mathsf{\Sigma }}^{-1}\mathit{y}+{\mathit{y}}^H\frac{\partial {\mathsf{\Sigma }}^{-1}}{\partial {\theta }_m}\mathit{y}+{\mathit{y}}^H{\mathsf{\Sigma }}^{-1}\frac{\partial \mathit{y}}{\partial {\theta }_m}\right).\hfill \end{array}$$

(28)

Since the value of $\mathit{y}$ and ${\theta }_m$ is independent, $\frac{\partial {\mathit{y}}^H}{\partial {\theta }_n}=\frac{\partial {\mathit{y}}^H}{\partial {\theta }_m}=\mathbf{0}$. The Equation (28) is determined only by ${\mathit{y}}^H\frac{\partial {\mathsf{\Sigma }}^{-1}}{\partial {\theta }_m}\mathit{y}$ and the item can be expressed as

$$\begin{array}{cc}\hfill \mathit{E}\left\{{\mathit{y}}^H\frac{\partial {\mathsf{\Sigma }}^{-1}}{\partial {\theta }_m}\mathit{y}\right\}& =Tr\left(\frac{\partial {\mathsf{\Sigma }}^{-1}}{\partial {\theta }_m\partial {\theta }_n}\mathsf{\Sigma }\right)\hfill \\ & =Tr\left[-\frac{\partial \left({\mathsf{\Sigma }}^{-1}\frac{\partial \mathsf{\Sigma }}{\partial {\theta }_m}{\mathsf{\Sigma }}^{-1}\right)}{\partial {\theta }_n}\mathsf{\Sigma }\right]\hfill \\ & =Tr\left[-\frac{\partial {\mathsf{\Sigma }}^{-1}}{\partial {\theta }_n}\frac{\partial \mathsf{\Sigma }}{\partial {\theta }_m}-\frac{\partial \left(\frac{\partial \mathsf{\Sigma }}{{\theta }_m}{\mathsf{\Sigma }}^{-1}\right)}{\partial {\theta }_n}\right]\hfill \\ & =Tr\left[{\mathsf{\Sigma }}^{-1}\frac{\partial \mathsf{\Sigma }}{\partial {\theta }_n}{\mathsf{\Sigma }}^{-1}\frac{\partial \mathsf{\Sigma }}{\partial {\theta }_m}-\frac{{\partial }^2\mathsf{\Sigma }}{\partial {\theta }_m\partial {\theta }_n}{\mathsf{\Sigma }}^{-1}+\frac{\partial \mathsf{\Sigma }}{\partial {\theta }_n}{\mathsf{\Sigma }}^{-1}\frac{\partial \mathsf{\Sigma }}{\partial {\theta }_m}{\mathsf{\Sigma }}^{-1}\right].\hfill \end{array}$$

(29)

As a result, Equation (26) can be written as

$$\begin{array}{cc}\hfill \frac{{\partial }^2}{\partial {\theta }_m\partial {\theta }_n}lnf\left(\mathit{y};\mathit{\theta }\right)& =Tr\left[{\mathsf{\Sigma }}^{-1}\frac{\partial \mathsf{\Sigma }}{\partial {\theta }_n}{\mathsf{\Sigma }}^{-1}\frac{\partial \mathsf{\Sigma }}{\partial {\theta }_m}\right],\hfill \end{array}$$

(30)

where

$$\begin{array}{cc}\hfill \frac{\partial \mathsf{\Sigma }}{\partial {\theta }_n}& =\frac{\partial ((\mathit{I}\otimes \mathit{A}){\mathit{C}}_s{(\mathit{I}\otimes \mathit{A})}^H+{\mathit{C}}_w)}{\partial {\theta }_n}\hfill \\ & =\frac{\partial (\mathit{I}\otimes \mathit{A})}{\partial {\theta }_n}{\mathit{C}}_s\left(\mathit{I}\otimes {\mathit{A}}^H\right)+\left(\mathit{I}\otimes \mathit{A}\right)\frac{\partial \left({\mathit{C}}_s\left(\mathit{I}\otimes {\mathit{A}}^H\right)\right)}{\partial {\theta }_n}\hfill \\ & =\left(\mathit{I}\otimes \frac{\partial \mathit{A}}{\partial {\theta }_n}\right){\mathit{C}}_s\left(\mathit{I}\otimes {\mathit{A}}^H\right)+(\mathit{I}\otimes \mathit{A}){\mathit{C}}_s\left(\mathit{I}\otimes \frac{\partial {\mathit{A}}^H}{\partial {\theta }_n}\right).\hfill \end{array}$$

(31)

Additionally, we have

$$\frac{\partial \mathit{A}}{\partial {\theta }_n}=\left[\mathbf{0},\frac{\partial \mathit{a}\left({\theta }_n\right)}{\partial {\theta }_n},\mathbf{0}\right],$$

(32)

where

$$\frac{\partial \mathit{a}\left({\theta }_n\right)}{\partial {\theta }_n}=\left[\begin{array}{c}0\\ -j2\pi (d/\lambda )cos{\theta }_n{e}^{-j2\pi (d/\lambda )sin{\theta }_n}\\ \cdots \\ -j2\pi (N-1)(d/\lambda )cos{\theta }_n{e}^{-j2\pi (N-1)(d/\lambda )sin{\theta }_n}\end{array}\right].$$

(33)

Similarly, we can obtain the value of $\frac{\partial \mathit{A}}{\partial {\theta }_m}$.

Substituting Equation (30) into Equation (25), we can get the Fisher Information Matrix, and CRLB equals to the the inverse of the Fisher Information matrix. With K signals existing in the ULA system, the CRLB of the DOA estimation can be written as

$$var\left\{\mathit{\theta }\right\}\ge \sum _{k=0}^{K-1}{\left[{\mathit{I}}^{-1}\right]}_{k,k}.$$

(34)

5. Simulation Results

In this section, we provide some simulation results to demonstrate the capability of the raised method.

Table 1. Simulation Parameters.

Parameter	Value
The number of samples M	30
The space between antennas d	$0.5$ wavelength
SNR	20 dB
The number of antennas N	20
DOA range	$\left[-70{}^{\circ },70{}^{\circ }\right]$
The minimum DOA between signals	30°
The number of signals K	2

shows the simulation parameters. We use MATLAB R2015b to realize all the simulations on a workstation with 3.40 GHz Intel Core i7-6700 and 32 GB RAM.

• SOMP: The SOMP method is proposed in [55]. The SOMP algorithm is also called synchronous OMP algorithm. The main idea is that similar atoms have the same sparse characteristics. Therefore, when the sparse representation of similar atoms is assumed, the sparse atoms are located at the same position, and the selected atoms in the overcomplete dictionary are the same. The OMP algorithm is a special case when the original signal is an atom.
• MUSIC: The MUSIC method was first proposed in [56]. Based on the feature structure analysis, the MUSIC method is a classical super-resolution spatial spectrum estimation method. In this algorithm, characteristic decomposition is conducted for the covariance matrix of any array output data, which leads to a signal subspace orthogonal with a noise subspace corresponding to the signal components.
• TAN: The TAN method is proposed in [39]. The TAN method provides an approach to deal with off-grid signals. By converting the atomic norm minimization into a SDP problem and introduces the constraint of toeplitz matrix, the DOA estimation performance is improved significantly.
• OGSBI: appeared in [57]. The OGSBI can process off-grid signal model. The method can be applies to both single measurement vector (SMV) and multiple measurement vector (MMV) cases, while exploiting the combined sparsity among different measurements through supposing a Laplace prior for any measurements.

The above methods are classical DOA estimation algorithms, so we choose them as a comparison to show the estimation capability of the method raised above. Additionally, many other articles like [45] shows that some simulation parameters such as SNRs, the number of antennas and samples will influence the simulation results significantly. In this paper, we also choose these variables to be our simulation parameters.

The range of phase uncertainties are from $-5{}^{\circ }$ to 5 °. The gain errors are from 0.9 times to 1.1 times of original value. Both gain and phase errors obey the Gaussion distribution.

In this paper, The numerical results is achieved by much quantities of Monte Carlo trials and the results can be reproduced by using the given parameters. The DOA estimation performance is measured by the root-mean-square error (RMSE)

$$\mathrm{RMSE}\triangleq \sqrt{\frac{{\sum }_{i=1}^{N_i-1}{∥\widehat{{\mathit{\theta }}_i}-{\mathit{\theta }}_i∥}_2^2}{K{N}_i}},$$

(35)

where we use K to denote the number of signals, and $N_i$ to denote the number of Monte Carlo simulations. ${\widehat{\theta }}_i$ and ${\theta }_i$ respectively denote the estimated and ground-truth DOA vector in the i-th Monte Carlo simulation.

First, Figure 2 shows the estimated polynomials of proposed method processing narrow-band signals and wide-band signals. Narrow-band signals are generated by wide-band signals passing through low-pass filters. The DOA estimation errors (in deg) of proposed method in processing narrow-band signals and wide-band signals are $0.0971{}^{\circ }$ and $0.1218{}^{\circ }$, respectively. This figure reveals that performance can be improved by raised method in processing narrow-band signals than wide-band signals, which indicates that the proposed method is more applicable to deal with narrow-band signal recovery problem. Thus, the narrow-band signals are used as our received signals in the following simulation experiments.

Figure 3 shows that the performance of estimating for arrival direction can be influenced by the change of correlation coefficients and an increased value of the correlation coefficient the estimation degradation. Furthermore, the proposed method can performs better compared with existing methods such as MUSIC, SOMP, OGSBI and TAN. The performance of traditional methods deteriorates much faster than proposed methods, as the correlation coefficient increases.

In Figure 4, we compare the estimated polynomials of the proposed method and TAN. The DOA estimation errors (in deg) of proposed method and the traditional atomic norm-based method are $0.2663{}^{\circ }$ and $0.6307{}^{\circ }$, respectively. According to this figure, we can know that the proposed method can reach a much better performance than the traditional method both in gain and phase estimation.Then, estimated spatial spectrum of 2 target signals are shown in Figure 5. In this figure, the raised method is compared with 4 traditional DOA estimation methods including MUSIC, OGSBI, SOMP and TAN. We can see that the signals recovered by the proposed method are more closer to the target signals than the other four methods.

Table 2. Simulation time.

Methods	Signal 1	Signal 2
Ground-truth directions	$-{42.6461}^{\circ }$	${33.0028}^{\circ }$
OGSBI	$-{42.7868}^{\circ }$	${33.0568}^{\circ }$
TAN	$-{42.7868}^{\circ }$	${33.0568}^{\circ }$
MUSIC	$-{42.7201}^{\circ }$	${32.9852}^{\circ }$
SOMP	$-{42.7250}^{\circ }$	${32.9750}^{\circ }$
Proposed method	$-{42.7154}^{\circ }$	${32.9966}^{\circ }$

gives the corresponding estimated directions for 2 signals. The DOA estimation errors (in deg) of proposed method, SOMP, OGSBI, MUSIC and the traditional atomic norm-based method are $0.0492{}^{\circ }$, $0.0592{}^{\circ }$, $0.0538{}^{\circ }$, and $0.0538{}^{\circ }$ and $0.1066{}^{\circ }$, respectively. Thus, the direction performance is better using the raised method than the traditional method such as SOMP, OGSBI, MUSIC and TAN.

Moreover, Figure 6 shows the comparison of direction finding capability of raised method with other four traditional methods in with different SNRs, and the SNR falls in the range of 0 dB to 40 dB. The capability of direction finding improves greatly with the increase of SNR in the case that the SNR is smaller than 10 dB. Besides, the direction finding capability almost unchanged in the scenario with the SNR is larger than 10 dB. According to this figure, the proposed method behaves better than the other methods, including SOMP, MUSIC, OGSBI and TAN. The associated CRLB of DOA estimation is also shown in Figure 6. We can conclude that the proposed method is closet to the CRLB among the existing methods including SOMP, MUSIC, OGSBI and TAN.

As shown in Figure 7, the direction finding performs better with the sample number increases. The better performance can be reached by the proposed method than the existing methods including MUSIC, SOMP, OGSBI and TAN with different sample numbers.

Figure 8 shows the direction finding performances with different quantities of antennas. With the increasing number of antennas, the DOA estimation performance improves. When less than 20 antennas exist in the ULA system, raised method performs similarly to other traditional DOA estimation methods including MUSIC, SOMP and OGSBI and TAN. When the number is larger than 30, the proposed method outperforms the others.

Finally,

Table 3. Simulation time.

Method	MUSIC	SOMP	OGSBI	TAN	Proposed Method
Time(s)	0.10510	0.10858	0.09077	1.1268	0.8487

compares the computational time between raised method and the other 4 state-of-the-art methods. OGSBI has the least computational time, but the performance is much worse. TAN has the longest computational time and the proposed method is the second. Compared to the TAN method, the proposed method has the less computation complexity and better estimation performance.Thus, at an acceptable cost of computation complexity, the off-grid DOA estimation performance with inconsistent antennas and correlated signals are improved using the proposed method.

6. Conclusions

This paper has investigated a novel gain and phase errors compensation method that is capable of the direction of arrival (DOA) estimation in the ULA system. The system model in the scenario with inconsistent antennas is formulated by introducing a diagonal matrix whose elements are gain-phase uncertainties. Thus, the DOA estimation problem has been transfered to a sparse reconstruction one. Then, the proposed method is developed concentrating on the dual of the atomic norm minimization problem and the corresponding convex optimization problem is addressed using semidefinite programming (SDP). We also derive the CRLB of the system model. According to the simulation results, the cost-efficient method raised in this paper shows better performance than the traditional DOA estimation methods. Processing uncorrelated signals by the raised method will be the focus of our future work.