Efficient Cumulant-Based Methods for Joint Angle and Frequency Estimation Using Spatial-Temporal Smoothing

Abstract

Most non-Gaussian signals in wireless communication array systems contain temporal correlation under a high sampling rate, which can offer more accurate direction of arrival (DOA) and frequency estimates and a larger identifiability. However, in practice, the estimation performance may severely degrade in coloured noise environments. To tackle this issue, we propose real-valued joint angle and frequency estimation (JAFE) algorithms for non-Gaussian signals using fourth-order cumulants. By exploiting the temporal correlation embedded in signals, a series of augmented cumulant matrices is constructed. For independent signals, the DOA and frequency estimates can be obtained, respectively, by leveraging a dual rotational invariance property. For coherent signals, the dual rotational invariance is constructed to estimate the generalized steering vectors, which associates the coherent signals into different groups. Then, the coherent signals in each group can be resolved by performing the forward-backward spatial smoothing. The proposed schemes not only improve the estimation accuracy, but also resolve many more signals than sensors. Besides, it is computationally efficient since it performs the estimation by the polynomial rooting in the real number field. Simulation results demonstrate the superiorities of the proposed estimator to its state-of-the-art counterparts on identifiability, estimation accuracy and robustness, especially for coherent signals.

1. Introduction

Sensor arrays have been used in radar, sonar, wireless communications, seismology, etc. Joint direction of arrival (DOA) and frequency estimation of incident signals have been attracting considerable attention due to their application to various fields. For instance, the two parameters can be applied to locate the mobile subscribers and allocate pilot tones in space division multiple access systems in wireless communications. Besides, an accurate estimation of these parameters is helpful with channel estimation and thus enhances the system performance.

The maximum likelihood estimator (MLE) can achieve the optimal estimates, but is computationally cumbersome as it inevitably relies on a multidimensional search [1]. To reduce computational expenditure, numerous subspace estimators based on the linear algebraic theory have been proposed to extract the information efficiently from a batch of observations. In [2], both parameters were obtained by performing a two-dimensional (2D) spectral search of the MUSIC algorithm, providing an automatic pairing between the DOAs and frequencies, and it calls for a considerable amount of computations, despite lower cost than MLE. To further reduce the computing load, one-dimensional (1D) subspace-based algorithms were developed. By a similar principle, Lin et al. proposed a tree-structured frequency-space-frequency (FSF) MUSIC-based algorithm [3], allowing multiple one-dimensional (1D) grid searches in the frequency and spatial domains, respectively. In addition, ESPRIT-based joint angle and frequency estimation (JAFE) methods have been proposed to further alleviate the computational load. Zoltowski and Mathews provided a solution to radar applications from engineering perspectives [4], causing some compromise on mathematical rigour and theoretical depth. Haardt and Nossek discussed the problem in the context of mobile communications for SDMAapplications by using unitary ESPRIT, which transforms the complex data to real-valued matrices [5]. Multiresolution ESPRIT, referred to as MR-ESPRIT, has been developed to handle JAFE in different geometries, such as uniform linear arrays and uniform circular arrays, by Lemma et al. [6], and they subsequently presented an intensive performance analysis of the MR-ESPRIT JAFE algorithm [7]. These ESPRIT-based JAFE methods can provide satisfactory estimation performance, but require an additional pairing, which may fail to work at low signal-to-noise ratios (SNRs). In addition to ESPRIT-like approaches, Zhang et al. cast the JAFE problem as a trilinear model and performed trilinear decomposition to obtain angle and frequency without any eigen-decomposition to covariance matrices or singular-value decomposition to observations [8]; Liu et al. proposed a unitary JAFE algorithm, termed U-JAFE, in conjunction with the frame method and the Newton iteration method, resulting in improved computational efficiency and automatic parameter pairing [9].

It is worth pointing out that the existing methods for JAFE are generally restricted to white Gaussian noise environments. However, in many applications, the additive noises in each channel are correlated with each other, i.e., the noise becomes coloured. This coloured noise model is reasonable in practice since the noise can be coupled due to the internal circuitry of the antenna elements [10,11] or the prevailing external noise environment such as reverberation noise in sonar or external seismic noise [12,13,14,15,16]. Since high-order statistics of wireless communication signals can suppress coloured Gaussian noise and potentially extend the effective array aperture, they have been investigated extensively for DOA estimation in the presence of unknown coloured noise [17,18,19,20]. However, to the best of our knowledge, the previous effort has not been devoted to JAFE in coloured noise environments. This motivates us to address such an issue using fourth-order cumulants (FOC), and unitary cumulant-based estimators are devised in this work to tackle the problem. To be specific, if the incident signals are independent of each other, an augmented cumulant matrix is first constructed from the temporally-smoothed data, and its complex-valued entries are then transformed to the real number field by exploiting the centro-symmetric property of the matrix. To determine the DOA estimates with computational efficiency, the rank reduction (RARE) estimator in conjunction with polynomial rooting is developed, and the corresponding frequency estimates are obtained in the least squares manner with the DOA estimates already acquired, avoiding the parameter-pairing operation. For coherent signals, the generalized steering vectors and frequencies are first estimated, by a new dual rotational invariance arising from a series of dedicated cumulant matrices, to blind separate coherent signals into different groups. Then, the inherent rank deficiency due to the generalized steering vectors is recovered by FBSS, and finally, the DOA estimates are resolved by the unitary ESPRIT algorithm.

To show the contributions of this paper clearly, the differences between the state-of-the-art methods [6,7,8,9] and our work are highlighted as follows:

• The most important difference is that the operation of FOC with respect to JAFE is developed for the first time, and the corresponding existence conditions, and the number of achieved DOFs is analysed accordingly.
• In [6,7,8,9,10,11], only the temporal and spatial white Gaussian noise is considered, while in this paper, we study the JAFE in a more general case, i.e., the temporal and spatial coloured Gaussian noise, and propose an FOC-based estimator to jointly determine DOA, as well as frequency robust to white or coloured noise.
• In [6,7,9], the number of coherent signals that can be dealt with is up to $\lfloor \frac{2M}{3}\rfloor$, where M is the number of array elements, which is a strict condition. In practice, one base station serves multiple users simultaneously, where the propagation from each user composes multipath, so the total number of coherent signals may exceed that of array elements. The approach developed in this paper is capable of handling such a tough challenge that cannot be overcome by the previous techniques.
• To reduce the cumbersome computations arising from the processing by FOC, we transform the complex-valued information to the field of real numbers and perform the resultant RARE estimator by solving the roots of a polynomial, avoiding the multidimensional search.

The remainder of this paper is organized as follows. In Section 2, an array model for signals corrupted by unknown coloured noise is introduced. In Section 3, the RARE estimators for JAFE of uncorrelated and coherent signals using fourth-order cumulants are developed. Section 4 provides numerical examples for demonstrating the validity and efficiency of our proposed algorithms. Finally, some concluding remarks are given in Section 5.

Throughout this paper, the following notations will be used: the operators ${(·)}^T$, ${(·)}^{\ast }$, ${(·)}^H$, ${(·)}^{+}$, $\mathrm{cum}\{·\}$, $\det \{·\}$, ⊗, ∘, and ${\parallel ·\parallel }_2$ denote the operation of the transpose, conjugate, conjugate transpose, pseudo-inverse, cumulant, determinant, Kronecker product, Khatri–Rao (KR) product and Euclidean (${\ell }_2$) norm, respectively. The symbol $\mathrm{diag}\{z_1,\cdots ,z_N\}$ represents a diagonal matrix with diagonal entries $z_1,\cdots ,z_N$, and $\mathrm{blkdiag}\{{\mathbf{Z}}_1,{\mathbf{Z}}_2\}$ represents a block diagonal matrix with diagonal entries ${\mathbf{Z}}_1,{\mathbf{Z}}_2$. The symbol ${\mathbf{I}}_K$ stands for an identity matrix of size $K\times K$. The symbol $\mathbf{Z}(a:b,c:d)$ refers to a constructed submatrix by the entries from a to the $b\mathrm{th}$ row and c to the $d\mathrm{th}$ column of $\mathbf{Z}$, and the symbol $\mathbf{Z}(a,b)$ denotes the entry in the $a\mathrm{th}$ row and the $b\mathrm{th}$ column of $\mathbf{Z}$.

2. Problem Formulation

2.1. Signal Model

Consider Q narrowband non-Gaussian signals impinging on a uniform linear array (ULA) with M identical omnidirectional sensors. Let the signal from the $i\mathrm{th}$ far-field source be denoted as $e^{j2\pi (f_c+f_i)t}{s}_i(t)$, where $f_c$ is the carrier frequency, $f_i$ is a small frequency offset (central frequency) for the $i\mathrm{th}$ signal with $f_i\ll f_c$ and $s_i(t)$ is the amplitude of the $i\mathrm{th}$ signal, for $i=1,2,\cdots ,Q$. After demodulation to intermediate frequency (IF), the signal due to the $i\mathrm{th}$ source becomes $e^{j2\pi f_{i}t}{s}_i(t)$. We assume that there are K groups of coherent signals, which come from statistically independent far-field sources ${\left\{s_k(t)\right\}}_{k=1}^K$. In the $k\mathrm{th}$ coherent group, the signals coming from direction ${\theta }_{kp}$, $p=1,2,\cdots ,P_k$, correspond to the $p\mathrm{th}$ multipath propagation of the source $s_k(t)$, and the complex fading coefficient is ${\alpha }_{kp}$. It is apparent that the total number of coherent signals is $Q={\sum }_{k=1}^K{P}_k$. Since the K sources are mutually independent, the signals between coherent groups are also independent of each other, while the signals in each group are coherent. It is worth noting that if the independent sources arrive at the sensor array undergoing a line of sight channel (no multipath propagation), the incident Q signals can be regarded as Q coherent groups each of which only includes one signal, i.e., $K=Q$ and $P_1=P_2=\cdots =P_k=1$. The noise is spatially coloured and statistically independent of the signals. Let P be the sampling rate that is much higher than the largest IF, then the $M\times 1$ array output vector is given by [7]:

$$\begin{array}{cc}\hfill \mathbf{x}(\frac{n}{P})& =\sum _{k=1}^K\sum _{p=1}^{P_k}\mathbf{a}({\theta }_{kp}){\alpha }_{kp}{e}^{j2\pi f_k\frac{n}{P}}{s}_k(\frac{n}{P})+\mathbf{w}(\frac{n}{P})=\mathbf{A}\mathsf{\Gamma }{\mathsf{\Phi }}^n\mathbf{s}(\frac{n}{P})+\mathbf{w}(\frac{n}{P})\hfill \end{array}$$

(1)

where $\mathbf{a}(\theta )={\left[1,\beta (\theta ),\cdots ,{\beta }^{M-1}(\theta )\right]}^T\in {\mathbb{C}}^M$ is the steering vector, $\beta (\theta )=e^{j\frac{2\pi d}{\lambda }\sin \theta }$ with $\lambda$ and d being the wavelength of the carrier signal and the spacing between adjacent elements, respectively, $\mathbf{A}=\left[\mathbf{a}({\theta }_1),\cdots ,\mathbf{a}({\theta }_Q)\right]$ is the array manifold, $\mathsf{\Gamma }=\mathrm{blkdiag}\{{\mathbf{\alpha }}_1,\cdots ,{\mathbf{\alpha }}_K\}$ with ${\mathbf{\alpha }}_k={\left[\begin{array}{c}{\alpha }_{k1},\cdots ,{\alpha }_{k{P}_k}\end{array}\right]}^T$ containing attenuation information of the $k\mathrm{th}$ coherent group, while a special case $\mathsf{\Gamma }={\mathbf{I}}_Q$ if the incident signals are independent of each other, $\mathsf{\Phi }=\mathrm{diag}\left\{e^{j2\pi \frac{f_1}{P}},\cdots ,e^{j2\pi \frac{f_K}{P}}\right\}$ contains the information of the IF frequencies of incident signals, $\mathbf{s}(\frac{n}{P})={\left[\begin{array}{c}{s}_1(\frac{n}{P}),\cdots ,s_K(\frac{n}{P})\end{array}\right]}^T\in {\mathbb{C}}^K$ and $\mathbf{w}(\frac{n}{P})$ is the spatially-coloured Gaussian noise. Besides, we assume that the array manifold $\mathbf{A}$ is unambiguous, i.e., the steering vectors ${\left\{\mathbf{a}({\theta }_i)\right\}}_{i=1}^Q$ are linearly independent for any set of distinct ${\left\{{\theta }_i\right\}}_{i=1}^Q$. Collecting N samples of the array output $\mathbf{x}(\frac{n}{P})$ at a rate P and stacking them into a row, one has:

$$\begin{array}{cc}\hfill \mathbf{X}& =\mathbf{A}\mathsf{\Gamma }\left[\mathbf{s}(0),\mathsf{\Phi }\mathbf{s}(\frac{1}{P}),\cdots ,{\mathsf{\Phi }}^{N-1}\mathbf{s}(\frac{N-1}{P})\right]+\mathbf{W}\in {\mathbb{C}}^{M\times N}\hfill \end{array}$$

(2)

where $\mathbf{W}\in {\mathbb{C}}^{M\times N}$ collects N samples of the noise vector $\mathbf{w}(\frac{n}{P})$.

2.2. Temporal Smoothing

In this section, we consider a data stacking technique, referred to as temporal smoothing, which adds structure to the data model for the implementation of the JAFE algorithm. An m-factor temporally-smoothed data matrix is constructed by stacking temporally-shifted versions of the original data matrix. This results in the following matrix of size $mM\times (N-m+1)$ [7]:

$$\begin{array}{cc}\hfill {\mathbf{X}}_m& =\left[\begin{array}{c}\mathbf{A}\mathsf{\Gamma }\left[\mathbf{s}(0),\mathsf{\Phi }\mathbf{s}(\frac{1}{P}),\cdots ,{\mathsf{\Phi }}^{N-m}\mathbf{s}(\frac{N-m}{P})\right]\\ \mathbf{A}\mathsf{\Gamma }\mathsf{\Phi }\left[\mathbf{s}(\frac{1}{P}),\mathsf{\Phi }\mathbf{s}(\frac{2}{P}),\cdots ,{\mathsf{\Phi }}^{N-m}\mathbf{s}(\frac{N-m+1}{P})\right]\\ ⋮\\ \mathbf{A}\mathsf{\Gamma }{\mathsf{\Phi }}^{m-1}\left[\mathbf{s}(\frac{m-1}{P}),\mathsf{\Phi }\mathbf{s}(\frac{m}{P}),\cdots ,{\mathsf{\Phi }}^{N-m}\mathbf{s}(\frac{N-1}{P})\right]\end{array}\right]+{\mathbf{W}}_m\hfill \end{array}$$

(3)

where ${\mathbf{W}}_m$ represents the noise term constructed from $\mathbf{W}$ in a similar manner as ${\mathbf{X}}_m$ is obtained from $\mathbf{X}$. Assume that the signals are narrowband, i.e., $\mathbf{s}(t)\approx \mathbf{s}(t+\frac{1}{P})\approx \cdots \approx \mathbf{s}(t+\frac{m-1}{P})$. In this case, all the block rows in the right-hand term of (3) are approximately equal, which means that ${\mathbf{X}}_m$ has the following factorization:

$$\begin{array}{cc}\hfill {\mathbf{X}}_m& \approx \left[\begin{array}{c}\mathbf{A}\mathsf{\Gamma }\\ \mathbf{A}\mathsf{\Gamma }\mathsf{\Phi }\\ ⋮\\ \mathbf{A}\mathsf{\Gamma }{\mathsf{\Phi }}^{m-1}\end{array}\right]\left[\mathbf{s}(0),\mathsf{\Phi }\mathbf{s}(\frac{1}{P}),\cdots ,{\mathsf{\Phi }}^{N-m}\mathbf{s}(\frac{N-m}{P})\right]+{\mathbf{W}}_m\hfill \\ & \triangleq {\mathbf{A}}_m{\mathbf{F}}_s+{\mathbf{W}}_m\hfill \end{array}$$

(4)

where ${\mathbf{A}}_m$, referred to as the extended array manifold, is given by:

$$\begin{array}{cc}\hfill {\mathbf{A}}_m& =\left[\begin{array}{c}\mathbf{A}\mathsf{\Gamma }\\ \mathbf{A}\mathsf{\Gamma }\mathsf{\Phi }\\ ⋮\\ \mathbf{A}\mathsf{\Gamma }{\mathsf{\Phi }}^{m-1}\end{array}\right]\hfill \end{array}$$

(5)

and:

$$\begin{array}{cc}\hfill {\mathbf{F}}_s& =\left[\mathbf{s}(0),\mathsf{\Phi }\mathbf{s}(\frac{1}{P}),\cdots ,{\mathsf{\Phi }}^{N-m}\mathbf{s}(\frac{N-m}{P})\right]\hfill \end{array}$$

(6)

is a matrix collecting $N-m+1$ samples of the K sources.

3. Proposed FOC-Based Joint Angle and Frequency Estimator

3.1. Independent Signals

First, we consider the case of independent signals, where $K=Q$, $P_1=P_2=\cdots =P_k=1$ and $\mathsf{\Gamma }={\mathbf{I}}_Q$, so $\mathsf{\Gamma }$ in (1)–(5) can be neglected. Considering the received signals are assumed to be non-Gaussian, one can establish the array FOC matrix between the received data blocks as:

$$\begin{array}{cc}\hfill {\mathbf{C}}_q& =\mathrm{cum}\left\{\mathbf{y}(n),{\mathbf{z}}^H(n),{\mathbf{z}}^{\ast }(n),{\mathbf{z}}^T(n)\right\}\hfill \end{array}$$

(7)

where $\mathbf{y}(n)={\mathbf{H}}_q{\mathbf{X}}_m(:,n)$ and $\mathbf{z}(n)={\mathbf{H}}_1{\mathbf{X}}_m(:,n)$. Herein, ${\mathbf{H}}_q=\left[{\mathbf{0}}_{M\times M\left(q-1\right)},{\mathbf{I}}_M,{\mathbf{0}}_{M\times M\left(m-q\right)}\right]$, $q=1,2,\cdots ,m$, and ${\mathbf{X}}_m(:,n)$ is the $n\mathrm{th}$ column of ${\mathbf{X}}_m$. The entries of ${\mathbf{C}}_q$ in the $\left[\left(k_1-1\right)M+k_2\right]\mathrm{th}$ row and the $\left[\left(l_1-1\right)M+l_2\right]\mathrm{th}$ column are defined as:

$$\begin{array}{c}{\mathbf{C}}_q\left(\left(k_1-1\right)M+k_2,\left(l_1-1\right)M+l_2\right)\hfill \\ =\mathrm{cum}\{y_{k_1}(n),z_{l_1}^{\ast }(n),z_{k_2}^{\ast }(n),z_{l_2}(n)\}\hfill \\ =E\left[y_{k_1}(n)z_{k_2}^{\ast }(n)z_{l_1}^{\ast }(n)z_{l_2}(n)\right]-E\left[y_{k_1}(n)z_{k_2}^{\ast }(n)\right]E\left[z_{l_1}^{\ast }(n)z_{l_2}(n)\right]\hfill \\ -E\left[y_{k_1}(n)z_{l_1}^{\ast }(n)\right]E\left[z_{k_2}^{\ast }(n)z_{l_2}(n)\right]-E\left[y_{k_1}(n)z_{l_2}(n)\right]E\left[z_{k_2}^{\ast }(n)z_{l_1}^{\ast }(n)\right]\hfill \end{array}$$

(8)

where $y_i(n)$ and $z_i(n)$ are the $i\mathrm{th}$ entries of $\mathbf{y}(n)$ and $\mathbf{z}(n)$, respectively. Substituting (3) into (7) and utilizing the properties of cumulants [CP1]–[CP5]in [18], one can further get:

$$\begin{array}{cc}\hfill {\mathbf{C}}_q& =\mathrm{cum}\left\{\mathbf{A}{\mathsf{\Phi }}^{q-1}{\mathbf{F}}_s(:,n),{\left(\mathbf{A}{\mathbf{F}}_s(:,n)\right)}^H,{\left(\mathbf{A}{\mathbf{F}}_s(:,n)\right)}^{\ast },{\left(\mathbf{A}{\mathbf{F}}_s(:,n)\right)}^T\right\}\hfill \\ & =\mathrm{cum}\left\{\sum _{m=1}^Q\mathbf{a}({\theta }_m)e^{j2\pi f_m\frac{q-1}{P}}{F}_{s,m},\sum _{p=1}^Q{\mathbf{a}}^H({\theta }_p)F_{s,p}^{\ast },\sum _{n=1}^Q{\mathbf{a}}^{\ast }({\theta }_n)F_{s,n}^{\ast },\sum _{r=1}^Q{\mathbf{a}}^T({\theta }_r)F_{s,r}\right\}\hfill \\ & =\sum _{m=1}^Q\sum _{n=1}^Q\sum _{p=1}^Q\sum _{r=1}^Q\left(e^{j2\pi f_m\frac{q-1}{P}}\mathbf{a}({\theta }_m)\otimes {\mathbf{a}}^{\ast }({\theta }_n)\right){\left(\mathbf{a}({\theta }_p)\otimes {\mathbf{a}}^{\ast }({\theta }_r)\right)}^H\mathrm{cum}\left\{F_{s,m},F_{s,p}^{\ast },F_{s,n}^{\ast },F_{s,r}\right\}\hfill \\ & =\sum _{i=1}^Q{e}^{j2\pi f_i\frac{q-1}{P}}\left(\mathbf{a}({\theta }_i)\otimes {\mathbf{a}}^{\ast }({\theta }_i)\right){\left(\mathbf{a}({\theta }_i)\otimes {\mathbf{a}}^{\ast }({\theta }_i)\right)}^H\mathrm{cum}\left\{F_{s,i},F_{s,i}^{\ast },F_{s,i}^{\ast },F_{s,i}\right\}\hfill \\ & =\sum _{i=1}^Q{\gamma }_i{e}^{j2\pi f_i\frac{q-1}{P}}\left(\mathbf{a}({\theta }_i)\otimes {\mathbf{a}}^{\ast }({\theta }_i)\right){\left(\mathbf{a}({\theta }_i)\otimes {\mathbf{a}}^{\ast }({\theta }_i)\right)}^H\hfill \\ & =\left(\mathbf{A}\circ {\mathbf{A}}^{\ast }\right){\mathsf{\Phi }}^{q-1}{\mathbf{C}}_s{\left(\mathbf{A}\circ {\mathbf{A}}^{\ast }\right)}^H\hfill \end{array}$$

(9)

where $F_{s,i}$ is the entry in the $i\mathrm{th}$ entry of ${\mathbf{F}}_s(:,n)$, ${\gamma }_i\triangleq \mathrm{cum}\left\{F_{s,i},F_{s,i}^{\ast },F_{s,i}^{\ast },F_{s,i}\right\}$ and ${\mathbf{C}}_s\triangleq \mathrm{diag}\left\{{\gamma }_1,\cdots ,{\gamma }_Q\right\}\in {\mathbb{R}}^{Q\times Q}$.

By the principle of aperture extension via FOC [18], some rows of $\mathbf{A}\circ {\mathbf{A}}^{\ast }\in {\mathbb{C}}^{M^2\times Q^2}$ are redundant with respect to the rest in the case of a ULA, resulting in the effective array aperture being $2M-1$ elements. Considering the regular distribution of the redundancy in $\mathbf{A}\circ {\mathbf{A}}^{\ast }$, one can reduce the problem dimension by performing a linear transformation as:

$$\begin{array}{cc}\hfill {\overline{\mathbf{C}}}_q& =\mathbf{G}{\mathbf{C}}_q{\mathbf{G}}^T\hfill \\ & =\mathbf{G}\left(\mathbf{A}\circ {\mathbf{A}}^{\ast }\right){\mathsf{\Phi }}^{q-1}{\mathbf{C}}_s{\left(\mathbf{G}\left(\mathbf{A}\circ {\mathbf{A}}^{\ast }\right)\right)}^H\hfill \\ & =\mathbf{B}{\mathsf{\Phi }}^{q-1}{\mathbf{C}}_s{\mathbf{B}}^H\hfill \end{array}$$

(10)

where:

$$\begin{array}{cc}\hfill \mathbf{G}& \triangleq \left[\begin{array}{cc}{\mathbf{I}}_M& {\mathbf{0}}_{M\times \left(M^2-M\right)}\\ {\mathbf{0}}_{\left(M-1\right)\times \left(M^2-M+1\right)}& {\mathbf{I}}_{M-1}\end{array}\right]\in {\mathbb{R}}^{\left(2M-1\right)\times M^2}\hfill \end{array}$$

(11)

and $\mathbf{B}\triangleq \mathbf{G}\left(\mathbf{A}\circ {\mathbf{A}}^{\ast }\right)=\left[\mathbf{b}({\theta }_1),\cdots ,\mathbf{b}({\theta }_Q)\right]$ with $\mathbf{b}(\theta )={\left[{\beta }^{1-M}(\theta ),\cdots ,{\beta }^{M-1}(\theta )\right]}^T\in {\mathbb{C}}^{2M-1}$.

To take full advantage of the information embedded in ${\left\{{\overline{\mathbf{C}}}_q\right\}}_{q=1}^m$, one can design a new cumulant matrix, of size $m(2M-1)\times m(2M-1)$, as:

$$\begin{array}{cc}\hfill {\mathbf{C}}_x& =\left[\begin{array}{cccc}{\overline{\mathbf{C}}}_1& {\overline{\mathbf{C}}}_2^H& \cdots & {\overline{\mathbf{C}}}_m^H\\ {\overline{\mathbf{C}}}_2& {\overline{\mathbf{C}}}_1& \cdots & {\overline{\mathbf{C}}}_{m-1}^H\\ ⋮& ⋮& \ddots & ⋮\\ {\overline{\mathbf{C}}}_m& {\overline{\mathbf{C}}}_{m-1}& \cdots & {\overline{\mathbf{C}}}_1\end{array}\right]=\left[\begin{array}{c}\mathbf{B}\\ \mathbf{B}\mathsf{\Phi }\\ ⋮\\ \mathbf{B}{\mathsf{\Phi }}^{m-1}\end{array}\right]{\mathbf{C}}_s{\left[\begin{array}{c}\mathbf{B}\\ \mathbf{B}\mathsf{\Phi }\\ ⋮\\ \mathbf{B}{\mathsf{\Phi }}^{m-1}\end{array}\right]}^H=\tilde{\mathbf{B}}{\mathbf{C}}_s{\tilde{\mathbf{B}}}^H\hfill \end{array}$$

(12)

where $\tilde{\mathbf{B}}\triangleq {\left[{\mathbf{B}}^T,{\mathbf{B}}^T\mathsf{\Phi },\cdots ,{\mathbf{B}}^T{\mathsf{\Phi }}^{m-1}\right]}^T$. It can be readily verified that:

$$\begin{array}{cc}\hfill \overline{\mathbf{J}}{\mathbf{C}}_x^{\ast }\overline{\mathbf{J}}& =\left[\begin{array}{c}\mathbf{B}{\mathsf{\Phi }}^{1-m}\\ \mathbf{B}{\mathsf{\Phi }}^{2-m}\\ ⋮\\ \mathbf{B}\end{array}\right]{\mathbf{C}}_s^{\ast }{\left[\begin{array}{c}\mathbf{B}{\mathsf{\Phi }}^{1-m}\\ \mathbf{B}{\mathsf{\Phi }}^{2-m}\\ ⋮\\ \mathbf{B}\end{array}\right]}^H=\tilde{\mathbf{B}}{\mathsf{\Phi }}^{1-m}{\mathbf{C}}_s^{\ast }{\mathsf{\Phi }}^{m-1}{\tilde{\mathbf{B}}}^H\hfill \end{array}$$

(13)

where $\overline{\mathbf{J}}\in {\mathbb{R}}^{m\left(2M-1\right)\times m\left(2M-1\right)}$ denotes an exchange matrix that has unity entries on the cross-diagonal and zeros elsewhere. It is apparent that $\overline{\mathbf{J}}{\mathbf{C}}_x^{\ast }\overline{\mathbf{J}}={\mathbf{C}}_x$, i.e., ${\mathbf{C}}_x$ is centro-Hermitian, as ideally, ${\mathbf{C}}_s$ is a real diagonal matrix. However, to “double” the number of snapshots and decorrelate possible sample correlation in an arbitrary matrix ${\mathbf{C}}_s$ due to a limited observation window, the centro-Hermitian property is sometimes forced by means of the so-called forward-backward (FB) averaging, that is,

$$\begin{array}{cc}\hfill {\overline{\mathbf{C}}}_x& =\frac{1}{2}\left({\mathbf{C}}_x+\overline{\mathbf{J}}{\mathbf{C}}_x^{\ast }\overline{\mathbf{J}}\right)=\tilde{\mathbf{B}}{\tilde{\mathbf{C}}}_s{\tilde{\mathbf{B}}}^H\hfill \end{array}$$

(14)

where ${\tilde{\mathbf{C}}}_s=\frac{1}{2}\left({\mathbf{C}}_s+{\mathsf{\Phi }}^{1-m}{\mathbf{C}}_s^{\ast }{\mathsf{\Phi }}^{m-1}\right)$. Then, one can transform ${\overline{\mathbf{C}}}_x$ into the field of real numbers for the sake of computational efficiency. We define a new matrix ${\mathbf{C}}_r$ as:

$$\begin{array}{cc}\hfill {\mathbf{C}}_r& ={\mathbf{Q}}_u^H{\overline{\mathbf{C}}}_x{\mathbf{Q}}_u\hfill \end{array}$$

(15)

where ${\mathbf{Q}}_u$, $u=m\left(2M-1\right)$, is a sparse unitary matrix given by:

$${\mathbf{Q}}_u=\left\{\begin{array}{cc}\hfill \frac{1}{\sqrt{2}}\left[\begin{array}{cc}{\mathbf{I}}_v& j{\mathbf{I}}_v\\ {\mathbf{J}}_v& -j{\mathbf{J}}_v\end{array}\right],& u=2v\hfill \\ \hfill \frac{1}{\sqrt{2}}\left[\begin{array}{ccc}{\mathbf{I}}_v& {\mathbf{0}}_{v\times 1}& j{\mathbf{I}}_v\\ {\mathbf{0}}_{v\times 1}^T& \sqrt{2}& {\mathbf{0}}_{v\times 1}^T\\ {\mathbf{J}}_v& {\mathbf{0}}_{v\times 1}& -j{\mathbf{J}}_v\end{array}\right],& u=2v+1\hfill \end{array}\right.$$

(16)

where ${\mathbf{I}}_v\in {\mathbb{R}}^{v\times v}$ and ${\mathbf{J}}_v\in {\mathbb{R}}^{v\times v}$ are the identity matrix and exchange matrix, respectively, and v is an arbitrary natural number. Referring to Theorem 3 in [21], one can identify that ${\mathbf{C}}_r$ is a real matrix. Performing the singular-value decomposition (SVD) of ${\mathbf{C}}_r$, one has:

$$\begin{array}{cc}\hfill {\mathbf{C}}_r& =\mathbf{U}\mathsf{\Sigma }{\mathbf{V}}^H\hfill \end{array}$$

(17)

where $\mathsf{\Sigma }=\mathrm{diag}\left\{{\lambda }_1,\cdots ,{\lambda }_{m\left(2M-1\right)}\right\}$ consists of $m\left(2M-1\right)$ singular values satisfying ${\lambda }_1\ge \cdots \ge {\lambda }_Q>{\lambda }_{Q+1}=\cdots ={\lambda }_{m\left(2M-1\right)}=0$. The columns of ${\mathbf{U}}_s\triangleq \mathbf{U}(:,1:Q)$ are the singular vectors corresponding to the Q largest eigenvalues, while the columns of ${\mathbf{U}}_n\triangleq \mathbf{U}\left(:,Q+1:m\left(2M-1\right)\right)$ are the singular vectors corresponding to the $m\left(2M-1\right)-Q$ singular values, which are all zeros. The signal subspace is spanned by the columns of ${\mathbf{U}}_s$, whereas the noise subspace is spanned by the columns of ${\mathbf{U}}_n$. Defining $\tilde{\mathbf{b}}(\theta )\triangleq {\left[\mathbf{b}(\theta ),e^{j2\pi \frac{f}{P}}\mathbf{b}(\theta ),\cdots ,e^{j2\pi f\frac{m-1}{P}}\mathbf{b}(\theta )\right]}^T$, one can construct the following function:

$$\begin{array}{cc}\hfill p\left(\theta ,f\right)& ={∥{\left({\mathbf{Q}}_u^H\tilde{\mathbf{b}}(\theta )\right)}^H{\mathbf{U}}_n∥}_2^2\hfill \\ & ={∥{\left({\mathbf{Q}}_u^H\left({\mathbf{I}}_m\otimes \mathbf{b}(\theta )\right)\mathbf{g}\left(f\right)\right)}^H{\mathbf{U}}_n∥}_2^2\hfill \\ & ={\mathbf{g}}^H\left(f\right){\mathbf{T}}^H(\theta ){\mathbf{U}}_n{\mathbf{U}}_n^H\mathbf{T}(\theta )\mathbf{g}\left(f\right)\hfill \\ & ={\mathbf{g}}^H\left(f\right)\mathbf{M}(\theta )\mathbf{g}\left(f\right)\hfill \end{array}$$

(18)

where $\mathbf{g}\left(f\right)={\left[1,e^{j2\pi \frac{f}{P}},\cdots ,e^{j2\pi f\frac{m-1}{P}}\right]}^T\in {\mathbb{C}}^m$, $\mathbf{T}(\theta )={\mathbf{Q}}_u^H\left({\mathbf{I}}_m\otimes \mathbf{b}(\theta )\right)\in {\mathbb{C}}^{m\left(2M-1\right)\times m}$ and $\mathbf{M}(\theta )={\mathbf{T}}^H(\theta ){\mathbf{U}}_n{\mathbf{U}}_n^H\mathbf{T}(\theta )\in {\mathbb{C}}^{m\times m}$. We now show how to estimate DOAs based on the determinant or the smallest eigenvalue of $\mathbf{M}(\theta )$. It can be found that the size of ${\mathbf{T}}^H(\theta ){\mathbf{U}}_n$ is $m\times (m\left(2M-1\right)-Q)$; and if $m\le m\left(2M-1\right)-Q$, the matrix ${\mathbf{T}}^H(\theta ){\mathbf{U}}_n$, in general, is of full row rank, and $\mathbf{M}(\theta )$ is of full rank. However, when $\theta$ coincides with any one of the Q desired DOAs, i.e., $\theta ={\theta }_i,i=1,\cdots ,Q$, the expression in (18) becomes zero by the orthogonality between the signal and noise subspaces. Since $\mathbf{g}\left(f\right)\ne \mathbf{0}$, (18) can hold true only if the matrix $\mathbf{M}(\theta )$ is rank deficient or, equivalently, its determinant (as well as its smallest eigenvalue) is equal to zero. Now, it becomes clear that the determinant of $\mathbf{M}(\theta )$ can be utilized to estimate DOAs as:

$$\begin{array}{cc}\hfill \widehat{\theta }& =\arg \underset{\theta }{min}det\left\{\mathbf{M}(\theta )\right\}.\hfill \end{array}$$

(19)

One solution to (19) is to perform a one-dimensional spectral search to determine the DOA estimates corresponding to the minima of $det\left\{\mathbf{M}(\theta )\right\}$, which approximates to zero. However, it involves more computational cost than JAFE, which exploits ESPRIT without any grid search. If the value of m is small, which is the common case in practice [7,8,9], one can turn (19) into a polynomial rooting to avoid the search process and resolve DOA estimates directly. Denote $z=\beta (\theta )$, then $\mathbf{b}(z)={\left[\begin{array}{c}{z}^{1-M},z^{2-M},\cdots ,z^{M-1}\end{array}\right]}^T$. As a result, a root-RARE polynomial can be constructed as:

$$\begin{array}{cc}\hfill h(z)& \triangleq det\left\{{\left({\mathbf{Q}}_u^H\left({\mathbf{I}}_m\otimes \mathbf{b}(z^{-1})\right)\right)}^T{\mathbf{U}}_n{\mathbf{U}}_n^H\left({\mathbf{Q}}_u^H\left({\mathbf{I}}_m\otimes \mathbf{b}(z)\right)\right)\right\}\hfill \\ & =\sum _{k=1}^{4m(L-1)+1}{r}_k{z}^{k-1-2m(L-1)}\hfill \end{array}$$

(20)

where $r_k$ is the coefficient of the polynomial. Therefore, the roots of (20) ${\left\{z_i\right\}}_{i=1}^Q$ that are closest to the unit circle indicate the desired DOAs of independent signals. The DOA estimates related to the polynomial roots can be determined as:

$$\begin{array}{cc}\hfill {\widehat{\theta }}_i& =\arcsin \left[\frac{\lambda }{2\pi d}arg(z_i)\right].\hfill \end{array}$$

(21)

Once the DOA estimates are obtained, one can perform the frequency estimation as:

$$\begin{array}{cc}\hfill \widehat{f}& =\arg \underset{f}{min}p\left(\widehat{\theta },f\right)=\arg \underset{f}{min}{\mathbf{g}}^H\left(f\right)\mathbf{M}(\widehat{\theta })\mathbf{g}\left(f\right).\hfill \end{array}$$

(22)

Equation (22) is a quadratic minimization problem. Rather than spectral grid search, we provide a closed-form solution to frequency estimation that is more computationally efficient. Let ${\mathbf{e}}_1={\left[1,0,\cdots ,0\right]}^T\in {\mathbb{R}}^m$, then one has the constraint ${\mathbf{e}}_1^T\mathbf{g}\left(f\right)=1$, which can remove the trivial solution $\mathbf{g}\left(f\right)=\mathbf{0}$, and (22) can be rewritten as:

$$\begin{array}{cc}\hfill \widehat{f}& =\arg \underset{f}{min}{\mathbf{g}}^H\left(f\right)\mathbf{M}(\widehat{\theta })\mathbf{g}\left(f\right),\mathrm{s}.\mathrm{t}.{\mathbf{e}}_1^T\mathbf{g}\left(f\right)=1.\hfill \end{array}$$

(23)

By using the Lagrange multiplier $\mu$, one can construct the following cost function [22]:

$$\begin{array}{cc}\hfill \mathcal{L}\left(f\right)& ={\mathbf{g}}^H\left(f\right)\mathbf{M}(\widehat{\theta })\mathbf{g}\left(f\right)-\mu \left({\mathbf{e}}_1^T\mathbf{g}\left(f\right)-1\right)\hfill \end{array}$$

(24)

Setting the partial derivative of $\mathcal{L}\left(f\right)$ to zero, i.e.,

$$\begin{array}{c}\hfill \frac{\partial \mathcal{L}\left(f\right)}{\partial \mathbf{g}\left(f\right)}=2\mathbf{M}(\widehat{\theta })\mathbf{g}\left(f\right)+\mu {\mathbf{e}}_1=\mathbf{0},\end{array}$$

(25)

we find that when $\mathbf{g}\left(f\right)=\nu {\mathbf{M}}^{-1}(\widehat{\theta }){\mathbf{e}}_1$ where $\nu =-\frac{1}{2}\mu$, $\mathcal{L}\left(f\right)$ will achieve its minima. Given ${\mathbf{e}}_1^T\mathbf{g}\left(f\right)=1$, one can derive:

$$\begin{array}{cc}\hfill \nu & =\frac{1}{{\mathbf{e}}_1^H{\mathbf{M}}^{-1}(\widehat{\theta }){\mathbf{e}}_1}\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill \mathbf{g}\left(f\right)& =\frac{{\mathbf{M}}^{-1}(\widehat{\theta }){\mathbf{e}}_1}{{\mathbf{e}}_1^H{\mathbf{M}}^{-1}(\widehat{\theta }){\mathbf{e}}_1}\hfill \end{array}$$

(27)

Given Q DOA estimates ${\left\{{\widehat{\theta }}_i\right\}}_{i=1}^Q$, there are ${\left\{\mathbf{g}\left(f_i\right)\right\}}_{i=1}^Q$ available. To estimate the frequency $f_i$, one can extract the phases from $\mathbf{g}\left(f_i\right)$ as:

$$\begin{array}{cc}\hfill {\mathbf{h}}_i& =\arg \left(\mathbf{g}\left(f_i\right)\right)={\left[0,2\pi \frac{f_i}{P},\cdots ,2\pi f_i\frac{m-1}{P}\right]}^T.\hfill \end{array}$$

(28)

Since the solution of $f_i$ in (28) is overdetermined, one can construct a fitting relationship to find out the desired frequency in the least squares sense, that is,

$$\begin{array}{cc}\hfill {\widehat{\mathbf{c}}}_i& =\arg \underset{{\mathbf{c}}_i}{min}{∥\mathbf{P}{\mathbf{c}}_i-{\mathbf{h}}_i∥}_2^2\hfill \end{array}$$

(29)

where ${\mathbf{c}}_i={\left[c_i,f_i\right]}^T\in {\mathbb{R}}^2$ with $c_i$ being an augmented parameter, and:

$$\begin{array}{cc}\hfill \mathbf{P}& =\left[\begin{array}{cc}1& 0\\ 1& \frac{2\pi }{P}\\ ⋮& ⋮\\ 1& \frac{2\pi (m-1)}{P}\end{array}\right]\in {\mathbb{R}}^{m\times 2}.\hfill \end{array}$$

(30)

The least squares solution of (29) is ${\widehat{\mathbf{c}}}_i={\left[{\widehat{c}}_i,{\widehat{f}}_i\right]}^T={\mathbf{P}}^{+}{\mathbf{h}}_i={\left({\mathbf{P}}^T\mathbf{P}\right)}^{-1}{\mathbf{P}}^T{\mathbf{h}}_i$, and one can directly obtain ${\widehat{f}}_i$ from ${\widehat{\mathbf{c}}}_i$ accordingly.

3.2. Coherent Signals

As emphasized earlier, the above method is restricted to the case of independent signals, and its performance would deteriorate severely provided that there exists coherence between signals due to the multipath propagation. In this subsection, another FOC-based estimator is proposed to tackle coherent signals by extending a dual rotation-invariant property to blind signal separation. More specifically, the dual rotation-invariant property in terms of frequency and spatial signature is first utilized to estimate the frequencies and blindly separate the coherent groups. Then, the DOAs of coherent signals in each group can be estimated after rank recovery by FBSS.

Denoting $\overline{\mathbf{A}}=\mathbf{A}\mathsf{\Gamma }=\left[{\overline{\mathbf{a}}}_1,\cdots ,{\overline{\mathbf{a}}}_K\right]$, referred to as the generalized array manifold, where ${\overline{\mathbf{a}}}_k=\left[\mathbf{a}({\theta }_{k1}),\cdots ,\mathbf{a}({\theta }_{kPk})\right]{\mathbf{\alpha }}_k$ is the generalized steering vector, we first construct a series of cumulant matrices, distinct from (7), as follows:

$$\begin{array}{cc}\hfill {\Delta }_q& \triangleq \mathrm{cum}\left\{{\mathbf{X}}_1(1,n),{\mathbf{X}}_1^{\ast }(1,n),{\mathbf{X}}_q,{\mathbf{X}}_1^H\right\}\hfill \\ & =\mathrm{cum}\left\{\sum _{m=1}^K\overline{\mathbf{A}}(1,m)F_{s,m},\sum _{p=1}^K{\overline{\mathbf{A}}}^{\ast }(1,p)F_{s,p}^{\ast },\sum _{n=1}^K{\overline{\mathbf{a}}}_n{e}^{j2\pi f_n\frac{q-1}{P}}{F}_{s,n},\sum _{r=1}^K{\overline{\mathbf{a}}}_r^H{F}_{s,r}^{\ast }\right\}\hfill \\ & =\sum _{m=1}^K\sum _{n=1}^K\sum _{p=1}^K\sum _{r=1}^K{e}^{j2\pi f_n\frac{q-1}{P}}\overline{\mathbf{A}}(1,m){\overline{\mathbf{A}}}^{\ast }(1,p){\overline{\mathbf{a}}}_n{\overline{\mathbf{a}}}_r^H\mathrm{cum}\left\{F_{s,m},F_{s,p}^{\ast },F_{s,n},F_{s,r}^{\ast }\right\}\hfill \\ & =\sum _{i=1}^K{e}^{j2\pi f_i\frac{q-1}{P}}{\left|\overline{\mathbf{A}}(1,i)\right|}^2{\overline{\mathbf{a}}}_i{\overline{\mathbf{a}}}_i^H\mathrm{cum}\left\{F_{s,i},F_{s,i}^{\ast },F_{s,i},F_{s,i}^{\ast }\right\}\hfill \\ & =\sum _{i=1}^K{\gamma }_i{e}^{j2\pi f_i\frac{q-1}{P}}{\left|\overline{\mathbf{A}}(1,i)\right|}^2{\overline{\mathbf{a}}}_i{\overline{\mathbf{a}}}_i^H\hfill \end{array}$$

(31)

$$\begin{array}{cc}& =\overline{\mathbf{A}}{\mathsf{\Phi }}^{q-1}{\overline{\mathbf{C}}}_s{\overline{\mathbf{A}}}^H\hfill \\ \hfill {\mathsf{\Xi }}_q& \triangleq \mathrm{cum}\left\{{\mathbf{X}}_1(2,n),{\mathbf{X}}_1^{\ast }(1,n),{\mathbf{X}}_q,{\mathbf{X}}_1^H\right\}\hfill \\ & =\mathrm{cum}\left\{\sum _{m=1}^K\overline{\mathbf{A}}(2,m)F_{s,m},\sum _{p=1}^K{\overline{\mathbf{A}}}^{\ast }(1,p)F_{s,p}^{\ast },\sum _{n=1}^K{\overline{\mathbf{a}}}_n{e}^{j2\pi f_n\frac{q-1}{P}}{F}_{s,n},\sum _{r=1}^K{\overline{\mathbf{a}}}_r^H{F}_{s,r}^{\ast }\right\}\hfill \\ & =\sum _{m=1}^K\sum _{n=1}^K\sum _{p=1}^K\sum _{r=1}^K{e}^{j2\pi f_n\frac{q-1}{P}}\overline{\mathbf{A}}(2,m){\overline{\mathbf{A}}}^{\ast }(1,p){\overline{\mathbf{a}}}_n{\overline{\mathbf{a}}}_r^H\mathrm{cum}\left\{F_{s,m},F_{s,p}^{\ast },F_{s,n},F_{s,r}^{\ast }\right\}\hfill \\ & =\sum _{i=1}^K{\gamma }_i{e}^{j2\pi f_i\frac{q-1}{P}}\frac{\overline{\mathbf{A}}(2,i)}{\overline{\mathbf{A}}(1,i)}{\left|\overline{\mathbf{A}}(1,i)\right|}^2{\overline{\mathbf{a}}}_i{\overline{\mathbf{a}}}_i^H\hfill \end{array}$$

(32)

$$\begin{array}{cc}& =\overline{\mathbf{A}}{\mathsf{\Phi }}^{q-1}\mathbf{D}{\overline{\mathbf{C}}}_s{\overline{\mathbf{A}}}^H\hfill \\ \hfill {\breve{\mathsf{\Xi }}}_q& \triangleq \mathrm{cum}\left\{{\mathbf{X}}_1(2,n),{\mathbf{X}}_1^{\ast }(1,n),{\mathbf{X}}_1,{\mathbf{X}}_q^H\right\}\hfill \\ & =\mathrm{cum}\left\{\sum _{m=1}^K\overline{\mathbf{A}}(2,m)F_{s,m},\sum _{p=1}^K{\overline{\mathbf{A}}}^{\ast }(1,p)F_{s,p}^{\ast },\sum _{n=1}^K{\overline{\mathbf{a}}}_n{F}_{s,n},\sum _{r=1}^K{\overline{\mathbf{a}}}_r^H{e}^{-j2\pi f_r\frac{q-1}{P}}{F}_{s,r}^{\ast }\right\}\hfill \\ & =\sum _{m=1}^K\sum _{n=1}^K\sum _{p=1}^K\sum _{r=1}^K{e}^{-j2\pi f_r\frac{q-1}{P}}\overline{\mathbf{A}}(2,m){\overline{\mathbf{A}}}^{\ast }(1,p){\overline{\mathbf{a}}}_n{\overline{\mathbf{a}}}_r^H\mathrm{cum}\left\{F_{s,m},F_{s,p}^{\ast },F_{s,n},F_{s,r}^{\ast }\right\}\hfill \\ & =\sum _{i=1}^K{\gamma }_i{e}^{-j2\pi f_i\frac{q-1}{P}}\frac{\overline{\mathbf{A}}(2,i)}{\overline{\mathbf{A}}(1,i)}{\left|\overline{\mathbf{A}}(1,i)\right|}^2{\overline{\mathbf{a}}}_i{\overline{\mathbf{a}}}_i^H\hfill \end{array}$$

(33)

$$\begin{array}{cc}& =\overline{\mathbf{A}}\mathbf{D}{\overline{\mathbf{C}}}_s{\left(\overline{\mathbf{A}}{\mathsf{\Phi }}^{q-1}\right)}^H\hfill \\ \hfill {\mathsf{\Omega }}_q& \triangleq \mathrm{cum}\left\{{\mathbf{X}}_1(2,n),{\mathbf{X}}_1^{\ast }(2,n),{\mathbf{X}}_q,{\mathbf{X}}_1^H\right\}\hfill \\ & =\mathrm{cum}\left\{\sum _{m=1}^K\overline{\mathbf{A}}(2,m)F_{s,m},\sum _{p=1}^K{\overline{\mathbf{A}}}^{\ast }(2,p)F_{s,p}^{\ast },\sum _{n=1}^K{\overline{\mathbf{a}}}_n{e}^{j2\pi f_n\frac{q-1}{P}}{F}_{s,n},\sum _{r=1}^K{\overline{\mathbf{a}}}_r^H{F}_{s,r}^{\ast }\right\}\hfill \\ & =\sum _{m=1}^K\sum _{n=1}^K\sum _{p=1}^K\sum _{r=1}^K{e}^{j2\pi f_n\frac{q-1}{P}}\overline{\mathbf{A}}(2,m){\overline{\mathbf{A}}}^{\ast }(2,p){\overline{\mathbf{a}}}_n{\overline{\mathbf{a}}}_r^H\mathrm{cum}\left\{F_{s,m},F_{s,p}^{\ast },F_{s,n},F_{s,r}^{\ast }\right\}\hfill \\ & =\sum _{i=1}^K{\gamma }_i{e}^{j2\pi f_i\frac{q-1}{P}}\frac{{\left|\overline{\mathbf{A}}(2,i)\right|}^2}{{\left|\overline{\mathbf{A}}(1,i)\right|}^2}{\left|\overline{\mathbf{A}}(1,i)\right|}^2{\overline{\mathbf{a}}}_i{\overline{\mathbf{a}}}_i^H\hfill \\ & =\overline{\mathbf{A}}{\mathsf{\Phi }}^{q-1}\mathbf{D}{\overline{\mathbf{C}}}_s{\left(\overline{\mathbf{A}}\mathbf{D}\right)}^H\hfill \end{array}$$

(34)

where $q=1,2,\cdots ,m$, ${\overline{\mathbf{C}}}_s=\mathrm{diag}\left\{{\gamma }_1{\left|\overline{\mathbf{A}}(1,1)\right|}^2,\cdots ,{\gamma }_K{\left|\overline{\mathbf{A}}(1,K)\right|}^2\right\}$, and $\mathbf{D}=\mathrm{diag}\left\{\frac{\overline{\mathbf{A}}(2,1)}{\overline{\mathbf{A}}(1,1)},\cdots ,\frac{\overline{\mathbf{A}}(2,K)}{\overline{\mathbf{A}}(1,K)}\right\}$. Then, one can build three cumulant matrices, of size $mM\times nM$, with these blocks as:

$$\begin{array}{cc}\hfill {\mathsf{\Pi }}_1& \triangleq \left[\begin{array}{cccc}{\Delta }_1& {\Delta }_2^H& \cdots & {\Delta }_m^H\\ {\Delta }_2& {\Delta }_1& \cdots & {\Delta }_{m-1}^H\\ ⋮& ⋮& \ddots & ⋮\\ {\Delta }_m& {\Delta }_{m-1}& \cdots & {\Delta }_1\end{array}\right]=\left[\begin{array}{c}\overline{\mathbf{A}}\\ \overline{\mathbf{A}}\mathsf{\Phi }\\ ⋮\\ \overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\end{array}\right]{\overline{\mathbf{C}}}_s{\left[\begin{array}{c}\overline{\mathbf{A}}\\ \overline{\mathbf{A}}\mathsf{\Phi }\\ ⋮\\ \overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\end{array}\right]}^H\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill {\mathsf{\Pi }}_2& \triangleq \left[\begin{array}{cccc}{\mathsf{\Xi }}_1& {\breve{\mathsf{\Xi }}}_2& \cdots & {\breve{\mathsf{\Xi }}}_m\\ {\mathsf{\Xi }}_2& {\mathsf{\Xi }}_1& \cdots & {\breve{\mathsf{\Xi }}}_{m-1}\\ ⋮& ⋮& \ddots & ⋮\\ {\mathsf{\Xi }}_m& {\mathsf{\Xi }}_{m-1}& \cdots & {\mathsf{\Xi }}_1\end{array}\right]=\left[\begin{array}{c}\overline{\mathbf{A}}\mathbf{D}\\ \overline{\mathbf{A}}\mathsf{\Phi }\mathbf{D}\\ ⋮\\ \overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\mathbf{D}\end{array}\right]{\overline{\mathbf{C}}}_s{\left[\begin{array}{c}\overline{\mathbf{A}}\\ \overline{\mathbf{A}}\mathsf{\Phi }\\ ⋮\\ \overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\end{array}\right]}^H\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill {\mathsf{\Pi }}_3& \triangleq \left[\begin{array}{cccc}{\mathsf{\Omega }}_1& {\mathsf{\Omega }}_2^H& \cdots & {\mathsf{\Omega }}_m^H\\ {\mathsf{\Omega }}_2& {\mathsf{\Omega }}_1& \cdots & {\mathsf{\Omega }}_{m-1}^H\\ ⋮& ⋮& \ddots & ⋮\\ {\mathsf{\Omega }}_m& {\mathsf{\Omega }}_{m-1}& \cdots & {\mathsf{\Omega }}_1\end{array}\right]=\left[\begin{array}{c}\overline{\mathbf{A}}\mathbf{D}\\ \overline{\mathbf{A}}\mathsf{\Phi }\mathbf{D}\\ ⋮\\ \overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\mathbf{D}\end{array}\right]{\overline{\mathbf{C}}}_s{\left[\begin{array}{c}\overline{\mathbf{A}}\mathbf{D}\\ \overline{\mathbf{A}}\mathsf{\Phi }\mathbf{D}\\ ⋮\\ \overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\mathbf{D}\end{array}\right]}^H.\hfill \end{array}$$

(37)

Arranging ${\left\{{\mathsf{\Pi }}_i\right\}}_{i=1}^3$ in an appropriate manner, one has:

$$\begin{array}{cc}\hfill \mathbf{Y}& \triangleq \left[\begin{array}{cc}{\mathsf{\Pi }}_1& {\mathsf{\Pi }}_2^H\\ {\mathsf{\Pi }}_2& {\mathsf{\Pi }}_3\end{array}\right]=\left[\begin{array}{c}\overline{\mathbf{A}}\\ \overline{\mathbf{A}}\mathsf{\Phi }\\ ⋮\\ \overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\\ \overline{\mathbf{A}}\mathbf{D}\\ \overline{\mathbf{A}}\mathsf{\Phi }\mathbf{D}\\ ⋮\\ \overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\mathbf{D}\end{array}\right]{\overline{\mathbf{C}}}_s{\left[\begin{array}{c}\overline{\mathbf{A}}\\ \overline{\mathbf{A}}\mathsf{\Phi }\\ ⋮\\ \overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\\ \overline{\mathbf{A}}\mathbf{D}\\ \overline{\mathbf{A}}\mathsf{\Phi }\mathbf{D}\\ ⋮\\ \overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\mathbf{D}\end{array}\right]}^H=\tilde{\mathbf{A}}{\overline{\mathbf{C}}}_s{\tilde{\mathbf{A}}}^H\hfill \end{array}$$

(38)

where $\tilde{\mathbf{A}}={\left[{\overline{\mathbf{A}}}^T,{\left(\overline{\mathbf{A}}\mathsf{\Phi }\right)}^T,\cdots ,{\left(\overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\right)}^T,{\left(\overline{\mathbf{A}}\mathbf{D}\right)}^T,{\left(\overline{\mathbf{A}}\mathsf{\Phi }\mathbf{D}\right)}^T,\cdots ,{\left(\overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\mathbf{D}\right)}^T\right]}^T$ is the extended array manifold.

Now, one can estimate the frequencies and generalized steering vectors via the dual rotation-invariant property. Let ${\tilde{\mathbf{J}}}_1=\left[{\mathbf{I}}_{M(m-1)},{\mathbf{0}}_{M\left(m-1\right)\times M}\right]\in {\mathbb{R}}^{M\left(m-1\right)\times mM}$ and ${\tilde{\mathbf{J}}}_2=\left[{\mathbf{0}}_{M\left(m-1\right)\times M},{\mathbf{I}}_{M(m-1)}\right]\in {\mathbb{R}}^{M\left(m-1\right)\times mM}$, then one can extract two cumulant matrices from $\mathbf{Y}$:

$$\begin{array}{cc}\hfill {\mathbf{Y}}_1& =\left({\mathbf{I}}_2\otimes {\tilde{\mathbf{J}}}_1\right)\mathbf{Y}={\tilde{\mathbf{A}}}_1{\overline{\mathbf{C}}}_s{\tilde{\mathbf{A}}}^H\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill {\mathbf{Y}}_2& =\left({\mathbf{I}}_2\otimes {\tilde{\mathbf{J}}}_2\right)\mathbf{Y}={\tilde{\mathbf{A}}}_2{\overline{\mathbf{C}}}_s{\tilde{\mathbf{A}}}^H\hfill \end{array}$$

(40)

where ${\tilde{\mathbf{A}}}_1=\left({\mathbf{I}}_2\otimes {\tilde{\mathbf{J}}}_1\right)\tilde{\mathbf{A}}={\left[{\overline{\mathbf{A}}}^T,{\left(\overline{\mathbf{A}}\mathsf{\Phi }\right)}^T,\cdots ,{\left(\overline{\mathbf{A}}{\mathsf{\Phi }}^{m-2}\right)}^T,{\left(\overline{\mathbf{A}}\mathbf{D}\right)}^T,{\left(\overline{\mathbf{A}}\mathsf{\Phi }\mathbf{D}\right)}^T,\cdots ,{\left(\overline{\mathbf{A}}{\mathsf{\Phi }}^{m-2}\mathbf{D}\right)}^T\right]}^T$ and ${\tilde{\mathbf{A}}}_2=\left({\mathbf{I}}_2\otimes {\tilde{\mathbf{J}}}_1\right)\tilde{\mathbf{A}}={\left[{\left(\overline{\mathbf{A}}\mathsf{\Phi }\right)}^T,{\left(\overline{\mathbf{A}}{\mathsf{\Phi }}^2\right)}^T,\cdots ,{\left(\overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\right)}^T,{\left(\overline{\mathbf{A}}\mathsf{\Phi }\mathbf{D}\right)}^T,{\left(\overline{\mathbf{A}}{\mathsf{\Phi }}^2\mathbf{D}\right)}^T,\cdots ,{\left(\overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\mathbf{D}\right)}^T\right]}^T$. It is readily identified that ${\tilde{\mathbf{A}}}_2={\tilde{\mathbf{A}}}_1\mathsf{\Phi }$. Then, one can perform the eigen-decomposition of ${\mathbf{Y}}_2{\mathbf{Y}}_1^{+}$ to obtain the frequencies, as well as the scaled column vectors of ${\tilde{\mathbf{A}}}_1$ by the following proposition.

Proposition 1.

The principal eigenvectors and eigenvalues of the generalized space-frequency matrix${\mathbf{Y}}_2{\mathbf{Y}}_1^{+}$are the generalized steering vectors of${\tilde{\mathbf{A}}}_1$scaled by nonzero coefficients and$\mathsf{\Phi }$, respectively, i.e.,${\breve{\mathbf{U}}}_s={\tilde{\mathbf{A}}}_1\mathsf{\Lambda }$and${\breve{\mathsf{\Sigma }}}_s=\mathsf{\Phi }$, where the columns of${\breve{\mathbf{U}}}_s\in {\mathbb{C}}^{mM\times K}$are the K eigenvectors, corresponding to the K largest eigenvalues of${\mathbf{Y}}_2{\mathbf{Y}}_1^{+}$that are contained in the diagonal matrix ${\breve{\mathsf{\Sigma }}}_s$, andΛis an arbitrary diagonal matrix with nonzero entries.

Proof of Proposition 1.

Along with (39), (40) and the rotation-invariant property between ${\tilde{\mathbf{A}}}_1$ and ${\tilde{\mathbf{A}}}_2$, one has:

$$\begin{array}{cc}\hfill {\mathbf{Y}}_2{\mathbf{Y}}_1^{+}{\tilde{\mathbf{A}}}_1& ={\tilde{\mathbf{A}}}_2{\overline{\mathbf{C}}}_s{\tilde{\mathbf{A}}}^H{\left({\tilde{\mathbf{A}}}_1{\overline{\mathbf{C}}}_s{\tilde{\mathbf{A}}}^H\right)}^{+}{\tilde{\mathbf{A}}}_1\hfill \\ & ={\tilde{\mathbf{A}}}_1\mathsf{\Phi }{\overline{\mathbf{C}}}_s{\tilde{\mathbf{A}}}^H{\left({\tilde{\mathbf{A}}}^H\right)}^{+}{\overline{\mathbf{C}}}_s^{-1}{\tilde{\mathbf{A}}}_1^{+}{\tilde{\mathbf{A}}}_1\hfill \\ & ={\tilde{\mathbf{A}}}_1\mathsf{\Phi }{\overline{\mathbf{C}}}_s{\left({\tilde{\mathbf{A}}}^{+}\tilde{\mathbf{A}}\right)}^H{\overline{\mathbf{C}}}_s^{-1}\hfill \\ & ={\tilde{\mathbf{A}}}_1\mathsf{\Phi }\hfill \end{array}$$

(41)

where ${\tilde{\mathbf{A}}}_1^{+}{\tilde{\mathbf{A}}}_1={\tilde{\mathbf{A}}}^{+}\tilde{\mathbf{A}}={\mathbf{I}}_K$ as both ${\tilde{\mathbf{A}}}_1$ and $\tilde{\mathbf{A}}$ are of column rank. By definition, the eigenvector ${\breve{\mathbf{u}}}_k$ and eigenvalue ${\zeta }_k$ of ${\mathbf{Y}}_2{\mathbf{Y}}_1^{+}$ satisfy ${\mathbf{Y}}_2{\mathbf{Y}}_1^{+}{\breve{\mathbf{u}}}_k={\zeta }_k{\breve{\mathbf{u}}}_k$, $k=1,2,\cdots ,K$. Stacking ${\left\{{\mathbf{Y}}_2{\mathbf{Y}}_1^{+}{\breve{\mathbf{u}}}_k\right\}}_{k=1}^K$ in a row, one has ${\mathbf{Y}}_2{\mathbf{Y}}_1^{+}\left[{\breve{\mathbf{u}}}_1,\cdots ,{\breve{\mathbf{u}}}_K\right]=\left[{\zeta }_1{\breve{\mathbf{u}}}_1,\cdots ,{\zeta }_K{\breve{\mathbf{u}}}_K\right]$, and it can be rewritten as:

$$\begin{array}{cc}\hfill {\mathbf{Y}}_2{\mathbf{Y}}_1^{+}{\breve{\mathbf{U}}}_s& ={\breve{\mathbf{U}}}_s{\breve{\mathsf{\Sigma }}}_s\hfill \end{array}$$

(42)

where ${\breve{\mathbf{U}}}_s=\left[{\breve{\mathbf{u}}}_1,\cdots ,{\breve{\mathbf{u}}}_K\right]$ and ${\breve{\mathsf{\Sigma }}}_s=diag\left\{{\zeta }_1,\cdots ,{\zeta }_K\right\}$. Comparing (41) with (42), one can deduce that ${\breve{\mathbf{U}}}_s={\tilde{\mathbf{A}}}_1\mathsf{\Lambda }$, where $\mathsf{\Lambda }$ is an arbitrary diagonal matrix with nonzero entries, as well as ${\breve{\mathsf{\Sigma }}}_s=\mathsf{\Phi }$. This completes the proof of Proposition 1. ☐

Denoting the eigenvalues of ${\mathbf{Y}}_2{\mathbf{Y}}_1^{+}$ as ${\left\{{\eta }_k\right\}}_{k=1}^K$, then all frequency estimates, ${\widehat{f}}_k$, of coherent signals can be obtained by:

$$\begin{array}{cc}\hfill {\widehat{f}}_k& =\arg \left({\eta }_k\right)\frac{P}{2\pi },k=1,2,\cdots ,K.\hfill \end{array}$$

(43)

Let ${\tilde{\mathbf{J}}}_3=\left[{\mathbf{I}}_{mM},{\mathbf{0}}_{mM\times mM}\right]\in {\mathbb{R}}^{mM\times 2mM}$ and ${\tilde{\mathbf{J}}}_4=\left[{\mathbf{0}}_{mM\times mM},{\mathbf{I}}_{mM}\right]\in {\mathbb{R}}^{mM\times 2mM}$, then one can get another two cumulant matrices as ${\mathbf{Y}}_3={\tilde{\mathbf{J}}}_3\mathbf{Y}={\tilde{\mathbf{A}}}_3{\overline{\mathbf{C}}}_s{\tilde{\mathbf{A}}}^H$ and ${\mathbf{Y}}_4={\tilde{\mathbf{J}}}_4\mathbf{Y}={\tilde{\mathbf{A}}}_4{\overline{\mathbf{C}}}_s{\tilde{\mathbf{A}}}^H$ where ${\tilde{\mathbf{A}}}_3={\tilde{\mathbf{J}}}_3\tilde{\mathbf{A}}={\left[{\overline{\mathbf{A}}}^T,{\left(\overline{\mathbf{A}}\mathsf{\Phi }\right)}^T,\cdots ,{\left(\overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\right)}^T\right]}^T$ and ${\tilde{\mathbf{A}}}_4={\tilde{\mathbf{J}}}_4\tilde{\mathbf{A}}={\left[{\left(\overline{\mathbf{A}}\mathbf{D}\right)}^T,{\left(\overline{\mathbf{A}}\mathsf{\Phi }\mathbf{D}\right)}^T,\cdots ,{\left(\overline{\mathbf{A}}{\mathsf{\Phi }}^{m-1}\mathbf{D}\right)}^T\right]}^T$. Evidently, another rotation-invariant property is also valid for ${\tilde{\mathbf{A}}}_3$ and ${\tilde{\mathbf{A}}}_4$, i.e., ${\tilde{\mathbf{A}}}_4={\tilde{\mathbf{A}}}_3\mathbf{D}$. Proceeding similarly, one can resolve $\mathbf{D}$ and the scaled column vectors of ${\tilde{\mathbf{A}}}_3$ by performing the eigen-decomposition of ${\mathbf{Y}}_4{\mathbf{Y}}_3^{+}$.

Let ${\breve{\mathbf{J}}}_p=\left[{\mathbf{0}}_{M\times M(p-1)},{\mathbf{I}}_M,{\mathbf{0}}_{M\times M(2m-p-2)}\right]\in {\mathbb{R}}^{M\times 2M(m-1)}$, $p=1,\cdots ,2(m-1)$, then an improved estimate of $\overline{\mathbf{A}}\mathsf{\Lambda }$ is obtained by performing a series of averages as follows:

$$\begin{array}{cc}\hfill \overline{\mathbf{A}}\mathsf{\Lambda }& =\frac{1}{2(m-1)}\left\{\sum _{p=1}^{m-1}{\breve{\mathbf{J}}}_p{\breve{\mathbf{U}}}_s{\mathsf{\Phi }}^{1-p}+\sum _{p=m}^{2(m-1)}{\breve{\mathbf{J}}}_p{\breve{\mathbf{U}}}_s{\mathbf{D}}^{-1}{\mathsf{\Phi }}^{m-p}\right\}.\hfill \end{array}$$

(44)

By denoting ${\widehat{\mathbf{a}}}_k$ as the $k\mathrm{th}$ column vector of $\overline{\mathbf{A}}\mathsf{\Lambda }$, it is readily seen that the ${\widehat{\mathbf{a}}}_k={\kappa }_k{\overline{\mathbf{a}}}_k={\kappa }_k{\mathbf{A}}_k{\mathbf{\alpha }}_k$, $k=1,\cdots ,K$, where ${\kappa }_k$ is the $k\mathrm{th}$ diagonal entry of $\mathsf{\Lambda }$. Once ${\widehat{\mathbf{a}}}_k$ is obtained, one can apply FBSS r times to a Hermitian matrix constructed from ${\widehat{\mathbf{a}}}_k$ as follows:

$$\begin{array}{cc}\hfill {\overline{\mathbf{Z}}}_k& =\frac{1}{2r}\sum _{i=1}^r\left[{\mathbf{F}}_i{\widehat{\mathbf{a}}}_k{\widehat{\mathbf{a}}}_k^H{\mathbf{F}}_i^H+\mathbf{J}{\left({\mathbf{F}}_i{\widehat{\mathbf{a}}}_k{\widehat{\mathbf{a}}}_k^H{\mathbf{F}}_i^H\right)}^{\ast }\mathbf{J}\right]\hfill \\ & ={\mathbf{F}}_1{\mathbf{A}}_k\left[\frac{|{\kappa }_k|}{2r}\sum _{i=1}^r\left(\begin{array}{c}{\mathsf{\Phi }}^{i-1}\mathbf{\alpha }{\mathbf{\alpha }}^H{\mathsf{\Phi }}^{1-i}+{\mathsf{\Phi }}^{2-l-i}{\mathbf{\alpha }}^{\ast }{\mathbf{\alpha }}^T{\mathsf{\Phi }}^{i+l-2}\end{array}\right)\right]{\mathbf{A}}_k^H{\mathbf{F}}_1^H\hfill \end{array}$$

(45)

where ${\mathbf{F}}_i=\left[{\mathbf{0}}_{l\times (i-1)},{\mathbf{I}}_l,{\mathbf{0}}_{l\times (r-i)}\right]$ with $l=M+1-r$ being the number of elements of a subarray, and $\mathbf{J}\in {\mathbb{R}}^{l\times l}$ is the exchange matrix.

Referring to [23], one knows that $\mathrm{rank}\left\{{\overline{\mathbf{Z}}}_k\right\}=P_k$ after performing FBSS, which implies that the rank deficiency due to the coherence has been successfully recovered. As a result, by applying prevailing high-resolution DOA estimation methods, such as MUSIC or ESPRIT, to ${\overline{\mathbf{Z}}}_k$, one can get the DOA estimates of the coherent signals. Note that $\mathbf{J}{\overline{\mathbf{Z}}}_k^{\ast }\mathbf{J}={\overline{\mathbf{Z}}}_k$, i.e., ${\overline{\mathbf{Z}}}_k$ is the centro-Hermitian matrix, and one can perform the following linear transformation to relieve the computational load as:

$$\begin{array}{cc}\hfill {\mathbf{T}}_k& ={\mathbf{Q}}_l^H{\overline{\mathbf{Z}}}_k{\mathbf{Q}}_l\hfill \end{array}$$

(46)

where ${\mathbf{Q}}_l$ is defined in (16). Referring to Theorem 3 in [21], ${\mathbf{T}}_k$ is a real matrix, and thus, one can resort to unitary ESPRIT [24] to resolve the DOA estimates of the coherent signals. As expatiated above, the frequency, generalized steering vectors and DOAs are estimated by using the rotation-invariant property, so the proposed JAFE estimation method for coherent signals can be referred to as the generalized ESPRIT-like estimator.

Remark 1.

Since the necessary condition for (19) being valid is $m\le m\left(2M-1\right)-Q$, the maximum number of resolvable independent signals by our root-RARE polynomial estimator is $2m(M-1)$, which is twice as much as JAFE. This is because the virtual array manifold $\mathit{B}$ resulting from FOC doubles the DOFs, as well as the effective aperture. By the principle introduced in [23], up to $\lfloor \frac{2M}{3}\rfloor$ coherent signals in the $kth$ group can be resolved, provided that $l\ge P_k+1$ and $2r\ge P_k$. Therefore, considering at most $M-1$ coherent groups can be separated, the identifiability of the proposed generalized ESPRIT-like estimator is upper bounded by $\left(M-1\right)\lfloor \frac{2M}{3}\rfloor$.

Remark 2.

The cumulant requires many more computations than the covariance and, hence, the proposed FOC-based estimator works well on the condition that the observation window is sufficiently long and the target is (quasi-) stationary. We attempted to implement the proposed method in a test bed composed of a Virtex-7 series FPGA and a TMS320C6x series DSP, but found that it is infeasible to complete the joint DOA and frequency estimation within time less than the scale of milliseconds in the case of $M=5$, $m=2$ and $N=2000$ due to the cumbersome calculations of cumulants and the eigen-decomposition of a matrix, of size $18\times 18$. As a result, for some applications, like high-speed moving targets, the coherent processing interval of such cumulants is quite short, and it is hard and even unlikely to realize the proposed algorithm by the prevailing hardware available on the market within the period.

4. Simulation Results and Discussion

In this section, a series of numerical experiments under different conditions is carried out to examine the performance of the proposed algorithms with some state-of-the-art methods. Simulations were conducted for a small number of ULA with half-wavelength spacing between adjacent elements. The existing approaches MR-ESPRIT and U-JAFE and the Cramér–Rao lower bound (CRLB) of joint DOA and frequency estimates [7] were chosen for comparison. For simplicity, we assumed that all signals had an identical power. Similar to the settings in [20,25], the signals were modelled as $\mathbf{s}(t)=\mathbf{F}(t)\mathbf{r}(t)$, where $\mathbf{F}(t)=\mathrm{diag}\left\{f_1(t),\cdots ,f_K(t)\right\}$, $\mathbf{r}(t)={\left[r_1(t),\cdots ,r_K(t)\right]}^T$ with $f_i(t)$ and $r_i(t)$ being zero-mean Gaussian processes with unit-variance and ${\sigma }_s^2$-variance, respectively. The noise was assumed to be a first-order spatial autoregressive process, and the $(a,b)\mathrm{th}$ entry of the noise covariance matrix is given by $\mathbf{R}(a,b)={\sigma }_n^2{0.8}^{|a-b|}{e}^{j\frac{\pi (a-b)}{16}}$ [26,27]. The sampling rate P was fixed at 30 MHz. The accuracy of the DOA estimate, the statistical performance of the algorithms, was measured from 800 Monte Carlo runs in terms of the root mean squared error (RMSE), which is defined as:

$$\begin{array}{cc}\hfill RMSE& =\sqrt{\frac{1}{800Q}\sum _{n=1}^{800}\sum _{i=1}^Q{({\widehat{\theta }}_i^{(n)}-{\theta }_i)}^2}\hfill \end{array}$$

(47)

where ${\widehat{\theta }}_i^{(n)}$ is the estimate of ${\theta }_i$ for the $n\mathrm{th}$ trial and Q is the number of all signals.

4.1. JAFE of Independent Signals

In the first scenario, we considered that three independent sources from $\left[-{52}^{\circ },-{27}^{\circ },-{10}^{\circ }\right]$ impinge on a five-element array, and the corresponding small frequency offsets (central frequencies) for the signals were $1.2$ MHz, $1.6$ MHz and $1.8$ MHz, respectively, while the temporal smoothing factor $m=3$. The resultant RMSE as a function of SNR and the number of snapshots N in the first scenario is illustrated in Figure 1. It can be observed that the proposed FOC-based method outperformed its two counterparts up to 10 dB, while being inferior to them when the SNR became even larger. For the other two schemes, U-JAFE performed slightly better than MR-ESPRIT at low SNRs and the RMSEs of both techniques tended to be the same as SNR increased. In addition, the performance of the proposed estimator improved with the increase of the number of snapshots, whereas the RMSEs of both U-JAFE and MR-ESPRIT stabilized for all snapshot sizes, even though as many as 10,000 snapshots were collected, implying that the estimation performance of these two algorithms was mainly subject to the SNR while immune to the large number of snapshots.

Figure 2 depicts the RMSEs of the frequency estimates varying with changes of the SNR and snapshot size. Clearly, the proposed approach performed the best when the SNR was less than 7 dB or the number of snapshots exceeded 1500, whereas both U-JAFE and MR-ESPRIT were superior to our solution at moderate to high SNRs, and U-JAFE still achieved somewhat better accuracy of estimates than MR-ESPRIT under low SNRs, similar to the case of DOA estimation. Generally speaking, the proposed FOC-based method had a certain advantage over its second-order statistics counterparts in low SNR environments, whereas the latter provided better performance at high SNRs, which is consistent with the general results of the existing work. It should be noted that clear margins arose between the estimates, both DOA and frequency, and the CRLBs. This is mainly due to the fact that the cumulant-based estimator is inherently biased, and algorithms using second-order correlation are vulnerable to coloured noise, which will cause large estimation errors.

As discussed earlier, MR-ESPRIT, U-JAFE and the proposed FOC-based method were able to identify more independent signals than physical sensors, so it is necessary to investigate the performance of three approaches in such an underdetermined case. Assume that six independent sources from $\left[-{52}^{\circ },-{27}^{\circ },-{10}^{\circ },{10}^{\circ },{27}^{\circ },{47}^{\circ },\right]$ impinge on a five-element array ($Q\ge M$); the corresponding small frequency offsets for the signals are $1.2$ MHz, $1.6$ MHz, $1.8$ MHz, 2 MHz, $2.5$ MHz, 3 MHz, respectively, and the temporal smoothing factor $m=3$. It can bee seen from Figure 3 that the absolute estimation performance of all techniques became worse, which is also illustrated by the CRLB, while the advantages of our technique over the other two became more obvious than the overdetermined case, and the counterparts U-JAFE and MR-ESPRIT only worked well above 19 dB and 20 dB, very large SNRs, respectively. Besides, the RMSE of the proposed FOC estimator approached the CRLB as the snapshot size increased, while both U-JAFE and MR-ESPRIT obtained almost the same large errors and remained constant throughout the numbers of snapshots, similar to Figure 1b.

Figure 4 exhibits the performance of the three methods for the frequency estimation, as well as CRLB as a function of SNR and the total number of snapshots. One can observe that generally, the proposed method was distinctly better than the other two algorithms except at 20 dB and above, and U-JAFE outperformed MR-ESPRIT for all SNRs and snapshot sizes, which agrees with the results in Figure 2.

4.2. JAFE of Coherent Signals

In this section, we investigate the JAFE performance of the proposed cumulant-based algorithm first in the underdetermined coherent signals case, i.e., $Q\ge M$. It is assumed that two groups of six coherent signals uniformly distributed between $-{60}^{\circ }$ and ${60}^{\circ }$ are imping on a six-element ULA, and the frequency offsets are $1.2$ MHz and $1.6$ MHz, respectively, as each group of coherent signals was derived from an independent source. The fading amplitudes of the coherent signals were $\left[1,0.9,0.8\right]$ and $\left[1,0.7,0.6\right]$, while the fading phases were $\left[{48.74}^{\circ },{121.15}^{\circ },{35.66}^{\circ }\right]$ and $\left[{9.35}^{\circ },{251.47}^{\circ },{103.56}^{\circ }\right]$, respectively. The temporal and spatial smoothing factors were $m=2$ and $r=2$, and the SNR and number of snapshots were fixed at 15 dB and 5000. In such a scenario, neither U-JAFE nor MR-ESPRIT was able to resolve all DOAs as their DOFs were both upper bounded by four, whereas the proposed method can handle 20 DOAs at most. Figure 5a depicts the DOA estimation results of 50 independent trials by the proposed method. The solid lines mark the true DOAs. It can be seen that each DOA was correctly determined, and the frequency offset estimates were even more accurate, as shown in Figure 5b, since the DOFs used for the frequency estimation were larger than that for the DOA estimation.

Next, we examined the estimation performance of the proposed FOC-based estimator in the presence of multipath propagation. Assume that one group of coherent signals from $\left[-{52}^{\circ },-{27}^{\circ },-{10}^{\circ }\right]$ arrives at a six-element array; the corresponding frequency offsets for the signals are 1.2 MHz, 1.6 MHz and 1.8 MHz, respectively. The fading amplitudes of the coherent signals were $\left[1,0.9,0.8\right]$, while the fading phases were $\left[{48.74}^{\circ },{121.15}^{\circ },{35.66}^{\circ }\right]$, respectively. The temporal and spatial smoothing factors were $m=2$ and $r=2$, respectively. Figure 6 shows the RMSE of DOA estimates in the overdetermined case. It can be found that as the SNR and the number of snapshots increased, the RMSE of DOA estimation decreased gradually for all of the methods, where the accuracy of our algorithm improved asymptotically, whereas the RMSEs of U-JAFE and MR-ESPRIT were close to each other at low and moderate SNRs and tended to saturate at $1^{\circ }$ and ${0.7}^{\circ }$, respectively. Evidently, the proposed FOC-based estimator was significantly superior to the counterparts for all SNRs and snapshot sizes tested, mainly because it exploited the dual rotational invariance of the frequency constructed by the rotation-invariant property in the generalized array manifold.

In Figure 7, we plot the RMSEs of frequency estimates of the three methods as a function of SNR and the total number of snapshots under multipath propagation. It can be seen that the proposed solution had a prominent advantage over the other two algorithms for almost all the SNRs and snapshot sizes, although its lead kept decreasing as the SNR rose and turned out to be lost at 19 dB and above.Unlike the illustration in Figure 2, the performance of frequency estimation of U-JAFE was strictly worse than MR-ESPRIT over the whole range of SNR and snapshot size for coherent signal estimation. Additionally, the RMSEs of the frequency estimates by our method approached the CRLBs more tightly than the results in Figure 2 and Figure 4 since all three coherent signals shared an identical frequency, and the number of frequencies to be estimated turned out to be one less than the cases studied above.

5. Conclusions

The joint angle and frequency estimation of non-Gaussian signals in coloured noise is addressed in this paper, and cumulant-based estimation algorithms are developed for independent and coherent signals, respectively. The proposed methods make efficient use of the dual rotational invariance of the newly-constructed array manifolds to enhance the DOF of a ULA, thus improving the estimation accuracy and resolving more signals than the number of array elements, either for the independent or coherent signal. For independent signals, we construct an augmented cumulant matrix, providing twice the DOFs of the covariance matrix formed by the state-of-the-art techniques, then transform it to the real number field and resolve the DOA estimates by using rooting of a polynomial resulting from the RARE property, and the frequency estimates can be resolved in a least squares manner given the DOA estimates already acquired, avoiding the spatial spectra search and reducing the consumption of calculations accordingly. For coherent signals, the generalized steering vectors (coherent groups) and the frequencies are blindly separated from each other by exploiting a new dual rotational invariance arising from a series of dedicated cumulant matrices. Then, FBSS is performed to restore the inherent rank deficiency of the generalized steering vectors, and finally, the DOA estimates are determined by using the unitary ESPRIT algorithm. Since the coherent signals are divided and conquered, into their associated group, more coherent signals than array elements can be handled. Compared with previous second-order statistics methods, the proposed solutions are efficient in the sense that by extracting the temporal correlation embedded in the high-order statistics, they further raise the array DOFs and effective aperture, as well as improving the estimation accuracy, especially for coherent signals, in addition to the robustness to coloured noise.