1. Introduction
The estimation of two-dimensional direction of arrival (2D-DOA) is essential in the field of array signal processing, and has been widely applied in military and civilian uses such as radar, sonar and wireless communication. After decades of development, many high-resolution methods are studied, including subspace-based algorithms [
1,
2,
3], sparse recovery (SSR) algorithms [
4,
5,
6,
7] and tensor-based algorithms [
8,
9,
10]. As the sensing distance decreases and scenarios become more diverse and complex in modern sensing systems, researchers are no longer limited to the idealized assumption of far-field target localization. As a result, extensive studies have been conducted on near-field DOA estimation and mixed near-field and far-field localization problems [
10,
11]. Furthermore, under challenging conditions such as signal coherency and underdetermined observations, limitations of the DOA estimation methods relying solely on second-order statistics have become increasingly evident. To further exploit the information embedded in array measurements, researchers have explored third-order and higher-order cumulant-based approaches, along with tailored array design strategies, to further enhance the degrees of freedom [
12,
13]. However, most algorithms studied focus on a scalar sensor array, where sensors receive signals omnidirectionally in space. As an inherent property of electromagnetic waves, polarization makes incident signals vary along the receiving direction. A vector sensor array is proposed to receive the signal difference due to the polarization in order to achieve better performance [
14]. Compared with traditional scalar sensor, a electromagnetic vector sensor (EMVS) collects more information of incident signals with better anti-interference ability, stronger detection ability, and higher system resolution. Normally, a full EMVS has six components including three dipoles and three loops co-located for measuring the electric and magnetic field of a signal, respectively. However, compared with a scalar sensor that has only one component, having more receiving components also leads to higher data dimensionality, which leads to a significant increase in computational complexity in signal processing [
15,
16]. An alternative solution is to reduce the components of the EMVS. At present, most research is still based on a crossed-dipole sensor array [
17], and X. Lan has theoretically proven that a uniform linear array (ULA) with tripole sensors can achieve 2D-DOA estimation [
18]. Compared with the crossed-dipole sensor, the tripole can obtain complete electric field information, and a tripole array can exploit the quantitative relationship between polarization in three different directions. Therefore, the research is studied on a ULA with a tripole sensor.
Most algorithms assume that signals are independent of each other. But in practice, there exist coherent signals in received signals due to multipath effects or electronic countermeasures. The coherency between signals results in the inability to properly separate the signal subspace and the noise subspace, making it difficult for subspace-based algorithms based on eigen-decomposition to accurately estimate DOA and polarization parameters of incident signals. To solve this problem, J. Li applied the spatial smoothing (SS) algorithm to EMVS arrays and achieved DOA estimation with the ESPRIT algorithm [
19]. SS correctly resolves the separation between the signal subspace and the noise subspace, but it causes a reduction in the effective array aperture due to subarray partitioning. In addition, the maximum number of coherent signals that can be handled is limited by smoothing times [
20]. Rahamim proposed and conducted many studies on the polarization smoothing algorithm (PSA) [
21]. The PSA is not limited by array structure without loss of array aperture, making it applicable to arbitrary-shaped arrays. But it ruins the structure of polarization steering vectors and cannot continue following polarization parameter estimation [
22]. In an acoustic vector sensor array, Palanisamy and Liu handled the rank-deficit issue by constructing structured matrices based on cross-correlation between subarrays to achieve decorrelation in an L-shaped array [
23] and a uniform rectangular array [
24], respectively. In scenarios where coherent and non-coherent signals coexist, Molaei did not explicitly resolve the rank-deficit issue, but instead proposed a clustering-based efficient separation method to distinguish the two types of signals, enabling subsequent DOA estimation to be performed separately [
25]. In addition, several non-subspace-based algorithms, such as subspace fitting algorithms [
26] and sparse recovery algorithms [
27,
28,
29], have also been employed to handle coherent signal scenarios, owing to their insensitivity to inter-signal coherency. To address the rank deficient problem, in addition to smoothing methods, another class of algorithms is the Toeplitz matrix reconstruction approach, which takes advantage of the fact that an ideal covariance matrix exhibits the Toeplitz–Hermitian property. By reconstructing a smaller-dimension, full-rank matrix that satisfies this property—using, for example, eigenvectors of the covariance matrix [
30] or the received array data [
31,
32]—as an equivalent covariance matrix, more accurate DOA estimation can be achieved. However, these methods only utilize partial information from the covariance matrix.
Recently, a method called multiple-Toeplitz Hermitian matrices reconstruction (MTOEP) and its improvements have been proposed to restore the rank of the covariance matrix and achieve superior performance in estimation accuracy [
33,
34], which exploit the shift-invariance property of the array and the Toeplitz structure in an ideal covariance matrix. Compared with the previously mentioned algorithm [
30,
31], MTOEP does not require auxiliary vector sensors or denoising processing and can fully utilize all array outputs. This more comprehensive use of covariance information theoretically improves estimation accuracy. It is observed that these state-of-the-art algorithms have not yet been investigated or applied in vector sensor arrays, and their potential remains largely unexplored, suggesting considerable prospects for further study. In this paper, we combine the ESPRIT algorithm with the block multiple-Toeplitz matrix reconstruction (BMTOEP) method, and propose a new algorithm to overcome the difficulty of 2D-DOA estimation of coherent signals in a uniform linear tripole array. Firstly, a novel preprocessing approach based on block-Toeplitz matrix reconstruction is applied to remove the coherency between incident signals. Then, the array is partitioned into different subarrays twice to obtain three rotational invariant factors using the ESPRIT algorithm. Finally, unambiguous DOA and polarization parameters are estimated from the invariants.
The structure of this paper is as follows. The coherent signal model for the EMVS array is presented in
Section 2 and the proposed method is developed in
Section 3. In
Section 4, we provide numerical simulation results. Finally, conclusions are drawn in
Section 5.
Notions: The superscripts , , and represent the operations of transpose, Hermitian transpose, conjugate and pseudo-inverse, respectively; ⊗ stands for the Kronecker product. ⊙ stands for the Khatri-Rao product; and are to extract the real part and imaginary part, respectively; and and are to get the expectation and phase angle, respectively.
2. Problem Formulation
Consider a ULA with
tripoles along the
y-axis as shown in
Figure 1. Each tripole consists of three co-located mutually perpendicular dipoles for measuring electric field information. Three components of each tripole are aligned with the
x-,
y-, and
z-axes, respectively. All tripoles are indexed as
. The interval between adjacent array elements is set to
d and the carrier wavelength of the arrival signals is
.
Assuming the array is impinged by
K narrowband, far-field fully-polarized waves (i.e., TEM waves) parameterized by (
,
,
,
),
.
,
,
and
represent the elevation angle, azimuth angle, polarization auxiliary angle and polarization phase difference of the
k-th signal, respectively. Assuming that some of the
K incident signals are coherent, the first
signals are coherent with each other, and the remaining
signals are independent to other signals. The complex envelope received at sensor
n can be expressed as:
where
is the complex envelope of the
k-th signal, and
is the additive white noise vector at sensor
n.
is the polarization response vector for the
k-th incident signal.
denotes the amplitude and phase difference between the
k-th coherent signal and the reference signal. In the above formula,
. For
,
takes the form
. According to the geometric symmetry, the array output at time
t is given by:
where
denotes a
signal vector. The correlation between signals can be arbitrary. The steering matrix
is composed of the steering vectors and can be expressed as:
As the
k-th column of matrix
,
has the following form:
where
denotes the spatial steering vector of the
k-th incident signal, which is formulated as:
For a tripole used in the array, following the standard model [
18], the polarization response vector
can be expressed as:
The additive noise
is a
zero-mean white Gaussian noise vector with variance
which is independent of all signals. The covariance matrix of the array output is given by:
where
is a
identity matrix.
is the covariance matrix of the source signal vector, which is defined as:
where
and
denote the covariance matrix of the first
P signals and remaining
signals, respectively. Notice that
, so
, which means that
is rank deficient. Consequently, signal subspace and noise subspace cannot be correctly separated, which leads to the failure of traditional subspace-based algorithms. Hence, necessary preprocessing is required.
4. Simulation Results
In this section, we conduct numerical simulations to verify the effectiveness of our proposed method in terms of root-mean-square error (RMSE). The average RMSE is defined as:
where
W is the replication number of Monte Carlo runs, and
is the estimation of
in the
w-th Monte Carlo trial. It is considered a successful resolution when the spatial angle error is less than 1°. For simplicity, we suppose that the power of all signal sources equals
, and the signal-to-noise ratio (SNR) is defined as
. In all simulations, consider two coherent TEM signals impinging on a ULA composed of seven tripoles with
. Each incident signal is assumed to be fully elliptically polarized. The polarization state is characterized by the auxiliary polarization angle
and the phase difference
, as defined in
Section 2. In the simulations, the polarization parameters of each source are fixed when generating the received data.
4.1. Effectiveness of the Algorithm
To validate the effectiveness of the proposed algorithm, both polarized-MUSIC and the proposed algorithm are applied under identical conditions. Two incident signals are parameterized by
,
, respectively. The number of snapshots is 500, and the SNR is set to 15 dB. A total of 100 Monte Carlo experiments are performed. According to the simulation results shown in
Figure 3, multiple spurious peaks and a broadened main lobe appear in the polarized-MUSIC spatial spectrum due to the coherency between incident signals, making it impossible to distinguish coherent sources. This indicates that decorrelation processing is necessary before applying subspace-based methods for parameter estimation.
In
Figure 4, the estimated DOA–polarization pairs form two compact clusters around the true values, and no noticeable outliers are observed, showing that both directions and polarization parameters have been identified and paired successfully with high estimation accuracy.
4.2. RMSE Performance Versus SNR and Number of Snapshots
In this part, we design experiments to test the accuracy of DOA estimation. To evaluate the performance of multiple parameter estimations, the average RMSE is used as the performance metric. First, we investigate the RMSE performance of three decorrelation preprocessing methods (ESPRIT-like [
35], FSS [
19], and the proposed BMTOEP algorithm), as well as the case where the ESPRIT algorithm is directly applied without any decorrelation preprocessing, with respect to SNR. The ESPRIT-like method represents a low-complexity decorrelation approach using a single reference sensor. The Cramér–Rao Lower Bound (CRB) [
37] is also included as a benchmark in the simulation results. It should be noted that the CRB is obtained via numerical evaluation based on the adopted data model, rather than a closed-form analytical derivation. The same signal parameters as in
Section 4.1 are adopted, while the SNR varies from 3 to 15. The predetermined number of snapshots is 1000. After 100 Monte Carlo trials, the RMSE of DOA estimation for two coherent signals is illustrated in
Figure 5.
As shown in
Figure 5, the RMSE of all methods decreases as SNR increases. The method without decorrelation preprocessing is ineffective due to its significant errors. The performance of the ESRPIT-like algorithm is poor and becomes effective only when the signal-to-noise ratio exceeds 10 dB. Besides, the performance gap between ESPRIT-like and the CRB gradually decreases as SNR increases. It can also be observed that the proposed method maintains a smaller gap to the CRB and produces slightly lower RMSE than the FSS method across the entire SNR range. This is because FSS uses only the autocorrelation information on the principal diagonal of the signal covariance matrix to obtain an equivalent covariance matrix. In contrast, the proposed method utilizes the total information to reconstruct the equivalent source covariance matrix.
To further assess the performance of the proposed algorithm, we compare the DOA estimation RMSE versus the number of snapshots.
Figure 6 illustrates the comparison of RMSE curves for different algorithms as the number of snapshots varies. The simulation conditions are the same as above, except that SNR is fixed at 15 dB and the number of snapshots varies between 100 and 2100.
In
Figure 6, the RMSE of all methods generally decreases as the number of snapshots increases, indicating improved estimation accuracy with more available data. It can be observed that the ESPRIT-like method exhibits a performance saturation when the number of snapshots exceeds 500. This is mainly attributed to the inherent limitations of the decorrelation strategy adopted in that method, where the available array information is not fully utilized, and thus increasing the number of snapshots does not further improve the estimation accuracy. It also demonstrates that the proposed method consistently achieves the lowest RMSE across the entire snapshot range, while FSS exhibits slightly higher errors but follows a similar downward trend. The method without decorrelation remains inaccurate regardless of the number of snapshots.
4.3. RMSE Performance Versus Angular Separation
To assess the resolution and probability of successful estimation, we evaluate the algorithm performance and success probability based on angular separation between two coherent signals. The azimuth angles of both signals are identical and fixed at 100°. The pitch angle of signal 1 is fixed at 30°. The polarization parameters of signal 1 and signal 2 are (20°, −80°) and (50°, 80°), respectively. The spatial separation angle between the two signals varied from 2° to 14° by changing the pitch angle of signal 2. A total of 100 Monte Carlo trials are performed with an SNR fixed at 15 dB, and the number of snapshots is set to 1000.
As shown in
Figure 7 and
Figure 8, both the DOA estimation accuracy and the probability of success improve as the angular separation between the two coherent sources increases. Compared with the CRB, the proposed method exhibits the smallest performance gap across the entire separation range. When the separation is below 4°, the RMSE curves of the proposed method and ESPRIT-like method are close to each other. As the separation grows, the proposed method maintains the lowest RMSE, while the gap between the proposed method and FSS gradually narrows. Moreover, the proposed method approaches CRB more closely as the separation grows because the steering vectors of the two sources become more distinguishable. For sufficiently large separations, the proposed method and FSS tend to achieve comparable estimation accuracy, while the ESPRIT-like retains noticeably higher errors.
Figure 8 shows that the probability of successful resolution increases with angular separation. The proposed method achieves the highest success rate when the separation is small. All algorithms achieve a 100% success rate when the spatial separation angle exceeds 12°. These results suggest that the proposed method has advantages in scenarios where coherent sources are closely spaced.
4.4. Computational Complexity Versus Number of Sensors
In the last experiment, the computational burden of different algorithms is compared in terms of CPU runtime. We evaluate the runtime performance of the proposed algorithm, FSS-MUSIC and the ESPRIT-like algorithm with the number of sensors
M varying from 7 to 19. The average runtimes were obtained from a PC with an Intel(R) Core(TM) i3-7100 3.9 GHz CPU and 8 GB RAM by running Matlab (Ver. 2021a) codes for 50 independent trials. We set the number of snapshots to
and the number of incident signals to
. For the FSS-MUSIC algorithm, the smoothing time is
. The comparison results in
Figure 9 show that the runtime of FSS-MUSIC increases rapidly with the number of sensors and is significantly higher than that of the other two methods. This is mainly because it relies on spectral peak searching, which leads to higher computational complexity. In contrast, both the proposed algorithm and the ESPRIT-like algorithm follow the ESPRIT framework and therefore avoid the spectral search procedure, resulting in substantially lower runtime. Although the ESPRIT-like method achieves the lowest runtime, the proposed algorithm maintains a comparable computational cost while providing improved estimation performance in previous experiments. These results demonstrate that the proposed algorithm achieves a favorable trade-off between estimation accuracy and computational efficiency, making it suitable for practical engineering applications.
5. Discussions
From the simulation results, it can be observed that the proposed BMTOEP decorrelation method provides improved estimation accuracy and resolution compared with classical approaches. It should be noted that the proposed algorithm is developed under the ULA model and relies on the shift-invariance property of the array, which leads to a block-Toeplitz-structured covariance matrix. Nevertheless, the core idea of covariance reconstruction is not inherently restricted to ULA. The proposed method can be extended to other array configurations that preserve similar translational invariance, such as uniform rectangular arrays (URAs), where a multidimensional Toeplitz structure can be constructed. For array geometries that do not satisfy such properties (e.g., nested or coprime arrays), the proposed method may still be applicable by transforming the array into an equivalent virtual uniform array through appropriate preprocessing techniques. However, such extensions require additional processing steps and are beyond the scope of this work.
In addition, this work assumes ideal electromagnetic vector sensors and does not explicitly consider mutual coupling effects. In practical array systems, mutual coupling may distort the array manifold and degrade estimation performance. This effect can be incorporated into the signal model by introducing a mutual coupling matrix that modifies the steering vector. If the coupling matrix is known or can be accurately calibrated, its impact can be compensated accordingly. Otherwise, more advanced calibration or robust estimation techniques may be required. The investigation of mutual coupling effects and their mitigation is an important topic and will be considered in future work.